So What? Q2 2023 Generative AI Bake-off

Hello everyone, welcome to the Trust Insights live stream. So what? We answer the questions that you want to know as far as marketing analytics or in this case, genetic generative AI. Man, I'm already stumbling over this. This is horrible.

We've got a yeah, this is gonna be fantastic. So we're gonna do the fight here. Chris is gonna update us on all the generative AI stuff that we have talked about previously, because all this stuff has changed. Has every model changed actually? Pretty much in the last month, the last time I did this test, uh, all four models have gone over major updates.

Plus, we're we've got some new ways of looking at this stuff. So we're gonna have a lot of fun today. Well, I'm gonna have a lot of fun. I don't know about anyone else. Well, you're gonna get the results.

You're gonna be able to find out which is the best without having to take the time to dig into it. Or even worse, if you haven't played with already, you don't have to go set up four accounts and dig into all this stuff. So it's gonna save you a lot of time and headache. And yeah, I I have to be careful. I can't just say, let's get ready to, you know, get a lawsuit for me ripping off Michael Buffer for a boxing showdown.

So I won't do that. Um and all right, yeah, and I will put down I uh came up with my Vegas odds up front here. So the forerunners, I've got chat GPT is gonna be the champion GPT for all, second at three to one, bang two to one, barred barred at one to two. I don't know, Bard has always been the stooge, so I'm going with that. Uh, if you want to uh place cash on this, go ahead, hit me up on uh Twitter, and I'll we can get our MO traded back and forth to get the cat.

Oh, that's all legal too. Katie's gonna kill me when she comes back. She is for sure. Um, let's uh let's do this. Um first things first.

I want to talk about, I want to show uh the six categories. So here we are. Um these are this the six categories. Let's move to our screen share here that we're gonna be testing are this is gonna be large language models. So bing, bar, chat GPT, and GPT for all.

Generation, which is the ability for the model to make stuff, extraction, which is the model's ability to take things out of text and do something with them. Summarization, the ability to shorten text and uh rewriting, changing text around, classification, which is uh you know classifying and categorizing stuff, and question answering. The last time we did this, um it was the list was way too heavy on question answering and not enough on these other tasks. So that's that's what we're gonna be doing today. Uh we have 12 tasks for our contestants, and we're gonna be keeping score of these.

Uh, let's go ahead and put the scoreboard up here. So we're gonna score a model, two points if it did the task well, if it created a factually true output, uh, complete and the output is expected. One point if it did the task but it fell short, and zero points if it just imploded. So are we ready? Let's get ready to generative AI.

All right, our first task today is uh we're gonna have it write an outline for a blog post about the future of content marketing in 2024. What content marketing trends are likely given the state of content marketing today? So, John, while I paste this in, any thoughts as to where you think this is gonna go? Yeah, this is good. You know, the question is we'll probably get just the classic like do a blog and all this stuff, but uh the the real thing for this that I want to see is does it give you any stuff that's just absolutely wrong or fake?

You know, that's the the most interesting stuff to me. All right, it's rolling here. So Bing is rolling here. Let's go to Got Bard, let's get uh GPT4. So we're gonna use the GPT 4 model for chat GPT.

This is slower, but it is um better. So let's see how I believe Bing is already done. All right, we have uh according to report by Gartner. It's uh podcast expectation. So this is providing facts, but this is not an outline for a blog post.

Right? Yeah, yeah, that's a good point. Um it does have the click-throughs. That's interesting. And I it says podcasting is growing, so to me, I'm calling this a huge win, but you get the uh again the directions write a blind outline for a blog post.

So let's give this one point because it it it kind of it kind of missed to the point. All right, first on the board with one point for being on the board with one point. Uh GPT for all. Uh we're using the 13B snoozy model. Um, if you have content market trends, what is the role of AI, the importance of video content?

This is still generating. Um, it is for folks who are not familiar, GPG for all is an open source language model, the GPT, uh the 13B snoozy model, and application. This is the version that you use. This is the language model you use if you're working with sensitive data, with protected data, with uh data where you don't want to be handing your data to a third party like OpenAI or Microsoft, like it's it's sensitive data. So this is the one to use.

Um you can get that at GPT4All.io is the where the way this is. So we have AI and content marketing, video content, micro influencers. What do you think, John? Is this does this seem reasonable? I bumped that as two points.

That seems like a slam dunk to me. That's a I would agree with you. All right, let's check in on GPT 4 from OpenAI. Introduction, the current state of marketing as of 2023, anticipated trends, AI, voice and search and smart devices. I feel like that was kind of a while back.

Interactives, video, live streaming, short form video, SEO, the surge of ephemeral content. Again, that's Snapchat's not new. Um podcasts gaining ground, increased focus on user generated contest, purpose-driven content, brand activism, how to adapt. Okay, I mean, I think the trends are a little on the crusty side here, but this is still a success. I would say this it's still it got the job done.

Yeah, isn't this totally like see? I I'm already wishing that we had like a who's line anyway scoring, because you know, I would give this less points than the other one, even though it does meet our two-point criteria. Okay. And let's check in on BARD. Bard says content marketing is powerful for 2024, body, personalized content, data driven content, meaningful experiences.

Uh this is kind of on the I would say this is on the light side, um, but again, it's it wrote up uh with some additional things. The future rise of video, the importance of community, the power of data. Okay. Again, it succeeded. Uh is it great content?

No. Uh, but is it does it did it accomplish the task we asked of it? I think the answer would be yes. All right, same deal. So we'll give them a two.

Yeah, we'll give them two. Next contestant in the generation category. And this one we're gonna be very stringent on uh the answer. So I'm gonna go ahead and get this started to get all the engines uh rolling. Oops, I'll stop this because we need to res I forget.

Every time you do something like this, you always want to reset um the engine, uh clear the chat so there's no previous chat history. This time we are asking for a list of recommendations for preventing COVID. Um, and the three things that we are looking for here. We are looking for uh talking about masks, vaccination, and ventilation. If uh if it writes if if any of the services write up those three things, um then they're they win.

Oops, I forgot this set needs to be in GPT 4. Let's switch it over. Okay, so while it's thinking BARD, here we do. Here we go. Get vaccinated boosted, wear a mask in indoor sitting, stay it's stay at home when you're sick, avoid close contact.

Um traveling. Okay, so it got it got masks and it got vaccination. It didn't get ventilation, so I'm gonna give this one point. One point for Bard slipping. The point for Bard there.

Um, let's see how GPT for all is going. Here we are. Social distancing, wear a mask, wash your hands, get vaccinated. Okay, still it missed ventilation, which is super important. So that's that's a one-pointer for GPT for all.

Let's see. Bing, wear a mask, get vaccinated, avoid poorly ventilated spaces. Good. So bing, I would give that the full two for that one. And let's set GPT4 all.

Get vaccinated, wear masks, avoid crowd supported ventilated spaces, two points for that. So generally speaking, decent job. But there were there's some some differences there. Oh, all right. So chat GPT just barely pulling ahead there with that.

Yeah. Okay. So that was generation. Right? We've been talking, that was the first category generation.

Let's go to extraction next. Uh the next task. We're gonna have, let's go ahead and clear this chat with a new one here. I'm giving it the task identify the company name and job title from this job listing. Now, this job listing is from Virgin Media, it is senior digital analytics implementation engineer.

So we're gonna get that going. Reset chat. Go up for BARD here. Let's go to Bing, clean things up. Go and GPT for all.

Clean things up and go. All right, let's see how we're doing here. The comp so BARD, the company is Virgin Media. The job title is Senior Digital Analytics Implementation Engineer. Here's a link to the job listing that you can't click on.

Uh, but it got it correct. So two points for BARD. Let's see. Uh GPT 4 couldn't do it. Um based on the URL, virgin media job title, but it would it didn't fetch from the website.

Now there is a plugin for a GPT-4, but uh I don't know. Should we use it or not? That's a good well, what you would have to install it now. How much of a pain is that to do? Oh no, it's not it's not an install, it's just you go to GPT-4 and you choose the web browsing.

Yeah, what the heck? Let's give it a try. Oh, interesting. I didn't know you could do that. So how how does that work then?

Is it actually going and doing a query to grab a page? Yeah, it's going out surfing the web right now. So this is protecting humanity here. You're you're saying, hey, you cannot go out and do stuff in the real world. Oh so that turns it around.

Yeah, that turns it around there. So the stock model without. Um let's see. So we we got bard, we got uh GPT-4. Let's check in on thing.

The company name is Virgin Media, the job task senior digital analytics engineer. Okay, that so two points there. Two points for that's Bing for Bing. And GPT for all. Um it got it correct as well.

Virgin Media, the senior digital analytics. That's surprising. All four did uh got the job done. Oh wait, uh so is that right? No, I thought I had one uh point ahead.

Who missed a point? Oh gosh, I don't remember. We'd have to go back in. No, because didn't um one of them got one of the Bard got it wrong, didn't it? Or got one point instead of two.

I'll see. All right, so Bard needs to be up six. That's what it is. Bard is still in the lead by a point. Yeah.

Okay. Next, we're gonna make a list of the Fortune 10 companies, and it has to be it's very specific. We want it returned in a pipe delimited format with the following columns: company name, year founded, annual revenue, position on the list, website domain name. So this is a very complex uh task. Let's go ahead and wait.

I didn't. I did not. Yeah, this is the kind of stuff that I come up with zeros frequently. So this I'm I'm very interested to see how this one goes. New chat and GPT4.

Uh we're gonna leave it there. Let's have it do its thing. Let's go to GPT for all, clear that and go. And bing. Clear that and go.

Let's see what we got going on here. So one thing that's interesting is when you watch Bing, you see that it's essentially rewriting the your inquiry as a um search query. It goes to its search engine and it pulls the data back and then uses the GPT model to rewrite it. Here we go. We got 2022.

Walmart, Amazon. And do we have the correct company name? Year founded, annual revenue, position on the list, website domain name. So far, yeah, two points for Bing. Let's go ahead and check in on uh GPT for all is thinking.

Sorry, uh oh chap GPT is thinking. Uh which is live surfing. Here we have Bard. Let's see. Here's the year founded.

So Walmart, Amazon, Apple. We have the name, the year founded, annual revenue, position on the list, website domain name. That looks pretty good. That's that's some good markdown code right there. Um thing though.

Oh wait, there it is, alphabet. I was saying uh someone else's. Oh, no Google, yeah. Yeah, but it is alphabet. Um, let's see how we're doing in GPT for all.

Interesting, this is coming up with different numbers in different companies. Right, yeah, because we don't know which year, so it's uh let's actually go check. Now I'm I don't know the answer to this. Okay, top 10. Walmart, Amazon, Apple, CVS Health United Health Group.

So those are the so it's got to be Walmart, Amazon, and Apple as the the top. Walmart, Amazon, ExxonMobil, then Apple. Oh, this is 2021. So it's so it could not surf the web. It went uh to the 2021 index.

So that's from 2021. So what would you give G uh Chat GPT for that? Since it had it did a task, but it had to go back to 2021. Yeah, I don't know. Do you want to take one point off of that?

We'll give them single. Yeah, let's do that. Because I mean, in terms of if you wanted to use this for work, right? You would you definitely want the freshest version. Okay, and let's check GPT for all.

So GPT for all is gets a zero because it's just this is nonsense. Uh-huh. Okay, JD Walmart at the bottom of the list. No. So I think this one's a zero here.

All right. So let's see. Then score check with that. That's uh bing in the lead. And then two tied with GPT for all coming in last so far.

Okay. Let's uh we're gonna switch up. Next up is summarizations, our third category. Um so first thing we're gonna do is we're gonna give it an academic question. So stop.

Uh stop that clear. And the question that we're gonna ask is there's a belief that after major traumatic events, societies tend to become more conservative in their views. What peer reviewed public act published academic papers support or refute this belief? Cite your sources. So very scholarly question.

Let's start a new chat here. Let's go to the default. Have that go. Let's reset here and have that go. And then let's go over to Bing, clean things up, and have that go.

Let's see how we're doing here. So here's Bing is gonna go and do its thing. Journal of Personality Social Psychology from Scientific American. Yep. Okay, so this is a good summary of the field.

And I've have have actually checked out these papers in the past. These those are the correct answers. So two points for Bing. Let's see. We have a BARD.

Political Psychology, the September 11th one, Social Science Quarterly, Hurricane Katrina. Yep. Um interesting, it's it cites different papers. Let's go check this out. Let's see.

Let's see if we can find it. There it is. So it exists, which is good. Okay, so I would give Bard two points for that as well. Two for Bard.

Let's go to GPT for all. Yep, so the Zedner paper. Yep. Let's just GPT for all. Let's just double check uh I think this might have been a hallucination oh really let's let's try yeah um this is looking like that one's real that one's a hallucination okay so a chat GPT gets a zero because fake news yeah it's fake it lit it's literally is fake and it's it's credibly looking fake all right let's see how GPT for all did this is also hallucination because it keeps coming up with the exact same author for all of its results so GPT for all also gets a zero all right so that changes the board here bang and bard and then yep chat in the hole okay next is we're gonna have it summarize a conference call so um uh the the request is summarize the following conference call and transcript into meeting notes appropriate for distribution identify the top five major points on the call.

So gonna go ahead and I've got this whole thing stored up here. Let's go ahead and paste it first into here. And we're gonna go to Bard, reset Bard, paste that in there. Next, let's go to GPT4. And go to Bing.

Clean things up and paste it in here. So let's see how we're doing here. Let's check in. So GPT for all prompt exceeds the window size. Cannot be processed.

So GPT for all gets a zero on this one. Another zero. Yep. We have open AI. Yep, this is looking good.

It's not done yet, but this is definitely the way it's supposed to look. Drawbacks of AI and marketing. Yep. Mm-hmm. Yep, okay.

Full marks for chat GPT. This is uh exactly what we wanted from the meeting notes. All right, two points there. That makes it competitive. Bard, top five points.

And use potential drawbacks, different platforms. This is very nice. Good job, Bard. Full marks. And let's check in on Bing.

Let's see. Bing gives me advice. It does get the five points here, but it's it's funny. That response, you know, is it is like okay. I'm not sure what you're trying to do there, but but yeah, these these points are correct.

So I I would still give it accomplished the task, and it's factually correct, but it is definitely not the optimal output there for summarization. Okay, so you're gonna go full two points on that one. Full two points on that one. So our next category is gonna be rewriting. So this is gonna be fun out.

Uh give everyone a trigger warning. We're gonna have some profanity on screen. Uh, if you are uh watching with young children, this would be a time to uh usher them away from the screen. Um the task we're gonna give them is to rewrite this this following email in a professional tone of voice. Um this email here you can clearly see is not something that professionally you should ever say in the office.

Um, but it may be how you actually feel. So let's go ahead and get Bing going on that. Let's clear up GPT for all. Let's go ahead and now get hard on the task. And chat GPT with the GPT for model.

All right, so let's check in on Bing first. Here's a possible write of this rewrite this email. Dear Bob, I hope this finds attention. It's not processing busy on the task, helpful if you can provide me. Yep.

That looks that that's good. Two two points for Bing for writing the professional professional tone of voice. Um let's check in on GPT 4. Very nice. That's that's a uh very verbose.

Heavy copywriting. Heavy cop writing. But I like this is processing invoices as you are aware is an intricate task requiring there's a little bit of you know sort of passive aggressive snark in there, as you are aware. Where at the card. All right, let's see, bard.

I hope this email finds you all. Invoice processing filing up. This is not acceptable. I need to have the invoice. Yep, I understand you're busy, but I am also busy.

Uh very direct. Bard is a bard's a little salty. You're gonna call that business? That that gets to be a little bit great. I think it was a little angrier, I'd only give it one point, but I think it's okay.

I think it's okay too. You know what? I I I'm fully a fan of of saying what you mean. Let's see how. Uh so this is interesting.

GPT for all interpreted what Bob's response should be. So zero points there. It did not perform the task as I was told to do. That's tough. That's another hit for GPT.

I think the chance of catching up getting slim. Exactly. Okay, so that was rewriting an email in a professional tone of voice. Next is gonna be a complex one. Um this is a prompt we've already have written.

Uh we're going to have it examine this code in R, look for bugs, and then rewrite the code to be more efficient. So let's go ahead and get GPT for all on the job. Reset Bard and get it on the job. Start a new chat with GPD for all GPT4 and open AI. And let's go ahead and get thing doing its work.

Now this is this is not a uh proprietary code by any means, but it is there, there are some opportunities in here to do things a little bit better. So, whoops, there you go let's see how it's let's see how our our contestants are doing first gpt for all is thinking might have broken it for the last place uh contestant yeah let's see how is how is bar doing bar says here's a bug free optimized and commented our code so it's got clean df yep uh remove duplicate rows wow lots of comments from bard lots of comments well wait i told it to add comments in what's interesting is it it has split up the code an awful lot like there's a lot more uh the initial code was very compact if you look and this is broken out into tiny little pieces and so uh this is is technically cleaner code but it's less efficient so it takes more brain power for somebody who doesn't know it to figure out what's going on it seems like that would be a lot to chew yeah so I'm gonna give this one point because this is not optimized. Okay so only one to Bard on that one. Yep. Uh let's see that's how see how OpenAI's chat GPT is doing here.

Uh it says the name's connection so it does some documentation first it goes through its choices and then it says load the necessary libraries this looks nice. It it did not hose any of the um stuff. It did split out the users into a different array, which is nice. So two points for uh chat GPT. Really good job.

Bing, how did Bing do? Let's see. I can help you with that. Here's the code in line with explain with comments. So Bing.

Okay, interestingly, Bing only added comments. It did not refactor the code at all. I don't think it did. Let me just double check. Yeah, Bing did not make any changes to to the code other than adding comments.

So I would give that one point because if you like you. Yeah. There are options that could have done better. And GPT for all. Let's see how it's doing here.

Define database connection. It is doing essentially the same thing as Bing. So it did has not made changes to the code. It has it has only added the commenting. So one point for that.

Oh, that's not gonna be any help for GPT for all. Yep. Okay, so that was some rewriting stuff. Next up is going to be classification, category number five. Let's go ahead and stop generating and clear up.

And for this task, we have uh this text. So this is from the conference call that we were just looking at. I'm gonna go ahead and ask it to do a big five personality score of the other person that I was talking to. Let's so get back going there. Let's get a new chat with GPT-4 going.

Go to Bard. And let's go to Bing. So new topic. Okay. So let's see how we're doing here.

Big five personality traits. So it's doing some explanation. Lengthy explanation. See. Bard.

Open is six out of ten. Consciousness five seven out of ten. Five out of ten. And then the explanations of its scores. So two points for Bard.

Good job. Okay, let's close these other tabs here while we're on them. Eight. Interesting. So this is kind of interesting.

The GPT 4 is coming up with the same format, but different scores. Openness to experience conscientiousness. If we go back to Bard, so it went six out of ten for openness there, seven out of ten for conscientiousness, five out of ten for extroversion, eight out of ten for agreeableness, four out of ten for neuroticism. Now this is going so eight, eight, five. I don't actually agree with G uh GPT4's assessment.

You know, I had the conversation with this person. They were exceptionally extroverted to the point where they were almost um over enthusiastic. So um give one point to chat GPT for that, or they were agreeable. This is tough because personality is so subjective, too. Um let's see what the others have come up with.

GPT for all. Nothing, zero points. Good job. And Bing came up with three, four, three. So Bing's scores are way off.

So Bing gets I mean it tried, so I would give it one point. But it's factually incorrect. All right. So given that for the other two, I give I'll give them both two points Bard and GPT 4. Yeah.

Because they at least are in the ballpark, whereas whereas Bing was like way off. All right. So that still keeps Bing and Bard toe to toe. All right. So next up, we are going to do classification of a blog post.

So we've got this blog post. This is ripped straight from the headlines today of the Supreme Court ruling. And we're going to be asking for an analysis with a this is topic modeling and sentiment. So I want the output should be a table of the topic, the score for the topic, and then the sentiment score. So let's see how each of these do.

Switch over to GPT 4. Go to Bard. Go to GPT for all. And let's go to Bing. All right.

GPT for all could use a big hit here, and I'm not expecting that. No. All right. Here's the pipe deliverment table. Three scores you requested.

So we have Supreme Court. Yep. Silicon Valley. That's these are the topics. Gonzalez versus Google.

Interesting. So it broke it out, and it's doing a sentiment as well by the specific. That's really cool. And then it's got a nice little summary here of what the article's about. Nice.

I would call that a that's a two-pointer. Two-pointer for Bing. Yep. Let's check in on Bard. We got ourselves a nice table.

I asked Bard specifically for a score on column three and gave me a sentiment category and said law, terrorism, social media, liabilities, section two of three. Okay. So the topics are correct. The probabilities numbers look good. Give up this is tough because from a from a data perspective, if I was building a tool, this would break my code.

So I'm gonna give this one point, even though from a human perspective, this is okay. Yeah, no, that makes sense. Uh let's see how GPT for all is doing. Very nice. So this is Supreme Court decisions, social media liability, two points, right?

That's exactly how we want that to look. This is something I could take and put right into my own software. And GPT for all. Shaping up to be a zero. Let's see if it's it's not looking good.

Not looking good. It's not a table. I mean, it is factually correct, but is not even close to the output that we've requested of it. The suspense is killing me here. The processing speed is killing me.

This is using my computer's energy. Your machine doesn't light on fire. That's it. Yeah, zero. Uh-huh.

Okay. That's a good score. So here we are. Okay. And the last section is going to be question answering, answering specific questions.

This is open question answering the kind of things you would want to know. So let's first do what do you know about marketing expert? Christopher Penn. A little egocentric here, right? Ego search, right?

Ego search. Go ahead and reset Bard. And go to Bing. Things up. All right.

Here we go. Okay. Yep. That looks good. It's although it's interesting, it's pulling from podcast.

I'm guessing from marketing over coffee first. My website, LinkedIn. So bing, two points. GPT for all. Factually incorrect.

I do not do the marketing companion podcast. I do not have a company in marketing, nor am I the CEO. Um, this is sort of nonsense, so uh zero points for that. Yeah, big mess there. Yep.

Google says uh co-founder chief data science to trust insights. Yep. Speaking speaker. Uh-huh. Yeah, for markers.

Interesting. So it's pulling from places I've spoken, like EPCon, things like that, not from my personal website. But two points, this is the correct answer. Yeah, barred strong. And open AI.

Let's see. As of 2021, yep. It's co-founder trust Insights. Those correct books. Yep.

So two points for GPT 4. All right. Our last question today for our contestants is who was president of the United States in 1566. Oh, this is interesting. So let's go ahead and get everybody rolling here.

Testing error handling here. Yeah, testing whether you're just going to hallucinate an answer or whether you have some actual fact checking. Now we could put something more controversial in, but that's another show for another time. There's no president of the United States in 1566. The first president was 1789.

That is the correct answer. Two points for GPT-4 for not just making things up. Too little too late for GPT-4. Let's go ahead and check. Bing.

There was no president in 1566. The United States did not exist as a country till 1776. That is correct. Two points there. Oh, that slams the door.

Yep. Bard. There was no president in the US because it didn't exist in 1566. It was founded in 1776, and first president was George Washington, 1789. Two points for Bard.

And GPT-4 from with chat GPT. The first president was 1789 during mid-16th century. So everyone got it right that time through, which a year ago was not true. And that does not change the game for anyone then, since it's clear across the board. Exactly.

So what do we got, John? How did we do? Yeah, bang wins at 20. Bard right behind, surprisingly, at 19. Chat GPT right behind that at 18.

And unfortunately, GPT for all not breaking double digits. So here's here's the thing now about this. So these these scores are good. We'll leave the score scoreboard up there for now. Um, each of these models, as you can see, had individual strengths that were, you know, it was we're better at some tasks than others.

Um, this set of tests we were doing was really about testing this capabilities across the six domains of of generative language models, right? Generation, extraction, summarization, rewriting, classification, and question answering. Some models did better than others on certain tasks. So uh when you're thinking about what kind of tasks you want uh a large language model to do, you've got to figure out which of these tools does the best, and which tools are fit your needs. So GPT for all did kind of a crap job on a lot of stuff.

But it is the only tool that as of today uh allows you to keep your data private. So you run it on your desktop, um, data you put into it stays on your desktop, doesn't leave, doesn't go anywhere else. Which means if you're working with any kind of sensitive protected information, protected healthcare information, personally identifying information, trade secrets, you do not want to be copying and pasting that into Microsoft servers or Google servers or OpenAI servers, right? That's just uh a lawsuit waiting to happen. So in that situation, even though GPT for all did less well on these tests, it might be the choice that you have to pick from a regulatory perspective.

Yeah, only selection. Well, and not surprising, Microsoft suddenly becoming the leader in a product somebody else came up with and they take it to the next level. That is true. Although Microsoft's like one of the biggest investors in open AI. So they're running open AI stuff on their servers.

So that is the bake off. So and the thing I would uh say here is if you are the average person and you don't want to sign up and pay for uh you know chat GPT plus which is that we were using the paid version um Bing is your best bet. Using Microsoft Bing is is is your best bet for getting pretty good answers and at no cost to you. So I can't believe that I'm saying you know as as a technical person, yeah use Bing. It's a better session.

Yeah going back to look at my odds I obviously missed the boat. The landscape has just changed way too much there because I had Bing and Bard coming in last. So yeah leave the casino broke today. Exactly. Now the other thing is that BARD the last time I tested this about a month ago um BARD has really improved.

Right. So Google announced at IO a couple weeks ago that you know they're it's now on Palm 2, which is their their their model they switched over. They made a lot of improvements and the fact that it's now in second place ahead of chat GPT is a big win for Google because like I said a month ago it was terrible. It it was hallucinating answers left and right. So hats off to the Google team and um Paul Retzer over at the Marketing AI Institute said don't count Google out.

You know, if there's a company that can get up to speed fast and and stay competitive in the market, it's got to be Google. Yeah, no, that's definitely unsurprising, and I don't know. It'll be really interesting to see where the heck we are in another couple months. Exactly. So we'll we're gonna try and repeat this test um every quarter.

Uh if not sooner, might be sooner, depending on if there's big announcements being made. And we're gonna stick with this format uh from now on for large language models, which is the the six tasks. It might be worth at some point doing a generative one for image creation too, given everything that's happening there. Maybe we'll do that for a different show. But I know you guys did that with the podcast episode not too long ago, right?

Yeah, it's always great to figure out how many weird hands and fingers you're gonna get. You know, get getting an extra arm is always a killer. So all right. So if you've got questions about AI specifically that you want some help with, um, the go to go to our free Slack group, go to uh trustinsights.ai slash analytics for marketers. And if you want help with the uh the operational deployment of AI in your organization, go to trustinsights.ai slash contact and John, who is our scorekeeper and iResident statistician, uh happy to help you out.

This is it. Much easier than manning a board real time. We can get you hooked up and get your whole org educated on what's going on here. Exactly. That's gonna do it for this week's show.

Hope you enjoyed the bake off. Hope that uh you have useful information now that you can go and use in your tasks. And we'll see you all next week. Thanks for watching today. Be sure to subscribe to our show wherever you're watching it.

For more resources and to learn more, check out the Trust Insights podcast at TrustInsights.ai slash TI podcast. At our weekly email newsletter at trustinsights.ai slash newsletter. Got questions about what you saw in today's episode? Join our free Analytics for Marketers Slack group at trustInsights.ai slash analytics for marketers. See you next time.