So What? Revisiting Prompt Engineering Basics

Well, hey everyone. Happy Thursday. Welcome to So What, the Marketing Analytics and Insights Live Show. I am Katie joined by Chris and John. How's it going, fellas?

Hello. Chris, you look concerned. I think something goes up with my audio during the intro because the music was really distorted, but everything sounds fine now, so I'm good. Okay. I mean, you've been having camera issues, audio issues.

I think really what it comes down to is you likely have a poltergeist. That's the only logical explanation that I will accept. Sure. Don't get sucked into your television then. That's the only advice I can give on that front.

Speaking of things, uh, it's a terrible segue. I didn't I didn't have one for that. This week we're revisiting prompt engineering basics. And so we wanted to go back to where we started with prompt engineering because what we found, and Chris, you specifically, is a lot of people got very excited and built frameworks and started prompting two years ago when generative AI became widely available. And you know, we have to give the system some instructions.

We have to tell it what we're after. And that's where uh we at Trust Insights were able to come up with some of our oh goodness, I can't even find them now. Uh, some of our basic frameworks of race, uh, that's where we started. So role, action, context, execution. Um, I don't see it in our uh tags, but that's okay.

And then, you know, we took it a step further with pair, and then we blended it together and made repel because why make it easy? Um, but Chris, and I would like to hear you speak to a a little bit of sort of what you're seeing, but it feels like as the large language models, the LLMs are evolving, people's skill set of prompting is not evolving with it. And so you brought this to us, and so that's why we said let's revisit prompt engineering basics. So, what is it that you're seeing that makes you feel that people are stuck in 2023? A few things.

First, people are still giving uh absurdly short prompts. It's like write me a blog post about B2B marketing, right? You know, that's it is 2023 still. And you see this, especially when you start looking at content on places, say like LinkedIn, uh where there's just so much slop, particularly in the comments, looking at you comment bots, um, that it indicates a lack of a level of sophistication. So that's one big thing.

The second big thing is that in you know, Chat GPT and GPT 4, GPT-4 came out in June of 2023. It is a toddler, right? When we think about this, it is still in the terrible twos, and yet all the platforms models have evolved from face rolling idiot two years ago to above PhD level capabilities today. And that means that the way you prompted a face rolling idiot and spelling everything out is different now than how you would prompt you know a PhD. You still need to provide things like context and details and what your desired outcomes are.

But you no longer have to rigidly adhere to one strict prompt framework because the models are smart enough if you tell them to ask you questions. And this is something where I've seen a lot of people run into trouble because they're they're doing what's called cognitive offloading, which is where you say, you know what, I don't want to do this task, I'm just gonna have AI do it. And in doing so, you kind of give up your own agency. And Katie, you and I have talked about this on the podcast, where people are, you know, I think we actually had an episode saying is is AI making us lazy. We did.

Um and so it sounds like if I can paraphrase for you, that despite the large language models being at PhD level, we're still talking to them like they're face-rolling idiots. We are, and more important, because they've evolved so much, we are not leveraging that those capabilities. So one of the things that we talk we we talk about a lot is the difference between offloading and enhancing. Offloading is when you are asking AI to do things, and you're like, bye, I don't want to do this anymore, which is fine for some tasks like expense reports. Um, but enhancing is when AI is making you do things and making you think more because the more you're thinking, the better you get as a professional.

So I want to call up this comment. Uh, how many prompters know how to talk to a human PhD, let alone an AI version? Which I'm I feel is tongue-in-cheek, but at the same time, if you're positioning these large language models as PhD version, I can actually see how that could be intimidating. Like just a real example, Chris, we run into this a lot. People in general struggle with how they feel they should be talking to you because of the level of expertise you have in AI, they themselves don't want to come across as uneducated or unprepared.

And so they just kind of don't bother at all. And so I feel like it's sort of the same thing. Like, if you're saying these are PhD level, you know, language models, use them as such. How do we take the intimidation factor out of it and make it approachable so that somebody doesn't feel like they're not smart enough to do it? The good news is that unlike a curmudgeonally human like me, um, the models are much more polite and helpful than I am.

And so what we need to do is we need to be clear in our thinking. And I will show what I clap what I call the magic trick to the extent that there is anything magic in AI, because there really isn't, but this is the magic trick. If you take away nothing else from this entire live stream, it is to paste this sentence at the end of your prompts, particularly your initial prompt in a conversation, to say, ask me one question at a time until you have enough information to complete the task. So here's uh just a simple example. Let's go into a stock chat GPT.

Nothing special here. Let's make it a little bit bigger here. And I'm gonna give it a terrible prompt. I'm gonna say, write me a blog post about B2B marketing. And uh let's see what it comes up with.

And here's a complete blog post draft for you. I mean, it did what you asked. It did what I asked, and it's and it's it's not wrong, but it is slop. Like this is this is just generic garbage. Now it's a list, you have a list.

I have a list, I have a great list. Now let's do this. Write me a blog post about B2B marketing. Ask me one question at a time until you have enough information to complete the task. Now we hit go.

Go, yeah. Well, no, go ahead. Sorry. I was gonna make an observation. Um, my observation is it I find it interesting because I personally I would get annoyed with the one question at a time strategy, whereas like I just want to know what all the questions are up front, and I feel like that comes down to the type of person you are in terms of how you consume information.

And so, you know, we've joked slash not joked about how you can't ask certain people more than one question at a time because they're only gonna pay attention to the last question you asked or one part of the information, and so you get frustrated continually following up and being like, well, what about this other thing? What about this other thing? And so I guess my question to you, Chris, the my one question to you, uh, is do you get the same result from a large language model if you use the ask me all of your questions up front versus ask me one question at a time? Meaning, does the info do you think the information that it's compiling and the questions it would ask would change given the strategy? Yes, you get slightly better answers when you have one question at a time.

And the reason for that is every time you're going back and forth and providing data, you're adding new tokens to the context window. You're adding more words to its memory. And as if you have it generate all the questions all at once, it doesn't have a chance to refine the subsequent questions based on your responses to triangulate faster on what you want. If you do one question at a time and you give an answer, it can triangulate faster and ask deeper questions faster because it can make some inferences based on your answers. Now, is it a substantial difference?

Depends on the task, but usually not. What I will often do, for example, before I go and work out is I will have it generate a long-ass list of questions, and I will take that with me with my voice memos app and you know, groan and mumble as I'm uh on the treadmill uh through all the answers and upload those back. But then I typically have two or three more cycles of okay, here's all the answers I gave you. Now give me another set of questions and stuff. Whereas if I'm doing it iteratively when I'm sitting at my desk not sweating to death, um, I can usually get to those second and third level questions faster.

Okay. John, um, question for you, one single question. Uh, how long would you say your average prompt is when you're using generative AI? Yeah, I try to keep it as simple as possible for the first run, but I you have used this trick every time of that's kind of the big key is you realize how complicated the prompt is or not, based on if you throw it a quick question and if it only gives you one or two things, or if you throw it a question and it gives you 10 or 15 questions, it like starts mining, you know, deep down the uh the rabbit hole of wherever you've sent it. Um but yeah, it's kind of a mix.

You you want to be directed to the point so that it can be on target, but you also have to give it enough information so that it won't just come back with slop and you know will at least go down uh get getting closer to the target. So as we're as we're trying not to get frustrated with ask me one question at a time, because I mean, and this is just my own personal personality, like an N of one. I'm like, let's let's just get to the point faster. Like, can we just move it along? Um, but I understand what you're saying, and how like, you know, it makes the output more detailed, better, etc.

etc. Um so we go through. Well, I guess you know, let's go back to the beginning. Because you said one of the things you said was you no longer need to use any kind of prompt engineering framework like race or pair or repel. Um, so it like, are those out the window?

Do you can you really just give it, write me a blog post about B2B marketing, ask me one question at a time? So the frameworks are good for us as humans to remember what it is that we're we should be including. They are still very valuable because we the human operators have issues remembering things. And so they're good. Um I've actually defaulted back to recommending race as a general framework to people because it's easier for them to remember.

And with the advent of tools like deep research, repel is kind of overkill these days. Um the however, with the models themselves, if you are using a reasoning model, so you're using chat GPT 5 thinking, if you're using Gemini 2.5 Pro, uh Claude uh and its series of models in the 404.1 generation, Deep Seek R2, uh, Alibaba Quen 3 uh 235 AB, um all those models have reasoning, all which is fancy for it does a rough draft uh before it presents the answers on screen. If you decompose what's going on in the reasoning of of those models, you can see they're trying to figure this out themselves. So they've got their own internal logic that they're trying to reason out, like, well, what is this user trying to do? What are the what are they thinking about?

What's the you know, what's going on under the hood that I should be attempting to guess at? Um and as a result, you don't need this the strict level of prompt framework that you did in twenty two, you know, 2023 when no models had any reasoning, and what you see is what you get. Does every model have reasoning built in at this point? For the ones that you're going to run into as a standard user, the answer is yes, with a lot of asterisks. Okay, let's well, and it because what I want to do in this episode is make sure people are clear.

Like if you if you take our advice, here's what's going to happen. And it sounds like there's some conditions in which that might not work. And so when you say the standard models people are going to run into, you're talking about sort of like the big names like Claude, Gemini, uh, Chat GPT, for example. Um, I'm sure there's others that I'm not thinking of at the moment, but those seem to be like the big names. Like most people I talk to, they're like, oh, I'm using Chat GPT, or I'm using Gemini.

Uh Gemini now built into Google Workspaces, which a lot of people use. Copilot, for example, um, is one of the major names that come up. And so it sounds like if nothing else, if they're not quite sure where to start, they can do the ask me one question at a time until you have uh until you have all of the information you're looking for. Is that sort of a is that a good assumption or am I missing something? That is generally a good assumption.

The two asterisks, one is within Microsoft Copilot, um, whether a query gets routed to a reasoning model or not is dependent on the prompt and how Microsoft is feeling that day. Um, and so it is an uneven experience uh inside of Copilot. Uh hence this lovely slide from from our talk that uh that I give frequently. In chat GPT, it is the asterisk is it depends on how you prompt it. And there's two ways to invoke reasoning if it's not doing reasoning.

The first is to make sure it's on thinking, uh so that uh, because that is a a model that forces reasoning. And the second is in your prompt to tell it, think hard. Uh which um which which tells the model router where to send the prompt to because chat GPT and a GPT-5 is not one model. GPT-5 is actually four different models with an invisible delegator up front that reads your prompt and says, which one of the four models here could do this task the the fastest and the cheapest. So I'm gonna keep picking this apart because I want to make sure that we're being clear.

So it's not enough to just say, ask me one question at a time until you have enough information to complete the task. Because you may not be using a reasoning model. So do you have to add on to that, you know, magic? I don't remember what you called it, your magic or whatever. Uh say think hard.

Like, is that those two things together the key to success? So if I sign up with the fast, you know, GPT and not the thinking or GPT 4.0, and I say, ask me one question at a time until you have enough information to complete the task, think hard. If I'm understanding what you're saying, it would switch to a thinking model and say that is correct. This is not the model you want. That is correct.

So by telling it to think hard, you can now see the little reasoning bubble pop up, which is the dumbest possible interface. Uh uh, but what it is done behind the scenes is basically flip from auto mode to thinking mode for this particular chat, even though the interface itself doesn't change. Gotcha. Uh Jim has a question. Um, are they going to undo the invisible router and let people choose their model?

No. Well, no, with an asterisk. The enterprise customers supposedly will retain full access to all the model choosing stuff. But everybody else supposedly will have 4.0 for paid customers, which is available, which is a non-reasoning model, but people really like how psychophantic it was. And then the current GPT 5 family.

So as usual, OpenAI making a hot mess of product market fit. So you said, you know, there was okay. So my question to you was if you say, you know, if you use this sort of magic question and then you add think hard, then it's going to do the thinking. But you said there was asterisks to every system having a reasoning model. So obviously with ChatGPT, we covered co-pilot and it sort of like iffy, you know, is there anything we need to be aware of with the other major models like uh anthropics Claude or uh Google's Gemini.

So Claude has um a toggle called extended thinking, which it does thinking all the time, but it does more thinking if you have that turned on. Um and it in the general best practice is just leave it turned on. Um you'll you'll do much you'll it'll it's just a better practice for that one. Gemini, uh, you don't get a choice. Gemini does is is reasoning all the time, um, which again is the sensible thing.

For and then there's this new crop. You don't you won't see it in the major three in the West, but there's a new crop of models like Mistral from France and Alibaba's Quen and Deep Seek that are called hybrid thinking models, where you tell them to think or not to think, depending on how you address them, kind of like the whole think hard thing. Um, and that hybrid reasoning model decides for itself whether it needs to do extended reasoning or not. And since we're getting back to basics, can you without getting too overly technical, define when you say thinking, what does that mean? Like to a layman, if you said this model is thinking, this model is not thinking, you know, to someone who's new to this, might be like, well, aren't don't they all think?

Like, what does that mean? So thinking and reasoning are synonymous in this case. What that means all the time is that the model is doing a rough draft behind the scenes of its answer first before it puts something out on the screen. So it wasn't doing that before when it wasn't a reasoning model. It was just like, here you go.

Correct. Okay. That's right. All right, let's continue on this very basic example. Who is our target audience for just B2B marketing blog?

Let's see. Our target audience would be marketing managers and mid-size enterprises who are confused about the basics of B2B marketing. That seems like a reasonable uh target audience. What are the odds that somebody would respond B2B marketers? Pretty high.

Okay. Just wondering. Yep. But you can see already by the words we're using in each answer it's going to dial in faster to what it needs to know next because some of a some people like me are very verbose in their answers and every word helps it figure out faster what it is it we're talking about what's the goal for the blog post educate inspire or persuade I want to create an foreboding sense of doom uh this this is why you're not our content manager or dark cautionary tale do you want the doom to feel uh uh realistic or dramatic and exaggerated let's go for dramatic and exaggerated a B2B business horror story I knew you were going for that uh let's see do I have a post a narrative format or a warning manual like a survival guide with obvious commentary let's go with the narrative format following a fictional marketing manager who spirals into disaster I'm actually shocked by that I thought you were gonna go with the warning guide no no no um uh what industry uh let's go with professional services like a law firm and a mid-sized law firm in the B2B space how should the ending feel complete catastrophe or a last minute bittersweet survival it's like zombie movie let's go with epyric victory uh and see, it has now had to flip into thinking mode. You can see that it we've done something in this prompt chain to to make it go, uh oh, um, my routing isn't is having issues.

Uh let's see. The law from Wins the Amp at such terrible cost, it feels like a tragedy. Should be straight five faced or stylized. Let's do a stylized gothic tale of woe. Well, let's see.

Should it be a harsh should the story teach real B2B marketing fundamentals? Yes, it should teach some basic fundamentals like remember to not rely on social media and things that you don't own. All right, here we are. The shadow of Ashford and Goal, a gothic B2B marketing tragedy. This is qualitatively and quantitatively different than best practices for B2B marketing in 2025, which is where we started.

So just in this series of questions, you can see how quickly this went off the rails in a good way to create something that's very different. What's interesting, um, and this is probably something that I would add to the initial prompt is you know, it didn't tell you when it was done asking questions. It it just was like it just decided in the background it was done and created the thing. But what if you had more context to provide? Like obviously you can give it feedback here to be like, oh, I don't like this or whatever, but you know, that to me is a little jarring where it's like, okay, I'm done.

Here's a thing. Like, there's no warning that it's done asking questions. That's right. And so you could uh give that feedback, or you could include it in the prompt and say, like, show me a progress indicator of uh the questions as you're asking them. Um, but because it is kind of almost a choose your own adventure now with the ask me one question at a time, uh you might not know where this I had no idea we were going to end up with a gothic marketing tragedy uh out of this.

I you know, we we started with just a terrible prompt just to illustrate the prompt strategy, but we ended up in this very strange place. So, John, when you do this tactic of ask me one question at a time, you know, do you find that the questions are helpful? Does it keep things on the rails, or does it kind of like go this route of uh the seduction of glittering platforms? Right, right. You're you're playing.

No, there's a whole mental shift that it it took me a while to to get to of this idea that you know it's all about probability, right? There's not absolute answers. It's the models take a look at everything out there and fig determine what they think the most probable answer is. So if you give it a huge prompt up front that's kind of just very general, it's gonna grab from the whole internet, it's gonna be all over the place. But with a lot of small individual focus questions, you know, you're keeping the focus very tight as it goes through the whole process.

So you know, in theory, it's gonna be closer to what you think, you know, the quote unquote truth is, um, or what's correct, you know, what's accurate. Um, but yeah, it it's it can go way off the rails too, depending on how you answer the questions, right? I mean, that's really the at the heart of it is if you start steering it in a direction where you know nobody else has ever gone before, yeah, there's kind of no telling what you're gonna pull out, but uh it has helped me as far as you know being able to land closer to where I want to go. You know, doing it in one shot is a real roll of the dice, but going multiple questions, it seems like it can at least dial in closer to where it's supposed to be. Yep.

So next, we're gonna move this into the canvas. So for folks who are unfamiliar, the canvas in OpenAI is essentially kind of like a Google Doc. And we're gonna we're gonna have the post be here. Uh let's see. In the Canvas, you will note that there's our our general post.

There's an edit panel here now. So I can say I make this longer, change the reading level, add final polish, add emoji if you wanted to do such a thing, um, suggest edits. So I'm gonna say, let's take this post and make it much, much longer. And hit go. And now it should revise our our content within the canvas to make this you know much longer.

So from in terms of prompting basics, since we're talking about that today, a lot of the tools today have built-in utilities to do things like changing style, changing length, um, changing tone, changing readability, uh, that you may not have known that those tools were there, right? So I can I could adjust the length even longer. I could change the grade level from you know to to college to graduate school, uh, I could change it down to kindergarten, uh, which is a gothic marketing tale for kindergartners. Well, I was so I was in uh Google Gemini yesterday, uh working on some stuff internally, and I was using its version of Canvas. And so as it was giving me the output, I could go into specific parts of the content.

And when I highlighted it, it gave me a little pop-up that says ask Gemini. So basically exactly that. And I could say, I want to tweak this specific to, you know, whatever without having to redo the entire thing, which you know, if you've ever had to do that, sometimes can really kind of go off the rails because you're like, no, no, no, I really liked two-thirds of it. I just wanted to focus on this one thing, and now they give you the opportunity to do that, which I personally have found incredibly helpful because it's usually like minor tweaks to things without having to do redo the entire thing and be like, well, let's you know, revert back. Like it's you have a little more control over the information that you're editing, which is very helpful.

Exactly. So part of prompting basics is just learning the tools, learning what capabilities are available in these systems. So things, for example, like in OpenAI, you have deep research agent mode, creating images, study and learn, web search, and so on and so forth. As you mentioned, Katie, if you go into Google's Gemini, uh, you have video and deep research, canvas image guided learning, which is their version of study and learn, etc. So those are all of the things that from a basic perspective, if you use the tools that are there, it will save you the time of saying, look out, I need to rewrite this for this.

However, you can always just specify and say, I want, you know, I want it to do this kind of thing. Um those are a couple of the basics. The the other basic I want to to spend some time on, and we talk about this in our mastering prompt engineering uh course, is knowledge blocks. If there was one other thing besides the asking one question at a time, that I would advocate everybody do, it is to have pre-built knowledge blocks. I call them digital Legos.

And what these things are are blocks of knowledge, blocks of information that you can drop into a chat uh that may be thousands of words long, so that you don't have to retype that information, but it instantly makes everything better because you know exactly what uh the model knows exactly what you were uh it's it's supposed to be know about you. So, for example, I have a trust insights knowledge block here that says, you know, here's who we are, here's what we do, and so on and so forth. We have our sales playbook, the 135-page uh document that says here's how we sell and market. And for us to be able to just drop that whole document in is huge time saver. So the the knowledge blocks I recommend people have, especially for the folks who are in marketing, uh, are make sure you have an about us, right?

That is a detailed uh explanation of who you are, have a how you do your marketing, um, any kind of reports that you have, uh, ideal customer profiles, which if you don't have them and you don't know how to build them, uh, contact us. We're more than happy to walk you through our process of building them or build them for you. Any kind of strategic planning documents that you have, any kind of frameworks. So, for example, I have knowledge blocks for my versions of a SWAT and a Porter's Five Forces and a BCG growth matrix because I have very strong personal opinions about how they should be done. And so when I when I do them, I have them written out.

It's like this is how I want you to do it. I don't care what the industry does, this is how I do it. Exactly. Well, for example, we used to work at a PR firm, and those folks did SWOT analysis, and I thought they did it horribly because they kept mixing up things like industry trends and stuff. Like that is not what a swap is for.

That's what a Porter's Fire Forces is for. Putting industry trends into SWOT analysis is stupid. Anyway, that go ahead. But that's well, yeah, I digress. That that's not your personal opinion of a SWAT.

That's just proper use of a SWAT. But anyway, yeah, I digress. Okay, but no, and um I think it's worth noting though, is well, I guess let me say rather than worth noting, the question, the one question that I have for you is obviously we can put about us and everything that's like public on our website, but do we need to go deeper and put in things that aren't on our website? Is that a risk? Is that a bad idea?

How detailed do we need to be? It depends. It depends on a the privacy of the system you're using. So for example, uh chat GPT right now is under court order to retain everything that's put into it. So if um if you have stuff that's super private that you don't want to end up in a court deposition somewhere, maybe don't put it in there.

Um Google's Gemini, there the Gemini workspace version says they don't train on your data, and a human will review it only if you do something dumb like ask it how to build a very bad thing. Um Anthropic's Claude uh has very strong privacy protections. Uh if you use the Chinese hosted version of Deep Seek, there are zero privacy protections on it. Everything that you put in there is going to be seen by multiple people in the People's Republic of China. Um, if you put anything into Elon Musk's X AI and Grok, you have no privacy there.

Um, everything's gonna be consumed, and you know, Elon may retweet it. Uh the the key lesson here is you need to check the privacy policies and the terms of service on every platform that you use. And what I would suggest you do is have a gem or GPT or something that you've written that evaluates privacy policies and says on a scale of zero to 100, here's how private your data is in this platform. And then you can make the decision like, yeah, this this is not good. This is usually about the time that I bring out my tinfoil hat and assume that everything is dangerous and don't use anything.

Um we all it's well documented that I am the least likely to take any kind of risks on the team. I am like the most conservative when it comes to business. So when I see people are putting like their strategic plans into these uh large language models, I'm like, ooh, ah, don't do that. Um, you know, but you know, Chris, to your point, if you evaluate the privacy policy, you know, and do your due diligence, then you can make those judgment calls of how much risk are you willing to take. Like if your strategic plan is, you know, make more money by posting more things on social, probably not an industry secret.

Probably okay to share. But like if you get deeper into the weeds, like those, you know, maybe there's the high level AI friendly version and then the more deeper detailed version of things. You know, just that's my little tinfoil hat talking. So just you know, be careful. One of the things that uh we show in our presentations about this is sort of the levels of privacy that you have.

Right. Local AI models, the ones that you run on your computer and you can unplug the internet, they are the only ones that you are guaranteed privacy. Because as we say often, there is no cloud, it's just somebody else's computer. The tools that you see in green, those are generally safe, mostly private. That means that they're not going to train any data.

And a human will only look at them if you do something stupid, uh, like violate the terms of service. Uh the ones in yellow can be made private, but are not private out of the box. And the ones in red are not private at all, cannot be made private. And uh in some cases, like with uh XAI's Grok, you probably shouldn't use them. If you want a copy of this for yourself to hand over to whoever's trying to force you to use tools that you're like, I don't know about this.

You can go to trust insights.ai slash AI data privacy. Uh, we try to keep this updated about once a quarter as things change, but I would say as of today, this is a fairly good representation of how secure your data is if you put it into a large language model. Exactly. And uh companies like OpenAI have released uh open models. So open AI made a big splash a couple weeks ago, not GPT 5, but they're their first open models since 2020.

Um the last model they released openly was GPT 2. Um, and then they released the poorly named GPT OSS 20B and the GPT OSS 120B, uh thereby demonstrating that there was no product marketing manager in the room. 120, 121, whatever it takes. It is widely suspected that these two open source models are actually GPT 5 nano and GPT GPT 5 Mini. They are very good models.

They are extremely fast. You can run them locally on your computer and you have the same control that you do in their uh developments uh studio. However, um they are they hallucinate approximately 90% of the time because they are trained on fluency and skill, not knowledge. So I could go in and say, write a blog post about B2B marketing. Ask me one question at a time until you have enough information to complete the task.

And inside, I'm using this inside LM Studio. Um LM Studio is private hosting. You can see there's its thinking, and it's now starting to do the exact same process as its cloud-based big brother. The difference is this is private. When I hear 90% hallucination, my first thought is like, oh man, you really are choosing to take an adventure today.

Like you don't know what you're gonna get. And maybe that's what you want. Maybe you want something totally wacky out of the box. Maybe it's a good brainstorming partner because it's gonna think of things that are like totally from a different universe. Maybe that's a good thing.

But yeah, that's 90% hallucination to me, sounds problematic. It is in the sense if you're if you are relying on generative AI to present factually correct information, uh, that is definitely a problem. However, you shouldn't be doing that in the first place. Uh generally speaking, what you should be doing is you should be bringing the data and then having AI manipulate the language. Um, that's the best practice in general.

Today's models that you use in like a chat GPT or a Claude typically have a web search tool. And so they will just go out and search the web to get information if they're not confident that they're they've come up with a good answer. But the hallucination rates, even in the big tools, still exist. Um it's one of the reasons why we tell people anytime you have AI AI output in finance, law, or health, please have a human subject matter inspect it. I would go so far as to say any AI output, but again, most conservative person on the team.

How about can you go into it? So you had mentioned that you that 90% hallucination rate is based because it takes fluency first over accuracy. But what exactly does that mean you know, by prioritizing on fluency? So fluency means that it's really good at manipulating language, so it can do things like tool calling. So if you're using it to write code, it can say, oh, you have a tool that does linting, you have a tool that does static typing, you have a tool that does this or that, and I know it exists and I know how to use it.

So in the past, we've talked on episodes of the podcast and stuff about MCP servers, model context protocol. And what that means is uh the model knows, oh, this tool exists and I can use it. Gotcha. Okay. So it's been trained on tool use, but they sacrificed knowledge to get those capabilities in it.

See, putting the platform first. That's right. Absolute wrong thing to do. That's a different episode. That is a different episode.

So, knowledge blocks. Um, the ask me one question at a time, and knowing what platforms you can use safely with uh your data are kind of the big three basics for prompting. We're you know, the basics of prompt engineering. Now, a lot of people are circulating uh you know, open AI's GPT-5 prompt optimizer, which is good. It's it's good for GPT-5.

And we recommend if you have custom GPTs that you put them through the prompt optimizer so that your instructions are updated. But in general, you don't necessarily need to use a special tool to do that. You can ask a model, say, here is my prompt. Using your knowledge of prompt engineering after January 1st, 2024. How would you revise this prompt to be most effective?

What is my intent of the prompt? What are the downstream effects of this prompt? Um, rewrite the prompt with this inform with after you've thought this through. And what it will do is it will essentially optimize your prompt for that model. And as a reminder, optimizing a prompt for one model does not optimize it for every other model.

So if you do that on a Gemini instance, you can't, you're not going to get as good a performance if you just copy and paste a Gemini prompt into Chat GPT. You'll see it, it's still language, it still understands what you're trying to do. Um, but it will look different than if you said, here's this prompt, optimize it for chat GPT. So it sounds like to paraphrase what you're saying, is you don't necessarily need a strict framework anymore, but what you do need is critical thinking, which has been sort of the theme for the past couple of weeks that we've been talking about. And so critical thinking in the sense of what could go wrong?

Did I provide enough information? Do I have the expertise in order to determine whether or not the information is correct that I'm getting back out? And so really thinking through what am I actually asking this machine to do? Like, am I being thorough enough? And you know, does it have to be structured in a framework?

It can be if that's how you personally organize information. Otherwise, just making sure you're kind of just covering all your bases. Exactly. Um, what I generally say for people who are working with AI is critical thinking, creative thinking, right? The tools can do just about anything.

You have to have the brain space to go, oh, I I I should ask it to try to see if it can do this, right? And then contextual uh thinking, uh, which is do you know where your data is and and how are you using it with AI? Do you understand the context of the request you're making? Could you provide it with more information? Um when you were and and Katie, you've been talking about this a ton lately with soft, aka durable skills, transferable skills.

Um these are the ones specific just for working with technology, and then you got many more for working with humans. Yeah, not everyone can work with humans, and I feel like those of us who, for the most part, can are going to be in demand because it's the narrative is flipping back to okay, if AI can do the hard skills, which is the technology, then what's left? That's the soft skills, which aren't really that soft. They're actually really difficult. I would say they're hard, harder than the tech.

Uh, because I don't know about you, but John's pretty unpredictable, and I don't know what I'm gonna get day to day with him. It's a grab bag, that's for damn sure. It's always a grab bag, it's always a grab bag. Um but yeah, it in terms of how you should be prompting in mid-2025, rely more on the knowledge in these tools. I mean, the the fact of the matter is that on most benchmarks now, like Diamond GPQA, for example, um, these tools all operate at or above human PhD range.

Uh, you can see here, GPT 4 started out as a face-rolling idiot, right? And then as of 2025, like Gemini 2.5, up where it's above PhD range in terms of its ability to pass this particular very difficult academic test. That means you should be using these tools with the understanding they have many more capabilities than they did two years ago. Perhaps offline or more appropriately, in our free Slack group, analogs for marketers, you can explain to me what exactly a face rolling idiot is. So many answers to that question.

And I look forward to hearing all of them. But yes, so that's that that's the thing is with revisiting prompt engineering. People are prompting like it's summer 2023 when really it's it we're we're well past that, and the skills that separate you from the rest of the herd now all revolve around critical thinking, creative thinking and contextual thinking. All right. Use your brain, people.

Bottom line. Use your brain. Any final thoughts? That's it. Go do some good stuff.

Get amazing uh prompts running. All right. Thanks, folks. We will talk to you all on the next one. Thanks for watching today.

Be sure to subscribe to our show wherever you're watching it. For more resources and to learn more, check out the Trust Insights Podcast at TrustInsights.ai slash TI podcast. And our weekly email newsletter at TrustInsights.ai slash newsletter. Got questions about what you saw in today's episode? Join our free analytics for marketers Slack group at TrustInsights.ai slash analytics for marketers.

See you next time.