Mind Readings: Change Your Prompts When AI Models Change

In today's episode, let's talk about models and changing your code. When we talk about generative AI and code, I'm specifically talking about your prompts. When you write a prompt, you are coding. And if you're writing really good prompts that are detailed that have examples of how things uh should work, you are writing extensive code. When you do that, depending on the model you're using, you may get different results.

You will get different results. So ideally, you're using the model to help you write prompts for that model. So if you're using Chat GPT to do stuff, ideally you're using Chat GPT to also help you write the prompt. But when the underlying model changes, for example, ChatGPT just replaced its GPT-4 Turbo model with its new GPT-4 Omni model. When a model changes, its underlying architecture changes, its knowledge changes.

Think about if you if we think of an AI model that powers something like ChatGPT as a library. And inside this model, there are these the technical terms called retrieval heads. Think of them as the librarians. You go to the library, you ask the librarian for a book. The librarian goes and gets the book and comes back with a book and says, Here's your book.

When a model changes, it's kind of like having the library rearrange and reorganize itself. And now, if you just tell the librarian to go to the third shelf on the back there, that might not be cooking books anymore. That might be geography books or that might be uh painting books. And the librarian comes with back with the book that you referenced from a physical place in the library, and you're like, this is not what I asked for. When a model changes, that's effectively what's happening.

Is it's a new library and maybe even new librarians. And if you're going and asking for same the same things in the same way, the librarian may come back and be like, that's not what I asked for. So when that happens, you want to have the new librarian and the new library read through your prompt and say, here's a better way of up of doing this task, because it will adapt it for the way that particular model works. This is especially important if you're using the Trust Insights Pair framework, where we tell you to prime the model by asking it what it knows about a topic. When a new model comes out, there's probably new knowledge, and you want to ask it, okay, well, what do you know about this now that you didn't know before or that is different from before?

And in doing so, it's going to change the model outputs. Now, for the average everyday user, this just means maintaining your prompts, keeping them up to date, testing them out, and paying careful attention to see whether the quality of the responses increases or decreases over time. For developers or for people publishing models and model apps, for example, if you use um OpenAI's custom GPTs and you built a GPT for this, that, and the other thing, if you built it under the previous version of one of OpenAI's models, you want to go back and revise it. You want to go back and take that prompt that you built the first time around and have the new model evaluate it and say, well, here's how you should do this now, so that your software works best with the new models. When Google's Gemini changes models under the hood, you go back and you have Gemini, the new version, rebuild its prompts so that it works better.

You rebuild those system instructions. This is all about change management and and data governance within AI. Just because generative AI is a new technology, this is something that my partner and CEO, Katie Robert says all the time. New technology doesn't fix old problems. Things like data governance and and data set governance and process management and change management are really important.

They're really important because the field of AI itself is moving so quickly and models are changing so fast that if you don't have good governance, you can very quickly find yourself with something that worked great six months ago and suddenly just doesn't work at all. Whereas if you've been maintaining it the entire time then as the models shifted and drifted you could move with them. It'd be kind of like being a surfer right you you move with the wave. You don't just stay in one spot in the ocean if you do you're gonna get you know you know taken under the same is true for using AI models. So regardless of whether you are just using prompts and storing them in your prompt library you are doing that right or you're building applications like custom GPTs or building full-fledged AI apps you absolutely need to make sure that when a model changes you change your code with it and that you have a process for keeping things up to date that is that takes advantage of new capabilities and addresses performance changes in the code you have that's gonna do it for today's episode.

Thanks for tuning in I'll talk to you soon if you enjoyed this video please hit the like button subscribe to my channel if you haven't already and if you want to know when new videos are available hit the bell button to be notified as soon as new content is live