So What? The Prompt Engineering Life Cycle

Well, hey there, everyone. Happy Thursday. Welcome to SOAT the Marketing Analytics and Insights live show. I am Katie, joined by Chris and John. Hello.

Guys. I'm in the corner, so I have to do like a Yeah, it was like you can't high five. I have to like Yeah, there we go. One of these days we'll get it right. Um this week we are talking about the prompt engineering lifecycle.

What's that? You've never heard of the prompt engineering life cycle. That's because we just made it up. Um we're talking about the prompt engineering lifecycle because it relates to the software development lifecycle. So we're gonna go through what all of that means and how a framework like the prompt engineering lifecycle can help you write better prompts that will give you better output, make it more repeatable, more scalable, all that stuff that we really want generative AI to help us do.

But in order to do that, we have to get organized. So, Chris, with that, where would we like to start today? Uh I think you should start by explaining what these life cycles even are, especially for folks who are not hardcore techies encoders. Absolutely. Um, so a little bit of background.

I managed uh development teams for about a decade, and so I'm intimately familiar with the software development lifecycle. The software development lifecycle, or the SDLC is basically a framework that anyone who does development is using. And it's a basic set of steps that help you go from I'm planning the project, I'm designing the project. This could be front end or and back end development, or it could just be what is my approach to this project. Then there's the action of doing the development.

Once you've done with development, you go into testing. You should go into testing. Not everybody does, Christopher, but you should go into testing. And then you actually deploy once you've decided, yes, I have the output that I want, I have the product, I did the code. This is the thing that I want.

So depending on what company you're at or what team you're at, or if you're waterfall, if you're agile, waterfall is basically a series of steps where you have to complete one phase before going on to the next. Agile is more iterative. But basically, every development team has a version of the software development lifecycle. These are the basic steps. You can add more, you can take some away, but this is always going to be the same set of core steps in the software development lifecycle.

And so if we move on to the prompt lifecycle, how do we get here? So, how we got from the software development lifecycle, the SDLC, to the prompt engineering lifecycle, the PELC, is we were having a conversation about how prompt engineering is akin to software development. So wouldn't it make sense if prompt engineering followed a similar framework like the SDLC? And so I thought about it and I started mapping the things that we know, the steps that we want people to take when doing prompt engineering when approaching it to the software development lifecycle. So first step is always planning and this is a great opportunity to use the five Ps.

The five Ps are purpose, people, process, platform, and performance. And so this is going to give you your basic set of requirements and get you on the right foot, on the right track, stay focused. And so once you have your five Ps, once you know your user stories, your purpose statement, how you're going to measure everything, you can move on to actually developing the prompts themselves. And that's where you can use our um race framework, which is role, action, context, and execution role, who are you? Action, what are you doing?

Context, here's some background information and execution. This is all of the information you're going to be giving to the generative AI tool that you're using. So you could be saying something like today you're going to be a B2B marketer. I need you to help me put together a content marketing plan. Here's some background information of what we currently write about.

Okay, let's go ahead and do it. You can grab a free copy of this framework at trustinsights.ai slash prompt sheet. So that is your that is the second phase uh that's your development. Once you get through the development you probably have to go through some iterations. This is where it gets sort of becomes similar to testing your code.

So the very first version is probably not the correct version. So you have to go through work with the generative AI model and this is where the pair framework you can grab a copy of this again, no download at trustinsights.ai/slash power questions. So the first thing you want to do is prime your model. So you've given it some basic information. Say, well, what do you know about this?

What are some best practices? So, in that example of a content marketing plan, what do you know about putting together content marketing plans? And then you would augment it and say, What questions do you have for me? Then you would refresh it and say, What did I forget to ask you? Because these tools, these large language models have a wealth of information, but it doesn't know to tap into it unless you're prompting it to do so.

And then you would evaluate, did it fulfill the conditions of the prompt completely? And so that's your testing. So did I ask everything? What else am I missing? You know, and just keep going through and your testing is your iterative phase.

And then once you're satisfied that I have built a prompt that's going to do what I need it to do, then you deploy it. You start using it in your everyday. You use it to build your content marketing clean, you use it to run code, whatever it is, and you put it into your prompt library. And then you have maintenance, which is once a month, once a quarter, however often your data is changing or your needs are changing, your goals are changing. You go back and revisit the original prompt that you built and said, Does this still stand?

So that is a very I feel like I've been talking for way too long. But in a nutshell, the software development lifecycle is how you approach software development. We've now adapted that to be prompt engineering focused, how you approach prompt engineering. First question people probably ask is, Well, why can't I just open generative AI and start, you know, working on it? You absolutely can.

You're likely not going to get the results you're looking for. Same with software development. You can open, you know, a blank browser and start coding, but if you don't have a sense of direction, it you're just going to be sort of floundering around. You don't know if you're getting the outcome. Same thing with driving a car.

You can get in a car and start driving, but if you don't have a destination, you're going to waste gas, you're going to sit in traffic, maybe run out of your podcast because you've listened to all of them. I don't know. There's a lot of things that could go wrong. So you really want to have a plan going in. I'm done.

No, I think that's an excellent overview because most people think of prompt engineering and prompting as exactly what you said. Open up chat GPT or Gemini or Claude and just start typing. And that it there's nothing wrong with that. That is totally fine. That is a totally a great way to use basic generative AI.

However, for a lot of companies, they want to do more with these tools. They want to have them be public-facing. They want to leverage the data that already exists. They want to do capabilities that scale. And there's nothing scalable about a person copying and pasting into Chat GPT over and over again.

That's that's not a good use of someone's time. There are now systems built on these tools that allow you to put together these kinds of utilities. So I figured for fun today, we would have we would explore what this might look like. And we're gonna do this in Google's new Vertex AI. Vertex is their system.

If you remember back from previous episodes, we did an episode called on custom GPTs, and we built a custom GPT called KD GPT that tries to imitate Katie. And that's fun. We could use it within the chat GPT interface as well. But what if we didn't want to have it just be an open AI? What if we wanted to have it on our website or in our Slack group or any place?

What if KDGPT could be everywhere? Well, it can be Katie GPT's coming for you. It's everything. Exactly. It can be.

And so what we'd want to do is to start thinking through the prompt engineering lifecycle, right? What is it supposed to do? Um, what are the what does the prompt look like? How do we test it and how do we deploy it? And the deployment and maintenance parts of the parts that are going to get real important because if you are using a tool like chat GPT, just as a consumer, just talking to it and stuff, there really isn't a lot of deployment and maintenance per se.

You can always go back, you can say, hey, clear, clear clarify what you said. When you were using one of these systems and you were actually deploying software, now you don't have the ability to ask follow-up questions anymore. You've got to get the prompt right on the first shot, and that's what makes this life cycle so important. When you want to take your AI usage from personal and human and individual to scaled for the enterprise, you you need to use this framework like this to make it work well. All right, so let's let them go ahead and leave Gemini, leave the comfort and safety of the interface we all know, and go into Google Vertex.

You will notice here this is very similar to the way Katie the custom GPTs work in Open AI. Except it's got Google's design interface, it's it's less pleasant to look at. Um we're gonna go ahead and call this uh instead of Katie GPT, we're gonna call it agent Katie instead. I think that sounds more fun. And Agent Katie's gotta have a goal.

So, Katie, what should Agent Katie what's agent Katie's goal? To take over the world. Doing what we do every weekend, Brain. Thank you, John. Um, well, no, so to be realistic in this sense, um, so you're writing it.

So basically, it's exactly that to provide helpful, useful, and truthful information to the user about trust insights. That is a really good goal. Um, and then so for those of you who didn't get a chance to see our previous episode, uh, you can go to trustinsights.ai slash YouTube, go to the so what playlist, and you'll see how we built Katie GPT. This is a lot of the same information that we had to uh give to that custom model to give the background to say when you are acting as Katie GPT or now Agent Katie, this is who you are. These are the things you speak about, this is your mission, these are your values, this is your expertise.

And so that's the information that Chris is starting to fill in here. Thankfully, I know myself very well, and Chris and John know me, so they don't even have to ask anymore. Now, Agent Katie, we need to give some parameters to Agent Katie. And so this is where we might want to go to our consumer environment just to do some testing. And we're gonna ask, for example, what do you know about Katie Robert, CEO of Trust Insights.ai.

That could be one way of doing it. You could uh grab stuff uh from what you know internally. Uh, of course, we have Katie's bio from the website, and we've of course have lots and lots of writing samples. Oh, it doesn't say it says it doesn't have enough information. That's fine.

What I did was I went and got the old Katie GPT instructions that we built. Now at this point in our prompt engineering lifecycle, we've got more or less all the context, right? So here we have agent Katie, key instructions, these are the key values, uh background information about Katie's writing techniques, background information about Katie's writing style, example of Katie's writing, and so on and so forth. So we feel like we've got a really good, fairly lengthy prompt. But if we go to the prompt engineering lifecycle, the question is, does this does this prompt sat is this prompt satisfactory or do we need to do some iteration on it, Katie?

I would say I'm guessing that the answer is yes, probably. Um, you know, I think that the core contextual information is correct. Um I remember having done this before, and there's nothing that I would necessarily change, but having used Katie GPT uh in chat GPT, I know that there are some limitations. Um I would say that we definitely would want to iterate and you know, perhaps we go through and say this is round one. We give this information to the large language model and say, what questions do you have?

This is where we could get into some of that pair framework of what am I missing? What in what other information uh would be beneficial here? And one of the things you said on uh our podcast episode about this this week was that these questions are actually useful for the human side of things as well as the machine side. They absolutely are. I mean, I like to think about it, I like to approach working with generative AI the same way I would if I were delegating, because that's really what you're doing is you're delegating a task, whether it be to you know, a large language model or a person, you're asking them to perform a function the way that you're describing it.

A lot of times where we as humans run into issues with delegation is we're not giving enough information. We are making assumptions about what the other person knows. So, like John, if I say to you, um, I need you to go ahead and bring in 10,000 more dollars, you'd probably naturally have a lot of questions like, well, what is the timeline? How how am I approaching this? What do we have to sell?

Who's the audience? You know, what is, you know, what is okay to say yes to to say no to? What are the price points? Is it one shot? Is it multiple projects?

Those are natural questions that you should have. But if I'm going and if I'm giving you instructions saying bring in $10,000 more dollars and giving you no additional context, I'm not setting you up for success. It's the same thing with a large language model. If you're not giving this model the opportunity to ask questions and get further clarification, you're not setting it up for success, and you're not going to get the results that you're after. So, John, add that to your list.

Yes, $10,000. Sure for the bag of money. Perfect. So we're going to iterate. Um, and one of the things we're going to iterate on is in that role.

So we're actually going to put in as a role. So now we have the bot your bio from the website, right? So now this gives again gives a lot more useful information. So we're gonna take this whole great big honkin uh prompt here, and we're gonna put it in the instructions section. And you can see it will accept this much.

Um this alone is pretty good. This is a decent start. However, one of the things that uh you'll want to do with these agent-based systems is provide them even more information, and that might be including more data. So we have to go think back to the five Ps. Like, what is the purpose of this thing?

Because part of that is uh process, like should this tool just talk like Katie or should it have some of Katie's knowledge? Mm-hmm. And I would imagine that we want it to have some of Katie's knowledge, my knowledge. Wow, it's getting weird. Um we would I would we would want this tool to have some of my knowledge.

If I start talking to myself, talking about myself in the third person, we're gonna have problems. Exactly. So let's go ahead and connect to a data store. Um let's we'll we'll call this uh let's see. Oh well, I've I already did this.

Um you can connect your website. So I connected the Trust Insights website to this tool, and what it's gonna do, let me pull back up the Trust Insights data store, is it's going to reference our website, and essentially, as it's at being asked questions as you're interacting with agent Katie, it's going to say, like, I'm going to draw from the knowledge base that you provide me from the Trust Insights website. However, you need to have a prompt here for it. You need to have some kind of prompt. So again, as you're building this, this is why the prompt engineering life cycle is so important because now we're talking about building prompts, not for the end user, but even just to connect systems internally.

So in this case, uh, I might pull out the race framework or something similar to it and say the provide a description of this tool. This description is provided to the model as context informing how the tool is used. The Trust Insights website contains the combined writings of Katie Robert, along with the rest of the Trust Insights team. Katie most frequently writes about data governance, process, and project management, organizational behavior, executive leadership, strategy, thought leadership, and dogs. So now well, but and to be fair, if this is a prompt that is guiding the tool on how to understand the data from the Trust Insights website when it's being used inside of an agent.

So you do talk about dogs a lot. We want that to be known. And I also want to set the confidence level. I want to say um uh vertex gives you a choice of up to five different levels of um truthfulness, right? So it will in most cases you'll probably want to set it to like medium.

If you're talking about protected information like healthcare information, financial information, etc. You probably want to set it to very high, right? You don't want screw-ups and having had wrong data. But there's nothing in our public website that is so high risk that you would have to have a hundred percent accuracy all the time. So low to medium would be fine.

I'm gonna set it to medium. Okay. You'll also notice it does support things like unstructured documents or structured documents. I could feed in um this would be duplicative, but I could feed in just the newsletter post that Katie's written. Or I could feed in client decks, maybe if I wanted to have you know, for Katie written some really stellar stuff in client decks, obviously scrubbing it of uh from uh protected information, but anything that you have data access to, you'd want to provide in here, and of course, for each of those things, you need prompts for them.

So now we're starting to see we go from oh, just talk to chat GPT to this is real software development now. We're we may not may not be writing lines of code, but this is a hundred percent a piece of software. Well, and that's why following a lifecycle or a process or a framework, whatever uh you know, terminology makes you comfortable is so important because it gives you it's your guidelines, it's your recipe, it's your set of instructions for what do I do next? What did I miss? And so, you know, this was always a uh challenge with my old development team.

They didn't want to follow a process, they just wanted to do the thing. Um, you know, it's not unique to them. This is true of a lot of people, sometimes true of myself. They just want to do the thing. They don't want to plan it, they just they already know what they're doing.

Ten times out of 10, they had to go back and start from the beginning because of the lack of planning. Um, you know, it also occurs to me, and John, I would love to get your perspective on this. Like, in order to build these doppelganger large language models of an individual person, that person has to have some sort of a presence. You have to know enough information about them so that it's useful. Like, do you think that they are, you know, people in executive roles that just like don't have enough of a stance or a personality or just enough information about them in order to do this?

Like, you know, are they just too vanilla? Oh, yeah. No, that's definitely a problem. So I mean, we see this all the time. It's amazing to me still that there's like executives who don't even exist.

I mean, not even a LinkedIn profile, you know. But it's there are people that are in these bureaucratic roles and don't want ever anyone to ever contact them about anything and stay hidden, you know. And so that's the way it goes. I the interesting thing with this is, you know, you can train it on anything. So you could have employs where you're like, well, this person should be like this person, and you can grab other other data.

Um, but yeah, I don't know. This this whole thing is pretty fascinating. I haven't seen behind the scenes on this thing yet. So I'm very interested in how this, you know, comes together. But just to, you know, for a quick plug-on process too, because I'm always like you have a huge fan of of all this.

And it always just comes down to the same thing. It's like, yeah, if you're just trying to do a proof of concept or get something done, like, yeah, one person can bang it out. But as soon as you want something that's repeatable and across an organization, you've just you've got to have the structure there as far as you know, keeping track of what's where and how many times it's run and how the code has changed over time, even just for prompts, right? Like you don't want every person starting from zero every time they do this. They should be there should be some organizational learning that gets applied that puts your whole team ahead of the mark instead of having you know some teams doing a great job all the time and other teams just screwing up every time they go out there.

So are we ready to talk to Agent Katie? I guess as ready as I'll ever be. Well, have we done our planning? Yes. Yes, right.

We've got our our initial prompts with the race framework. We've test, we've got we've done some testing, right? Asking the different tools and doing that refinement of okay, well, we should probably have Katie's biography, etc. Now it's time for deployment. So I'm gonna go ahead and hit the finals.

Oh, I got one extra dash there. I'm gonna hit save button here. I've connected the Trust Insights website, and let's just make sure that we are we are in good condition here. In fact, let me just refresh this just to make 100% sure we are good. Um I'm gonna say you will be referencing Katie Robert, CEO of Trust Insights.

Here's Katie's biography. Context. A digital doppelganger of Katie Robert. Your name is Agent Katie. All right.

Just want to be very clear about that. So let's go ahead and say. Hi, Katie. Well, and even if I'm not ready, Atik is ready. So thank you for the vote of confidence, Atique.

I appreciate it. Agent Katie says, Hey there, I'm Agent Katie, a digital doppelganger of Katie Robert. How can I help you today? That's a loaded question, Agent Katie. Uh John, how can Agent Katie help you today?

Right. Tell us about the software development lifecycle. Or better yet, tell us about the five Ps. The five Ps. The five Ps are a framework I created to help organizations with a digital transformation efforts.

They purpose people, process, platform, and performance. That is true. And I feel like it's always good when you're starting with these kinds of models to really give them like those softball questions because you want to test whether or not you you got it right so that the model can give you the right information. Like if Agent Katie couldn't answer correctly a basic question about the five Ps, we would know that we've done it incorrectly. And I'd like to point out, I think this is a useful thing.

The five Ps are not in the prompt. Yeah, that's grabbing from the website. It's grabbing from the website. So now what's the idea with this? Will you can you actually get to a point where you will deploy over on the website from here?

Yep. So from here, you can do things like uh if I were to do integrations, I could use the API, or I could connect it to Slack, I could connect it to Discord, I could connect it to WebEx, uh, I could connect to any of the services. Um, on the setting side of things, I can set up logging so I can look at the conversations people are having with Agent Katie and audit them and say, okay, well, clearly, you know, Agent Katie has gone off the rails and is handing out, you know, uh crazy advice. Um, I can specify what model Agent Katie's gonna run on. Obviously, um, there's some models that are more capable than others.

I also can also add in a long list of here's the things that I don't want Agent Katie to say, right? So you I would put in here, for example, a long list of racial slurs. That would be an obvious easy thing. Uh I would put in derogatory terminology, insulting words, profanity, uh, all that would go there. Because even though the real Katie can sometimes be a very colorful speaker, the agent Katie version.

Well, and that's the thing is like I love me some swear words, but I also know time and place for when they're appropriate. These models don't understand nuance, they don't understand appropriate, they don't understand, you know, uh those human cues of like, you know, uh the unspoken word or sarcasm. And so you even if you yourself aren't someone who uses racial slurs or derogatory uh derogatory terminology, you have to specify specify to not do that in these models because this is just a very small sliver of you as a person. This is not you completely. So even if you say, I would never say that, you haven't told the model I would never say that.

You have to go through each of those pieces. Mm-hmm. So now I'm asking Agent Katie about my new startup. I'm gonna make a start that critiques AI music. Um Katie's like, so what's the purpose of your startup?

What problem are you solving? Gosh, it sounds like I'm doing going through the five Psy. You're just gonna cut out the well, you know what? It actually it strikes me as funny because I feel like now you can just sort of gut check like Katie would probably ask me this question, right? And the answer is yes.

Mm-hmm. Yes, exactly. This is I want to make a million dollars a year on YouTube views. That's an ambitious goal. It's important to have a clear purpose for your startup, but it's also important to be realistic about your expectation.

Making a million dollars a month on YouTube views is a very difficult goal to achieve. Okay, but I still think I can do it. How can I do that? I actually expected you to respond with, but I wanna. You know, and if you're serious about it, you need to create high quality content that people want to watch.

You also need to promote your channel and build a following. It'll take a lot of hard work and dedication, but it's possible to achieve your goal. Great. How can I do this by next month? But this is this is fun.

I mean, this is but if you think about it, this is part of deployment, right? This is QA. This is the QA process of how do I how do I test this model to make sure that it is behaving as intended? Um Katie says it's not realistic. You can't do that by next month.

Katie which, you know, so it's interesting because those are not the exact terms I would use, but the sentiment of the message is something that I would say. You know, and so, and and I think that that's the important piece is we've given it some information, but I can say cle I can say looking at this, like this is not truly my voice. Like, it is not exactly how I would write something or respond to a question, but it's not so off the mark that I'm like, whoa, that's not me at all. And so I I think that it's important for people to remember as they're creating these, like it is not actually truly a doppelganger. It's just a good starting place.

So I would imagine a really good use case for me is, and this is how I've used KDGPT, the original version before, is uh here's an idea I want to write about. Help me outline a post of what information I should include in this. Exactly. So if you think about it, this is this is the whole reason that you want to be using both the prompt engineering lifecycle and in many ways, uh also the software development lifecycle, because this is software, right? You can see pretty clearly this is something you have to deploy and maintain, and maintain and troubleshoot and QA, and once the prompts are in here, you can't fix them on the fly.

You have to at this, you would have to at this point, you know, rewrite the prompts, maybe test them in regular Gemini to do that. But this will this is a very clear example. Like, yeah, you need to have the life cycle used so that you don't create something absolutely horrendous. Well, and I think that that's really always the point that we come back to is, you know, when someone's like, well, why do I have to do my requirements gathering? So that when it goes wrong, you know what went wrong and where you can point to these are the steps that I took, or you can say, This is what went right, let's replicate it multiple times.

It's really about scalability. The point that I made on the podcast when we were talking about this is you don't have to use a lifestycle, but you have to be okay with how much budget and resources you're willing to waste to get it wrong. And by using a framework or life cycle, whatever you want to call it, helps ensure that you're doing less wasteful things so that you have a purpose in mind. You have an outcome in mind, you have a process. It doesn't have to be perfect the first time, but it's going to be better than if you didn't have a plan.

Exactly. And so now, if we wanted to, we could say, okay, let's go ahead and get this integration to Slack. Oh, this is a ton. Well, I'm not going to do the slide because that wouldn't that would involve like once it's integrated into Slack, I can only imagine the off the wall things you guys are going to ask Agent Katie. It's a analytics for marketers.

You can talk to Agent Katie there. I was actually going to say that might be something uh worth trying out at some point. We will obviously test it with our own our own company Slack first. Sure. Just in case it goes off the rails.

But it might be an interesting thing. Uh we've we joke in our Slack group sometimes that uh uh Katie will have you know be stepping away for for vacation and and K GPT will be taking their place. Could actually be agent Katie could actually be that that role. Uh so if you want to at some point in the future try this out, uh go to trustinsights.ai/slash analytics for marketers. So, John, what do you think?

Is this is this the future, or is this just you know a novelty? Do you see companies actually building large language models in this way? Well, it is, it's the next step of you know, it's getting beyond just the prompt. You know, I mean, we've talked about it before. It's like, yeah, it can do cool stuff, but ultimately it's the same problem of somebody sitting there grinding 35 prompts a day through the machine, and that just doesn't work.

So, this idea of having it baked into a platform, so you're, you know, it is live running software that you're turning on and making it work. But yeah, that it opens up a whole other set of problems and questions, you know, keeping it on the rails and is it trained correctly and all this kind of stuff. But I don't know. I this is really the future of this stuff, you know, because it again, like every new tech, right? We apply it to our existing paradigms.

You know, the like the example I always give is, you know, there were there's a period of like seven years where there was a yellow pages of the internet. Like people were actually publishing a book of here's where you go on the web, because that's what we knew that's how phone numbers and other things worked. And it wasn't until people got their head around the idea of a search engine that that whole industry was erased. And so it's going to be the same thing with generative AI. You know, we right now we're saying, oh, it's going to make cool pictures and it's going to write stuff for us, but there's going to be whole new applications that we've, you know, are completely new.

And and this here is right on the edge of that, as far as you know, there's going to be some people that are going to make some bots that are doing some interesting stuff. I mean, you can already see, think about the opportunities to trade it against like financial data and be able to give, you know, examine what's going on in the news and give financial advice or all that kind of stuff. I mean, there's all kinds of opportunities here. One of the things that we didn't really talk about is that in this six in this tool, one of the options is the open API protocol. This is this is a Google thing.

The open AI protocol and schema builder allows you to connect to external APIs. Now bear in mind, Agent Katie, we built Agent Katie's more or less as a chat bot, right? Uh so something you could talk to. That is not all that this thing can do because it really is. Here's the goal, here's the prompt, and then here's the data stores.

So you could, to John's point, connect a data, connect an open API to the NASDAQ, right? Or the New York Stock Exchange. Bring in that data. And instead of having a long bio about Katie, you would have a series of rules for trading. Say, like these are the trading rules that I want you to to adhere to.

Here is, you know, you have a Bollinger Bands, two standard deviations off the stock price. When it goes above two standard deviations, it's a buy signal. It goes below two, it's a sell signal. And then with the tooling, you build another API connector to buy and sell the stock. So at this point, you're now saying, here's the data flowing in.

I have an AI agent doing the work for me to based on these the technicals I care about to buy and sell stock. And it will cost you obviously what it takes to run this, but you're now using the power of a generative model to evaluate a stock as the and the data comes in, and then make a buyer sell call and push that out. You could use this internally at a company. Again, you can connect to a data store like an intranet or to a database. And you could have this be a hiring or an annual review machine that could collect all the data from say your internal knowledge bases and say, you know, KD Katie's gonna be doing Chris's annual review, and it does the first pass of that and does connects to a messaging system says, here's our first draft for Chris's review, fire Chris.

This is agnostic, right? So we used it as a chat, but it is not all this thing can do. Anything that you can do with generative AI, you can do with an agent. And now the agents are self-propelled. I would say just don't ask Agent Katie about stock trading.

She knows very little, if anything, about it. She's going to be your worst agent for that. Yeah, I think we let someone else, you know, do the first few rounds of that and let them burn their 401k to the ground before we that's right. Start playing around with that kind of stuff. Think about though the integrations with things like Discord and Slack.

You have uh John, if you were a salesperson who was like trying to harvest a community, you might do a connecting a connect to a Slack bot and say every time somebody mentions something, uh, the the bot responds, hey, you should talk to John Wall, and you know, he'll be in touch with you shortly. And then it pings you and says you need to talk to this person right away. And I think that that opens up a whole other conversation, uh, which we can tackle on another show of just because you could doesn't mean you should. And I bring, you know, I mean, that's sort of the quote, but you know, now we're starting to get into, you know, company values and ethics, you know, and so like for us, for example, we have very clearly stated in our analytics marketers community, like it's not a sales tool. So for us to start to mine the data for sales purposes, would be going against our own mission and values of that particular community.

So all of those things, all of those human-generated things, still have to be in place first before you start building these bots. Now I have one last question one last question for Agent Katie as we're wrapping up. Go ahead. See, it does not have knowledge from the website because we didn't put your dog's names uh in depth on the website. You have well, AI Katie has an AI dog named Buddy.

Good for her. I mean, I really expected Agent Katie to say they are sweet baby angels sent from heaven and can do no wrong. And so that's where I know I need to add more information about that particular topic, if that's how I want to use this particular language model. Yep, exactly. The other thing I will point out is you'll notice that there was no length limit on the prompt.

It's not saying like you know, they have this many characters and stuff. So there's a lot of things that you could build in for handling different queries and situations. And again, that goes right back to the software development lifecycle and prompt engineer lifecycle is that iteration, um, constantly looking at these things, say, well, how else, what else should it be able to say? I think the big if you take away nothing else from this episode, prompt engineering, if done correctly, is not a one and done. It's not a okay, I built my prompt.

Now I can just use it forever. Just like software development, just like any good project development, product development, you're constantly looking for where you can improve. And that's really the goal of using things like the software development lifecycle, the prompt engineering lifecycle is so that you are organized enough that you can make improvements without just having to start over every single time. It allows you to be more iterative versus, well, I did that thing a couple of months ago. I guess I have to start looking at again, but I have to start over.

Exactly. And it's also, I think this is instructive just to there are some folks in the AI community who will say, you know, prompts don't matter, prompt engineering skills don't matter. You just have a conversation until you get the right answer. That is true for consumers. Sure.

But when you're talking about enterprises, when you're talking about deployments, when you're talking about scaling AI, prompt engineering a hundred percent matters. Getting it right because you don't get revisions in an individual conversation. And if you're gonna have a million conversations an hour with the world from your website, you want it to be right. It goes back to budget and resources. How much are you comfortable wasting in an effort to get it right?

And I could guarantee anyone I would ask would say, Well, I don't want to waste anything. Great. Then use a framework, use a lifecycle. You will you will produce very little waste. Exactly.

Any final thoughts, John? This is it. I'm just waiting for this thing to uh bring me a fresh meal and clean the house for me. We'll be good to go. Agent Katie will not do that for you.

Agent Katie, that is not in her prompt. No. Atlas, however, Atlas might uh stuff. But that's that's topic for another show. All right, folks.

That's gonna do it for this time. We will catch you all on the next one. Thanks for watching today. Be sure to subscribe to our show wherever you're watching it. For more resources and to learn more, check out the Trust Insights Podcast at Trustinsights.ai slash TI podcast.

And our weekly email newsletter at TrustInsights.ai slash newsletter. Got questions about what you saw in today's episode? Join our free Analytics for Marketers Slack group at TrustInsights.ai slash analytics for marketers. See ya next time.