So What? How to identify use cases for Generative AI at Agencies

Well, hey there, everyone. Welcome to So What the Marketing Analytics and Insights Live Show. I am Katie, joined by Chris and John fellas. Hooray. So I'm just amazed.

I mean, that it was um, I'll give you like a three out of five for that one. Wasn't a total failure. It's pretty good. Um this week, we are talking about how to identify use cases for uh generative AI at agencies. And so this is something that's been coming up more and more is you know, if you're working on a marketing team, if you're working on a data analytics team, a data science team, those tend to be more clear-cut use cases.

But when you have an agency and you have a variety of different clients and your workflows are all kind of all over the place, it's harder to pinpoint how you start to figure out well, where can generative AI fit in and help me? Because a lot of times you're just like running, running, running, running. You don't have a lot of time to like stop and assess and then retrain and reteach. It feels very daunting. So what we want to do today is sort of go through here's how you could approach it if you want to start to incorporate generative AI into your workflow in a non-overwhelming, daunting uh way.

So, Chris, where would you like to start? Uh I think we should probably start. I'm so excited for this episode, by the way, because we used to I can tell you're like literally like I am. I'm excited because we used to work in an agency, and I could just thinking back to all the time that we spent doing stuff that to looking at the experiences back then with the tools we have today makes me go, oh man, we could have done like 15x this amount of stuff that we used to do because these tools enable so much. And it breaks out into two categories, right?

There is the client side of things, doing stuff for the client, right? Um, and that's something that I will very strongly caution. You will want to talk to your legal counsel, you'll want to talk to your operations department about whether there are things that contractually you even can use generative AI with clients and client work, et cetera. But the other side is the operational side. This is how you manage your team, your agency, et cetera.

There's so much opportunity there that it literally does make me giddy. So I think the first place we need to start, unsurprisingly, is the five Ps, right? Which is to say, what exactly are we doing right now? And is it a candidate for to be used with generative AI? So, Katie, you want to talk through that?

Yeah, absolutely. So, at a high level, the five Ps are it's a framework to help you get organized, make decisions. Um, so the P's are purpose, people, process, platforming, and performance, purpose, always starting at the top. You never want to skip your purpose. What the heck are we trying to do?

What is the question we're trying to answer? What is the problem we're trying to solve? Hard stop. It should not be what is the problem we're trying to solve with generative AI. You'll get there.

It's just what is the problem we're trying to solve? Period. People, who needs to be involved? Is it internal stakeholders? Is it a larger group?

Who are the players? Is it your customers? Is it board members? You know, consultants, so on and so forth, making sure that you're getting feedback from all of those different people before you spend the money to do something with the platform. The third P is the process.

How are we going to do the thing? What is our approach? Is it repeatable? Is it sustainable? Is it a one-time thing?

How do we get the information? So on so forth. Then you can go move on to platform. In this case, is it generative AI? Is it, you know, robotic process automation?

Is it something else? And then performance. Did we answer the question being asked? Did we solve the problem? Exactly.

So the first part of figuring out, you know, the generative AI use cases is process documentation, right? So you you have to say, okay, well, we know we want to make things more efficient, right? That's a good, that's a good purpose. You might want to be more specific, but like we know we're wasting time on X. Let's let's let's see if we can fix that somehow.

We know who's involved. To what Katie's saying, you don't want to go generative AI, that's the answer. No, you want to say, what are we currently doing today? So I'm going to show you an example of a process audit that we did in partnership with uh a fellow agency. I've redacted it using generative AI to make it a lot uh to make it safe to show on the air.

And what we asked people to do, and you can do this, you should do this, is to say, look at your week, look at your calendar, talk to your team. What are the things you do each week that take the most time that are repetitive? Right? There's obviously things like you know, talk to the client, which you you cannot and should not automate right now. But for example, uh number one list here, administration taking call notes and summarizing action items.

A person takes manual notes during the calls, then summarizes action items to what needs to happen next. This happens for 30 minutes per call times 11 weekly calls, then 30 minutes to delegate and determine next steps to 11 clients. That is at 11 hours of someone's time per week to do a task that a hundred percent is a machine-based task. But to the five Ps, the purpose would be stop doing this with people, right? The people I would I would never phrase it that way.

Because basically, you're cutting out you know, the second P, which is the most important P. It's a rephrasing and a reframing of what the people are doing, to be clear. Exactly. So the people in this case, instead of doing that manually, the people would be trained to do this in partnership with AI. And this is something that instead of taking 11 mind-numbing hours a week, could probably be done in 11 minutes a week uh pretty easily.

They documented the process, what happens pretty pretty easily. So now we get to the platform part. Is this a task that generative AI can do? Yes. And it happens in two one or two phases, depending on uh the software you're using.

So, for example, let me pull up uh Gemini here. So, Gemini is uh is one of Google's products, and there's two flavors of it. Um actually, let's do this before we do it with Gemini. Let me bring up a sample of a different system. So this system is a system called Otter.

Uh, it is transcription software. And what you can do with Otter is it, and Zoom has this pretty much every major company has the ability to send an AI agent to a call, listen and take notes. So I'm going to go ahead and take uh this. This is just an example of uh an interview done with Kara Swisher and Ted Serrandos from Netflix. Um and here, Otter has created a lovely text transcript.

Now you could just if depending on your tool, just export this and say, okay, I'm gonna now feed it to a tool like Gemini and summarize it. Or I could say, hey, what are the what are the action items from this this particular meeting? And it will go through and build you a nice little summary. So this is an example where um this tool sort of is all in one. But if you have an existing tool, maybe you don't want to buy another tool, you would take that transcript, just copy and paste it and say, summarize the meaning notes and action items.

So that is an example of the the the process, and the the and the fifth P is we got 10 and a half hours back uh uh of this person's time every week. What do you say to uh people who push back? So this is a really straightforward example, but even in this example, we've gotten pushback from other uh peers in our, you know, industry to say, well, I don't trust that the machine is taking the notes the way that I would take them or that my team take. Are they detailed enough? Are they, you know, do they really understand the nuance to delegate it?

So I don't trust the machine to do it the way that a human does it. What do you say to that when you get that pushback? That it's a valid criticism. And the reason it's a valid criticism is because it is one of those things where the difference is going to be made by prompting, by how you prompt. Um, so let me let me uh I have I actually have an example prompt, but I have to anonymize it quickly because it it discloses um some proprietary information.

Um but while while I do that, you can these tools will do pretty much anything, right? So you have to decide what it is that you want it to do and be clear in it. And again, this is where the five Ps really comes in handy is having those requirements gathered, right? To say, like, this is what we're gonna do. Um, so I'm gonna bring this back on screen that I've taken some time some time to sanitize it, I think.

Well, I feel like, you know, so you're saying, and you know, and I'm not trying to be difficult, I'm really trying to understand because we do get a lot of pushback. Like I could say to John, John, I can do anything. And he's gonna be like, Yeah, but that doesn't mean you do it well. You know, I could say, well, I can tap dance. Yeah, I can totally tap dance.

I cannot do it well. The difference between humans and generative AI is that because generative AI has such a ridiculously large skill set in terms of the latent knowledge it has, its limitations are mainly on how you prompt it and not how it um and not on how capable it is. Because inherently it's it's the world's smartest, most forgetful intern. Right. So this is an example here.

I have, and this follows very cle uh clearly the race framework, which you can find on Trust uh the Trust Insights website. If you go to trust insights.ai/slash prompt sheet, uh, you will be able to find uh PDF version, no form to fill out, just grab your copy. So I say at the beginning, the role you're gonna role play as an executive assistance for Fortune 100 company. You you are good at this stuff, right? Here's uh action, you're gonna analyze the conference called transcript produced meeting notes and action items.

Context, here's the background information. But this is the transcript. This is who is in the call, right? So I'm gonna specify, and I could add more detail there. I could add team members of my team and tell the model here's what they're good at, because one of the things that was commented in the original piece was um, you know, delegate and determine next steps.

Well, if I specify and Katie is really good at uh project management, the SDLC uh and oh and high-level business strategy, and John is really good at business development and marketing and media, and Chris is good at pushing buttons, uh, then when this thing produces its digest, it will take into that inf that information into account as part of the prompt. And then I say, you know, this call is in reference to this company's migration to this thing's the platform. Here is the the format that I want you to produce this. So this is a much bigger prompt than people are used to seeing for like summarize this conference call, right? Or what are the action notes from this conference call?

This is a couple of pages of instructions. And when I use this or reuse this with, you know, if this was being used in an agency, I would have a prompt like this for each of my clients. I'd say, This is my client, this is what they do. These are their major pain points. Uh, if you go back to the podcast we did a couple weeks ago on the Trust Insights Podcast, about building an ideal customer profile, you would use the exact same process to build your client profile.

So and that would be part of this to say, like, hey, going to the task. Delegate and determine next steps. Well, if you provide the model with who the client is and what they what they care about, then it will do a really good job of of writing those next steps in that tone. Something you said that was I think was really important, and John, I want to ask you about this is reusing. And I think that there's still sort of that lack of understanding of creating your prompt library.

So creating it once and then reusing it. Uh John, do you so I know you experiment a lot with these uh different tech platforms? Do you have you started to build your own prompt library? Or are you just sort of uh creating net new every time? No, I have a running, you know, in my notes, just a bunch of prompts that we've used for stuff.

And it's not as organized as it should be. But it doesn't even need to be either. As long as you've got the prompts all there in a pile and you just search for whatever you know, a keyword or two that you know is going to be in there, you grab it and throw it in there. Um and yeah, I mean, part of it too is that with the way the tokens work when they're run. I mean, you can just rerun uh a prompt and you'll get different results, and you can see if it's closer to where you want.

I mean, as far as what you're talking about, as far as like, does it work? I've it's just been testing, you know. I've like run enough um transcripts through it to know that it does the job. Like it will find the five points that are top of mind. And I've never had a situation where there was something that came out where I was like, oh no, it should have picked that one up.

Like it does always catch what's in there. So um, yeah, I know that's you know, um the school of hard knocks is a tough way to go, but it does, you know, has proved to me I at least trust it. And I guess that's kind of a weird way to think about it, you know, that just that you trust results having gotten positive results, uh, you know, X number of times. Exactly. Um I'll show you another example.

Um, one of the things that we always used to have trouble with was you know, we're so busy as as agency owners, we're so busy as as people working on client accounts, that we don't have time to stay up on the latest developments to say, okay, well, what's new, what's different? There's so much news. How do we how do we stay current so that we can do a good job for our clients? For example, let's say you do SEO. We do some SEO for some of our clients.

There is a podcast Google does called Search Off the Record. It's a great podcast. It's with Google search team. These folks talk an hour or two, almost sometimes two hours every single episode. And they these are Googlers.

They're telling you, hey, here's what we're doing inside the system. Uh and what you should, and and so you, you know, as as SEO professionals, we should be like listening avidly to this. Do you have time? This is the last uh 15 episodes, right? This is uh a hundred thousand words from the transcripts, which they helpfully provide.

Do I have time to go through this? I do not. So I say, hey, here is the transcripts of the search off the record podcasts, broadly summarize the transcripts, right? And here it goes through and it does this, and then I give it some very special instructions about how to build protocols and say, okay, tell me, based on the last year's worth of episodes, what things should I be doing that they've mentioned on the show. Tell me which show it's from if I need more detail, so that I am so I'm I'm keeping up with best practices.

Again, this is not rocket surgery. This is, you know, this is not something super crazy, but this is a very specific problem to say I've got this information. What do I do? How do I how do I use this so that as a professional at an agency, I can stay up to date. So I think that the this is a really great use case.

And so I'm I'm starting to think back to sort of where we started was you know, we're trying to find ways to make the inefficiencies more efficient. And so it strikes me that you know, we should at least acknowledge the classifications of use cases of generative AI and what generative AI is really good at. You know, so you started with um, you know, we spend 11 hours a week on calls. Well, that to me, you know, when I would then be like, all right, well, can so you know, you're saying yes, generative AI can do that, but you know them really well. Someone like me is like, I don't know generative AI as well, so let me at least consult like what is generative AI good at?

It's really good at summarization. Okay, then that probably makes sense for me to at least look to see if generative AI can do it. You know, you're talking, yeah, so generation, extraction, summarization, rewriting, classification, and question answering. So as you're categorizing the processes that you have, you can start to match them to these six different classifications of use cases and say, you know, I'm stuck, you know, generating, you know, follow-up emails to every single person who has like downloaded a PDF. Okay, generative AI can help you do that.

Extraction. I need to pull out all of these data points from all of these different research studies to put together a talk. Great. Generative AI can do that. Summarization.

This is something we've now, this is our second use case that we're talking about. So the first one being the call notes. Now we're talking about summarize these 15 different podcasts, which is what, 30 hours of content. Summarize 30 hours of content, distill it down to give me a high-level view of like what it is, what happened, because I know I don't have time to listen to those, nor do I have the focus to pay attention and retain the information. And then also summarize and tell me what the heck do I need to do?

Because that's really what people want is just tell me what I need to do and I'll do it. But they don't know how to get to that information. And you just hit upon something really important. So we teach this in our webinars and in our AI course as the six categories. These are the basics.

Where people should be starting to see things now is now you start chaining these tasks together. So first you do the summarization and then you do question answering on it. Like here's the podcast. Hey, what about this? Did they talk about this?

What if I have this kind of situation? What if my client does this? How does this knowledge base that we've accumulated turn into usable answers for our clients? We just did this the other day in our analytics for marketers' Slack group. I took, no joke, about a hundred different documents, uh, blog posts from LinkedIn Engineering podcast interviews with their engineers on different podcasts and threw it all in the machine and said, okay, we're gonna make protocols for what really works on LinkedIn based on the technical explanations from LinkedIn's staff.

And we came up with a nice five-page document, which if you join those analogs from Marcus Slack group, you can get a copy for free. Um, it's join free to join the Slack group too. That is extraction, summarization, and question answering all in the same task. So we're chaining these use cases together now to make the these tools even more powerful. And so that's what I would call that like the 201 of generative AI use cases.

So when you're doing, but it it is all heavily reliant on the five Ps and this process gathering. If you don't do this part, you can't use generative AI. I mean, you can, but it'll go usually about how it does for me when I'm trying to tap dance. Exactly. So let's look at the second task here.

This is a research task, identifying context to pitch based on the topics we have and building a list of them. We use a combination of you know this tool and search engines to identify who's discussing relevant topics and developing media pitches. While this thing is good at it, sometimes it's outdated or doesn't exist. If starting from scratch for new clients, it can take up to 10 hours. 10 hours to build a media list, and these media lists are 15 or 20 names sometimes, not even that.

So this is not a 10-hour task. Based on this, I I'll actually put this to you, Katie and John. How would you tackle this task? I'd probably find a new job. I know it is so funny.

You said that. I'm like, oh, this is so painful. That was my initial. All right, but in all seriousness, John, what do you think? Well, yeah, I mean, you've got to grind it out and get the pitch straight, get the list right, and then figure out, you know, are they going to send emails and how are they going to follow up on that stuff?

Um, but I yeah, I don't know. It's normally just creating a huge list of stuff to do and then dole it out and micromanage. There are databases that you can subscribe to, but they're very expensive and they're not always super up to date. And so, you know, if you're like us, you're a small business and you can't afford to subscribe to one of those services or platforms, you know. Honestly, it would be like, all right, let me open up a web browser with a search bar.

Who covers this topic? What is their contact information? Like this is me typing very slowly with all with my thumbs too. And it would just be long and painful. And so, you know, I am assuming we're talking about it because I'm assuming, Chris, there is a more efficient way to do this.

There absolutely is. So there's a lot of different generative AI tools out there. Uh, we are familiar, of course, with uh chat GPT, et cetera. But one that is ideally suited for this particular use case is a tool called Perplexity. So perplexity is an AI-based search engine.

Um, so let me show you an example. I pulled this up. I said make a list of marketing podcasts that talk specifically about account-based marketing more than 50% of the time. The podcast should contain contact information, return your results like this podcast name, most recent episode title, contact, email, or URL. I want it to be, I want the podcast in English.

And I'm using the pro search feature, which every account, including the free accounts, gets five free for searches a day that are five free pro searches per day. And if you pay the you know 20 bucks a month or whatever, um, then it becomes uh you get like 300 of these. So it will have links to the different episodes and things. You'll want to spend some time refining this, but this is a phenomenal search engine because it is good at finding data that is um that has good sources, right? Um, Christian uh comments in the in the comments uh loves perplexity.

It's it is for what it does, it is very, very good at uh this, and it gets even better if you pay the money for the the pro version. But this would be one example. You can also use things like Gemini, right? So Gemini is Google's uh language model, it is connected to the web. Uh so let's go ahead and pull up this and let's see if we can give it the same query and see what it thinks.

But this would be a good starting point, right? So there's some of these where it's you can tell it's hallucinating the URLs, so you'd want to go and Google uh where if that is in fact surveyal. But this is instead of having to just raw googling and hoping you find stuff, um, it will help you find data faster. So that's a task where there's a generative AI will assist, but it can't do it for you. Well, and I think that you know, in this particular ask, one of the challenges of you know doing a regular, call it almost like an old school uh internet search for podcasts that talk about marketing.

The primary thing you're gonna get is a bunch of lists, and a lot of those lists are people who have paid to be on those lists, um, unlike marketing over coffee, the world's best and oldest um marketing podcast, but you have people who have said, I need to be on that list so that I can get awareness. It has nothing to do with whether or not I'm in the right place, I'm relevant, I just want the awareness. And so being able to get in some ways almost, I wouldn't call it unbiased, but almost like a third party coming in saying, okay, here's what I found instead. You could probably even specify, don't give me, you know, top 10 lists or you know, whatever, uh, as results. Yep, you could do that way.

Also, remember that these tools are fantastic data processing tools. So if you've got data in some fashion that maybe isn't uh formatted ideally, we go to use case number two, which is extraction. So, for example, uh, if I go into the TalkWalker uh system, which is one of our favorite uh media social medium and social and media monitoring tools, and I put in account-based marketing and podcast. I'm going to get some results. I can take those results uh and actually export the raw data as a CSV file and say, okay, I don't really want to go through the what is it, 8,000 results here.

Can I take this CSV file now and put it into Gemini and say, extract from this file the domain names of the podcasts that talk about account-based marketing? And this is a case where you will get much less hallucination because you're now not asking the tool to generate something new. You're saying just here's the existing data, it's just a lot of it. Process it for me. So this is yet another example of where we have we have to tie things together.

We can't just use generative AI in a vacuum. We have to know, again, the purpose, people process, platform performance. Part of platform is not just the tool, but is also your data that surrounds the tool. John, this strikes me as so, you know, Chris is obviously using the research for a media list, but as someone who does business development, um, is this a process that you would use or have used similarly to try to find potential customers? No, we haven't had anything that's been, you know, we have a hard time getting data sets like that.

You know, it's such a um, you know, there's no specific titles that people look for. I think the best stuff that we've done is at least getting closer to an ideal customer profile, you know, finding out the organizations that fit the match of size, number of departments, number of ad spend, revenue numbers. So it we can at least make it a lot easier to figure out where to start all the manual digging. Um, but yeah, no, I we haven't cracked the code on, you know, can we look at some LinkedIn profiles or what are some things that our customers have in common? Because it's just we deal with everything from you know the CEO to marketing analysts to people that are trying to prove product versus web team, tech team.

So yeah, unfortunately, our customers don't easily fall into a single bucket. Well, and I think that that's important to note because I there's this uh assumption that, well, I can just find out that information with generative AI. But as you know, you stated, John, as we're going through and defining our ideal customer profile, it's diverse enough that there is no one single, like, let me just put a prompt in and suddenly get this magical list of customers who are going to hire us tomorrow. Yeah, if only. All right, Chris, what do you got?

Well, no, so uh piggybacking off that comment though, if you have access to some data and you're not sure what the patterns are, you absolutely could be using it to dig through that data and in particular to find relationships in the data that you might not have thought about. Um of the challenges that that traditional marketing automation has struggled with is highlighting who's a likely prospect. I mean, there are entire companies uh out there that that try to build AI around that. Demand based, you know, for example, is is one of those companies. Um you can use these tools to help you digest your data, particularly if you don't have a ton of it.

So, like in our HubSpot instance, we have we have a decent amount of data, but we still only have like 80-ish thousand contacts. Now, granted, if you're a company like I don't know, Walmart, yeah, you have billions of contacts. Uh, it's a little harder to do, but you also have the the money to build your own data center. Uh whereas if you're us, you will use commercial tools. But as an agency, you have that data probably in a CRM somewhere.

Whether you have the people who can digest it for you, that's a different question. Right. So here, this just cruised through those 10,000 results that TalkWalker delivered and came up with uh a decent set of of podcast domains talking about account-based marketing. Now, if I was building going back to our use case, if I was going to our uh, hey, how do I build that media list? I built that while you and John were talking.

Is it that it's a good first draft, it's a good starting point. And I think that that's really the key is a lot of these tasks that you're doing with generative AI, even if we go back to the first one, which was, you know, transcribe and summarize my call notes, it's a really good first draft, and it's gonna get you, depending on how good your prompting is, it's gonna get you 70 to 90% of the way there, and that's where you're gonna find the efficiencies. It's not gonna wholly replace someone reviewing the information, editing, finalizing the information, but all of that upfront work of you know sitting there with your pen, taking the notes, trying to remember what people said, trying to pay attention, being engaged in the conversation and writing everything down, like it takes away that part of it, and then you can focus on the more valuable tasks of getting shit done. Here's one. Sometimes prompts don't work well.

This is a content creation. We had to start from scratch our bylines. The real work often lies in finding relevant research and statistics to support claims. If we could use AI to source some of those relevant statistics, you could speed up the content delivery process. We just showed with perplexity how straightforward that is.

Um, in fact, um, even it's kind of a funny act. My youngest kid uh had to write a term paper recently on uh the free one of the free trade acts, and uh he was saying, uh, I don't know where to start. I said, just use perplexy and write the whole bloody paper um and get it done in 15 minutes so that you can get on with your day. And he's like, Can I do that? Like, of course you can do that.

That that's that's the way the modern world works. So um that's a perfect example of taking this particular part of this task and reducing the amount of time you spend on it by having AI-based tools generate that that those those research pieces that you can then read through or have it summarized and then incorporate your content. Well, and I I again sort of want to with the caveat that it's a first draft, it's not AI is not completing the task for you start to finish. So if you have to write a report, for example, it's a good place to start to source the information to get a rough outline, but you yourself, the human still want to be a part of the process to actually write and finalize the content, especially where you need to do fact checking. I've run into, and I don't know, John or Chris, if you've run into, where you ask it for some information, and it's like, yeah, with confidence, I'm gonna tell you exactly that this is the information that the United States was founded in 1991 by a guy named Jerry.

I'm to I'm saying it with such confidence that you're gonna believe me, and you're like, wow, well, that was easy, and so you still need to, you know, make sure that you're even with really good prompts, even if you're giving it the data, it still can give you back big fat lies, yes or hallucinations. Yes, because it is returning statistically correct answers, not factually correct ones. That is all those Jerry fans are screwing things up, yep. And to your point, Katie, that that a big part of that is how you word the prompt. So let me show you an example of ambiguity in a prompt that you want.

So I wrote this. This is part of my Thomas the Critic uh prompt. And I said, using Thomas's background, identify up to three things, there may not be any that this post does well. Identify up to three things the post does poorly, they may not be any, then make a list of the up to three things the author could do to strengthen the post again. There may not be any.

If you tell a machine, give me five things, it will say, here's five things, and it will hallucinate those five things because it is following the directions that you gave it. If you say, give me up to five things, and there may not be any, you are creating a bit uh enough basic logic for it to say I couldn't find any, right? And and so that's a part of a nuance of prompting that again. We would say this if we had a human intern, like, hey, go find me five things. Now the human intern would go, I couldn't find anything.

Um the machine's like, I will do as you I'm told. Um, author, here's a straw. Like, here's a cup of coffee. It's like, great, I needed a media list. Exactly.

Um it machines do as they are told. So we it kind of like the you know, the the fabled genie in a lamp kind of thing. Um you gotta be real careful what you ask for. It this is a little off topic, but it is also, you know, uh I I saw this post one day where you know, this woman was saying, I asked my husband, you know, to help me, and he's like, Well, how can I help? And I told him he could pick things up.

And it's just a video of him like literally like picking things up and then putting them back down and picking them up and putting them back up because that's what she asked him to do. She said, You could help by picking things up. And that was sort of the end. And that sort of like it's a funny example, but it's a real life example of how you're interacting with these machines. It will do exactly what you say.

It is a literal, there is no room for interpretation. It's not saying, oh, you know, Katie said I need a list of five things, but I wonder if she really meant like six things or if she really meant four things. Like it's not going to try to assume what it thought I meant. It's going to do exactly what you say. Exactly.

Um let's show you another example. This is uh an example uh from uh grading papers. So this is something that I use because we homeschool. Um I said, you're going to evaluate a term paper written by a student. Here's the criteria and how you're going to score it.

So thesis, statement, argument, organization and structure, evidence and analysis, grammar, mechanics and style, sources and citation. Here's how to grade the paper. Here is the scoring mechanism and how you'll format your response. Up to three things the paper did well, and so on and so forth. We had a comment in our Slack group earlier today saying, hey, one of the things that I struggle with is getting people to do things the way that I want.

So I feel like I can't delegate because I I'm not, I know it's not going to be the way I want. A rubric like this, that if you invest the time to say, here's how I expect things to be done, and then you put it into a prompt structure, you can then give that prompt to uh someone else that you're working with and say, do the thing, and then run this prompt against the thing to see how well you did compared to the criteria that I expect. Again, I'm finding a new job. That is so unfun. I mean, I get I get where you're coming from.

I do, I really do. Because, but you know, um, having someone instantly have to grade their work is very demoralizing. Um, it's probably the opposite of what you're trying to do in terms of training and delegation. However, in the context of actual education and grading papers and giving feedback, it totally makes sense. Yep, because otherwise, I mean, we talk about this in our use cases of generative AI for agencies about doing things like grading RFP responses.

You put it on RFP, you get 82 responses. You're like, oh my God. And of course, each response is like 48 pages long, and the first first 46 pages are, oh, we have an office and we have these awards and we want, you know, we have a lake, and here's a picture of our staff. Like, we don't care. Can you do these five things?

You write a rubric, say these are the five things I care about. Score my RFP responses. You can just give them one out or another, and this produce the course, the scores. That's your rough draft. Then you look at the scores, you validate a couple of them, just take a quick look, say, okay, yes, the model did what it was told.

And then there you have your short list. You have your top three, then you can you can human review the top three and say, Yep, this is the one that I want. And now, see that I think that use case makes a lot of sense. So it really depends again, goes back to your purpose, you know, your five Ps, what is it you're trying to do? So training someone to do a task and then asking them to grade themselves, probably not great.

But if you are sending out RFPs and you're getting responses, this is an excellent way to cut down on a lot of the noise. Or if you yourself are writing RFP responses, you can run it through this kind of thing. So you can see what did I miss? Or if you're an agency that does a lot of award submissions, either for yourself or for your clients. Again, it's like, okay, run it through, score my award submission before I even submit it so that I can work on making it stronger.

I don't know about you, Chris. I always hated doing award submissions. Um, John, do you love it? Do you hate it? Yeah, no, that's like uh I prefer the dentist that's over in 40 minutes.

But you are a hundred percent correct. You have the criteria from the award from the award ceremony body. Like, here's what we're looking for in the award submission. You build a rubric just like the paper grading one, right? And you say, here's the here's the requirements for the this award, here's my award submission.

Score me on how well I meet these things and tell me the three to five things I could do to improve it based on this. And if you are an agency that has any experience that you've been around for a little while, say here are past submissions that won. Here are past submissions that lost. Tell me uh improve the grading rubric to help me, you know, make my my award submission more like the ones that won and less like the ones that lost, right? Again, this is all language stuff.

And so that is case where uh there this uh agency was saying two to four hours of writing per award submission. That should be 15 minutes for the first draft, then a two hours to human review and polish it for sure. But your first draft should not be that much time. And all of these prompts, and again, I just want to reiterate uh and John, what you were saying is, you know, you can just have a pile of them. Like you can get to organizing them, you can be like me, everything has a folder, everything has a name, and you know, a whole convention, or you can just have a pile of them that's searchable, but you don't have to rewrite the prompt every single time.

Because Chris, you're showing multi-page prompts, but if you it's like anything, it's like any requirements which is why you do the five piece up front if you put in the work up front to be clear about what you want to do how you're gonna do it and what the outcomes need to be I mean this is like requirements gathering 101 the actual execution goes so quickly goes so fast and is so efficient and so that I think is sort of part of you know the other half of the conversation is people are saying well you know I'm trying to find efficiencies but now you're telling me I need to spend hours writing prompts I could have just done the thing already and I've had this conversation with you Chris I've actually pushed back and be like I could have done the thing in the time but your point is right however I can now rerun the prompt an infinite amount of times and like you know run so far past what you're doing that I'm gonna do 10x what you're doing and I'm gonna do it better. Exactly here's a let's look at this one last use case I think this is an important one. It's very similar to the one we just did the podcast I do a news scan every day for this client based on keywords and competitors that takes me an hour every day that one takes me forever because I have to send a summary of each article tool finding them as a pain search engines sometimes don't don't show certain things I miss important pieces some days. Okay so in this case this is a combination so this is extraction right this is classification in some ways it is summarization and it is question answering so it's four different categories all at once. And if I was advising this person, I would say you would should use a tool that gathers this data well.

And search engines may or may not be it. A system like Talkwalker or a brand 24, which is my other system that I really like, would can give you the raw extracts. Here's the text, right? And then you feed that into a system like Gemini and say, here give me a roundup, a roundup summary in outline form of the trends for this week. That's your first draft.

And then say, okay, give me the the three most here's here's what my client cares about. Here's my client profile, here's what my client cares about. Give me the top three articles and one paragraph summaries about each article for this client. And to your point, Katie, this is a prompt. And every week you just add new data, run the prompt, take the findings.

That is an hour every day. You don't have to spend doing that. That the time savings uh will add up super fast. So, fun fact, I will actually be doing that. I've been using that technique for so I um am the corporate community manager for the women in analytics community, and we have um about a dozen different interest groups, and about half a dozen of those interest groups I am responsible for content curation uh and scheduling content for.

Pretty straightforward stuff, but it's something that historically had fallen to the bottom of my list because it's a pain because you have to find the article, read the article, summarize the article. Now I've refined my process to include Google Gemini to give me what are the three takeaways from this article so that when I'm sharing the article in the different interest groups, because I'm not an expert in IoT and cybersecurity, but I'm sourcing content and trying to share interesting content with these groups. So I'm saying, help me help me do this thing faster so that I can then give them the information so that they can benefit from it. So I am the first one to raise my hand and say, I am absolutely using that technique because it is been so much, it has been so much more helpful to get at least that initial summary. Because I can still glance over the article and be like, oh, is this actually what the article's about?

But it's already done the hard work of writing the key takeaways, it's already picked out all that information for me. Exactly. And it's it's so straightforward. So to recap, number one, you've got to use the five P to and do that process audit at your company, your agency. What are the things on the client side and on the internal operation side that are the biggest time sucks?

And especially look at them with a critical eye of I'm pretty sure a machine could do that. Like, hey, transcribing calls and not adminotes, I'm pretty sure a machine could do that. Second, look at the six use cases for generative AI to figure out what combinations of those use cases fit the processes you've outlined. Because some of them, so Katie's point at the start of the show, not a fit for generative AI. Generative AI is not a magic wand, it is a tool and it has a specific use.

Third, I let it take one of those things and say, let's figure out how do we make this thing work. How do we use generative AI? And you may not be able to use it as we saw today. You may not use it for the entire thing. You may use it for pieces of the process and still have other parts in the process.

And if you are an agency and you want some help with this, we do this. Look, this is literally the client notes we had from my client. So we do this. So if you would like us to do this for your agency, drop us a line. John, thoughts?

Yeah, he hit two things I was gonna throw out right there. One is that you know, if you want to skip all this and just jump to getting a file every week of the articles, you know, we do that. And it's not all about generative AI. We've been doing machine learning and other you know advanced data analysis. If you're looking for predictive to help with SEO and which topics you should be writing about on what weeks and all that kind of stuff, again, you can just uh throw us a check and have that done for you and start working instantly.

You don't have to waste time with it. Final thoughts, Katie. Honestly, just start somewhere. It can it can, you know. I started the conversation with it can feel overwhelming, it can feel daunting.

You know, once you take a look at your processes, the things that you're doing every single week, start with the lowest risk. Start with the thing that if you change the process, things won't break that bad. And just start trying it, start experimenting with it, write down like here's what worked, here's what didn't work, because that's the best way that you're gonna find out. Um, okay, this is a good solution, or we need to keep doing it this way for now. But here's what I've learned for a future reference.

Um yeah, I would say start with like transcribing notes. You can have that running in parallel to you actually taking notes and then actually do almost like a face-off of okay, which one got the notes better, or which one was faster. And that's a very low risk way of trying testing out these systems. Exactly. If you've got other questions and stuff, you know, feel free to reach out to us.

But that's gonna do it for this week, folks. We will see you on the next one. Thanks for watching.ai slash TI podcast. And our weekly email newsletter at TrustInsights.ai slash newsletter. Got questions about what you saw in today's episode?

Join our free Analytics for Marketers Slack group at TrustInsights.ai slash analytics for marketers. See you next time.