So What? Integrating AI into your company

Well, hey everyone, happy Thursday. Welcome to So What The Marketing Analytics and Insights Live Show. I'm Katie, joined by Chris and John. Hello. This week we are talking about integrating AI into your company.

Everyone's talking about artificial intelligence right now. Chat GPT, BARD. I mean, I've personally lost track of all of the different systems that have come out just even in the past week, or the updates and you know, so we don't, you know, that's an important topic, but we want to really talk about you and your company because with all of this technology disruption comes a lot of people disruption, comes a lot of companies grasping onto shiny objects and trying to figure out how does this work for me, or if I start using this thing, what do I need to do? Am I even ready for it? So that's what we want to tackle today.

You know, what are the latest developments with AI? Uh, what are the use cases for integrating AI into your company? AI is not gonna solve every problem. So we want to make sure that we cover that. And then best practices to evaluate the readiness.

Uh, there are ways to tell whether or not your company is ready uh to start to bring a technology like artificial intelligence in. So Chris, where do you want to start? I think we have to start with the AI portion itself, which specifically means AI is gonna find its way into your company in two different ways. One is purposeful, right? Uh purposeful meaning that your company decides, hey, this stuff is important.

Uh we need to, we want to go and we want to buy this stuff. We want to get this stuff. Let's let's put the stuff in the company, right? Uh a good example would be like our friends over at Gochar. You are a content marketer and you say, you know what, we need to get more of our stuff um operationalized or changed to different model to you know different formats and stuff.

I want to use a purpose built piece of AI to do a thing. That's one way that AI is going to find its way into your company. The second way that AI is going to find its way to the company is it's going to be forced on you, right? So you upgrade to the latest version of Microsoft Office, or you use Bing, or you use uh you know Google Docs, and suddenly boom, there's a whole bunch of like really fancy new AI there, and you didn't get a vote, right? It's it just kind of appeared.

Uh same with like Google Analytics. Google Analytics has AI built into it uh for specifically for its anomaly detection. Uh if you were to go into to GA, you would see very you know pretty clearly like, hey, these are some of the insights, these are things that you need to pay attention to, and and Google will, you know, GA will yell at you uh about things. And again, you don't get a choice in this, it just kind of appears. So I think the important thing to start uh thinking about is do you have a plan for each?

Right. So what's your plan for purposefully introducing AI? And what's your plan the I which I think is a bigger problem for when AI finds its way in, whether or not you want it. Well, and I think that there's the plan for bringing it in, but first you need to see if your organization is even ready to bring AI. So sure, there's the purposeful in that example with Go Charlie, like, you know, our content marketing, we want to do more, but are you even set up with clear processes and data collection that would allow for bringing AI?

One of the big challenges that I see across the board is that there's a misconception that artificial intelligence is going to solve a lot of problems that you already have existing in an organization. Um, and what that really translates into is you don't know what the problems are, so you're just gonna slap another piece of technology right on top of it. Um so let's just take AI out of the equation for a second. John, you've worked at a lot of startups. Have you seen sort of this phenomenon of let's just keep bringing in tech and it'll solve the problems?

Yeah, you know, in um dysfunctional culture, yeah, that happens all the time, you know, like everybody's looking for the next big thing. Um but for startups, yeah. I don't know. I just keep coming back to the idea that these are tools for experts, you know, you kind of need to know what you have to do and what you want to do. And then if you find something that you can automate and cut cycles out of the workflow for you, that's great.

But yeah, this is not like, hey, we don't know what this is or what it does, but let's just like start using it today. And like that's just kids running with scissors. It's true though, because I was having a chat with a friend this morning, actually, um, who's a music student, and uh she was saying her professor was just ranting about you know chat GPT and AI uh and you know and people using the software to to write like concert reviews and just how inept you know these tools are. And I said, Well, what's the prompt that they were using? It was like a two-sentence prompt.

I said, Well, that's why you know you you're getting bad results because the the person who came up with this to rant about it specifically doesn't know how to use the tool. I said I I wrote uh a one-page prompt back and sent it with you know with the review, and you know, she showed it to a professor and he's like, Oh shit. He's like, This is a better review than I wrote I'm like yep if you know what you're doing with the software so that's a big part of the the five Ps when it comes to AI and readiness for AI is do people have the skills to even be able to use the tools and again I think for purposeful AI when you're going to go out and buy an application you can take the time to do that evaluation say okay yeah we're ready you know or this department's ready to use this tool. When it gets forced on you by you know big tech vendors that's when I think there's gonna be a lot more potential for chaos. So if we take the example of the five Ps and sort of understanding the readiness of artificial intelligence you know you've already started to cover a couple of them Chris and so you know the people is going to be the biggest um stumbling block in terms of introducing artificial intelligence because big surprise people run the show you have to program the system you have to judge the output you have to do something with the information and as these interfaces get more user friendly it doesn't change the fact that it's still artificial intelligence and that just because I can write a prompt or someone else Chris in this example can write a prompt say you know give me a music review that doesn't mean that it's any good you then took the time to write a one page prompt because you've been studying prompt engineering and actually wrote something useful.

Those are two different levels of skill trying to accomplish the same thing. And so first thing you need to do is a skills gap assessment of do I have anyone in-house, or can I bring in someone who actually knows how this thing runs? So you need to do that skills gap assessment first and foremost. It's not good enough for you to say, hey, marketing analyst, here's a piece of AI software. Good luck.

Or hey, you know, content marketer, here's Go Charlie. Good luck. There's a lot of skill that goes into it. And Chris and I covered this somewhere, sometime. I don't remember now.

Oh, probably on Monday when we recorded the podcast, where you actually have to start thinking about do you have software development skills within your team, not just analyst skills, not just writing skills, not just management skills, because these versions, this new school of artificial intelligence, requires some level of understanding of software development. So that's number one to determine the readiness of your organization. In terms of the people, uh Chris and John, what else would you consider? The other thing you need is in addition to technical skill, you also need domain expertise, right? You need to be able to know if the prompt is if the software, the systems are spitting out what you intend, if you know you know what something means.

A real simple example again, going back to our friend Google Analytics. You know, Google Analytics does not have any of the fancy large language model stuff, it's got some basic anomaly detection things uh that you know you will show up when when there are alerts. And the challenge with that, even for um even for it, you know, like uh just basics is do you know what this means? Like, so what does this thing mean? So, yes, AI detected an anomaly in our web analytics.

So what? Right? That's literally the name of the should we do about this? This is a really good example. Sessions from direct traffic spiked.

This is a literal so what because you can't do anything about it because direct traffic has no attribution. If you don't know what that means, this you might think, is this bad? Is this good? What you know, what do we do about this? Um, going to the example of um the the prompt.

Um, you know, this is the example the example prompt that I wrote. Um this you need to have a some domain knowledge about music, not just prompt engineering, but to the music itself to be able to write uh the prompt necessary to generate a good review. So I had to to you know dig around and rummage around in my head to figure out like, well, what are the things that would you know you would want to have in uh a good music uh concert review? Um it's not just you know write a review of a concert. That's that that's not going to get you anything very specific.

So you need that, and then again, Katie, going back to what we were talking about on the podcast this week. There is also then you have to go back and fact check the output to say, like, did this in fact do the job that we told it to do? And B, is it factually accurate? Because it it might not be. Well, and you know, the really simple example of that that we gave was we asked Chat GPT what it knew about me, Katie Robert, and it was about 90% of the way there until it said that I was a professor at Rutgers University, um, which I'm not, and I've never been to, and at this time don't have any affiliation with.

And so I happen, you guys are lucky. I happen to be a subject matter expert in what Katie Robert does and does not do. So I could easily fact-check it, but someone who isn't me who doesn't know me that well might just look at that and go, Oh, okay, that makes sense. That's a really simple low-risk example. Um, you know, but Chris, to your point about the prompt engineering, people need to understand that these systems are not search engines, and a lot of those prompts that people are generating, they're asking the questions that they would ask of a search engine to generate a piece of useful content, a music review, some data points.

And that then goes back into that skill set of do you feel equipped to use these systems the way that they're intended? You know, I can say that I'm a novice and I would need more training before I felt comfortable sort of leading the charge for the company on these different systems. Um so yeah, I feel like it's a huge thing that needs to be evaluated first, regardless of whether it's purposeful or forced upon you. Exactly. And you know, that goes squarely in the process section.

So, what are the processes that you need to uh to have? If you think about the machine as sort of this black box, what are the things that have to happen before stuff goes in the black box? And what is happening to happen after it? So, fact-checking, for example, be something that has to happen after the black box, and you take that process out and say, Okay, well, who are the people that need to fact-check the what the machine has done? Because so again, someone who's the good a prompt engineer may not have the domain expertise to look at that and say, This is correct.

I did a bake-off uh last night. You can go see the video on the Trust Insights YouTube channel, um, where I put in um, I said, you know, what are some uh effective ways to to prevent COVID, right? And I put it into Google Bard, do chat GPTs, uh GPT-4 model, and then uh Microsoft Bing, and the funny thing was so Bing kind of hosed it, you know, it said you know, wear mask and social distancing and stuff like that. Uh, it did not get ventilation is one of the answers I was looking for. Uh no, it did not get vaccination, although the answers Google said vaccinate and wear a mask, but no nothing on ventilation.

GPT-4 got all three. So even though it's not a search engine, it was more correct than the actual search engines. And so again, that fact-checking, you know, that domain expertise to go, okay, these are the things that should be in there that that in these cases were not, is problematic. You need to have that domain expertise. So part of integrating AI into your company is who's doing the fact-checking.

Well, and so now you're starting to talk about the other gap that you need to assess, which is your processes. Um, so first you need to evaluate whether or not the people have the skills, but then okay, let's say you have people who have skills to run these systems. What is the actual process so that you know that these systems are effective? And so we talked about um how to A-B test some of these things a few episodes back, but it's more than that. So if we look at the Go Charlie example, and that's you know, purely writing content, what is your content creation process and where does this AI tool fit into all of that so that you can determine this is saving us time, or this is taking us more time, or this is just sort of equaling out, or we can scale or whatever the situation is.

And so the process is also something that needs to be evaluated. Do we even have a repeatable process where we can start to bring in artificial intelligence? Or if we don't, what does that process look like? Because again, sort of thinking about now artificial intelligence is this sort of shiny object for a lot of companies, they're just signing up for it and saying, okay, what can it do? And then that's a big time waster.

We actually have uh a comment from our friend uh Brian Piper. Sorry, John, this covers your face. Uh he says great points. You also need to be able to track the impact uh that adopting new tools has on your productivity and conversions. That's exactly it, because bringing in new tools, there's a learning curve, there's probably you know, set up an integration, there's maintenance of this thing.

All of those pieces get factored into the overall cost, not just the five dollars a month that you might spend on any given system. How did it go? How did we hear? Uh, still not great. Nope.

Sorry, it doesn't pass muster. Um so that takes us to the I think it's a good good uh segue into performance, right? So, how how do you measure? There's obviously the cost side, the time to retrain and things like that. How do you measure the performance, particularly for stuff outside of marketing?

Let's say, like, you know, your janitorial service is putting in timesheets through Excel, and now Microsoft Copilots there, and uh they have the opportunity to do that. Or uh John's using ChatSpot as part of the new Hubspot uh CRM, and and you know, John can now update deals uh just by talking to uh uh a large language model. How would you measure that performance? What you what are the things that you would be looking for? Well, it goes back to, and we probably skipped over the most important P, which is purpose, um, because we were talking about purposeful, but not purpose of AI.

So the first question you need to ask is what problem is this solving? Why are we doing this in the first place? What's the so what of this? And so, you know, to use John as the example, you know, maybe his purpose for using the chat spot uh versus through HubSpot is to you know convert deals faster, or for him to be able to do even more outreach because he's not bogged down in the admin side of things. And so by stating that purpose first, by using that user story as a business development partner, I want to use chat spot through Hotspot so that I can close my deals faster.

And then that would become the measurement. The so that the outcome is the thing that you start to measure the performance on. And you would say, okay, am I closing my deals faster now that I'm using chat spot? If the answer is no, then you know that that's your performance metric. You know that this is not helping you.

Yeah, I would argue, you know, there's there's sort of the baseline. Does the software do what it's supposed to do? Like, does it do what's uh what says on the box? Um the folks over at HubSpot, Darmesh uh Shah, who's been building this, uh, has been very clear. This thing is still highly experimental, right?

Like it will break more than it doesn't. So you know, that's totally understandable. Um but then part of it for like a salesperson or a sales administrator, you know, which is this is the bane of CRMs, is getting your salespeople to use the CRM to keep it up to date, right? So if you can look at something like you know, uh record completion or number of updates in the system, if you suddenly see when you roll out this tool that suddenly, you know, your sales force has gone from 40% of salespeople updating the system, you know, each week to like 80% because they can just now tell it, hey, tell you know, this deal is moving forward, and instead of having to click around in the interface and and click all the buttons, that would be a win because you have a very clear metric. Like, yeah, we want we we care about efficiency here, but you've got to have those measures of efficiency, uh, wouldn't even know to look for them in order to measure them.

So, John, as the designated Hubspot user on our team, what would be your use case for using uh chat spot in HubSpot? Yeah, for a lot of this stuff, it's party tricks, you know. I mean, updating records wouldn't be the thing, but where it could really be fantastic is in report generation. You know, like you said, like that prompt that Chris had just tried there. If you can, you know, show me everybody that's in this time zone or show me everybody that's in the Baltimore area because we have an event coming up there or whatever, being able to slice and dice data without having to go through a report interface is fantastic.

And then there is other value too, if it's able to do something like if I could say, okay, take all the contacts from a specific geographic area and set them up with a task uh, you know, over the next week uh to follow up or to kick off uh a series of emails, you know, something like that is kind of cool if you're able to do that. Um but yeah, I don't know from everything I've seen, it is like it's real alpha software, like I would even be afraid to use it because I wouldn't want it to start sending Muppet you know gifts to everybody from the New York area or something like that, like it just could totally go off the track. So I would would let it ride for a little while, get at least get to beta. You're gonna go in and change it because I accidentally moved you to an opportunity. See, this is what happens when you don't have domain expertise or a process.

Well, when you're playing with alpha toys because again this is sort of the low risk example of okay now we're just kind of playing with the system to see what it does. But some companies don't have the same luxury that we do where we can just back out of something easily because all three of us have access to the same the same level of access in the systems. You know, I can imagine in enterprise sized companies with all the different layers of permissions if Chris changed John's status on an opportunity and maybe even the dollar value of it accidentally but Chris is in an admin and he can't roll that back. He then has to go through the chain of command to let someone know hey I need to roll this back but now you know enough time has gone by or the board got an alert like hey we suddenly have a $5000 opportunity. That's amazing let's start making plans around this money like it moves really quickly and there's a lot that can go wrong if there's not a proper process in place for managing these systems.

Yep sandbox are your friends remember that use the sandbox it's true though that's why those things exist the sandbox the staging environments do all of your testing outside of a live environment like don't do things on a live website don't do things with your live data that's just a bad idea. So how do you tackle it when AI is forced on you right? So you come into work on July one and suddenly everyone has Microsoft co-pilot at their desk uh and you're like, uh, well, we didn't even know this was going to happen, and now everyone's got the thing. Um, you have a whole bunch of people who have no training uh on it, but they're like, hey, there's a chat box here, and they're you know, they're talking to it like it's clippy. Uh, and then suddenly clippy's like, Oh, I will delete all your files.

As you as your question, I will delete all your files because you somebody typed in F this out, I don't want to work you anymore. And so Clippy's like, sure, I'll take care of that for you. Um, the first thing is to like just stop and take a breath, because you have to understand when this kind of software is forced on you, then there's gonna be some sunk costs, there's gonna be some failure points, there's gonna be some risks. Because to your in that example, if nobody saw this coming and suddenly, oops, you know, we suddenly have this big piece of software uh that we have to learn how to use, and 90% of the company uses Microsoft Suite, and the majority of the work that we do is powered by Microsoft Suite, then you have to pause and take a breath. You have to stop for a minute, you have to designate someone, doesn't matter who, someone has to very quickly figure out what this thing does.

And if you can't, if you can't designate an expert, you pick up the phone and say, Who's my Microsoft expert? Who actually knows what this thing does? Hey, Chris, you've been talking about this for the past six months on your blog. Do you think maybe you could you know do an emergency session, come in, talk to my team about what the heck it is? Um, because to your point, if people just start playing with it with live data with live files, they could accidentally delete really important things without knowing how to retrieve them.

So I would say my advice when it's forced upon you is to just like stop for a minute, take a breath, look around, see, does anyone know what this system is supposed to do? No, okay, call in an expert and have them explain it. Katie is embracing the vanilla ice school of management of crisis management. Yes, you have to stop, collaborate, and listen. ICE is back with a brand new edition.

Um you have to be careful because this is like prime ground to have a new coke scenario. You know, somebody will come out with something and everybody will get so angry with it that they'll start tagging it, Clippy Jr. and you know, they'll switch over to Google Docs. So it's uh, you know, let's uh see who's got their CX uh outfit on and does it right. Yep.

So in specific cases where it's it's forced on you too. The other thing is how do you manage? So one of the things that tends to happen is that we would we tend to replicate what we know, right? So what we're seeing with, for example, generative AI models is people are asking prompts, you know, creating prompts and asking things of these machines that are familiar territory for them, right? So make it a picture of a dog wearing a tutu on a skateboard.

Um the the actual capabilities of these tools are far beyond that, but because people are not aware of what those capabilities even are, um, they don't know to ask for it. Um we were looking at some examples this morning. I was showing one in um the video saying, okay, make me a list of the Fortune 10 companies in order by domain name, because that's actually a very hard thing to find online. There's actually it's so funny, you there's a like a Fortune 500 uh service out there that just that provides a data file, the Fortune 500. You have to pay money just to get a list of the Fortune 500 by domain names.

Well, now you can just ask GPT-4 for it and get exactly the same thing and pay no money for it. Uh well, you have to pay the 20 bucks a month. But if you didn't know that that was possible, that is still closed off to you. So for companies that want to be agile, the companies that want to move ahead quickly, how do we help them open their minds to what's possible? How what's what is what does that process look like for introducing innovative thinking among the people who are talking to these machines?

I think if you do that skills gap assessment and you find that you don't have that skill set in-house, that you know, person who is sort of the the cutting edge of technology, the person who's staying, you know, involved in what's happening, that's okay. There are people out there who can do that. So I would say that you know, I would recommend to my team like if you know, if I had a team of people and you know, we suddenly were had to start using this new technology, and people said, How do I even know what it's supposed to do? The first thing you do is you start to figure out who are the experts in the industry that you can trust when they say this is the thing that it's doing, you can follow. So you don't have to necessarily hire a Chris Penn, but you could probably follow his YouTube channel.

You can probably sign up for the newsletter and figure out okay, Chris publishes once a week. I can at least stay about a week behind on the new tech. I'm being serious, uh, on the new technology, and then I at least have a fighting chance to understand what the thing does. And then you can say, okay, we've at least figured out what the team is capable of, and then we at least can figure out what the process is for using this. Now we're ready to move on to the next phase, and that's where you'd bring in an expert to consult to say, these are the capabilities.

Help me understand your problem statements, your use cases, and I will help you figure out what kind of AI technology solves that problem. Okay. It strikes me though, there's probably not enough actual experts to go around. Not at this time, probably not. I mean, it's still for as not new as artificial intelligence is the version of artificial intelligence that we've seen over the past few months is new in terms of its capabilities, in terms of its utilities, in terms of the quote unquote ease of use.

And so, you know, Chris, you're doing your best to be an expert in these things, but you actually still have a full-time job at Trust Insights being the chief data scientist. Don't please don't forget this. So you're so you're trying to sort of pull, you know, two roles at the same time, because keeping up with everything that's happening with artificial intelligence minute by minute, literally minute by minute, is a full time job. And then you also have your regular full time job, and then you also have a personal life that you want to make sure you're keeping all of those things balanced. Um, and so you're right, there's not enough experts who are respected authoritative experts.

Anyone can say, I've written a chat GPT prompt. Anyone can say, here's my free ebook of the top 50 most effective prompts. I've never heard of half these people. They just came out of the woodwork. Just like the people who could suddenly magically migrate you to Google Analytics 4 just came out of the woodwork.

Guess what? All those people faded right back in. And so I think, you know, it's gonna be hard because there's gonna be high demand for experts, but so few experts who are actually truly experts who can speak a non-technical language to people who aren't technical people. I feel like this episode is really making me rant. How do you know the difference?

If you don't have subject matter expertise, how do you know who's who's got the goods and who's full of shit? That is I mean, that's a great question. I'm fortunate that I work with you and you know, a small group of others who I trust implicitly who have never bided me. But I guess I would turn the question around on you since you actually let's ask John. All right, let's ask John.

John, how do you know when someone's got the goods versus someone's you know just full of shit and you know, like they they were a crypto bro last week and now they're this week they're an AI expert all of a sudden. Well, you have to either there's two ways to do it. I mean, one is either if you know it's just probing questions, you know, you just keep poking until you break through to the other side. The other one is you have to find another expert who can give you a trap question to ask, you know, and then you can throw something out there and see if it hits. But yeah, that's you know, that's a whole skill set in itself.

That that's a classic game theory poker thing, you know, being able to try and figure out who knows what they know and don't know. And yeah, if you have the knowledge, then you're in a power seat and you you can go there. But if you don't know yourself, then you're uh yeah, you may find yourself with a pork barrel. I think you you started to say something though that I think is really important to hit on is look at the person's resume. And if they were a you know, crypto bro last week and an NFT person the week before that and a web3 expert the week before that, and now they're an AI expert.

That to me raises a red flag because given how quickly the technology is changing, unless you are the one single person developing all of it, which is impossible, there is no way that you are an expert in all of these things all at once so quickly. So I would say definitely, you know, use your best judgment in but do your research on the person. What have they spoken about before? Chris Penn, you know, sitting right next to me, Chris Penn has been talking about artificial intelligence for the past what, seven, eight years now? Ten years.

And so you've never wavered from I'm an expert in artificial intelligence, I'm an expert in machine learning. You haven't jumped on the Web3 or NFT or crypto or clubhouse or whatever the thing is, bandwagon. You've said, cool, all of those distractions can happen around me. I'm gonna continue to be an expert in my lane. So you can trust me when I say I know what I'm talking about when it comes to these things.

And you're someone who you've demonstrated through your content, through our work that you are willing to dig into the systems. And so if someone's calling themselves an expert, but they also don't have any portfolio to show that they've actually used these things, that's also another red flag. The thing I keep coming back to though is that one of the challenges, and this this now branches into an entirely different episode, uh, which we'll tackle maybe some other time, is that that inherently creates biases towards the established folks. Uh and there may be new folks who really do know that their stuff, um, but you don't know that they know their stuff because you don't have the domain knowledge. Um, one of the things that I looked at when the pandemic was first starting is looking, you know, who's already working in in places that are generally restrictive, like you know, Yale University's Virology Lab, Dr.

Akiko Iyosaki, right? Not the kind of person who's gonna suddenly be a pandemic expert overnight because she's already a pandemic expert, has been for 30 years. Um, and then looking at what she starts talking about, right? Um, then looking at a cadre, in my case, about 53 people who you know, looking at their, like you said, looking at their credentials, looking at their backgrounds, and then looking at what they talk about and what they generally agree on, right? There, you know, everyone has kind of their own take, but like Eric Tapal, for example, uh Iosaki, uh Eric Ding, all these folks were talking about the same general thing, like hey, the S1 region of the spike protein is is the problem area.

This is the thing that's causing all the problems with this with this virus, and then from there, once you've built a vocabulary of these are the topics that the the fo that the all this collection of people who seem credible all seem to have in common. Then you can look at the what the fringe folks are saying, go, okay, these people they're not saying the same thing. There's and there's not enough evidence to back up what they're saying, right? There's you know, the the in the in the AI world, these would be the folks who are who are making either outlandish claims about AI or just doing you know silly stuff like use your your your book of chat GPT beer prompts. Um, and not looking at here's what the core technology does.

You know, folks who are talking about uh attention is all you need and the attention window in how a transformer works. This is not unique to one person. This is a generally agreed upon industry standard of how a transformer model works. So folks who know their stuff will have that somewhere in their content, and you can look at that, go, okay, well, this person at least knows the basics of of how uh a transformer-based model works. That I think for if you don't have the domain expertise, at least getting the vocabulary is a good starting point so that you can immediately spot, like, okay, well, this person is saying stuff that's wildly off base from what other folks who you know seem credible are talking.

And that list of trusted folks will drift from time to time as you know, ideally as more people come in um to the space. But I find that technique tends to help identify the people that you need um to make the you know, to to make the integration of AI easier and and make the your BS detector better. Well, and you know, again, I think you're right, it's a different topic, but back to your point about you know, it creates bias for the new people who may be experts but don't have the longevity in the industry to sit and make a name for themselves. I'm someone who I want you to show me, don't just tell me. And so if you are in a position to try to find that expert, I would say I recommend looking for people who can show their expertise, as in they've demonstrated they can use the tool.

They have their case studies, they have their sample data, they have their examples versus someone who's just navel gazing and writing about the thing and you know, doing you know, fluffy prompts that don't really amount to anything. That's not demonstrating to me that you're an expert. It's demonstrating to me that you can do the same thing that I can do, which is hey, chat GPT, write me a limerick about beer. Anyone can do that. That doesn't make me an expert.

Just because I can figure just because I figured out how to sign into the system and get it to give me something back does not make me an expert. And 98% of the people that I've seen who are writing about these things are those navel gazers. There's very few people who are demonstrating the ability. So to the back to the original question of let's say that this technology is forced upon you because the systems that you've been using have made a major update, you're suddenly using Microsoft Copilot, but you didn't know it was coming. You have to start looking for those experts really quickly.

And there's definitely sort of that short checklist, as Chris mentioned, of ways to find those people, especially if nobody on your team, nobody in your company either has the time or the capabilities or the skill set to do to sort of do this investigation of what the heck does this thing even do. That gets expensive though. How do you how do you start growing your own experts? So I would say in this example, I think anyone who works in a company who doesn't think artificial intelligence is going to affect their job is sorely mistaken. We don't know exactly what that's going to look like for every single company, but you can make a safe assumption that in some way, shape or form, artificial intelligence is going to have some kind of an impact on your company in the next six to twelve months, if not sooner.

So I would say the first thing you need to do is figure out where to get that information, where to get those really good um updates on what's happening. And I would say, Chris, you know, our content, our newsletter, your newsletter are really good. Our friends over at the Marketing AI Institute have really kept their finger on the pulse of what's going on with artificial intelligence. It's literally what they've built the whole business around. Um and so now, before the technology is forced upon your team, is the time to start finding those experts, finding those resources that at least tell you what the thing is in a way that you can understand it.

I think the additional thing that again, most companies don't do, and it costs you nothing, costs you zero dollars. It does cost you time, is what we used to do with our team at our the agency we used to work at, which was essentially lunch and learns where we required people who didn't have expertise to earn a little bit of expertise that we can then present it to the rest of the team. I could easily see a situation where you know, if we're talking about generative AI, you have uh, you know, sort of a prompt lunch and learn like this week, uh Katie's gonna show the prompts that she worked on that got good results. And you know, next week John's gonna show a prompt that he worked on, and you kind of just create any impromptu knowledge management system where people start sharing things that work better, and that over time, as people learn from each other, you know, none of you are experts in this scenario, but as you find stuff purely through experimentation, you will start to increase the the capabilities of the entire team. You want to talk more about that process about how you coordinate those kinds of things?

Yeah, it doesn't have to be a long, laborious thing with your team, and it doesn't have to be something that people lose sleep over, especially if public speaking isn't really their forte. Hey, here's what I learned this week. And it doesn't even have to be in person. It can be over email, where you just start to give people on your team the opportunity to share what they learned about a specific topic. So in this context, we're talking about artificial intelligence.

I might say to you and John, hey, so Chris, on Monday, I want you to share with the rest of us just one article about what's going on with artificial intelligence. John, on Tuesday, I want you to find two different products and write a quick summary of what they are. And on Wednesday, I will summarize what's going on with, you know, the big dogs and the funding. And, you know, I will also share with you resources. And so you can sort of divide and conquer that way amongst your team and say, just one article a day, share it with the rest of us of what you found is happening with this topic.

You know, for our team at the time, it was what's going on with digital marketing, analytics, predictive forecasting, attribution analysis, the things that our team was really involved with with the clients. But Chris, you were really the expert. And so to give our team a very low risk, low cost way to become experts, we said, Hey, um, you know, you marketing analyst, I want you to spend 15 minutes at our team meeting next week showing our team how to set up a tag and tag manager. So then the task then becomes does that person know how to do it? If not, they have you know some time to figure it out.

There's no rules to say you can't ask other people how is it done. But the goal is that that person who it is assigned to is the one who then presents what they found. And I would say if you are not in a situation where you have access to that team, that doesn't mean you don't have access to those people, right? So you can go to something like you know, this is our our Slack community, Analytics for Marketers. Um find a couple of study buddies, if you will, of you know, folks who are would be interested in in doing something just like that and and join up, joint forces with them and say, like, hey, you know, here's uh who wants to to to do a lunch and learn on on generative AI this week.

Absolutely. And I think that that's a communities are where you're going to find a lot of those like-minded folks and where you're going to stay up to date on a lot of what's happening. Um, you know, Chris, you're a partial to Discord because you can find the major companies that are creating these technologies. That's where they put out a lot of their updates. So if you're someone who enjoys the Discord experience for communities, then I would say definitely start to join some of those if that's a topic that you want to know more about for your team or for yourself.

You can join our Slack group, trustinsights.ai slash analytics for marketers, um, where we're also sharing a lot of the information that we're finding. And we're also trying to put out uh content like this podcast, this live stream, where we actually walk through the scenarios, the real life scenarios that you're going to encounter as artificial intelligence starts to affect your day to day. Exactly. So to wrap up, um the five steps you need to take to integrate AI into your company. You need to have a purpose.

Um, and sometimes it is purposeful. You're going to buy an application to do a thing. Sometimes it's not, where oh, uh it has been forced upon you, and you get to enjoy that. Um, you have to audit the your folks, your people. What skills do they have?

Uh who should be in charge of you know various things. Um, three is you got to look at your processes. What pretend AI is a black box. What do you have to have in place before things go into the box? And what do you have to have in place after things come out of the box in order to make AI safe and effective?

Uh platform, there's a gazillion platforms to choose from. That really should be as it is here, fourth on the list. You've got to get your purpose, people, and process in place. And fifth is look at what are the things that uh from a performance perspective, how you're going to measure that you're doing the thing and that the investments have been worth it. John, final thoughts.

Your Jerry Springer thoughts. You know, um, I was actually talking with a friend in academia who had asked yesterday because he said Chat GPT kept forcing him to have a degree from Harvard. And I was like, oh yeah, Katie was provost at Redcare. And um, and it was interesting. I said, Yeah, we I seem I see a lot of problems with these hallucinations, and uh and especially people using it as search.

And he, of course, was like, no, no, even though people are like, don't use this for search. And really the model is about plausibility, it's coming up with plausible things. And I was like, Oh, that's a nice way to spin it. That sounds a lot less pejorative than hallucination hallucinations, but but that is the thing. This is a way to make things more plausible and to get where you want to go.

And so, yeah, get ready to improvise. That's really because we don't know what's coming at us, and uh, you know, get ready to try some new stuff and hopefully you don't get in too much trouble. Um, I just want to acknowledge one last comment. Our friend Brian uh said communities are also a great place to ask about potential experts to get those qualifying questions. And that's exactly right.

You know, so that's tying all the different pieces together that we were talking about in terms of finding the experts. If you've joined a community like analytics marketers, that's a great place to sort of ask the question. Hey, my company wants to start using Microsoft Copilot. Are there any experts that you trust, anyone that you would recommend to help us with this integration into our company? So that's a really great point.

Um I'd say the shout-out, Brian's book, uh Epic Content Marking. Go grab a copy of it. It's excellent. Um, I would say my final thought is you know, yes, there's the sort of gloom and doom, artificial intelligence is coming, blah, blah, blah. Don't panic.

Just take a breath, and you will have time to figure out how this works in your company. If it's forced upon you, still stop, take a breath, stop, collaborate and listen, is really the big aim here. Um, I need to create a meme, but anyway, that's a different day. Um, but make a plan, start making those backup plans for if that day comes. Start finding those experts, start figuring out how you can get your team trained up, start signing up for those newsletters.

Trust insights newsletter is a great one to start with. Uh, start following podcasts and blogs because a lot of people are talking about there's a lot of information out there, so you need to start distilling it down into your specific use case. All right. Now on that note, we will see you all next week. Thanks for watching today.

Be sure to subscribe to our show wherever you're watching it. For more resources and to learn more. Check out the Trust Insights Podcast at Trustinsights.ai slash TI Podcast and our weekly email newsletter at TrustInsights.ai slash newsletter. Got questions about what you saw in today's episode? Join our free Analytics for Marketers Slack group at TrustInsights.ai slash analytics for marketers.

See you next time.