You Ask, I Answer: How to Evaluate AI Solutions?

In today's episode, Chiba asks, how do you evaluate AI solutions with everything that's happening right now? How do you know what's real and what isn't? This is a really important question because as you've likely seen on LinkedIn and in the news, there's a gazillion new AI companies every single day promising point solutions for just about everything. And we've seen this happen before, right? In the marketing technology space, we've seen this with the Martech 9000, Scott Brinker's annual survey of the number of marketing technology companies, and it's like over 9,000 different companies that have all these point solutions.

The way you evaluate AI solutions is no different than the way you evaluate any other solution. The framework that I use that that tends to work best is one from Trust Insights. It's the five Ps, right? Purpose, people, process, platform, performance. And very quickly, first, what problem are you trying to solve?

That's the purpose, right? If you want to just use AI for the sake of using AI, you're gonna have a pretty rough time of it, right? Because there's so many different solutions that will let you use AI, but they don't really, you know, that doesn't really give you any focus. What's the specific problem you're trying to solve? And is an artificial intelligence-based tool the right tool to solve for that problem?

If you just need to create content to create content, then yes, generative AI is a great solution. Um there's no shortage of companies that will help you crank out mediocre content. If you want to create award-winning content, that's a different story, and AI probably is not the solution there because creating something that is truly original or award-winning kinda is not what the tools are meant for. They are really good at summarizing or extracting or rewriting or generating from existing known topics and content. Uh, they're not really going to create something net new that's never been seen before.

So that's the first P purpose. Second is people. Who do you have on your team and what skills do they have? That's gonna really dictate what solutions you look at. There are technical solutions and non-technical solutions.

There are solutions that require a lot of babysitting, solutions that are are turnkey. And if you don't have a skills inventory of the people who work for you, you're gonna have a rough time figuring out what solution to choose because every vendor's gonna tell you the same thing. Oh, is it it it's fast, it's easy, it's convenient, it's turnkey, all this stuff. And that's usually not true. Um so knowing who you have on your team and how technically technically competent they are uh will dictate what choices you can and can't make.

It's a it's a constraint, right? If you have people who are non-technical on your team, that rules out an entire section of artificial intelligence tools that require technical expertise and developers to be able to implement. And that's not a bad thing. It's you know it's it's not a knock on your company, it's just that's the reality. Um the third is process.

What processes do you have in place to be able to use this tool? Right? Think about it like a kitchen appliance. It how do you operate your kitchen right now? Yeah, what are the things that you're used to?

You're gonna put a new appliance on the counter. You need to figure out you know, how's it gonna change what menus you decide you're gonna to cook that week? Uh, how's it gonna change where you put dishes away in your own kitchen? How's it gonna change the flow when you're cooking? If you've got this new appliance?

Does it shorten the time from you know in a recipe? If so, you better make sure that your other dishes are are changed to accommodate that timing change. So there's a whole bunch of processes that happen with AI. The question that people ask the most, and first, which really shouldn't be, is the platform. Like, what tools should I be using?

What vendors should I be using? That's the last question you ask, right? That's the question you ask after you figured out the people and the processes and the purpose. Um, because there's no shortage of tools. The question is, is it the right tool for your budget, for your technical capabilities, for your data?

That's an important set of considerations. And finally, is the performance. How do you know that AI is working for you? How do you know that it is improving what you're trying to do? Uh and is not reducing your performance.

So, what are the performance metrics that you that you're going to measure success by? If you do this first before you start talking to vendors, if you do all five Ps, you'll be in a much better place to be able to uh say to a vendor, here's what I'm looking for. And the vendor, you know, the reputable ethical ones will say, nope, that's not us. We can't do that. You know, we we can't do this here, we can't do this here.

The unethical ones will tell you whatever uh you want to hear. But if you've gotten the five Ps down in writing and you're very clear, you can say, Great, you know, you promised this tool can do this. I want that in writing. And I want you know a service level agreement that says if it doesn't do this thing, you're gonna give us our money back plus some. And that's at that point, the vendor will be like, uh maybe, maybe uh maybe we can negotiate on that.

Um, but that's the process I would use to evaluate an AI solution or any technology solution. What's the purpose? Who are the people that are going to be involved? What are the processes needed to support the tool? Which tool or vendor are you going to choose?

And how do you know that you're going to be successful? Answering those questions in detail will save you so much heartache and so much heartbreak and keep things from going wildly off the rails and wasting a ton of time and money. So really good question. Thanks for asking. If you like this video, go ahead and hit that subscribe button.