So What? Data Horror Stories!

Well, hey everyone. Happy Thursday. Welcome to So What, the Marketing Analytics and Insights Live Show. I am Katie. Chris is traveling.

John is somewhere, but today I am joined by maybe I think. Did you say your name was Clark? Dan, I didn't want to violate the copyright laws playing uh John Williams hit. But yeah, I figured I could wear my costume, you know, rather than just get two hours Halloween evening out of it. I could, you know, get one more play out of it.

I mean, I love it. I'm I am here for a Halloween costume. I I use you know, I usually go big on Halloween. I used to when I worked in an office, I would always dress up. But you know, now that it's just me by myself in my house with my dog, she really doesn't care what I do.

Yeah, I I don't know. It's uh I think it's part of truly a marketing creative thing of just like costumes are a challenge, you know. It's kind of like what can you what kind of story can you tell? What can you pull off? And then the thing for me that has been just crazy is I got into it heavy with my kids, you know, with starting from when they were babies, we would get them dressed up as stuff.

And so now they're both teenagers, and so I have all these pictures of all these ridiculous costumes over the years, which is just fantastic. It kind of keeps me going. So yeah, I'm I'm all for uh a little bit of uh Halloween creative muscle exercise. So looking forward to it. Um well and our buddy Brian uh says he's wearing his full transformer costume right now, working at home by himself.

He said, just kidding, but I don't think he is. I think he's very serious. I'm gonna take that one. Autobots roll out. Um, so today on this week's live stream, in the spirit of spooky season, we're talking about data horror stories.

Uh, and so we asked our analytics for marketers slack community, which is for you to join, trust insights.ai/slash analytics for marketers. And John, you also posted on LinkedIn asking people for their data horror stories. And unsurprisingly, we want to be able to tie this back to something like the sixcata quality framework. So let me see. I'm gonna make this a little bit easier to see.

There we go. Uh so the six C's of data quality are clean, complete, comprehensive, calculable, chosen, and credible. So, John, you did the due diligence of collecting a bunch of these horror stories. So let's talk about some of them. What were some of the better stories that you got?

Yeah, well, you know, unsurprisingly, uh, many people were reticent to share tales of stuff that they've really screwed up and crashed and burned, you know. So um, I did get a couple that were in there, and you know, thankfully, at least I'm already gonna, you know, admit to a bunch of disasters. So I have a uh trove of burning the whole village kind of tales that I can throw out here. Um, I had eight that came down. I know we didn't get to do as much of a rundown.

How many do you want me to just give a quick overview of them? And you want to run down through the stack first, or what's how do you want to roll it? Yeah, no, I mean, I'm open because basically, you know, I think what what I got from collecting the stories was okay, we're not alone. A lot of people have these stories, but I think where people find challenges with these types of things is so I have these stories, but what do I do about it? You know, so I'm I'm happy to have you just sort of, you know, go for it because when we get um towards the middle of the show, I actually have a little bit of the solutioning.

But first we got to set the stage. Like, what are the problems? All right, that sounds good. So I'll do I've got eight tales. We'll start at the bottom, number eight, you know, the least horrifying of the batch.

And that is this is for all marketers. Everybody loves this when you know you sweat when you hit that send button. You know, you're sitting there with the email marketing automation, and you just you've checked it a dozen times and you click the button, and you know, 15, 20 minutes go by, okay. You've moved on to the next thing, you're all good. And then suddenly the emails start coming in that oh, the link in this message is bad.

And so what are you gonna do about that? And you know, if you've just mailed 60,000, you know, 250,000 addresses, you've now created, you know, a huge mass of inbound customer service complaints and just everything across the board. Who knows how much damage you can do? So that that's the first uh tale of panic right there. And I, you know, usually it comes down to unclean, right?

It's just a matter of somebody broke something somewhere along the way. So that was the you know, the stories that came in that was enough to be frightening, but was the bottom of the list. Um let's see if this works. I feel like this deserves a uh sound effect. We're not gonna we're not getting the clunk that we need, or at least it's not coming through to my side.

Oh man, that's all right. It's that's definitely a sad trombone. I gotta see on the board if I shouldn't have a yeah. Did that come through? If I yeah, there we go.

There you go. Um it I feel like that's such a relatable thing. And this is why, you know, we've talked about in other past episodes of the live stream and the podcast and pretty much anything, you know, people will listen to us on. Don't QA your own stuff. You know, you're gonna look at it 10 times, and that 11th time is when someone else is gonna look at it and go, Oh, I see the mistake immediately.

That happens to me all the time because we move too fast, we think we did the right thing, we click on it, we have 8,000 tabs open, like, you know, but to your point, then the shameful, hey, I clicked on your link and it goes to a 404, or hey, I clicked on your link, and I don't know if you want it to go to that picture of you from your summer vacation, or hey, I clicked on your link and nothing happened. Yeah, yeah, it's just uh nothing but pain comes from that that outgoing run there. And I think another thing with this is you hit a great point in that you've got to have a process that has secondary validation, just somebody else has to do it. Um, and you know, over time pick out somebody in your org. Like I have to, I'll give a shout out to Erica Halloran, who I you know I haven't worked with in over a decade, but she was the one person that everybody knew in the company that it didn't matter, it could go through five people who none of them found any mistakes, and it would go to her and she'd find three mistakes in the thing.

You know, she was just, there are some people that are born that have that skill set in the eye for that kind of work, and so you want to find some of those people and then start buying them lunch or pizza or whatever it takes to, you know, get them to help out, but find the experts and roll with those folks, because that's you know the easy way to avoid that train wreck as it comes along. And I would say people are gonna say, ask the question of, well, can't I just use generative AI to do proofreading uh and test? You can, but with the caveat that it's going to also miss obvious things. I, you know, usually try to run a lot of my writing through like basic spell checking and you know, sentence structure, grammar, that kind of stuff, but it always AI always misses something because it's not human. And so, you know, things that like it like it versus is, like it won't correct those things necessarily.

Uh, not because it's not looking for grammatically correct, but it's not so blatantly wrong that it's not going to pick it up. So you're still gonna publish things with errors if you don't have that human intervention. Um, and so you know, that's that's the case for like you still need a human doing that. You can build really elaborate prompts to catch all that stuff, but I would say you still need a human to go through and make sure that like links are working, uh UTM tracking codes are set up correctly and being caught that your, you know, tag manager or Adobe Analytics is set up correctly. Like there's a whole bunch it that goes into that one little link being in your email.

Absolutely. Um, all right. So in at number seven, I have having to install an Oracle database to be ready for tomorrow. So this is a project thing where there was a group of people going off to an event in Chicago, and they knew that they basically had to have two or three demo systems built, you know, to go there. But the problem was they hadn't calculated that the demo software had so much data in it that it would actually take about two days to build the machines to do the demo.

So when they showed up the day before the event and started working, they had to work all through the day. And you know, by 6 p.m. that night, they were like, oh, wait a minute. No, we're still like not even half installed on these machines. And so the team had to stay up through the night all the way through till the next morning.

And it was literally a matter of packing a bunch of laptops in a bag and somebody like driving them to the airport to meet the team as they were flying out because they weren't able to build the machines, you know, in time. Normally you would want to have them built at least two days ahead of a time so that the team could then, you know, play with the machines, make sure everything's working, get used to it. But yeah, this this was um, and I I filed that one under not calculable. You know, you need to figure out like how much data you've got, how long does it take to process, how long does it take to run reports or whatever? Because if you have just way too much data, it's not in the right format, and it takes, you know, 10x what you thought it would do to get the job done, you're gonna get yourself in a in another train wreck.

I mean, there's a lot to unpack with that. I would I agree. I think that that's calculable. That's also complete. Um, and probably comprehensive and probably credible and probably clean.

So let's just say that that one hits all the marks. Um, and so if that's only number seven, I'm nervous for what's to come, but that's definitely, you know, a case for, you know, I know this is gonna come as a huge shock to you, John. But you know, the five Ps could have helped. And within the process P is when you would have outlined, you know, so we know we need these people. The process is this.

Let's back into a work back plan and a timeline. Oh, we probably need more than 24 hours. And I'm not shaming whoever it is that submitted this story. Things happen. People ask for last-minute crazy banana stuff all the time.

I get that. That one to me, yeah. If that's number seven, I'm terrified for what's to come. Well, the good new, you know, that's only, and in fact, they actually did get the gear there. So uh, but yeah, it's it's the uh the landings get worse as we go here.

Um so in at what are we at with number six right now? Um this one is more there are a bunch of different stories that fell under this category. And anybody who's done marketing ops in a smaller org has done this. This is just the spreadsheet is off by only one row, right? So this is you've got this thing where you've got a spreadsheet and you're trying to clean it up to load maybe into your email uh system or for some kind of survey, but whatever you're basically massaging a spreadsheet, right?

Like that's the thing. And everybody always does this where they grab the the columns that they want to sort and they miss one of them, you know, and then they do the sort and now that last column stays in the order it was in, but the everything else gets shuffled. So now you've polluted the data across the board. And the uh, you know, this came up a dozen different ways, but one that comes over multiple times is this where it gets shuffled, and so you do your email blast and it goes out to the email address, but the first names are all off by one. So nobody gets the right first name in their message, right?

And so the link one is kind of is bad, but you can fix the link. And in fact, today on better email systems, you can update a link and that will just work. But something like this, you polluted the whole data pile. And so everybody gets a bad email address and it just looks horrible for everybody. Um, and one upside of this one I did here, this was the number one driver for people to switch to CRM systems.

You know, this is something where you this happened one too many times, and then they finally said, no, we just need, you know, whatever our CRM is to automatically dump the data into our email system so that it's not a manual process and we you know remove all chance for error. But so yeah, the spreadsheet jockeying coming in at number six. I remember a friend of mine telling me uh about his wife's job. She worked in marketing uh for a large grocery store chain in New England, and he said that he was always amazed at her ability to like basically wrangle her spreadsheet because if like she was in like the, you know, there was so many columns and it covered so many things, like and it was all in one sheet, and so you had like all of the hidden columns, and then you had things that were like were locked and you couldn't see all in one sheet. And so she was in like the triple letters.

So so like once you go like A through Z, it goes AA through Z. And then you start to get into like the and it just bonkers beyond like my ability to comprehend how you can do your job with something that big. And to me, that kind of falls under, you know, spreadsheets when they get that big and unwieldy fall under, you know, the comprehensive, must cover the question being asked. That's too much data. Like, how do you how can you answer a question with that much data?

Yeah. I mean, you should never, if you've got anything and you're going beyond Z Z, like that needs to be in a database. Like that's what databases are made for. The spreadsheet is not a database. You gotta get out of that.

Yeah. All right. Uh let's see. All right. So at uh number five, um, this we've seen hands on with a bunch of people.

It's basically your AI project failed. You know, that you wanted to do something with AI. Um, but then you, you know, you went through all the hassle of building the thing. You you use your data and you just output is all garbage. You know, everything it kind of gives you hallucinations.

So everything is just fake and false or looks bad. But yeah, that was a that one came up many times. And that one was a little bit of a it wasn't the scariest ones because a lot of these are pilot projects and they're you know new things that people are trying to do. So the stakes were kind of low. But you know, for everybody that does have a crash and burn, now your ability to get selected to do the next really crazy pilot project, you know, goes to zero as you tank that.

So um, but yeah, you you've seen this firsthand too with a bunch of stuff. What would what's your take on this this one here? Well, I'm going to uh tap into my spooky side and be a fortune teller for a second and say that if people stick around, there may be a solution to that exact problem if they want to see what it is. Um, which is at TrustInsights.ai slash AI ready data quality audit. That's the demo we're gonna do shortly.

But that's the you know, this is what we talk about a lot. And by no means are we the only ones to talk about this. I feel like it's a well-known thing that AI is going to amplify the things that are wrong. It's not going to fix them unless that is your goal. So really it's gonna say, this thing is wrong, so I'm just gonna run with it.

It's not gonna look at your data and go, you know what? They could do better, or they could be giving me something else, unless you're prompting it to do that exact thing. So if you're giving, if you're, I don't know, let's say if you're using your marketing automation, you know, export from you know, your Salesforce uh, you know, system and you want to find out like how many clicks led to uh conversions, but you don't have conversions set up, it's not gonna make magically make that happen. Or if your data was corrupted for three days that month and you weren't aware of it, and so you're missing that information, like it's not gonna guess and be like, hey, I wonder what happened on those days. Let me just fill it in for you.

It's gonna amplify. And what is it that Chris says that AI is a very over eager intern who's like a people pleaser? And so it's gonna try to give you the answer regardless of how bad the data is. And so that's you know, you have to work on that data quality before you give it to AI. It's just that's just a it's a non starter.

If you haven't done any kind of data quality checks first, don't let AI, you know, just go ahead and make decisions. That sounds good. So at number four was I was not expecting this, you know. You know, everything that I had originally run with was, you know, stuff where we're handling the data, you know, you kind of get data and something gets screwed up. But there were a ton of votes that just came in for data breach.

You know, they were somebody broke in, took the data, and you know, everything went south from there. Um, I don't know. I my the big thing with that is like that's data security. That's kind of a whole, you know, that's not along the lines of anything that we do directly. There's a whole bunch of vendors that you work with to kind of lock up your website, lock up your internal IT.

But um, but yeah, I can see how that scares the hell out of people. Oh, for sure. And, you know, so we to your point, John, we don't do that work specifically, but we do have a framework that we would recommend to people to at least start thinking about it. And this isn't going to replace any kind of, you know, uh data privacy or governance, but uh we recommend the raft. So responsible, accountable.

Oh gosh, I should probably know what they all stand for. And now I'm putting myself on the spot and exposing myself. Uh but ideally the idea is that if you don't have any kind of policy in place, and I highly recommend that you work with a proper legal team to put that kind of thing in place. Um Raft is at least going to get you started to think about it. So it's respect.

Here we go. Respect, accountability, fairness, and transparency. And so we frame it in terms of responsible AI use, but I've also used it to think about data privacy because really when you go through all of those steps, it's how are we asking people to give us data? Are we being clear about how the data is being collected? Are we being transparent?

Are would we want our data collected in that way? How are we securing that data? And a lot of these systems, so if you build like a contact form in WordPress or in HubSpot, a lot of them have some of those security controls built in. Don't skip them just because you think it's going to be annoying to you know the end user to have to check I'm not a robot or to fill in those like you know six scrambled letters to validate who they are. That's those are there for a reason.

So you're right, it doesn't fall into data quality specifically, but data breaches are awful. And people can inject false data into your data, which then does become a data quality issue. Yeah, that's it that was like we were kind of in one genre of horror movie and somebody I was like oh yeah that's not in our thing but man that is absolutely horrible over there too. It's the Freddy versus Jason crossover. It's you don't want to see it coming.

It just it's a buzzsaw when it comes at you no one survives that. Okay it's coming in at number three now we're getting into the the big the big guns this is one I'm you know knocking what I've never done this one. So I I kind of am like wearing the badge of like oh this is good but the mailing the blacklist like this comes up all the time where you know you've got your email system and for some reason on some day somebody messes up and they or in loading too they'll do this where there's like that list over there of like okay here's the people that said they're gonna sue us and they hate us, and the crazies are always in that pile over there. And for some reason, somebody grabs that and throws it right into the next mailing so that they get the you know five dollar off coupon, and that's you those you always hear about from customer service, like customer service explodes immediately because those people are just you know, for some reason they can't take one extra email. But um, so yeah, I uh again, I was kind of psyched that I was like, Yeah, no, that that was one firecracker I've never lit myself, but um that's definitely one that's a whole lot of pain when that happens.

Well, think I mean, we can think about it this way, and this is something that happens to a lot of people when they think about their email hygiene. Um, so it falls under clean, it falls under complete. Um, you know, sometimes someone's unsub link just stops working. And so you think you've unsubscribed to something, and you know, they're not aware that you've unsubscribed, and you know, they're sending out their weekly emails or whatever, and you keep getting the email. That is a data quality issue on their side because they don't have a clean list of people who have opted in, they don't have a complete set of people who've opted out.

So that is a data quality issue, and it goes back to sort of number eight, I think it was, where testing the links, making sure that things like an unsub works. As an end user, I mean, you know, this there is nothing more frustrating than unsubscribing from something multiple times and having it continue to show up. It's you know, damaging to the reputation for the brand, it's damaging to their email deliverability, and it's really poor data quality on their side. So there's a lot of things wrong with that. So I would say it's not even as you know drastic as sending to the wrong list.

Like just not even having a clean unsubscribe process is a huge data quality issue. Yeah, absolutely. And it's in most cases, it's like those those people are just a waste of resources in every you know way and form. I mean, they're raising their hand saying, don't talk to me. So if you ever have to talk to them against a waste of time, and there's never a chance of them ever buying anything.

So yeah, that's like uh uh just playing with uh quicksand, you know, you're gonna get stuck over there, which is another horrible thing, which is really overrated. Um, but I'm I'm getting pulled aside on quicksand. Well, yeah. I mean, when we were growing up, how how dangerous did we think quicksand would be when we were adults? Yeah, like every TV show, you know, there was one quicksand episode that was just like a an easy target.

Um but yeah, no, there's not much quicksand in New England, that's for sure. I think we're okay. Um, all right. So coming at number two. Now this one was interesting because this is I mean, it's not urban legend, it was real.

I guess the the thing with this one was there were tons of people that it happened to that pointed out, but actually it hasn't it doesn't happen as much um in current state of affairs. Like people who had this problem in the last three or four years were very few, but there were tons of people that had it, which is just the these backups don't work. Like something goes south, and so you know, you send somebody off to grab the off-site backup of whatever, and you find out that you know the kid next door has been using as a magnet, or you know, the tapes got flooded, or uh another one too of like systems saying the backups are running, but they're not actually running. So that whole idea, and even too, you'd see uh family members, normal people with this would like, you know, their computer gets hit with a virus or something weird happens, a hard drive crashes, and they just have no backups of any kind anywhere. And so they're like back to zero, complete clean slate.

So yeah, obviously, process, huge process thing, right? Like you've got to have a backup process. And of course, like I said, now with everything in the cloud, people tend to trust the cloud to do it. And I, you know, I I get that, you know, it's a lot of work and it's a pain to do backups. So if you want to trust the cloud, I mean, these are huge companies, it is their job.

I have heard occasionally really weird stories of something getting lost here and there, but not that often. Um, but yeah, you know, if you do take a backup, and even if it's just your um time machine on Apple, just like go in there once in a while and just open up a file from three weeks ago just to make sure that if you lost something, you can get to it. Because yeah, there's kind of one of the top nightmares is just waking up to a blank computer and having lost everything. And I think that, you know, in terms of data quality and process, to your point, you know, we as consumers and as businesses have become very reliant on third-party tools to be doing that work for us. And so, in terms of our data quality and you know, the credibility and it must be collected in a valid way, it must be, you know, complete.

Um, you know, we need to make sure that we understand if we're using a third-party tool like a CRM or marketing automation, what are their backup policies? So we do some backup of our own data at Trust Insights, but we're a small team. And if Chris is doing backups, I couldn't tell you where they live. So that is a huge data quality issue for us that we need to address uh when he's back from wherever he is. But it's true for a lot of for a lot of companies, they don't know if there is a backup, where it lives, who has access to it, what's on it, how much is being backed up, is it being overwritten every 30 days, or is it completely comprehensive of everything you've ever done?

And those are really good questions to ask. So when you're thinking about data backups, data quality, you can ask all of the six C questions like, is it going to be clean when I get it? Is it gonna be a one for one? Is it gonna be complete? Is it gonna be comprehensive?

And so on so forth. Because what's the point of having a backup if you can't use it when you need it? Yeah, absolutely. It's when you need it, man, you bet it's gotta be there. That's just a no-fail thing.

I will give a shout out for WP Engine, our website provider. We've used them for years. Chris and I have both been big fans of theirs. And you know, just daily backup is just part of the thing. You know, kind of anything tanks, you can just go back there and click a button and say, make it like it was yesterday.

And you know, that kind of SaaS service is a wonderful thing. It's worth the extra bucks to get that and go. Um, all right. So at number one, this was the the biggest one. This was the complete nuclear blow.

I know this is nail biting. I'm I'm nervous. So this was this is like an old marketing case study that was around. So it is basically a company that did holiday promotions and sales. You know, the the holiday period was their greatest point of sales that made the year for them for the whole year.

It was catalog mailing. So, you know, the one of these companies where they're gonna print up a couple million catalogs, drop them in the mail, again, pay a couple million for postage, and then the orders come in and everything's great. Well, somebody in the data department screwed up as they were putting the mailing together. And normally they would they do this mailing for the top 10% of customers, you know, the the best of the best customers, the folks that buy like crazy, the the top decile is who gets this catalog. Well, they screwed up and instead of the top 10%, they got the 90 to 100th percent, you know, which somebody thinking 90 to 100 is great, but no, actually that's the 10% worst.

You know, the people who never really buy by one, you know, one candlestick every five years, kind of people. And so by mailing the bottom decile, it tanked the entire company. They ended up going bankrupt. Like the catalogs went out, none of the business came in, they didn't have the money to cover everything, and it completely burned the business to the ground. That was the end of the place.

They had to close up shop. So that's the uh, you know, they kind of uh selected the wrong uh button and burned the whole place to the ground, which was the worst. The chosen data was incorrect. Yes, the select, they grabbed the wrong select, and because of that one file, it you know, everybody lost their job. That's brutal.

Yikes. I mean, and and that's the thing is like we talk about data quality as these six aspects, but when you bring it to that larger scale, it can have devastating business impact if you're not thinking about basic data quality. So you've covered, you know, the eight horror stories that you collected. And I feel like we've covered all of the um the the tenants of data quality. Um when we were talking in the pre-show, you didn't feel like you had a good example for credible.

Yeah, yeah. Credibility is um, you know. We talked about it as far as validity, you know, it's kind of like how you grab the data. Is that correct? I mean, I did have, you know, mailing the blacklist is is burning your credibility, but that's not the same thing.

So I mean, it if you could give us an example of where you would see credible, how often does that come up and what usually is that involved? Yeah, absolutely. So, well, a couple of weeks ago, we did actually do a very quick example, uh, which you can find at trustinsights.ai/slash YouTube, go to the so what playlist. Um, and we were able to show that our Google analytics for data was not being collected in a credible way, despite the way that we had set up the system. We, you know, we feel very confident with how the system is set up.

Um, you know, it's one of the things that you know we offer as a service to our clients, but despite that, we don't have control over the system. And so the data, when we took a look at it compared to um, I forget exactly what we compared it to, but we were able to see that there was enough missing data, there was enough uh unusable data that it made it not credible and collected in a valid way. And so, you know, think about your third-party systems and the data that you're using to make decisions. Do you know the definitions of those data? Do you know if they're sampling versus pulling everything?

Another really good example of this is survey data. So when Chris and I worked at the agency, one of the things that we used to do for the other teams was run the Google Google consumer surveys, which I loved. I love writing surveys, I love running surveys. You know, you can do an employee feedback survey, any kind of survey. The challenge that we ran into is that a lot of teams would approach us and say, I want to run a survey, and I'd be like, Great, let's talk about it.

And they'd say, So my client has a headline that they want to run. I just need the data to back it up. That is not how you run a survey because you're already going into the survey with a bias of how you want it to be answered. So therefore, you're asking questions that support the end result versus asking unbiased questions to find out what people actually think and feel about something. So that's a big thing in terms of credibility.

So think about anytime you see a methodology statement or how we collect this data or how we use this data, that's where you would factor in credible. So a lot of it credible really, you know, it fits nicely with market research. How are you collecting the data? I like to joke with you and Chris and everyone that like I'm an N of one. I am not a representative sample.

I am not enough data to say I represent this many people. And so that also is credibility is if you're pulling a small sample of information, is it representative enough? Is it statistically valid in a way that it covers, you know, me and you and Chris and Kelsey and our analytics for marketers, you know, uh community? My one opinion is not credible enough to cover everybody. We need more information to say, okay, now this is how everybody's feeling about something.

So that's you know, I could go on my soapbox all day about credibility, but it really is, especially when we're using third party tools like our SEO tools and our web, you know, tracking tools. You have to think about how is that data collected? Where are they getting it from? Are they inferring it using now in the day and age of AI and machine learning? If they can't collect it, are they inferring it?

Basically, are they sort of guessing at what it should be to make a complete data set? So those are the things that you really want to dig into. That sounds good. Yeah. So it's funny, Chris is on his flight to London, but somehow he is still, he's either got enough bandwidth to check in or whatever.

He put his backup process over in Slack for you. So he is uh reporting in and validating for us. I will have to look at that later. But uh thank you, Chris. We appreciate that.

Um, you know, so we've talked about the horror stories, and I feel like everybody has a horror story. Um, you know, it doesn't have to be something that took down a whole company, it doesn't have to be something that, you know, got you fired. It could just be a small niggly little thing. But the next question is how do we fix this? How do we get ahead of it?

Especially when we're trying to use AI paired with art, um, wait, what did I just say? AI paired with our data, we want to have data that is ready. Um, and so that's where things like where did it go? The AI ready data quality audit comes in. And I am gonna do a quick demo of this.

And so um, you know, with the examples, John, that you were able to get from people, I think they were fantastic. What we didn't get is we didn't get samples of the data so that we could say, and this is how we would fix it. So I am manufacturing another data horror story. Um, and so the way in which I would try to get ahead of, you know, the data slasher coming to murder me in my sleep is I would do something like use Google Colab. And I like Google Colab, and Chris has talked about Google Colab because I'm not someone who's about to sit down and write code in order to fix my data.

I'm just not gonna do it. It's gonna take me way too long. It's a really poor use of my time. And I have access to Colab, which with our Google, you know, uh, we're a Google shop in terms of our workplace, Google Workspace. We have access to Google Colab and people I believe can use a free version of Google Colab.

So now that I've said Google Colab about 20 times, and I have sort of much like Beetlejuice invoked its presence into uh our data horror story episode. What I'm gonna do is I'm gonna start a new notebook. So when you open this system, when you open Colab, you are given this, you know, uh conversation box. And I'm just gonna click on new notebook, and it's just gonna be fresh, maybe. What I'm going to do, and I need your help with this, John, is um let's say I wanted to look at the SEO data for Katie Robert.com.

I want to take, I want to see if it's clean, I want to see if it's complete, I want to see if it's comprehensive. And so what I have, and people can't see because I'm sharing my whole screen. Let me see if I can fix this. I'm not as savvy with this stuff as Chris is. He is way more uh, he's way better at this stuff than I am.

Here we go. What I have is a set of instructions, and so the prompt analyze the included data using the six C Data Quality Framework. And so this has already been written out, and this is what we can do for other people is the six C data quality framework clean and it goes through a bunch of questions that you would naturally ask about the data. The thing that I have to do is uh what's included here, and what I always recommend people do is have a sense of the five Ps. You following me so far?

Is this confusing or is this straightforward? No, we're we're right with the air. Okay. Uh so the five P's purpose, people, process, platform performance, purpose. What's the question you're trying to answer?

What's the problem you're trying to solve? People, who is this for? Uh, who's doing the thing, you know, who needs to be involved, process. How are you doing the thing? How are you collecting the data?

How are you using the data? Platform, what tools? So, in this case, where did you extract the data from? Where is it going? And performance, what is your expected output?

So, this is all part of your prompt as well. So, the first part of the prompt are the six C's and the questions you would ask. The second part of the prompt is, you know, your five Ps. So, purpose, the intended user of this data is a business user preparing SEO reporting. Makes sense.

People, the end user is non-technical. Process, the outcome from the end user is an SEO report explaining what happened in the company's SEO program for the last 60 days. The time frame of the data is the last 60 days. Platform, the tooling of the user is a spreadsheet and slide software. And then performance, process the data file, and produce a detailed, complete, comprehensive audit of the 6C data quality framework, which again, that's the first part of the prompt up here.

Uh the final audit should include a document in markdown format in an outline with a diagnostic of each of the six C's and rating on a scale of zero to 10 for each of the six C's based on your analysis. What that means is Google Colab is going to take my data file, which I've already downloaded from Hrefs, our SEO tool. Use whatever SEO tool you have or whatever tool you're looking at, and it's gonna say, okay, based on this prompt, I have six aspects to judge this data file on. So out of a score of 60, how did it do? And it's gonna go through piece by piece.

And because I'm giving it the 5P context, it's gonna know can this person do what they're intending to do? Because otherwise, if you're just saying, is it clean and free from errors? Is it is there no missing information? You haven't given it the context for it to be able to answer that question. So you need to pair the five Ps with the six C's.

So now I feel I've been talking a lot. You still following me? Do we still make sense now? Yeah, this is it. We've got a prompt ready to run here.

Are we uh ready to get the magic happening? We're gonna get the magic happening. So again, I all I have done so far is I have opened Google Colab and I have said new notebook. So what I have down here, just like any other chat GPT or Google Gemini or Claude, is you have the chat window. This is where you put your information.

So I'm going to upload a file, and I've already done this. This is on my machine. Um, and you can put it over here in the side. Uh, you can upload the file into here and just select it from there. I'm just uploading it straight from my machine.

I am keeping this as simple as possible. And now I am going to copy the entire prompt that we just talked through. I'm going to paste it into the chat box. And I'm going to hit send, just like you would any other system. The difference here is that Google Colab is going to write the code that another system would struggle to do and would probably ask you to do it for you.

So it's gone through. It says, happy to help. And see, there it is. It's the helpful people pleasing intern. Happy to help.

I can analyze the data using the 6C framework. Colab is going to give you a plan and it's going to say, I'm going to generate based on what you've asked. Here's what I'm going to do. The first thing I'm going to do is I'm going to load the data, then I'm going to analyze the clinginess, the completeness, the comprehensiveness, the chosenness, the credibility, and so on and so forth. It goes to go through the actual 6C framework.

It's going to generate the document, it's going to finish the task, and you're going to say, Yep, that's amazing. Accept and auto-run. Are you sure you want to auto-run? Yep, I do. And then what you're going to see down here is this is the plan that it just outlined for you.

It's going to go through step by step. So this might take a second, John. Hopefully not too long. While we're doing that, Chris is saying, You're doing great, boss. Thanks.

I'm very comfortable doing these kinds of technical demos. I'm just not comfortable doing them. Why like I usually have everything prepared in advance. I'm sort of, you know, I don't like to have it run live. So my little bit of nervousness is just making sure that the system doesn't crash because I want to be able to show people something.

But so John, when you're so you primarily responsible for sales, new business. What kind of data do you think that we would want to run through, like a data quality audit that you use on a daily basis? Well, the big thing is it's the whole CRM file, right? I mean, you know, how many of those emails are good? How many have we talked to recently?

You know, that that's always that's the whole data pile that you live and die by every day. It's just a matter of what's in there. And yeah, if you can have a scoring system to help bubble up stuff to the top, and then something else too to clean out the ones that are garbage to try and in fact, that was one of the data stories that we had come in. Um who was it that had that one? Sonny Hunt was talking about she had a project where they deduped a record database from seven million down to three and a half million records in five months.

So, you know, that kind of stuff just goes straight to productivity in the bottom line. If you can clean the data up so that all the activity you're taking is, you know, that's just when you're looking at a funnel diagram, you're you're doubling that point of the funnel if you can clean out that much junk. Mm-hmm. Yeah, I think that makes sense. And you mentioned lead scoring.

And so I think perhaps that might be we can cover that in next week's live stream if you're game. Uh, you know, so just as a little preview, one of the things that we did on a previous episode, while this is still this is just about done almost. Um, if you go to trust insights.ai/slash youtube uh, you and Chris actually walked through putting together a sales playbook. And the playbook is really comprehensive in terms of our services and our approach and the sales frameworks and all of those things and so we put that together and so now that exists and what we don't have in our CRM at the moment is lead scoring setup. We just we haven't gotten around to doing it.

We've been busy you know servicing clients and speaking and being on the road and you know doing things like what we're doing in Google Collab. And so it occurred to me to experiment to see could I take our sales playbook and could I take what we know about what we can do in our CRM and marry those things together to come up with a plan for lead scoring and the answer was yes. Now is it valid? I don't know. I gave it to you to look over so I'm still waiting for your feedback.

Yeah that well and then the big question mark for us is you know what does our CRM provider support right? There's all kinds of different pricing and unfortunately our provider is a big fan of like oh every time you want to do something else it's like oh well you're gonna have to run the credit card if you want to turn that on. So um yeah if there's enough there for us to kick around I am definitely up for talking about that because it's always interesting to see if you can pull some you know quality leads out of the stack. But um yeah yeah it it's um always a question of like well how much is it how much you're really willing to pay for that stuff you know because unfortunately the majority of that stuff is second tier right it's like the customers and prospects that come up and want to talk and have a deal to get done, you don't need scoring for those folks, you know. You just work them through the cycle and you go.

So lead scoring is great for trying to find stuff that's slipped through the cracks, but it's usually stuff that's not as warm. So it's yeah, it's never top priority, but it is, you know, the larger you get, the more kind of money you can find hidden. So I think that it's a great segue back into data quality because you've just touched upon a lot of potential data quality considerations when doing lead scoring. So we can talk about that next week. This week, we're talking about a data quality audit.

And so now everything has run. You see, everything is green. So I'm just gonna go ahead and close that and take a look at what we've got. So what it's been doing in the background is taking my data file. This is something else that you know, if you aren't aware, but you may have already run into this yourself.

Regular, I'm gonna call them regular, um, large language models like an open like a chat GPT or a Gemini or a Claude or Nord of Using, they are not great at reading CSV files or Excel files. They kind of struggle with that. Google Colab, because it's more of a developer focused tool, is the place where you would want it to read a CSV file. So that's why that's what I gave it. And so what it's been doing in the background is John and John and I have been sort of been chit chatting, is taking the prompt that I gave it to evaluate the data file through the six C's paired with the five Ps, and it's gonna give me the results.

It's written the code for me, it's done in the background. I didn't have to do any of that. So take a look at what we have. So the task analyze the data in the file using the six C data quality framework. So it's done that.

So it's gone through step by step, and we saw those green check marks happening. So it loaded the data, it takes a look. This is sort of a snapshot of what it looks like and what the system was seeing. You see all the data. So step one, analyze the cleanliness.

So check for duplicate records, outliers, inconsistent formatting, so on so forth. And so it did that. This is all the stuff that I was not interested in doing myself that I'm happy to have the system do. And so it did all of those things. And so we can keep going.

And so examine unique values. And so it basically checked through based on the prompt that I gave it for the six C's of data quality. And so I won't bore you with all the technical details. Happy to answer those questions over in our free Slack community analytics for marketers, analyze comprehensiveness. And so if we keep going, the thing that I'm most interested in is the end result because what you get at the end is an audit document.

So create a markdown document summarizing the data quality audit based on the six C's. Score each C on a scale of zero to 10. And at the end, you get this result. So I could take this document, paste it into a text file, and start to go through it, which is probably something I'm going to do. I can see at a high level, you know, cleanliness, eight out of 10, completeness, five out of 10, comprehensive, six out of 10, chosen, seven out of 10.

These are way higher than I thought they would be. Credibility, eight out of 10, and calculability, nine out of 10. Overall summary. And so you have this information here. And then you get this really nice, easy to read summary.

All of this work was done for us, John, while we were, you know, kind of shooting the shit about lead scoring. This all happened in the background. So I have my diagnostic for each of the six C's. I have my justification for why it was scored that way. And then what I also have is the more detailed finding and my insights and next steps.

I have my roadmap of if I want to increase the score, let's say it's 40 out of 60, and I don't want to use anything that's below a 55. I have my insights and next steps of what I need to do to fix the data. So when presenting this data to non-technical user, provide clear definitions for each SEO metric. Okay, great. Uh explore options for addressing the missing data in August, the crawled pages.

So it's not a complete data set. That is a huge red flag. So if I was saying I need to make big decisions about my website, and you know, I need to increase my SEO, my awareness, I need to do all the things, I would be using incomplete data. That's that's a big problem. So, you know, again, I've just sort of like run through everything pretty quickly, but any thoughts on that, John?

Yeah, well, it's solid, right? It's a great way to just take the data file, throw it in there, and find out, you know, where the red flags are, and you've got your kind of next steps to dig into and clean up. So yeah, it the big thing with this too is it's definitely a go no go report. You know, you can see if you're getting ones and twos on everything, you're like, okay, we have to totally go back to the drawing board. Like this is gonna kill the whole project.

Whereas if you the scores here are actually pretty high. So I would be interested in like running the reports and see if you know the resulting reports kind of pass the smell test too. But yeah, overall, it looks like it's a pretty solid file. It's not that angry with it at all, that's for sure. No, not at all.

It says the overall summary, the data set provides a high valuable high level look of website SEO performance prevented in a format highly compatible with a spreadsheet and slide software. It is generally clean and credible, likely originating from a reputable SEO tool. The chosen metrics are relevant for basic reporting. However, the data has significant limitations in completeness. So it's saying, yeah, you can use it, however, proceed with caution because these things could be a problem.

So I just wanted to sort of give sort of the rounded, we talked about the data horror stories, things that can happen commonly to all of us, but there are ways to get ahead of it. So the data, the AI ready data quality audit, which you can get more information at trustinsights.ai slash AI data ready. AI ready data quality. I've now said like certain words so many times that they've kind of lost meaning a little bit. Um, it's not gonna solve all the problems, but it is going to help you, especially with that horror story, John that you told, of I gave my data to AI and it just kind of like fell apart.

It just kind of failed. You don't want to do that. I feel like you know, in classic horror form story, we've like laid everything out and now things seem to be good again. I needed to have like somebody drag me off, you know, like some last shock close, so we could get back to the fact that you know the data's always there and can always be a problem, but we can stick with the happy ending for this one. All right.

Uh any final thoughts? Uh, you know, so you're gonna be Clark Kent, Superman for Halloween. What are your kids gonna be? Uh my son is gonna be Nightwing. He's we've gone DC comics this uh year, so it'll be Superman and Nightwing.

My daughter is uh she had a uh a 50s outfit that she had for school, but she's going as a Lorax tomorrow for uh trick or treat. So she has a bunch of friends that are all gonna do it. So there'll be a pack of Loraxes. So um, yeah, we'll see how that one goes. She's always uh running with the pack, which is usually amusing.

So we'll have to get some pictures and see how that goes. I love it. Well, if you want to tell us about your Halloween costume or join tomorrow's question of the day, which is Halloween themed. Uh, you can join our free Slack community at trustinsights.ai slash analytics for marketers. I don't know, John.

I'm exhausted. I did more talking this week than I usually do. Chris needs to come back soon. We need to get Chris back on. Chris who's somewhere over the Atlantic, but still watching.

Safely. Thanks for watching, everyone, and we'll see you next time. Thanks for watching today. Be sure to subscribe to our show wherever you're watching it. For more resources and to learn more, check out the Trust Insights Podcast at TrustInsights.ai slash T I podcast.

At a weekly email newsletter at TrustInsights.ai slash newsletter. Got questions about what you saw in today's episode? Join our free analytics for marketers Slack group at TrustInsights.ai slash analytics for marketers. See you next time.