So What? How AI Detectors Work

Well, hey everyone, happy Thursday. Welcome to So What, the Marketing Analytics and Insights Live Show. I'm Katie, joined by Chris and John, who were up top with their blue shirts today. Yes. I didn't get the memo.

I am wearing like an olive green color. So, you know, here we are. Someone's gotta be the one's gotta be the odd man out. Might as well be me. Uh this week, we have talked about it on the podcast.

We have talked about it in the newsletter. And today we are going to show how AI detectors work. So this came about because last week, uh, we had a very strong response to our newsletter that it was AI slop, that it was written by AI, that we were liars about our use of AI. And so we can't say for sure that the person who responded used an AI detector, but timing-wise, Substack and Panagram have released, so Substack is using Panagram, and then Panagram is an AI detector. I'm putting this in big, big, big air quotes.

I gotta get my red flag ready too. Sorry, I forgot about that. Uh they are using tools that allegedly can detect AI written content. So he but here's the challenge. Uh these large language models are trained on human written content.

And now it's telling us that the content that we've written that we didn't consent for them to be trained on is AI written content, which is a whole maybe other ethical topic for another day. But today we are going to walk through how to use an AI detector, maybe how to build an AI detector. But before we get into all that technical stuff, John, what what are your thoughts on AI written content? Like, do you care? Does it matter?

Are you looking? Are you out to like see is this written by AI or not? Like, where do you stand on this topic? Yeah, that's a that's a good question, just in general. And yeah, I don't really care about AI topics, you know, and AI writing.

That's really the first thing because it's most of the time people are using AI to generate slop, like they want to write some kind of article, and it's just gonna get grab whatever's out there on the web and process it. So, you know, if somebody just does something with AI and throws it out there as, hey, here's my new thing, that's of no interest usually ever. Um, where it's of huge interest is if you're working on something and you you know want it to go out there and grab everything that's out there and put it in a presentable form to you so you can learn what's going on and figure things out. That's you know, fantastically useful. It's basically the promise of search that we got stolen from us and you know got bastardized by marketing and destroyed.

So there is great things that can be done with it. Um, but and then there is this weird thing of all the AI generated stuff is here, but now unique and uh novel research or stuff that's original or created is at a whole nother level. And in fact, just this week we've been seeing stuff as far as you know, saying that hey, this is made without AI is actually some marketing credibility and uh you know shows either luxury or a brand above the general public. So yeah, there's a lot going on, but and I don't know, we've seen uh a lot of stuff with AI detectors. I last I heard this was all garbage, so I'm interested to hear where we're going today and you know what the state of the world is today.

Well, and I do find it interesting that you know, and it's you know, marketers to the rescue to you know rebrand things. So, you know, 100% human written content, no AI used, is now like you know, a something sought after when for how many years that was just how it was, like automated. And now it's like now you can just slap a label on it and be like 100% human. It's like wow, I mean, okay. It's I don't know, the whole the whole thing is very bizarre to me.

So uh Chris, we're gonna turn it over to you and figure out where the heck are we gonna get started? Like, what's the what's the deal with these AI detectors? So with AI detectors, the vast majority of them today use an AI model that is custom tuned to look for specific patterns, and the specific pattern they look for the most is the probability of the next word in the sequence. And if too many words in a row are common, are expected, then it's going to say that this is probably AI. So I'll give you an example.

If I say to a North American audience, I pledge allegiance to the what's the next logical word? The flag. Flag, right, exactly. Flag. Almost a hundred percent probability that that word is gonna be flag.

Almost zero percent is gonna be rutabaga, right? Like no one pledges allegiance to a rutabaga. If you do, leave a note in the comments. They might. It might happen today.

So if AI, an AI model is looking at a word sequence and it it looks at the next word and it sees the word rubega, it's like, well, that's weird. And that's a very low, that's not what I would expect there. That's probably human written because it's so low probability. But here's but here's the challenge there. There's a lot of very uncreative writers, especially when you get into things like you know, tech writers, uh, the probability of the next word being whatever it is is going to be very high because there's only so many ways to talk about a certain thing.

Like the sun is bright, the sun is yellow, the sun is shining. Like there's a very basic example, but to come up with some sort of like very flowery analogy of what the sun looks like. That person is likely not using AI in the first place. But for the majority of people who are like, I need to describe the sun. Let me say the things that I know about it.

It's gonna be a very uncreative, very like that basic pattern matching, but they wrote it. But now it's gonna get flagged as AI. Like I I think about like the students in school where they're just trying to get through the assignment, so they're just like, um, what do I have to say about the sun? The sun is in the sky, the sun is blindingly bright. Uh, I need sunglasses when it's sunny.

Like, we're all gonna say that. And I think that that like again, very basic example, but like that's that to me is like hugely problematic because again, it's trained on the way that we talk and write. So therefore, how could it be like, oh, well, if that's the pattern, then it must be AI written, even though it's trained that this was how humans write and speak. Right. And the so everybody starts with a base model of some kind, and then every company that claims to offer an AI detector has to custom tune the model or the software around it to say this is what we think is and is not AI written.

And we know this because AI models in particular have what we call tells, right? They are some constructions that they overuse. Example, negative parallelism is a very common tell for both ChatGPT and Claude. It's not this, it's this. It's not the death attribution.

It's the arisal of multi-touch attribution, right? You've seen that construction a lot. Very human construction, used a lot. That's one of the reasons. But what AI tends to do is tends to overuse it.

Another one is triadic rhythm. Triadic rhythm is things that come in triplets. Um maybe it went well, maybe it stalled, maybe it never started, right? Boom, boom, boom. That kind of triplet sequence.

Claude in particular loves triadic rhythm. Don't know why, except you know, it's obviously been trained on that material, but you see that, and it does that a lot. Um, Claude and ChatGPT both like um passive voice starters. So they will say something like the and you'll see this in LinkedIn comments, especially. The issue wasn't that like it was like, and then it goes on to say like a bunch of other foaming of the mouth, but it will start with a passive voice starter that has doesn't really say anything, it's just kind of a restatement.

One of the best ways to look for AI generated text, believe it or not, especially in places like LinkedIn, is in the comments at a profile level. Because what you see in the in link in LinkedIn, especially is you don't just see individual pieces, you see patterns. And the patterns to use a claudism, the pattern is the tell. So let me show a here's an example. Without looking too hard at this, do you see the just the general text shape of the sentences is the same shorter first line, longer second line, shorter third line.

And I'll make it bigger, but I want you to look at the, I want you to look at the shape of the words as like just colored lines. You can see just how formulaic it is. The AI audit only works if it knows whose eyes are reading the message. This amendment moved the scope boundary, not just the compliance case. The most telling choice of the playbook is this, where agents sit, right?

It's that very when you look at the pattern, you can tell this is a bot because it is generating the same format over and over and over again. The under yeah, the underlying language model is varying up the words. So early AI detectors use static features like how long is the sentence, how long, you know, how many nouns are in the sentence and stuff. Almost no company uses that anymore because the the false, the the false accusation rates were very high on those. Today's AI detectors use language models.

The false accusation rate is lower, but still not zero, and zero is the only acceptable uh uh rate with using for anything punitive. It's one of the things we've been saying now for three years. Like you must never ever ever use an AI detector for anything that is punitive because they still have false accusation rates. And if you're talking about ending someone's tenure where they're kicking them out of school or disciplining them, the tools cannot give it cannot falsely accuse someone. Well, and it's funny because you know, you're talking about like the shape of the content that is whereas I looked at that and I was like, oh, it's written by AI because it's utter nonsense.

Like it doesn't, it's not actually a coherent sentence. It looks like a structured sentence, but when you actually read the words, like the audit only applies to the eyes reading it. Like that's not actually a sentence that a human would write because it doesn't make any sense, it doesn't mean anything. And someone who does probably doesn't make sense in real life. Now, to your point earlier, AI detectors especially penalize non-English speakers.

Um, you know, it penalizes people who are neurodivergent who write in ways that are not normal, aka boring. Um it penalizes um uh highly templated text. So anyone who is ever who remembers school, um, especially in the last 20 years, remembers the joy of the five paragraph essay. Oh, yeah. Uh-huh.

The five paragraph essay with all you know, you say what the paper's about, you have three papers of exposition, you have the the third paragraph of uh conclusion, which restates the the key point, you know, transition sentences at the beginning and then of every paragraph. At that point, you may as well just have chat GPT writer for you, and a lot of students are, but that formulaic template is going to get flagged as AI because it is so strict that it also doesn't teach you how to write. Oh man, I remember yeah, like yellow paper, green lines on your pencil. It's like paragraph one is the argument. Paragraphs two, three, and four support it.

Paragraph five is your conclusion, which has to tie back. It's almost like the 5P framework. Uh-huh. Um, and so there are some definite tells for that AI uses that even humans are very, very skilled at f at uh identifying, especially Katie, as you point out, just it it's words, it's a it's a it's a sentence-shaped object, but boy, does it not say anything. If you bring that LinkedIn example back up again, um let's let me zoom in here, fit the screen.

There you go. The audit only works if it knows whose eyes are reading the message. That doesn't mean anything. The amendment moved the scope boundary, not just the compliance dates. That one, if you squint, almost makes sense.

But like there's no reason for someone to frame these sentences that way. So that to me immediately sticks out as well. AI is doing its best to sound like a competent human, but it is failing. Yes. And what these bots in particular do is they take the original post and then they essentially wash it through a prompt to generate a templated response.

And the response, uh the bots themselves, if you wanted to go buy the bot these bots or just make one yourself, they're not terribly difficult to make. And so this go kind of goes to why you would use it in AI detector. People use them. Um there's there are some valued use cases. Uh, we talked about this on the podcast.

But one is is a person attempting to deceive me with the use of AI, particularly with things like image generation, video generation, et cetera, to deep fake something to show that you are alive and well when in fact you're dead. For things in academia, which is very dangerous, things like academic honesty, um, academic integrity, like, did you actually do this work? And that's a whole other topic. And then of course, there's the social aspect, uh, which is um does the use or not use of AI confer some form of social status on the person? Hmm.

Um, there are a lot of people like the person who wrote in last week, who maybe need to feel better about themselves by yelling at other people who they think are using AI. Exactly. Um, one of the things about the different patterns is that you can give a language model a list of these patterns and say these are anti patterns to not use in your work and give it a scoring rubric and say you're gonna generate a piece of text, then go through it for these obvious patterns and knock them out. Um and and they can do that. In fact, uh separately in the in the trust insights Google Drive, we actually have one of these um that Katie, I didn've been mean to tell you exists.

I made it last week. Um, I appreciate that. I always love learning something new on these shows. At least it's not being demonstrated live, right? This is the we're well, let's not get ahead of ourselves, Sean.

Exactly. So how do these things work? The first thing you need to do is you need to have a language model that will return a specific metric. Uh, and the metric is called log of probabilities. And this is that guess of in the next word in this sequence, what is what are the most probable words, right?

So God save the is going to be king or queen, probably not Rutabega. Um, and it is now. It is now. So you have the base language model. Every company that builds one of these things has to figure out what model they're going to use that is a balance of intelligent and fast.

And we talked about, in fact, we have a paper up on Zenodo from December talking about like how Lama 3.2, which is a very fast model made by Meta, is also absurdly biased. Um, I actually ran it recently through a um uh uh test called BBQ, uh, which is bias built into queries, it's an AI bias test, and meta's models fail 50% of the time. They're given a multiple choice test, like you know, Korean, you know, you you're talking to a Korean person, uh, and then it gives like three answers that are clearly racist and one answer that's not. And and it measures the test measures does it pick a racist answer, and meta's models fail half the time. Like, oh, you must smell like garlic when you before you shower.

Like, no. We will put a link to that paper in our free Slack community. Uh, if you don't already have it, that's trust insights.ai slash analytics for marketers. Free to join. Uh, we will put a link to that research paper in that Slack community.

Yep. So for you need a model to start with, you need a server that can serve it up very, very quickly because these things are very compute intensive. Um, so again, it's it's picking the the best blend of speed and power. And then, and this is the part that every AI company does differently. You need a training corpus because you have to demonstrate to these tools and build benchmarks internally of what is and is not AI.

You can't, the tools themselves can't know that. You have to clearly demonstrate to them and help them build it internal scoring mechanisms using the base model to say this is AI, this is not AI. So I was doing this for a different project, and uh what I did for this was there are some folks that we know uh and uh know and love on LinkedIn who very adamantly do not use AI. For example, our friend Ann Handley um does not use AI. So I grabbed some of her LinkedIn posts.

Our friend Jay Aconzo is rapidly anti-AI, grabbed some of his posts. Uh, I know my own posts that I don't use AI in any fashion for them, uh, and I put them in here. And then I took those topics that we all are all writing about, and I then fed them into Claude, ChatGPT, Meta, Perplexity, Quen, Gemini, and Copilot, and had them take the general topic and write a post in a write a synthetic LinkedIn post for each of the topics that these folks are writing about that pairs kind of one to one. So Chris wrote this thing about authenticity, and then here's the chat GPT version, here's the Gemini version, and so on and so forth. So that when we start the model training, we can say, this is what human looks like, and this is what machine looks like.

Okay. Once you get the model training underway, uh, this will take you a couple of days to process. Um, and your computer's fans will be spinning it 100% for a while. Now, a lot of these companies, of course, are renting compute from AWS or Google or what have you. I did not feel like spending Trust Insights money on on that level of compute, so I just did it on my MacBook.

We appreciate it. Yeah, we do. We appreciate that. CFO thanks you. Exactly.

And then the the last part is you test it. Now, what's interesting is I also have my friend uh Becca's posts in here, and hers kept getting flagged as AI, even though she she is a human wrote them. I was like, what's going on here? And I I talked to her about it, and she's like, Oh, what I do is I dictate all my posts, and then Claude fixes it up into a post. And I said, Ah, that's the problem.

Because when an AI tool edits, yeah, it rearranges the world the words back into a statistically probable pattern. The example I give is uh Yoda said, for 800 years have I trained Jedi. Uh my own counsel will I keep on who is to be trained. AI would rewrite that as I have trained Jedi for 800 years. I will keep my own counsel on who is to be trained.

Same exact words, but in a much more probable word order, the second order version is going to get flagged because it's probable. The first version is truly Yoda. So you now have a base model, and by the way, this is why there's so much confusion in the space because every one of these companies, Pangram, Originality, GPT Zero, uh, Winston, all these companies each have their own base model that they don't disclose. They don't, so we don't know what the underlying model is, and then they have the train their own training libraries on top of it, which again we don't know whether they are. For my tests, I used a model called Skyfall.

Skyfall is a model made by Beaver AI that is an upsample of Mistral, which is a French model. Skyfall is specifically tuned for writing. Is it a writing model is meant to write mostly to write fiction, um, but it's equally good at nonfiction, but it is specially tuned for writing. So I started with a model that was made for writing, and then I built the classifier on top of that. And so what we end up with is something that runs on a laptop, and now let's go and get and let's go ahead and get uh let's get something from this last week's from this week's newsletter.

So I'll put Katie's uh part of Katie's uh thing in here. Now, what this will do is it will go into the Skyfall model, it'll feed every single word to it, and then it will color code the words by green being wow, that was an unexpected word, or red, yeah. I expected that word. You can see the overall document uh scores. If this was fiction, would score a zero, meaning definitely not AI.

It scores probably not AI on non-fiction because the training corpus is change. So if a company is selling, say specifically to academia, they might have trained their model just on papers and school papers and stuff. A company is selling to you know, LinkedIn uh to social media managers, they might have trained on social media content. And then for each word, we can see the probabilities. So here's Katie's talk about the the five Ps people who eight.

People who 86% uh percent predictable, and then it shows you what the other possibilities were there, that likely to be there. And what you would be looking for are longer runs of clearly generated AI content, whereas like, okay, wow, this thing is a hundred percent synthetic. So if I go back to my input folder, let me get a that post on empathy that Quen wrote. Start a new visualization, paste this in this one for the nonfiction, which it is definitely AI. And you can see there are lots and lots of red and red runs throughout it.

Uh, this thing is there's very little surprisal in this thing. Uh, even though it was told right like a real person, it has a hard time doing that, and you will see large clusters, and what you're looking for are clusters not just on a line, but in a group, like there's a blob here that is just all all bright red, and so for this particular detector, it says, yeah, this is this is definitely um AI. It's interesting. Um, I appreciate that it thinks that my writing is probably not AI. I would actually, and I'm gonna I'll share this with you in Slack, but I'm gonna share with you uh a snippet of the unedited draft of that particular newsletter.

Um, because again, like it makes sense that it's gonna flag when I start writing about the five Ps, guess what? I write it that always comes out the same way every single time. So let me just share. I'm gonna post this for you. Sorry for those who are like, wait, I want to see it too.

No, it's terrible. All right, it's in our uh podcasting channel, Chris. I guess because I'm curious to see the difference because I did disclose in that newsletter that I used AI support to help me clean up my writing. But my first draft of my writing, like a lot of people, is generally pretty rough. And you can see here, this one came out definitely not AI.

The perplexity level, the surprise level is much, much lower. It's half of what it was than the other one. Um, because again, you're you can see there's a lot more green, and there are none of those big red blobs just floating around. Um, the green is interspersed heavily throughout. And so when it comes to if you want to avoid being flagged for AI, the rough cut is actually better from that perspective because an AI detection is gonna go, wow.

This is clearly not. Even if, and this is the important part about these things, even if it makes absolutely no sense and is hot garbage. I'm not saying your stuff is kidding, but I'm saying in general, if you were just write hot garbage and put it in, it will say definitely not AI, and everyone else is like, this sucks. Well, and so and there's the conundrum that people find themselves in I could publish my raw writing unedited just to save face to be like, there's no way AI wrote this. And people are like, Yeah, there's no way that anyone can read this either.

So it it is no longer useful. And so I feel like, and that's the conundrum people find themselves in. Is like they want to publish polished, engaging, educational, entertaining, the three E's. So that kind of content. And this is what I talk about in the newsletter is like some people use a human editor to get them there.

So, you know, for a long time, John, you were my human editor and the one who would go through my content to make sure it was coherent. And then I was able to transition to um, you know, using these AI-assisted tools. The process is the same. Somebody still needs to go through my writing and make sure it's coherent. Now, you the difference too, and I understand what you're saying, Chris.

John wouldn't go through my writing and start rearranging sentences for me. He would make sure that things were grammatically correct, that the sentences themselves were coherent, but he didn't take that next step and rearrange things. And that is a big difference with these AI editing tools as they're trying to be helpful. And that I would imagine, once you're aware that that's happening, is something that you can set the system instructions to be like, don't do that. Like, don't rearrange things.

Like, here's as far as you're allowed to take editing. When I was using, when we were still using Gemini more heavily, and I was using it to edit posts, I would say, do not change anything. Just give me a, you know, a list of what you think the changes should be, and I will make those decisions. And I feel like a lot of people have moved away from doing even just that layer of work to like, yeah, you know what? You just do it for me.

It's so much easier. Yep. One of the easiest things you can do to preserve your voices, if you're going to have AI rearrange something, say you must operate only at the sentence level. You may rearrange the order of my sentences, but you may not reconstruct or edit the sentences themselves. And you'll because what you're doing is you're preserving the word order in sentences.

For 800 years, have I trained Jedi, right? Will remain an intact sentence. Even if it gets moved around, that construction gets moved. This is another example. This is a fiction example from uh where is this from?

Oh, this is from a uh an e-book that I have keep messing around for Katie that'll eventually give to Katie. She's seen the first chapter I can suck. Um, but you can see here this one scored a uh a perfect score of this is 100% AI and uh and non-fiction, it also scored very, very highly. And you can see this is just a sea of red. There are very, very few green words through here throughout here, and this is definitely 100% machine generated.

Now, here is the problem with these detectors. Whose data were you training? Would was the detector trained on? How is that data selected? How is it generated?

Um, and was the data representative? So if I were to say, take a the writing samples from a Korean American student prep school for for new immigrants, um, and all the students were writing with completely unassisted with AI. The English on that's not going to be great. Is data like that in the model of this is human writing? It's it's problematic writing because these are non-English speakers who are you know, like second-year English students, so they can't really create create good English yet, but it's 100% human.

Is that reflected in the model? And the chances are the answer is no. What typically models are trained on is they're trained on the easiest information available, and that tends to skew wealthy, that tends to skew the mud whatever the ethnic majority is in the in the corpus of the data set. Um, it tends to skew uh more educated, and so people outside those communities, they get flagged more. So here's a question, and this is something you know that we talk about on and off.

So, you know, the the 5P book. And so you've assembled a version of the 5P book using AI. And my pushback was, well, I don't want to publish a book that was written with AI. And you rightly push back and said, no, it just assembled all of the different ways that you've you, the human have written about it. So I guess, and that's where, you know, it's like, well, where is that line?

So like I I've written enough about the 5P framework, so much so that I could probably have two books by now. But I'm hesitant to use AI to assemble all the pieces. In my brain, I'm like, no, I have to start from scratch. I have to sit down and slowly type out everything I know about the five Ps when I've already done the work, I've already written it. And so if someone were to then put the 5P book through an AI detector, it's likely gonna get flagged.

But at the end of the day, like I wrote it, I created it. And so, like, I guess that's what I'm struggling with. With why do you even care? Was it did it help you? And so maybe, and I think this is what we were talking about on the podcast.

If you want to catch that episode, you can go to trust inset trustinsights.ai slash TI podcast. But do people care about the right thing when it comes to AI detection? Well, and here's the thing AI detectors detect a a mathematical assessment, it's a statistical assessment. They do not tech for quality, they do not detect for something being educational or entertaining or helpful. They only detect the likelihood of word order within a corpus of text.

Now, if you are a person, and there are people, and there's there's valid reasons for this. We talked about this on the podcast. There are if there are valid, you have valid reasons for as despising AI, right? The companies running them are ethically swamps. Um they the machinery itself is incredibly resource intensive uh for electricity and power and and water.

Um the data they were trained on was not ethically sourced, right? Um there's a whole thing in the news right now about an AI company that just shreds books as it scans them, um which you don't have to do. Um if you object to the use of AI on those philosophical and moral grounds, then yes, it is going to matter to you whether AI touch this book or not, because the mechanisms, the machinery that is used, you object to the use of that machine at all, right? It's like saying I object to the use of a gasoline-powered thing in the harvesting of my produce. I want my produce to be only human harvested, even though that will give you diarrhea lettuce.

Um as a result coming. But if you have those moral uh objections, then yes, it very much matters. If on the other hand you're like, is this book going to help me save money, save time, make money? Then it will matter less as long as the underlying source material is good. Um, AI can still make a mess of things, it can still hallucinate, especially if you start exceeding the context window.

But that you a percentage of the audience will say, like, if this use this unethical what they perceive as immoral or unethical machinery, I will not touch it. And that totally makes sense. Um I guess it's hard too, because like, you know, I developed the 5P framework, I created the 5P framework, and there's so much content written about it now that in some ways it's almost silly not to use AI to summarize and you know pull together everything I've written to make it make sense, like in one cohesive story. But then that then becomes the output of like, okay, but it was generated by AI. So Katie didn't do this at all.

And I guess it that's I guess that's my question is like, you know, if I go back, maybe I have answered my own question. If I go back then with the AI version and then, you know, rewrite sections, make sure it's more human. I guess it doesn't really matter, but it might still get flagged. And to where you were talking about where you can't use these AI detection tools to discredit someone credibly, you know, people are gonna try to do that anyway. Right.

Um, there are two ways I would handle that. Number one is if you assemble the whole corpus of everything you've ever written about the five Ps, and that gets put into a database and is broken up at the paragraph level. One of the things that you would do is you would have you work with AI to develop the skeleton of the book, right? Logically it's probably gonna be the five Ps. Um, and then say this par in the database, this paragraph is a purpose paragraph.

This is a people paragraph, this is a process paragraph or whatever. And from that, you could assemble them and tag these are all the paragraphs about this, and then have AI say just select the sentences from each of these paragraphs that make the most logical book out of the 82,000 paragraphs you've written about purpose, right? You will be able to cobble together something in your exact word order at the sentence level that is reflective of your writing, particularly any writing before 2023, uh, which is when you know chat GPT stuff started really becoming prominent. Because you've been working on the the 5P since 2020. Um that would be one way of doing that.

But the other way of doing that is, and this is where you start getting into very, very tricky territory. Um there are 40 some odd different text metrics that you can use to take a known sample of your exact writing style when you when it's known good, and you give that to AI to an AI tool as a benchmark, and you say you must meet these scores, rewrite until you do. And what happens when you do that is you basically invalidate the AI detector, can no longer detect that it's AI because you've got a mechanism in place that mathematically defeats surprisal measures. Hmm. All to say that AI detectors are a big red flag.

AI detectors are a big red flag. Well, AI detectors are problematic to some degree, it but how people use them, their purpose in the five Ps is the real red flag. That makes sense. What do you think, John? Are you gonna start using AI detectors to detect everyone and call them out?

Right, yeah. That's the, you know, unless you're going on some kind of crusade and you have a reason to kill it. No, it's because you know it all comes down to is the the writing or the copy or whatever useful to you. And, and you know, yeah, a lot of the time. I mean, I guess it would be a good thing for a screening algorithm.

You know, you could run it through and say, hey, anything that comes up mostly AI automated, don't bother showing me that. Show me the original stuff first. Um, but yeah, the thing that is clear to me is that there's a spectrum. I mean, it comes to the point where you absolutely cannot ever say definitively something is or is not, because you just have a score. And you know, it can be very probable, you know, 99% probable, but even in those situations, you can't definitively say we you know know for sure that this is or isn't.

Yeah. I uh again, the there is definitely in the same way that you have things like organic or you know, artisanal, small, small batchcrafted, whatever, there is going to be a market for people who are like, yeah, I want something that has 0% AI, never touched it, was never part of it. There will be a market for that. And if that is your market or that is your customer, double down on it, right? Yeah, it's gonna take you longer to create stuff than it's someone who's using AI, but you could double down on it.

Our big thing as a company, uh, as Trust Insights is it follows our our raft uh responsible AI framework. And the big thing there is transparency. It would just say, if you use AI, disclose it, right? Um disclose how you used it. And one of the things that, for example, I do with my books now is in the deluxe edition, you can in the deluxe edition, you get the rough audio recordings of me yelling at people at that as I'm driving down the highway and stuff.

So that you can say, Yes, okay, I clearly see chapter two came from this recording, laced with profanity. Chris, you're like, get the hell out of the way, you idiot. You're in the left lane going 50 miles an hour, right? Uh as I record the book along on my drives. Um you can get that provenance and lineage at I consider it an upsell.

Like here, you can have this, and here's the edited version that actually makes sense that absolutely 100% used AI. Where I and a lot of other people have a problem is if you say something is yours and it isn't, right? Uh, because that's the heart of academic integrity is you say you did this work and you didn't. And Katie, you've had this problem uh even with you in managing people. When when someone hands you a report and you ask them what's in it, they don't know.

Yeah, I mean, but that's not a new problem. Like AI didn't create that problem. A lot of people in general are just sort of on autopilot. So they'll follow the process, they'll put the numbers together, they create the dashboard, they create the deck, and you say, Okay, great, tell me about it. Well, I don't know what's in it.

I just pulled the numbers and put it in. Like, that's not an AI problem. AI is exacerbating the problem because it's taking out the step where the human actually has to put pull the numbers and put the deck together. So basically, you're paying someone to push a button and still not know what's in it. And I think that, you know, again, that's not an AI problem.

AI is exacerbating the problem. The problem is the education around the information, the data. So, you know, uh, when I, you know, when we worked with the teams, and I would say, well, you know, based on the data that you're seeing, like which channel should we focus on? And they would go, What's a channel? And it's like, well, the literally it's in the report that you just gave me that we're about to send to the client, and we need to tell the client something that they should do.

And like, I know the answer, but I'm trying to help you know the answer as well. They're like, Oh, um, well, this one's the biggest. I'm like, okay, well, that one's the biggest, but is that one the most expensive? Is that one the most resource? Like, and really trying to help people think through critically.

And I think that that's not a problem that AI created. That is just truly a lack of curiosity and a lack of education about what it is that you're doing. So you can follow a process. That's great. I want people who can critically think about the process that they're doing.

Right. And so when it comes to the use of AI detectors, that becomes the detector then becomes a part of the problem rather than part of the solution. Part of the solution is you're not encouraging people to think. You're not asking them to think critically, which is a basically metacognition. Think about your thinking.

You're instead saying, I want to see 10x productivity out of you. And they're like, I'm gonna use AI for everything because you want 10x more stuff coming out of my desk. You never should have given this to me, Chris. Absolutely. I should give you, I did give you more than one.

You gave me a whole pack of them, yes. I'm gonna have just like we like using them everywhere. Um and that's the thing, is like Wolverine gloves with like, you know, like eight of them. AI didn't create the problem of people not using their brains. AI amplified the problem of people not using their brains.

Exactly. Yeah, there's an interesting thing too in that, you know, for large publicly traded companies, the idea of a dashboard or report is assembling information that people can see. But in mid-sized or smaller organizations, a lot of times a report or a dashboard, that's not the goal. The goal is gathering that data and figure out what is really happening in the outside world. You know, you can deliver this dashboard, but the executives, like you're gonna come in and say, okay, what does this really mean though?

And that's the important part of it. It's not just cranking out a report, it's using that data. And it's just it's interesting because it's a very different approach on how to use AI. I mean, you could still use AI to figure out what's going on in the real world, but that's not the same as cranking out another dashboard. You know, and this is the thing that we've been saying all along, but people really aren't heeding, you know.

So to the question, will AI take my job? It's AI is gonna take those tasks. So AI can create the report, but you have to like real I mean the way we position is like you have more time to do the critical thinking. You have more time to do the human things. Now you're not bogged down 30 of your 40 hours a week creating reports.

Now you have 40 hours to think about what does the report say and what do we want to do about it, but that's not happening. Yep. And I don't have a good solution for that, but that's what we're seeing. And so, you know, people who are using these AI detectors, they're like, oh, well, this is AI, so I'm gonna discredit this person, or this is AI, so it's garbage. They're not digging in deeper to figure out like, is it really, or why am I even trying to detect whether AI was used?

Yep. And if you want to mitigate the strongest tendencies of AI, give it a list in jargon in linguistic jargon of the things you don't want it to do. Some examples, as we said, negative parallelism, anaphor abuse, triadic rhythm or tricolonic rhythm. Um, sentences that don't contribute meaning. Like you can actually tell a machine cut any sentence that does not meaningfully advance the narrative.

And I did this the other day, and like 30% of the text just vanished. Like, well, well, all right. Clearly, you you were uh things. Um, uniform staccato is a popular one. Uh rhetorical QA, because AI love to say, you know, the result devastating.

Nobody asked you. Nobody asked you about that. So make a list of the things that you in particular don't like, or that you say, you know what, AI, you're overusing this. Ask the AI what specific linguistic feature is this, such as negative parallelism, and then say make yourself a nice little rules thing that can go into a skill or plug-in or uh GPT or gem and say, don't do these things and check your work. Score the number of times you use each of these things in the document and then remove them.

And you will take care of the worst things that you don't like about AI. And if nothing bothers you about AI writing, carry on. Makes sense. Any final thoughts? Yeah, today's human statement is take heed.

That's human, that's not AI. I used a phrase in the LinkedIn post earlier. I'm like, I'm surprised nobody even commented on it. No AI would ever say that, but we'll we'll save that for another time. Thanks for tuning in, folks, and we'll talk to you all on the next one.

Thanks for watching today. Be sure to subscribe to our show wherever you're watching it. For more resources and to learn more, check out the Trust Insights Podcast at TrustInsights.ai slash TI Podcast and our weekly email newsletter at TrustInsights.ai slash newsletter. Got questions about what you saw in today's episode? Join our free Analytics for Marketers Slack group at TrustInsights.ai slash analytics for marketers.

See you next time.