Mind Readings: Hallmarks of AI-Generated Content

In today's episode, let's talk about two hallmarks of AI generated content, particularly written content. There's two measures, perplexity and burstiness. And these are both statistical concepts. Perplexity is the diversity of language used, and burstiness relates to changes in length of how much language there is. AI generated content today, out of the box without a lot of prompt engineering, tends to be low perplexity and low burstiness, right?

We all have seen and can pretty easily spot. Like, okay, that was written by ChatGPT. It just has that flavor to it. It's it's kind of like um like McDonald's fries that have not gone cold, but they're not hot either. They're kind of in the middle, like uh they're okay, but I wish that I had something better, right?

That's what AI content is because it's low perplexity and low burstiness. Now, what does this mean? How do how do we explain this? Well, let's let's explain in terms of sushi. There's a lot of food in today's episode.

Um suppose you're making like cucumber rolls or the sushi of your choice. If you're an amateur like me, what's your sushi gonna look like? It's gonna be kind of a mess, right? Again, you're gonna have wildly different size rolls. Some rolls are thin, some rolls are thick.

The the ingredients, the cucumbers are unevenly cut, they're not nicely cut in slices. Um, I remember we're during the pandemic, uh, during the early year early months when you know nothing was open, had to make stuff at home and made some homemade sushi, and it was pretty uneven. It was pretty amateur. Um, and that was a hallmark of something made by uh, you know, a human for sure, and an amateur. Now, suppose you are a sushi chef, you're a sushi pro, you've been making it for 20 years, or you're a sushi machine, like a literal machine.

You own a sushi machine. They they do exist. What's your sushi gonna look like? Every roll is gonna be nearly identical. The perfect amount of tuna or cucumber or whatever, it's good.

The rice is gonna be perfectly cooked, it's gonna be rolled exactly the same, it's gonna be cut exactly the same. When you put it on a plate, it's gonna look nice and orderly and neat. The variance in ingredients and amount and size, all of it will be nearly zero. Every aspect of it will be just perfect, uniform and identical. In sushi, that that's a good thing, right?

You want uniformity. You want uh the same quality fish all the time, you want the same quality of rice all the time, and it's easy to spot, right? You put a plate of you know, Chris's homemade sushi next to a machine made or or professionally made, and it's pretty easy to spot which one is was made by by Chris at home, right? With AI generated content, you can still see that uniformity, but it's less of a good thing, and it's also just as easy to spot. Go on to LinkedIn, see the same, you know, that you put up a post and you see the same LinkedIn comment over and over again from somebody from a group of people, like, hey, very insightful post about this thing, rocket ship emoji, right?

Uh, great explanation, thumbs up, Christopher. That auto-generated spam, it's got very low perplexity. The same general vocabulary is being used by these by these AI-based bot services, and so you can spot it, and it's got low burstiness. The comments are all the same exact length, like two sentences long. It's like, hey, really great insights, looking forward to more, blah, blah, blah, right?

It's the same all the time. And so you can spot that, particularly when you start getting a few of these adding up on the put on the same post. You're like, okay, that's that's a bot. Um, perplexity and burstiness. They have low perplexity, same language, they have low burstiness, same exact content length.

Now, can AI generate content that mirrors human content? Yes. Yes, it can. It can do anything it's trained to do. The reason why it's so easy to spot today is because the people who are using it like this, you know, basically spam bots to build their thought leadership on LinkedIn, um, they either have no training or no motivation to invest more than the bare minimum effort in learning how to use AI.

They're using a they're using a vendor who made the software as cheaply as possible with the same lack of effort and knowledge, they're probably using the lowest cost possible model. So you're gonna get fairly bland common results as opposed to using a custom tuned model. Like if I was going to go and build a comment bot for LinkedIn, I would not use the cheapest model possible. I would not have a short Python script that a developer, maybe even chat GPT wrote. I would invest a lot of time to make something that sounded like me, that had a unique perspective, that had you know the prompt would be like yes, long, because if I'm gonna build something like that, I want it to be built well.

Vendors who are building these tools as shortcuts, essentially, social media shortcuts, don't really have that motivation. If you're gonna use a tool, an AI-based tool, language model-based tool, for things like boosting social media engagement as a comment bot, please get great tools and invest a lot of time tuning them to be as close to you as possible. If a tool promises that it's it's turnkey, requires no effort on your part. Just push the button and go, and you'll be a thought leader, it's a bad tool, right? And I mean, this is probably a little bit rude, but if you're out to generate synthetic engagement with no effort on your part, you're a bad marketer.

So it's the use of AI tools is neither good nor bad. Because it's just a tool. It's how you use it and what you're using it for, and how skillfully you use it that determines whether the outputs are good or bad. But from what I've seen people doing so far, it's pretty bad. So please do take the time to learn how AI works, to learn how models work, to learn how to tune these tools and invest the time and the data to do it.

And you you will get better results, and everyone else will be happier with it. Thanks for tuning in. We'll talk to you on the next one. If you enjoyed this video, please hit the like button. Subscribe to my channel if you haven't already.

And if you want to know when new videos are available, hit the bell button to be notified as soon as new content is live.