Mind Readings: Anticipating and Mitigating Criticism Using Generative AI

In today's episode, Phil asks, I get the idea of asking a language model to draw on best practices picked up through its training data and shifting this into what you describe as the short-term memory, it makes sense. I still cannot get my head around this idea of role from the race model you mentioned. Why does Telling Limits an expert virologist in any way change the substance of what it can produce? The model might explain something in more technical language, but it doesn't suddenly have access to any new training data. Its ability to inability to create a credible commentary on virology remains stubbornly unaffected by my flattery, doesn't it?

So this is a really good question. And the answer is no, it actually is different. And here's why. And what the intersections are what the model spits out. So if I say, write a blog post about B2B marketing in 2024.

Fairly, fairly bland prompt. It's going to go into its database of probabilities and it's going to find all the probable intersections of all those words and spit out a blog post. If I say you're an award-winning content marketing writer, we've won multiple content marketing world awards and so on and so forth, do all that, the role stuff, those are more in different words that are going to invoke different probabilities. So let's think about the training data. Let's say you have two pieces of training data that mentioned B2B marketing.

One is your drunk Uncle Fred's shitposts on Reddit about marketing. B2B marketing sucks, right? And there's like a page of this, just drunken rambling. The other is an article on Content Marketing Institute on their website. And in the bio of the article, what does it say?

Christopher Penn is an award-winning content marketing expert in blah blah blah blah blah. If I prompt write a blog post about B2B marketing, drunk uncle Fred and his Reddit shit posts have the same technical weight as the article on the CMI blog. They have the same statistical probability. If I say award-winning CMI writer, suddenly the shit posts, there's a lower probability of invocation of that content from the training data. You'll still get some of the B2B marketing, but because I'm more specific about who the model is, I'm going to pick up content that's more like presumably better content that has those bios, those bylines, that just those descriptions in there in the same way that you would say for virology, you would use terms that you would find in an academic paper because you want to intentionally bias the model towards pulling a certain kind of content.

That content has associations with that specific prompt. Like you're an award winning, whatever. And that's why that role works. The same reason why politeness actually works in prompting. Not because the machine understands politeness, it does not.

The machine has no sentience, no self-awareness. But if you were to go on sites like Reddit or sites like Tumblr or whatever, and you look at the content that gets upvoted, the content that's helpful, what do you see? A somewhat of a propensity for politeness. Like, hey, that's a great question. Thanks for asking, and so on and so forth.

Polite content seems to have a statistical association in the training data for longer and richer content, right? Someone said someone is engaging in real substantial debate. They're probably not calling people names and speaking very brusquely. Not all the time, which is why it's a small lower probability. But that's why the role in the race model from Trust Insights matters, because it is all about making sure that you are you're aligning with what's in the training data, right?

So if you know for sure that the highest quality content in your field has those bylines, you want to use it. If you'd like the PDF that Phil is talking about, go to trustinsights.ai/slash prompt sheet and get the free PDF, no strings attached, no downloads, no forms to fill out, just grab the PDF to see what the race framework looks like. And I want to emphasize a starting framework for prompting, but that's why it works. It works because we're gathering up those associations for who the person is that you want this thing to emulate, and we're using language for that. This requires some subject matter expertise.

Go to the credible publications in your industry and see how those bios and blurbs and things are written because you want to mimic that, right? If I say Nobel Prize winning, okay, well, that's a very specific award. If I say you know, a Peabody Award or a Pulitzer Prize or whatever, those are very specific awards with very specific criteria. What's the award in your industry? That's what you should be using in your prompts.

So, really good question. It's an important question, and that's why it works. If you enjoyed this video, please hit the like button. Subscribe to my channel if you haven't already. And if you want to know when new videos are available, hit the bell button to be notified as soon as new content is live.