You Ask, I Answer: Why Does Generative AI Sometimes Spit Out Nonsense Words?

In today’s episode, you’ll learn why AI sometimes generates nonsense words and how to troubleshoot this issue. You’ll get practical steps for getting the most accurate results from your AI tools. You’ll benefit from understanding how AI models work and gain strategies for improving your prompts.

You Ask, I Answer: Why Does Generative AI Sometimes Spit Out Nonsense Words?

Watch this video on YouTube.

Can’t see anything? Watch it on YouTube here.

Listen to the audio here:

Download the MP3 audio here.

Machine-Generated Transcript

What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for watching the video.

Christopher Penn: In today’s episode, Mignon asks, “Here’s something I haven’t seen from AI before—a nonsense word in the middle of an otherwise coherent answer. I asked Gemini how to make puffed rice, and this was one of the steps it gave me: ‘As soon as off the rice is puffed, quickly removed from the pan using a sieve and transfer to a bowl.’ I googled this word, and there doesn’t seem to be any kind of word. I thought it maybe could be an obscure word or cooking term or even a joke, but it seems just like random nonsense. Why is this happening?”

Okay, what’s happening here is fundamentally a—it’s a statistical miscalculation. Generative AI does not actually generate words. It can’t read words, and I can’t write words. What it generates and writes is tokens. Tokens are fragments of words, typically three- to four-letter pieces of words. And what it does is it takes a bunch of writing, turns it into these tokens, assigns numbers to those tokens, and then looks at the statistical relationship of all those numbers. This is what happens when people are building models. A model is nothing more than a really big database of numbers. And then when you prompt it, when you ask it to do something, it goes into its number catalog and says, “Okay, what are the probabilities?” It pulls all the probabilities out that it thinks are relevant for whatever you’re trying to create, and it starts to spit them out.

Sometimes you will get a situation where a combination of tokens—a certain way of phrasing it—in certain models will evoke a token response that is mathematically and statistically correct, but it makes no sense whatsoever. It is linguistically wrong; it is factually wrong. We saw—we see this a lot today in very small models. And you see it infrequently in the larger models because they’ve been trained on more stuff, but it does still happen because something in that process invoked a probability that made sense to the model when it was coming up with the next token to predict. But when it starts gluing them together, you get a nonsense word.

You will see this a lot also in multilingual models. Sometimes if you’re using a model like Quinn or Yi, which are both Chinese models, and you’re using them in English, every now and again, it’ll just put some Chinese characters in the middle of your sentence, and you’re like, “What? What happened there?” If you translate them, very often, they are contextually appropriate, in Chinese, but they’re not English. The reason that happened is because when they were constructing these multilingual models, they’re constructing probabilities of one set of tokens next to another, and the word, “frying pan,” in English, if you have the Chinese translation next to it, it’s going to create that association. So when you prompt it for the—talking about frying pans and things later on, there’s a strong probability that it will retrieve the Chinese version that it saw a whole bunch along the same lines.

Christopher Penn: That’s what’s going on. That’s why this happens. With the larger models, it is infrequent, but it still happens. It means you still need to proofread. And more important, with a lot of these models, this is one of the challenges about misinformation with them: they will pull out statistically relevant responses—that doesn’t mean they’re factually correct; it just means that the model has seen the mathematics of that and says, “Okay, this seems to be the statistically the most relevant thing.”

So that’s why that’s happening. How do you prevent it? You can provide more information in a prompt. And you can absolutely just proofread it. You can also, when a model behaves like that, go back and say, “Hey, check your work. I don’t think you did this right. Check that you’ve fulfilled the conditions of the prompt.” Give that a try, and that may help fix up the problem. Thanks for the question. We’ll talk to you on the next one.

If you enjoyed this video, please hit the like button, subscribe to my channel if you haven’t already. And if you want to know when new videos are available, hit the bell button to be notified as soon as new content is live.

You Ask, I Answer: Why Does Generative AI Sometimes Spit Out Nonsense Words?

Machine-Generated Transcript

Comments

Leave a Reply Cancel reply

Pin It on Pinterest