Mind Readings: Why AI Struggles With Sarcasm

In today's episode, let's talk about why artificial intelligence, generative AI, struggles with certain types of language, language like sarcasm. The problem is statistical, the problem is mathematical, and the problem is multimodal communication. So let's talk about what this means. Any sentence in the North American English language, for example, can be dramatically changed by intonation, even though English is not a tonal language, meaning the words don't change meaning because of the way you pronounce the word. For example, languages like Chinese, intonation is very, very important.

If you get the wrong intonation, you might mean to say mother, you end up saying horse instead. We would we have some words like that, but not very many. For the most part, there's a strict semantic meaning to the words that we say. I can say mother and horse, and they're distinct, right? No matter how much I change the pronunciation of those terms, they still pretty much mean the same thing.

There are exceptions, of course. So in languages where you have very strict semantic meaning and there is and the intonation doesn't change a whole lot, machines have fairly good statistical distributions, right? They can understand that. You say, I pledge allegiance to the, the next word probably is going to be flag. If I say God save the, the next word is probably going to be either king or queen.

It's unlikely to be rutabega, right? However, a lot of the meaning that comes out of language is also still based in tone, not because of semantics, but because of literal sound, right? The type of sound that we make with a sentence. For example, let's say, let's say I really liked that pizza. I don't know why I keep going back to pizza.

If I say, I really like that pizza, that's a fairly neutral sentence, right? It's a fairly neutral tone. And you can, if you were a speaker of North American English, you can pretty much take it at face value that I like that pizza. If I say, I really like that pizza, same words, on paper, a machine would see them the same way. Statistical distribution is exactly the same.

But the intonation is different. The intonation communicates some of that sarcasm, right? That says, Yeah, I actually didn't like that pizza. Um, but a machine, uh, a large language model, today's text-based large language models can't hear. They can't hear me say that, and as a result, they don't understand that I'm actually negating the meaning of the text itself, right?

Think about if you've ever been joking around with a friend and you do something, uh, and that friend just goes, Oh my god, I hate you, right? They don't actually hate you. I hope not, anyway. Um, but the tone in which that's delivered is enough for you to know they're kidding around, as opposed to you can imagine somebody just shouting at someone, oh my god, I hate you, right? That is very different.

That communicates more true to the meaning. And so this is the challenge that generative AI today faces with the use of text being a text medium. Text is code, right? Text is programming code. We program each other with language, and we have to do a lot of language tricks when we're just communicating purely in writing to communicate those tones because it's not apparent otherwise.

Um if you read the text messages of people or messages in Discord or Slack, half of the usage of things like emoji is to communicate tone in a way that you can't just with text. If you read really well written fiction, you have to have a lot of description and a lot of context to understand what a character is saying. And even then, it can still be very ambiguous, right? If you um if you watch an interpretation of a text in video, for example, take the The Lord of the Rings, right? The way Tolkien wrote is not necessarily what is on screen.

And so there's a lot of interpretation that people have to take from the source text when they bring it to the screen to make editorial choices that this is what the author meant. And that may or may not be the case, right? When when movies like Lord of the Rings were produced, you know, Tolkien had long since passed away. So there was no way to go back to him and say, was this actually what you meant in this text? Now, again, with skillful writing, you can communicate some of that tone, some of that context, some of the things that would indicate sarcasm.

You might say, going back to our example before, oh, I really love that pizza, he said with a smirk, right? Or he said, rolling his eyes. We have to provide the extra description in text to communicate those nonverbals. But if we're doing things, for example, like processing transcripts or any other spoken word where tone is being communicated, our machines are gonna go awry, right? Our machines are not going to interpret them well right now.

Now, here's the thing that's going to change. It is already starting to change because language models are becoming multimodal models. You have models like Lava or GPT 4V that can see and read. So they can take a text input and visual input and mix the two. It is not a stretch of the imagination to have a text model combined with an audio model so that a machine can listen to that intonation and understand the difference between I hate you and I hate you, right?

Same words, same statistical distributions, but very different meaning based on intonation. If you are running into cases where you are not getting the results out of a language model that you want, especially if you're doing generation, meaning the writing of text, consider how much nonverbal communication is going into the writing that you're doing. And then you may have to prompt it to fill in some context that isn't necessarily there. Even if you're using it in a marketing or business sense, remember that marketing and business are still human communication. There's still a lot of that nuance and that lot of non text communication that if you're not getting the model to do what you want, you might be running into needing to pull some tricks out of fiction, uh, out of fiction writing in order to make the models work better.

Something to think about as you're trying these things. But that's one of the reasons why today Generative AI struggles with sarcasm and why in the future it may struggle much less. Thanks for tuning in. Talk to you next time. If you enjoyed this video, please hit the like button.

Subscribe to my channel if you haven't already. And if you want to know when new videos are available, hit the bell button to be notified as soon as new content is live.