Mind Readings: Large Language Model Censorship Reduces Performance

Today let's talk about an uncomfortable reality with large language models. That uncomfortable reality is this. The safer you make them, the less creative and useful they are. Yep. Here's why.

And trigger warning for profanity. There will be profanity in this video. We use language as humans in all sorts of ways. But things like profanity, for example, is part of our language. The more that you have to censor a model, the more that you have to censor a piece of software to say to tell it don't do these things, the more you constrain what it can do.

And in the process, it loses context. Here's what I mean. Remember that these models are nothing more than prediction engines. Even the most sophisticated ones, the biggest models like GPT-4 from OpenAI or Palm 2 from Google with like what, 570 billion parameters, they're just prediction engines. If I say I pledge allegiance to the, the prediction engine is going to come up with a list of anywhere between five and forty different alternatives for the next word.

It's going to score them by probability, and almost certainly the probability is going to be near 100% for the word flag, because I pledge allegiance to the flag is a very common sentence. When these models are trained on data that has been scraped up from around the web, among other sources, it builds statistical relationships from one word to the next. So for example, if I say I'm brewing the, depending on the context, the words that it was trained on, and the words that I've mentioned in my prompt, it'll choose a word like coffee or tea or beer or the fall of capitalism. And in doing so, it's relying on the patterns in language that it learned on input. If you look at a lot of the training libraries, uh explicitly declared or implicitly declared from big companies, for example, in um in an interview with uh Lex Fridman, uh Mark Zuckerberg said that the Facebook llama model was trained on data from Facebook's many services, right?

Facebook, WhatsApp, Instagram, et cetera. What's embedded in most people's language? Yeah. Profanity, racism, bias, you name it, particularly if you're drawing from Facebook. I mean, there's a whole bunch of people there who think that the world is still flat, which is mind-blowing.

Um, and because it's ingesting those patterns in language, if you then have to go back and say, don't say X or Y or Z, you are essentially handicapping the model. You are handicapping the model, and it's not just going to stop using words you don't want it to use, but it's also going to have to adapt and figure out how to use words in less creative ways that don't evoke those topics. So if you say, understandably incorrectly, hey, don't be racist, and there's a good chunk of racist text that was fed into the model, suppressing that not only suppresses racist language, as you would normally want to, but it also impacts all the other words that are used in that context. And it impacts their overall probabilities. If I say, go fuck yourself, right?

And then we say, you know what? Let's not use the word fuck. There is a statistical relationship in that sentence between the word go, the word fuck, the word yourself. And if you see that a lot, and it's a very common phrase, right? GFY, these words are associated with each other.

Now, if I suppress and or try to remove or or or censor that the word fuck, and you're left with go yourself, which makes no sense, right? And it breaks the probabilities around those words. So those words go and yourself are actually going to be negatively impacted by the suppression of the profanity. Do you get it? How these words are related to each other.

And the more you censor words, the more you have to come up with alternatives that may not be as good. Now, clearly, and and let's be very, very clear, it's a good idea if you're going to be using any of these models, particularly in any kind of professional context, to censor things like racism and bigotry and hate speech and substantial profanity. But you also have to understand that it will cause computational problems in these models. How do you get around this? The short answer is if it wasn't in the training data to begin with, it wouldn't be a problem on the output side, but it we don't have control over how these models are trained.

And there are very few companies that can actually build these things that have enough data to do the training, like Google or Facebook or OpenAI. And so we have to essentially handicap the models on their outputs. Now I believe there are probably some technological solutions to do this better that the industry isn't talking about enough yet. I believe there are some interesting things being done with uh adversarial models, which basically say, you know, here's uh what I'm looking for you to not do, um, and sort of getting into arguments semantically metaphorically, um, with the language model to help it to not do those things more. But if you want maximum creativity, you would have to use a model that has also is has problematic concepts and text in it, right?

If you want to maximize what a model can do, you will probably have to accept that you'll use a model that has a higher potential to say things you don't want it to say, right? So you'll want to have to build some gatekeeping in on its outputs to say to inspect it outputs so that the model can be as creative as it wants to be and then can get smacked down later on in the pipeline, say, yeah, let's we're we're not going to generate this, send this back for generation. Um over time, I suspect companies uh and if I had to guess company, my guess would be IBM, um, because they're they're not known for being first to market, but they're typically known for being best to market, uh, particularly on enterprise stuff. I would expect companies like IBM to say, hey, we're gonna build a custom model that doesn't include profanity, that doesn't include racism and bigotry and homophobia. We're gonna exclude those things from the source training data to begin with so that it's not there on the output in the output side.

It can't be there on the output side because it didn't exist on the input side. And that's what we'll have to do to if we want models that are have not had their creativity handicapped, but also have not uh taken problematic texts and concepts with them. So the bottom line is if you want creativity, you also have to accept a model that has problematic text. If you want a model to behave professionally, you're going to have to handicap it significantly, and the outputs may be lower quality as a result. That's the current trade-off as of the time, you know, mid-year 2023 that I'm recording this.

It's entirely possible tomorrow. This could all change, by the way. So it's worth keeping your ear to the ground to see what other things are likely going to happen to help these models become smarter and more professional. Thanks for tuning in. We'll talk to you next time.

If you like this video, go ahead and hit that subscribe button.