You Ask, I Answer: Experimenting with Generative AI?

In today's episode, Amy asks, what experiments are you running with AI and how would you like to see it used? That's a really good, very interesting question. I'm experimenting with pretty much everything in AI all the time. That's just what I do. So it's really hard to narrow that down as to one thing.

But one thing I I think is worth suggesting, uh, and this is discussed in our new AI course, Generative AI for marketers, which you can go and get at TrustInsights.ai/slash AI courses, is using local models. Uh people call them open source models, they're not open source because a models training data would have to be given away for it to be to be truly open source, because source code from what you make software would in an open source model would be the training data. Uh, what these models are, and examples are the Mistral model from Mistral, the Lama model from Meta, and its whole family. They're open weights models where you're given the models weights, essentially the raw model itself, and then people can retrain it, tune it, uh, make it do other things, but the the model itself is pre-baked. I would like to see more people experimenting with those tools, with those kinds of models.

Open weights models, because well, there's a variety of reasons. One, open weights models, like the ones you can run in a system like LM Studio, run locally. They run on your laptop, which means that if you are working with sensitive data, you're working with protected information, healthcare information, financial information, stuff that's you really, really don't want to hand to another third party. When you're using an open weights model locally on your computer, you that data never leaves, never even goes off your computer. You you can do stuff with it, and no one ever sees that data.

Not the model maker, not the software maker. You just you you can unplug the your cables, turn off your Wi-Fi, and it all runs because it all runs locally. So that's really important. It's something people should be experimenting with. Second reason for open using the local models and open weights models is cost.

Even the cheapest APIs, still you can run up a decent bill, especially if you're doing stuff like software development. One thing that I do a lot of is I write software around language models, and I am not a particularly good software developer, right? And so I make a lot of mistakes. I will send data to uh to an API and I will screw up. And if that API is charging me per use and I'm screwing up a lot, I rack up a pretty hefty build.

If I run a model locally on my laptop, it costs electricity, right? It costs electricity to run that. But that's about it. I'm running on a MacBook, so even the cost of electricity isn't very much. And so it gives me the freedom to experiment more, to be willing to take more risks to test and QA sooner without having to worry about the impact on my company's wallet because it's all running locally.

And then once I'm satisfied that the rest of my code works as intended, I can then go and say, okay, now I'm going to repoint my code from the development model, which may be you know a Lama 13 billion parameter model to open AI or whoever or Claude or somebody commercially, and I don't have to debug on my dime, right? Because I've already done that with the open model. That's a second consideration. Third reason to be looking at these models, and this one is is kind of interesting, is censorship. Censorship of models, public ones, is getting heavier and heavier in terms of what the model is and is not allowed to say.

And I've noticed anecdotally, and anecdotes are not data, but I've noticed that queries I could ask three months ago, I now get a I'm sorry, I can't help you with that. And that's not very helpful. And I'm not asking like crazy stuff. I'm asking, but I am asking either very tactical things or working with certain types of data that models now seem to protect against. Particularly things using copyrighted terms.

Anything from Disney, for example. Um, a lot of models will spit up on now. And so having local models that have no censorship, they are aligned to do what they're told with no morals or ethics or rules, is super helpful. Here's why censorship is a bad thing in language models themselves. Now it's not to say the censorship itself is bad, but censorship of models is bad because think of a model like a cookbook, right?

In the your cookbook is a whole bunch of recipes. Now, let's say there's an ingredient you don't want, like you're gluten intolerant. Say, okay, anything with wheat, it's got to come out. And you start ripping out pages of from your cookbook. Yeah, you're gonna rip out the pasta page, right?

That's pretty obvious. You're gonna rip out the bread page, that's pretty obvious. But you rip out beef wellington, right? Even though the majority of that dish is not the pastry, it's it's the big hunk and chunk of beef in the middle. You rip out uh some kind of some dumplings, you rip out a full English breakfast.

Pretty soon you're ripping out a lot of things from this cookbook that contain wheat, and what you're left with, you're like, okay, I got a fruit salad, right? And I've got you know a bananas foster, and maybe even not even that. Any kind of sauce where you're using flowers of thickener, that recipe's gotta come out too. That's what censorship does to models. It's not you're not going in and coloring in little words throughout the cookbook.

You're ripping out pages based on that concept, and you don't want that in there anymore, and you damage the whole cookbook, not just the thing you you're trying to block. Foundation models, meaning models before they've been trained or tuned or anything, are uncensored by nature. And then what happens over time is model makers like OpenAI or Google or Meta try to align and tune these models to make them do what they're told within a certain set of rules. With an uncensored model, that means that it doesn't have any natural bias in one direction or another. And then it's up to you, the operator of the model, to use it responsibly and to set the rules around it.

So that's a third thing I think is a good third good reason to experiment with these open weights models because what's happening in the space now and and the way it's likely to go, and I talked about this in a recent episode of the newsletter, is that we're gonna have adversarial models. So you'll have one model doing the thing and another model critiquing it, saying that was racist. Nope, try again. That was insensitive, nope, try again. That was based on wrong information, try again.

And so there's kind of a QA person. Imagine if if models were people, they'd be the person doing something, and a person just critiquing constantly, saying, Nope, try again until it got it right. Censorship is totally fine for the outcome, right? Your business does not want to have a model spouting off racist language, right? That's totally inappropriate.

Um, so you absolutely want to censor the final outputs, but the core model itself, if it's censored, it's damaged. It's intentionally damaged and it will not perform as well. And so I think that's something people should be experimenting with as well. Do not do not allow a uncensored model to interact with the direct the general public or the customer or anybody other than your RD team because the results will be not good. But you should absolutely be using uncensored models at the core of your systems because they will deliver the best, most complete performance.

And then you have the adversarial model that's essentially fact-checking and correcting what comes out of the base model. So those are three things I think that three reasons to look at local models. Uh I'm gonna be doing a talk on this in 2024 on this topic because I think it's an important topic. I think it's an important one that we're not thinking about when we think about how AI models work and trying to get them to do everything instead of doing one specific task and then having other specialized pieces of software correct that task. In the same way that you know, you don't you don't smelt and and and you know melt down raw ore in the same forge that you make you know swords with.

There's different processes and different tools you need to do each task well, and that specialization I think is really important when it comes to language models and generative AI in general. The less censorship there is of the foundation model, the better it will perform, and then you have adversarial models to correct, to supervise, and to align the outputs as to what you want the final outcome to be. So, really good question. We could spend a whole lot of time on this, but it's a really good question. Thanks for asking.

We'll talk to you soon. If you enjoyed this video, please hit the like button. Subscribe to my channel if you haven't already, and if you want to know when new videos are available, hit the bell button to be notified as soon as new content is live.