You Ask, I Answer: Operating an LLM on Your Own Data?

In today's episode, Rich asks, there was a lot of discussion at a recent conference about implementing a large language model for on your own data. Is this a reasonable expectation for a small business? It depends. It depends on the small businesses' capabilities, on their technical prowess, and on whether they have problems that a language model could solve. I mean, one of the most important things to do is to think about how do you use a tool like the like a language model to improve your business.

There's productivity gains, there's obviously scaling content marketing and things. But the question is, would your data make a big difference in tuning one of these models? The process for tuning is you create a large data set. You take a large amount of your data and you create associated prompts that go with it, like write a blog post about this, and of course, there's your blog post that exists, and that would be the response to that prompt. And then taking hundreds of those and pushing them into a tuning facility that allows a model to learn the kinds of questions and responses it should be giving, and it will nudge the model weights, the probabilities of returning any given word towards the data you've given it.

Now, the big question you have to ask is what problems do you want to solve? If you're doing content generation, yeah, that might be a reasonable expectation because content generation is probably content you probably have a good amount already. Um you probably have internal content that could be uh useful for content marketing. Uh for things like operations, if you have existing templates, existing emails that you could use as training data, then you could it's reasonable to say uh that could be an expectation that a small business could do it. On the other hand, if you're not sure about the use cases, then the first thing to do is to use the foundational models like the ones behind ChatGPT or Claude or Llama, et cetera.

Um, and see if the the vanilla unflavored models can do those the tasks you want them to without having to fine-tune it. If you get great results from detailed prompts and maybe some few shot examples in your prompts, then yeah, you probably don't need to tune a language model on your custom data. On the other hand, if you are trying to coerce a model into giving you a specific result, and no matter how detailed you make the prompt, it's just not doing it, then implementing a model on your own data with the fine-tuning process might be the way to go. So you've got to be clear about your purpose, right? We're going to use the reference to trust insights five Ps.

You've got to be clear about your purpose. You got to have the people and the skills to do it. You should have a process for gathering your data. You'll want to choose a model, the platform, and then measure the performance of the tuned model to say, like, yeah, this tuned model works better than the off-the-shelf models. That process is important to go through.

It's basically requirements gathering before you spend a lot of time investing in tuning your own model. I'm not saying it's a bad thing to do that because it does it does work, it can generate great results. But you may not need to do it if you can get similarly good results out of foundational models instead. But it's an interesting question. It's a question that's worth going through that exercise, that 5P framework exercise, so that you can understand yes, this is something we should do, or at least plan to do, or no, is this doesn't make a whole lot of sense.

So good question. Thanks for asking. Talk to you next time. If you like this video, go ahead and hit that subscribe button.