You Ask, I Answer: Operating an LLM on Your Own Data?

In today’s episode, I answer whether small businesses can reasonably fine-tune large language models. We discuss the capabilities required, potential use cases, evaluating model performance, and alternatives to tuning. I recommend clearly defining your purpose and process before investing time in tuning models. Tune in to hear my full perspective on this key question for small business AI adoption.

You Ask, I Answer: Operating an LLM on Your Own Data?

Watch this video on YouTube.

Can’t see anything? Watch it on YouTube here.

Listen to the audio here:

Download the MP3 audio here.

Machine-Generated Transcript

What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for watching the video.

In today’s episode, rich asks, there was a lot of discussion at a recent conference about implementing a large language model for on your own data.

Is this a reasonable expectation for a small business? It depends.

It depends on the small businesses capabilities on their technical prowess, and on whether they have problems that a language model could solve.

I mean, one of the most important things to do is to think about how do you use a tool like the like a language model to improve your business, there’s productivity gains, there’s obviously scaling content marketing and things.

But the question is, would your data make a big difference in tuning one of these bottles? The process for tuning is you create a large data set, you take a large amount of your data, and you create associated prompts that go with it, like write a blog post about this.

And of course, there’s your blog post that exists.

And that would be the response to that prompt.

And then taking hundreds of those, and pushing them into a tuning facility that allows a model to learn the kinds of questions and responses it should be giving and will nudge the model weights, the probabilities of returning any given word towards the data you’ve given it.

Now, the big question you have to ask is what problems do you want to solve? If you’re doing content generation? Yeah, that might be a reasonable expectation.

Because content generations probably content you probably have a good amount already.

You probably have internal content that could be useful for content marketing.

For things like operations, if you have existing templates, existing emails that you could use as training data, then you could it’s reasonable to say, that could be an expectation that small business could do it.

On the other hand, if you’re not sure about the use cases, then the first thing to do is to use the foundational models like the ones behind chat GPT or Claude or llama, etc.

And see if the the vanilla unflavored models can do those the tasks you want them to without having to fine tune it.

If you get great results from detailed prompts, and maybe some few shot examples in your prompts, then yeah, you probably don’t need to tune a language model on your custom data.

On the other hand, if you are trying to, to coerce a model into giving you a specific result, and no matter how detailed you make the prompt is just not doing it, then implementing a model on your own data with the fine tuning process might be the way to go.

So you’ve got to be clear about your purpose, right? We’re going to use the reference to trust insights five P’s, you got to be clear about your purpose.

You got to have the people in the skills to do it.

You should have a process for gathering your data, you’ll want to choose a model, a platform, and then measure the performance of the tuned model to say like, yeah, this tuned model works better than the off the shelf models.

That process is important to go through.

It’s basically requirements gathering before you spend a lot of time investing in tuning your own model.

I’m not saying it’s a bad thing to do that, because it does, it does work, it can generate great results.

But you may not need to do it.

If you can get similarly good results out of foundational models instead, but it’s an interesting question.

It’s a question that’s worth going through that exercise that five P framework exercise, so that you can understand Yes, this is something we should do or at least plan to do or no is this doesn’t make a whole lot of sense.

So good question.

Thanks for asking.

Talk to you next time.

If you’d like this video, go ahead and hit that subscribe button.

(upbeat music)

You Ask, I Answer: Operating an LLM on Your Own Data?

Machine-Generated Transcript

Comments

Leave a Reply Cancel reply

Pin It on Pinterest