So What? Pre-requisites for Large Language Model AI Fine-Tuning

Well, hey, howdy, everyone. Happy Thursday. Welcome to So What, the Marketing Analytics and Insights Live Show. I'm Katie, joined by Chris and John. And John is, believe it or not, not sitting in front of a fake background.

John, that is a real background, not a Zoom background. Where the heck are you today? This is. This is the winner of the most annoying Zoom background contest that we had last week. So I am at a framework, the virtual workspace in Pittsfield, Massachusetts.

Actually, part of the Berkshires here. You can see the Berkshires make tables turn, which I I always thought that was what the bad guy does. I thought the bad guy always turns the tables on you, but what do I know? I have a ridiculous background. Here I am.

Well, we are we're happy to have you, uh, all three of us in one place for the first time in a few weeks, uh, which again we'll change again next week. Um, but while we're here today, we are talking about prerequisites for large language model AI fine-tuning, which is super easy and rolls right off the tongue. But essentially what we're doing today uh is we are going to look at a large language model like Llama 2 and tune it to specific use cases and needs. Um, it's a it's an interesting topic to be covering because once again, there's a lot of things that go into the prerequisites up front before you even get into the model. And I think a lot of the questions, Chris, that you're getting, especially as you're uh heading out to events and speaking about large language models, is okay, how do I build it?

What do I do? Like, how do I make sure that it you know works exactly for me and what I need? So let's start to get into that. So, where would you like to start today? Okay, let's start with let's start with the the strategic approaches, the general approaches first.

Uh talk about the process and then we can demo a little bit of it. But obviously, because some of these training processes can take days, we're not going to do it on the show. It's gonna be like the cooking show, like uh aha, and the turkey is fully cooked. Um magic. So when we're talking about fine tuning, we are specifically talking about taking a large language model of your choice and saying to it, I want you to work with my data.

I want you to, I want you to give answers that are relevant to me. So, for example, if you were if you were a company and you had, say, a really big HR handbook online and you wanted a language model that could answer employee questions. That's a really good example of when you would want to find uh some kind of fine-tuned model where you could say, this model is designed now, is gonna be is gonna we're gonna force it to statistically focus on our employee handbook and our benefits and all this stuff. And so the model will it's kind of like dogs, right? If you train a bomb sniffing dog, it can't become a drug sniffing dog because its brain is now retuned.

These models are exactly the same thing. When you say this is gonna be the HR model now, it can't tell jokes anymore, it can't write limericks, it won't, you know, it won't write a screenplay for your next big thing, but it will answer those questions really well because you have forced it to be focused and and give extra weight to your stuff. So that's conceptually what we're trying to do. Okay. And so I'm I'm an executive.

I come to you and I say, Chris, my team needs a large language model. I don't want to use what Chat GPT has publicly. I want to build my own that sounds exactly like me. I want, you know, the Katie bot, you know, 900, so that I don't ever have to sit down and write a blog post ever again. That's what we're talking about, right?

Exactly. We're talking about getting a model to be to be purpose built for a very specific use case. Okay. So let's say this, let's start with uh the best part of a use case, which is the user story. The as a persona, I want to sew that.

It's a three-part sentence that tells you a lot about what it is you need to know. Uh so the persona is the people, that's your audience. The want to is your intent, that is your process in your platform, and the so that is your purpose and your performance, how you're measuring, did you do the thing? So this use case would be as a CEO, I want to fine-tune a large language model to my voice specifically so that I can expedite writing content from my brand. That would be a that would be an excellent start.

Uh something like that. And then you probably do want to document the five Ps, right? You do want to spend some time saying, okay, well, you know, clearly it's your you're you want to have a blogging software. Who's going to administer the model? Because there is model administration.

There's work that needs to be done to make these things work. Um, there is this guy over here. John's going to administer it. Exactly. Excellent.

There is a question of general process. How frequently are things going to change. Um, if you are in HR, your employee handbook probably isn't going to change a ton from year to year. They may be updates, but it's not like say uh, you know, like a trust insights client where we might have new meeting notes from every single client meeting every week, and that that database would change a lot. So you'd want to have a an understanding of what the process would be to do that.

Uh the big question is platform. How are you? What are you going to use? How are you going to use it? Um, and there are so many different resources for this.

There's so many different ways of managing it. And well, we'll talk about some of the technology on that. And then, of course, the performance, like, does the thing actually work or not? And with language model tuning, that is not always necessarily uh immediately obvious. So, one of the things you want to have is a bench test to say, okay, we're gonna make this maybe the HR bot.

You should have a set of questions like how much PTO is an employee allowed? And and questions that are known that you know the answers to backwards and forwards, so that you can test out the the model and say, well, okay, it just hallucinated and said that employees are not permitted any time off, and they might they will they might not leave allowed to leave the office. So clearly this this bot's not working quite correctly. Um or this employees are allowed 365 days of PTO. Like I think we might have mistuned that.

So those are the the five big things. Now, let's talk about the two approaches to tuning a model because there's two very different approaches that you need to have a conceptual understanding of to know which one is right for you. One is called parameter efficient tuning or pet, um, where you take a model and you basically force your data into that model and knock away and chip away everything that isn't you, right? So again, you would if it was the employee handbook, you'd force a lot of that data into the model and you take out bunches of stuff that you didn't need. That's one branch.

The advantages of that approach is that the model is very fast, it is self-contained, it does not require um you know a ton of maintenance, the um, and it's computationally very efficient once it's running. It is computationally very expensive to do that kind of tuning. It takes time and a lot of processing power. If you think about it, it would be like taking a deck of cards. Like maybe I had like all red cards, and I want black cards in this deck.

It's gonna rip out a bunch of the half the deck, right? Throw it away, put black cards in, and now I've got a deck that's mixed the way I want, but I had to do a lot of sorting, or maybe I want none of the queens in the deck. So I'm gonna it's gonna take me time to customize this, right? That's one approach. So computationally expensive up front, computationally cheap to run, good for speed, bad for updates.

Because every time you made an update, you have to re redo the training process over and over again. That's approach one. The second approach is called, it's just another mouthful, um, retrieval augmented generation or rag. Um so you have pet and rag. Um, retrieval augmented generation is when you have a model, a base model that does not change, it runs, and it's got to be fairly big model.

And then you have a big document store in a very specific database format where all your stuff goes. And then every time someone talks to that model, it's like instead of having separate black cards, you know, taken out of the red cards taken out of the deck, you have the black cards put on top. So now you have to sort through twice as many cards to find the ones you're looking for, but you don't have to change the underlying base model. Um the other analogy I use is it's like the pet approaches you get a base pizza, you rip off the cheese, you put on the toppings you want, you put the cheese back on, put it in the oven, rebake the thing, as opposed to the retrieval augmented model where you have the base pizza, and then you have a big bowl of toppings, and every time you take a bite of pizza, you put different toppings on. It's inefficient, but you don't have to re-bake the pizza every time, right?

The the pizza can change with every single bite. The advantage of the retrieval augmented generation approach is it is computationally fast to get rolling, because there's really no tuning you need to do on the model. It is slower when it's actually running, is computationally more expensive when it's running, but the data can change really fast. Like if you have a client meeting, that week meet that week's client meeting the notes go right into your document store, and now the model knows everything that happened that week. So it depends on the approach you want to take.

There's these two big forks in the road, and they each have trade-offs, right? So you have to figure out what trade-off am I comfortable with. So let's let's step back for a second. Let's go back to that user story. Because I have a unsurprisingly, I have a lot of questions for you, Chris.

Uh, and I'm sure John has some questions too. We're both like pet and rag. That's about all we got from that spiel. But okay. Um, so let's say in this user story, I'm the CEO, I want to build a large language model to mimic my voice so that I don't have to do any more of the writing.

Of the two approaches. Well, actually, no, that's not even the right question. The question is really how the question that I would expect that you would ask me is if we are building a large language model that mimics your voice, how often do you think your voice and your tone and your opinions and your perspectives change? And I think that might start to determine which approach you would advise the company who wants to build a large language model would take. So I would think that that would probably be the first question is how often do you have new information to give this large language model?

Is that the question you would start with? I would I would start with that question. And also, and this is a really big one. Do you have the data? Right?

If you don't have data available, you can't do either because both approaches require you to have clean data that is categorized. Um, and and we're going to talk about what the preparatory process looks like for that, because you have to structure the data in a very specific way for either approach. Either approach um you need some structured data so that you can better understand how you're going to work with these things. John, what questions do you have this? This, I mean, this is a lot to take in.

Yeah, well, how about as far as training then? So it seems like the the rag approach, like you said, is better because you're going to be laying all your data into that, and obviously you could um have integrations feeding that as opposed to have to do it as a batch kind of thing through pet. I mean, is that the right way to think about that as far as getting it to be more accurate and training it over time? Um the the biggest thing with the training really is the data quality, right? It it it ends building the day and structuring the data in such a way that you can easily you you have good governance over it.

I mean, that's that is the the biggest hardest problem is people don't have good data. Um the data is messy, the data is not well labeled, the data is sometimes incorrect, um, or the you're using the wrong data. And there's a number of, you know, there's so many different tools that you can use to do this sort of thing, um, but without without that governance up front, that preparation, you can't make these tools work. They will, they will simply if you recall back about I think a two or three months ago, we did uh an experiment where we took a bunch of Katie's blog posts that she wrote and we fed it into uh a fine-tuned model of GPT 3. And you'll recall that it it got the tone right, but there was a lot of garbage in there and it would spit up weird characters and stuff because the data going in wasn't clean.

You know, it's funny. Um, this is a total uh intrusive thought that just needs to come out. But as you were talking about um like the HR bot and needing to ask it questions, my first question for you know, the Katie bought 9000 would be when I say stop, you say, and if it doesn't say collaborate and listen, then obviously we need to continue to fine-tune the large language model to be more accurate to the actual true authentic Katie. And yet that's actually a relevant point. Like, should vanilla ice's lyrics be in the document store?

Like, is is that a if that's a thing that you want in there, then that data should be part of the data set. You know, if if you have all 90s songs in sitcoms and weird cultural references and things, and you're trying to make something that is custom to your voice, that is knowledge that it probably should have because it's not going to be in all your writing. And when you pull one of these models off the shelf, they will have some knowledge of it, but not extensive. Part of fine-tuning a model is all about saying, well, what does this model what is this model supposed to do? You know, how is it going to sound?

And so much of our voice is a reflection of the books we've read, the shows we've watched, the songs we listen to, and that can be part of the document store. Interesting. Mm-hmm. So, you know, Villa Ice, Belbev Devo, all the the 90s classics might need to be a document store. We have a new jack store.

Yes. Oh, there is a use case for this. I I promise. Uh this is this, I mean, this is exciting. This is absolutely the large language model we're gonna be building.

Um you were talking about the data, the governance. And so I know that we covered this a couple months back on the uh first version of this show, but talk a little bit about the data itself. So, you know, when I think, you know, if I'm just going strictly off of the use case that I gave you, so as a CEO, want to build the model so I don't have to write anymore. I'm like, well, what do you mean data? Because when you say that, I'm like, well, I don't have spreadsheets of this work that I've done.

I'm gonna give you blog posts. So talk a little bit about like the kinds of acceptable data to train for like the different use cases. Sure. Um, so let's actually walk through an example so we can see this. So I have here the opening of um my most recent newsletter.

This is this past Sunday's newsletter. This is just a plain text file, right? And it's all about the marketing AI conference. This would be one document that you would use for fine-tuning. And again, depending on the approach you take, you might do different things with this.

So if I was taking the approach of uh retrieval augmented generation, I'd want to put some metadata in this file, like author name, uh, event, date, and stuff like that. Just plain text, kind of like what you would have in a regular blog post on your website, so that you would maybe some keywords so that the system would know, okay, that's what this is about. Uh for the parameter efficient tuning, I would actually almost need to use another model to create the prompt and the response. This blog post is the response. This is what we would want the system to generate if you know um in that example.

And so I would go to a system like uh Claud's and uh Anthropics Claude, and I would say, you are an expert in generative AI, you know, blah, blah, blah. Your prompt should follow this format. I give it the prompt, our the trust insights um uh prompt uh uh sheet, which uh if you have not gotten your copy, go to trustinsights.ai/slash prompt sheet to get your copy of this framework. I give it a what's called a few shot learning example, and then I attach that blog post to it. And what it does is then it generates prompts that you would give to like a chat GPT for which the post I've written is the answer.

And this would have to be turned into basically a big spreadsheet. So each prompt would have the blog post as the answer. And you would have dozens or hundreds, possibly thousands of these for the parameter efficient tuning version in one big honkin spreadsheet. And that's what then gets put into a tuning mechanism. Like this one here is a tool.

The name of it is ridiculous. It's called Oogabooga. Um, that is fantastic. And I I'm gonna try to make you say it as much as possible. Yeah, is that an acronym or what is that come from?

It I it's it's I don't know. And is it spelled as it sounds? It is spelled exactly as it sounds. That is the best thing I've heard all day. Um but you can you can see in the system itself, there's an opportunity options to uh load like your your training data set, your evaluation data set, the formats uh you know, and you have to obey specific formats, evaluations, and so your data is those documents, and most software these days.

So there's a um for the retrieval augmented generation. This is a piece of software called GPT for all, and there's a document store where you can say in your configuration file, I want my documents to be stored in this folder on my on my laptop, right? And you can put all your stuff, your meeting notes and things in that folder, and as long as they're compatible formats, it will start to do the processing of that for retrieval augmented generation. So either way, you need clean data that's formatted well, that is the kind of data that you would want. So, for example, you would not want to just rip out all the blog posts that you've ever written, Katie, because to your point, blog posts from Katie Robert of 2018 are different than blog posts from Katie Weber of 2023.

So you have to go back through and say, do I want this even in the James data set or not? Or is it like this really isn't me anymore? Well, and that was going to be one of my questions along those lines of you know, how much, and I know the answer is going to be it depends, but I would like you to expand on it. But how much data is enough data to train it? So if I have like two really good blog posts that I'm like super proud of, and I'm like, this is it, this captures my voice.

I'm assuming that that's not enough because what I would end up getting back is just a different version of those two blog posts over and over and over again, versus if I give it a you know, a thousand blog posts, I would theoretically get more variation. The question is really, so my understanding is that these models don't know what you don't tell them. So I couldn't necessarily give it a thousand blog posts written by me that I've never talked about, you know, large learning models, large language models, and say, write a blog post about large language models. Like what how does that work? Like, does it only know the information that it's fed based on my blog post, or is there other information that comes in?

There is other information in there, and that is a function of what model that you're what model you're using as the base model. That is that is a relatively well, it's not complicated. It's it's just knowing what's out there. Generally speaking, um, let me show you an example here. So I'm gonna go over to Hugging Face, the model repository, which is kind of like the app store for all this stuff.

And uh in this case, I'm gonna look for models by there's one guy who's goes by the bloke um who creates tons of models and look for a specific format. Look, you'll notice there are these numbers here 13b, 70b, 7b. Anything that ends like that, that is a model that indicates the number of parameters in a model. Yeah. Um the more parameters there are.

Think of parameters like this. You've ever go to Subway, the sandwich shop, um, and you get it and you order a Subway sandwich. Um, the parameters essentially are the number and the diversity of toppings you get on your Subway sandwich, right? So if you get a Subway sandwich and it's just like I want a subway sandwich with cheese, right? Just cheese.

Very small number of toppings, very fast for the sandwich artist to put together, not a particularly satisfying sandwich, right? If you, on the other hand, get a sandwich and you and you know maybe you're in the subway app and you choose every topping and you go to double toppings and you get a sandwich like this tall, right? That would be uh uh the equivalent of a model with more parameters, lots more parameters, so 13 billion parameters, 70 billion parameters, you know, that you're you're now like a uh a giant sandwich. It's gonna take forever for the same the sandwich artist to assemble that sandwich, right? But it's gonna be really flavorful and have a lot of different stuff in it.

Model parameters are like that. The larger the number, the more knowledge it has, typically, but the slower it is. If you want your your language model to be able to answer questions about stuff that you didn't provide information for, the bigger the model is, the more likely it is it can do that. So a 7 billion parameter model is going to have some big knowledge gaps. So it will be able to answer to rephrase and summarize information you give it, but it's not going to have a lot of outside knowledge.

A 30 billion parameter model or 70 billion parameter model, there's gonna be a lot of extra knowledge in this. So it might be able to quote vanilla ice without you providing the lyrics, right? It might just know that uh you might be able to quote all the 90s rap songs, right? So you should just know. It should just know, right?

So you you could literally have it weave in stuff. You could say, hey, I want you to incorporate the phrase you can't touch this somewhere in this blog, right? And the associated lyrics, and it would do so in the larger in the larger models. In the smaller models, it might be like, I don't know what that means, right? Um and so that's part of the trade-off, too.

What size model dictates the the computational power you need to run it, but it also scales the amount of information it knows. A model like uh OpenAI's GPT 4 or Google's Palm 2, Google's Palm 2 has 570 billion parameters, like it is a massive, massive beast. Claude's Anthropics, we uh or Anthropocks Claude, we think has in the 500 billion parameter range. Same for OpenAI's GPT 4. These are really, really big, and as a result, they have a lot of knowledge.

That's why you can ask Chat GPT, you know, give me some restaurant recommendations in Chicago, because it has that somewhere in it while it's scene. If you were to ask, you know, Vacuna 7B, that it would put like, I'm not really sure. I don't know the answer to that. I don't have that information. Or it would just hallucinate something that's wrong because it just doesn't have that that innate knowledge.

Everything you're saying continues to point back to doing those business requirements first. And setting and being able to set those expectations. And so, you know, if I go once again, if I go back to simple user story, there's a lot of information that needs to be unpacked and documented. So as the CEO, I want to create, you know, the Katie Bought 9000 so that I never have to write a blog post again. However, I also then need to set the expectation of it needs to bring in outside pop culture references.

It needs to understand, you know, and stay up to date on what's changing in the AI industry. It needs to, you know, be able to do this and be able to do that. And that's not necessarily the content that I've written that I'm giving it. I then need to know where to get all of those outside pieces. And I think if I'm following, that's where we're really talking about the fine-tuning of these large language models, because it's not a one and done.

You don't just stand up and go, okay, now it's gonna write, you know, blog posts forever, and they're always gonna be brilliant and up to date. This, when you say fine-tuning, you're really talking about updating and maintaining and making sure that the data that exists within the model is the data that you need it to be in order to produce the output. Now my brain hurts, but I think I got it. That that is correct though. That that is correct.

Those requirements are really important, and there's there's actually even more layers of requirements as well. So for example. Just kidding. For example, are you working with protected information? I mean, that's gonna change what architecture you use.

That's gonna change what models you use, because some models are are better at certain types of content than others. That's gonna change the the approach you use. You might not want to do, say, the parameter efficient tuning because that if customer information is changing frequently, you might need the retrieval augmented generation where you have a CRM that's dumping records in a format that these models can can understand. Guess what that does? Now you have security requirements for the document store.

Where are you gonna store these documents safely and securely because they contain person identified information, sensitive protected information, protected health information? That document store is radioactive, right? You've got to lock that thing down and be able to control it, be able to produce results for an auditor to say, like, yeah, we've we've guaranteed that this language model is secure and the data store that feeds it is also secure. So there's those are extra considerations that are part of that user story who had better be part of that user story because if not, you're gonna have a bad time. John, thoughts?

I can see like I can see the sunflowers behind you spinning, like the wheels and the gears in your brain. It's it no, your model has to be legit, too legit to quit. That's really thank you where it needs to be. Full circle. Get out.

Yeah. Um so that's that's sort of the prerequisites for fine-tuning. And again, it doesn't matter which approach you take. There you have to do a lot of this legwork up front. Um, if you want it to work, the risk that a lot of organizations are facing is they're just rushing ahead, like, oh, let's just let's turn this thing on, let's build it.

Like, oh, here's a model, let's slap that in place. Well, is that the right one? Are you, you know, do you do you even know what the underlying architecture of the model is and whether it's a it's well suited or not? So there's one called um, I swear to the the names of these models could easily be the names of cannabis strains. Um, there's one called Cherry Blast that is uh specifically good at coding, writing code.

So if you pulled that model off the shelf as your foundational model for fine-tuning and you're working with medical records, those two things don't go well together, right? So that would be that would be problematic if you picked the wrong model. And again, that's something that as part of the requirements gathering, hey, this model had better be able to code too. Oh, well, now we have to look at something very different. Go ahead, Sean.

Well, I was just gonna say, how does that play into then if you want to make the uh your architecture updatable, right? It it I don't know, I kind of it feels like doing the rag approach would give you a data set that can sit outside of the model. So now when you have a new model, it's just plug and play the new thing and you don't have to go back. But is that how that works? That actually is a really important point.

Um something that came up at the the at the Makeon conference. A lot of vendors who were there had built these very specific fine tuned approach, the parameter efficient tuning approaches on GPT 3, the Da Vinci model from three years ago. And they were so much of their infrastructure was tied into this thing now that as new models kept coming out, they could not adapt, right? So if you work with some of these these AI vendors, like, yeah, you're using a model that's three years old now and it's not the best in class for those particular tasks. When you take an approach like um the retrieval augmented generation, yeah, there's a higher compute cost, but the underlying model, you can swap that in and out, right?

As long as you thoughtfully figure out what the abstraction layer looks like between your data and your your your underlying engine, um, you can pull the old engine out, put the new one engine in. And that approach, as of today, makes the most sense because these things are changing so fast. Like if you look, you know, if we look, for example, the the Lama 2 model came out from from face from Meta um three weeks ago now. You are just let's take a look at Llama 2. There are 3,365 derivatives of Llama 2 available, right?

So you have uh this one is uh a QLORA fine-tuned version for French. Um, there's an e-commerce frequently asked questions model in here. So as companies like Meta or whatever release new models, you're like, well, there's an there's one that's even better. If I go the fine the parameter efficient tuning approach, I gotta restart my training all over again. And granted, it doesn't necessarily take long, it can take a few days, but you might not have a few days to spare.

Exactly. Um, and the other advantage of the the um retrieval augmented generation approach is all these models perform slightly differently and do slightly different things. If you receive complaints from customers, like, hey, your chatbot just said something really offensive, and you and you know you just swapped in a new model, okay. Okay, well, great, pull that one up, put the previous one back in, right? And and we'll we'll QA it and test it and stuff like that.

If you went the parameter efficient tuning approach, you would have to shut that chat bot down until you were able to figure out what had gone wrong either in the tuning or if there's something wrong in the base model. And so having the modular approach does make more sense for the environment today, but it comes with those trade-offs. It also occurs to me that what you're talking about, so you mentioned that you know you were at the Make Con conference, you talked with a lot of vendors who have built their applications on top of these fine-tuned models. A question that we hear a lot from our audience is what questions should I be asking a vendor? Uh and I feel like a lot of what you're talking about with these fine-tuned models, the type of model, the requirements, if you're if you are assessing vendors who have built some sort of an interface on top of a large language model, if they are not asking you for your user stories for your requirements, that's a huge red flag.

Because to your point, Chris, some of these models are going to pull in PII. Some of these models are going to pull in you know information on literal llamas when really your business is all about camels and they're different animals altogether. And so really just making sure that the vendor if you're choosing to rent from a vendor versus build your own that the vendor is able to partner with you and explain what they're doing, that they're not black boxing it. And if even if they're not black boxing, if they're like, oh, we're using, you know, Llama rama oogabuga Booga 2, they can they can tell you what is in that model and if it aligns with your requirements. Because that's that's the missing piece is do you know what you need and can they match you with that with those requirements?

That's going to be a big deal, especially to your point around privacy, protected health information. I can tell you from my past experience, dealing with the federal government with a HIPAA violation is long and expensive and painful, and that is not time that you get back. Exactly. Even asking a vendor, hey, what is your foundation model? Because the reality is the foundation models, only a handful of organizations are going to publish those.

Meta, Google, OpenAI, the government of China, the government of Saudi Arabia. Um, because these things are well, the uh actually no, it's the United Arab Aramis that put out Falcon. Um these foundation models are so huge and so cost compute intensive that you have to you you almost have to use one of the big ones that a big tech is put out because they're the only ones who have the infrastructure to even assemble such a thing. I think Llama is trained on like 1.3 million hours of compute time, just you know, uh data centers full of of uh compute cards. And if you when you talk to eventually say, Oh, we use a proprietary model, like okay, so either you have a very small model that you custom train that's gonna be horrendously underperforming compared to what's best in the market, or you have no idea what's in your infrastructure, and you can say, Well, well, cool, I understand you're fine-tuned it.

What's the foundation model you started with? And if then if then they suddenly get real uncomfortable, like, uh, we're on uh uh we're on open AI 800 zero two like, okay, so you're four years out of date, and and your foundation is is shaky um those are questions that you can ask uh those questions that you know if you want it if you are trying to vet a vendor and you and it's like mission critical and you want some help let us know we will talk to the vendor for you um probably well I mean it depends on how much you like the vendor or not if you don't like the vendor have me talk to them because it'll be a rough ride for them but we can we can ask those questions to say what's the foundation model how did you do your fine tune what kind of fine tune is it now that you've watched this episode you know what the two different branches are and you have a sense from your from the approach is which approach you might want to take these are all questions you can ask then you should ask vendors to say tell me more about what's going on under the hood you don't have to give away all your secrets but at least give me the broad scopes and if you're like me then I'm you just start making up names of models and asking the vendors if they're using them. If they say yes then you know then they're definitely full of it. So if they are using the Lama Rama ogaboga you know breaking three yeah breaking three electric boogaloo I was actually uh referencing that yesterday um then you know that there's definitely something wrong so John you know what would you call your uh large language model yeah is that break in three electric ogaloo boogaloo would be uh where I would go from that this is why Chris can't hang out with us we just don't take him seriously enough we do you know you know what it is it's a lot of information to take in and I think that the advice of you know, bringing us on to help vet those vendors because this is a it's a big decision. We've always talked about making sure you're doing your requirements, doing your due diligence, asking your vendors questions about things like a CRM or an email marketing system or you know, a financial system.

The stakes for a large language model are different. It's still a piece of technology that you still need to use the five Ps to figure out where within your organization it fits and who's going to be using it. But the data, the stakes for the data that you're bringing in and that you're pushing out are so much higher that you really need to make sure that you know what you're getting into. Uh, because it could be a very costly decision if you make the wrong one. Yep, it can be reputationally damaging, it can just blow up things.

Um, so all of those are all of those are risks that you can mitigate if you if you have some help. Um but that's a I mean that's that's the fundamentals of the prerequisites of fine-tuning. You'll note we did not actually fine-tune anything in this episode yet. And again, that's because the process can take days, um depending on the size of your data and and what foundation model you're starting with, and which approach you're gonna you're gonna do. But when you do it, um it's like the software development lifecycle.

So you also need to have a plan for the fine-tuning process. What are the technical requirements, right? Um, do are you going to do a pilot in MVP to see like, okay, I'm gonna give this a data set of 10 examples and and run it through real quick because that'll tune in like an hour. Um, and then you can check the results real quick and say, okay, the tuning process worked, or the tuning process did not work. Um, and and then from there you can iterate kind of like a scrum cycle, and and where you can do additional revisions and things.

All of those are are parts of the actual tuning process itself, which we did not touch on today, because this is just gathering the materials you need to be able to start the tuning process. Well, it's like any other piece of tech or software. Taking the time to do your requirements up front and spending more time on that saves you resources and budget and energy when you actually start to put your hands on the technology itself. It's always been true of any kind of software development. If you just go right into pushing buttons and you know, trying to make things happen, it's gonna take longer and cost more money than it would if you just sat down and did something like, oh, I don't know, the five Ps.

What is what is the question you're trying to answer? Who needs to be involved? What is the repeatable process? What pieces of equipment are you using, and how do you measure success? Exactly right.

So any final questions, Katie and John. I want to know when my Lamarama ding dong is gonna be available. First quarter of never. Perfect. I look forward to it.

John. We're out of here, peace. Can't touch this. We'll talk to you next time. Thanks for watching today.

Be sure to subscribe to our show wherever you're watching it. For more resources and to learn more, check out the Trust Insights podcast at TrustInsights.ai slash TI podcast. At our weekly email newsletter at trustInsights.ai slash newsletter. Got questions about what you saw in today's episode? Join our free Analytics for Marketers Slack group at trustInsights.ai slash analytics for marketers.

See ya next time.