You Ask, I Answer: Who To Hire For Large Language Models?

In today’s episode, I tackle the question of how to staff up your agency to take advantage of open source AI. I explain that leveraging these new technologies requires a cross-functional team, not just one specialist. From IT infrastructure to project management and requirements gathering, many key roles are needed to implement generative AI successfully. Discover the insights you’ll need to build a winning team!

You Ask, I Answer: Who To Hire For Large Language Models?

Watch this video on YouTube.

Can’t see anything? Watch it on YouTube here.

Listen to the audio here:

Download the MP3 audio here.

Machine-Generated Transcript

What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for watching the video.

In today’s episode, Chris asks, I run an agency and I want to get to the next level with open source, large language models and generative AI, who am I looking for? This is a very tricky question.

And it’s a tricky question, because we think of generative AI as this thing, like, you know, use chat GPT, or you use LM studio or something.

And the reality is, under the surface, it’s not one thing, it’s a whole bunch of things.

At a bare minimum.

This is a person who has experience with it with information technology.

So to run an open source model, you need to have compute infrastructure, you need to have a compute infrastructure that either runs locally on your hardware or runs in the cloud somewhere.

So if you were to deploy, say the llama to model, and you were to do so, say in Google Colab, or Azure or AWS, you need to have a server that has GPUs that can run that model because you don’t want to do it on CPU inference.

It’s bad idea, you’ll be waiting forever for like even simple answers.

So you need some horsepower.

That means you need people who are comfortable working within that infrastructure.

Now they may not be full time employees, but they better be people who know how to deploy that kind of hardware or services to do that.

Then once you’ve got a model, minimally operational, you need to be able to talk to it, you need to have some kind of interface to it, because I’m presuming that this is not this is not something that everyone in the agency is going to be working on, you know, command lines and issuing commands directly to an LLM.

It’s kind of like driving a car by, you know, manipulating the chains going in and out of an engine, you really wouldn’t do that.

You would, you would give your users an interface like a steering wheel and pedals to be able to use a car successfully, chairs, you know, no one wants to sit on a motor going down the road.

So an LLM needs some kind of interface, and there’s so many different options there.

So you’d want to have someone who has some experience evaluating different options, and figuring out which one best suits the the needs of the agency, which means you also need someone who can do requirements gathering really well, a good project manager with DevOps and it background, who can ask and users, hey, we’re going to be rolling this thing out.

It’s like chat GPT, but it’s our own, what would you like it to be able to do? And that requirements gathering is going to help you scope out what the project looks like, because it might just be as simple as we got all these documents, we want to be able to ask questions for our documents.

And you might say, you know, then we’re just going to wait for Microsoft Copilot to come out and have Microsoft and its infrastructure, handle that for Office 365.

That’s all people want.

That’s fine.

You don’t need to build a lot of stuff.

If you are saying to your stakeholders or your your key personal, hey, I want a custom model that does x, y, and z tasks that are unique to our agency, but does them really well.

Okay, now you have some better requirements.

If you say I want a system that can automate this set of processes, you then have to start looking, okay, well, what kind of developers do we need to do? Because, believe it or not, language models, language models are not all that capable.

I know that sounds weird.

So you know, everyone’s talking about chat GPT and, and what large language models can do and how they’re going to take over the world.

Their word prediction engines, that’s all they are.

They’re really good at language.

They’re really not good at things that are not language.

And so to make the most of these systems, they need other infrastructure support, they need a vector database of some kind to take your data and transform it so that it’s usable by language model, they need a system like Lang chain to be able to tie in external services, right? chat GPT can’t even browse the web.

So you’d want to have something like the ability for Lang chain to talk to say like a selenium browser instance to go browse the web and return the HTML to your language model to then process the language within it.

So a big part of this comes down to requirements gathering and system architecture.

And then you have DevOps, you have it.

And then you have your project management.

So this is not a person you’re looking for.

There is no person that I know of.

And I know some really, really, really smart people who are way smarter than I am.

There is no person like this who exists.

This is a team effort.

This is a team effort between project management, and it and DevOps, and marketing, and your and your stakeholders, right? This is this is sort of an all hands thing, starting with requirements gathering, then going into building a project plan, then figuring out the infrastructure and the coding and all the pieces you need.

And what the final vision looks like? What is the product at the end of the day? Is it a web interface on your company’s internet that people can use? Is it an app on their phones? Is it a public facing tool? Whatever it is, you need to be super clear on it so that people understand this is what we’re going to get.

And so it is it is fun to play with the models themselves.

I do it all the time.

I make them do all sorts of weird things, you know, turn, turn a role play software into a focus group and stuff.

But putting this stuff into production, particularly if you’re going to make it part of your secret sauce requires a lot of thought, a lot of budget, a lot of people, and a lot of good processes.

It’s a that we call the trust insights five P’s, what is the purpose? Who are the people that are going to be involved? What are the processes that are involved? What platform and technology you’re going to use? And then what’s the performance? How do you know that you’ve succeeded? Building out this kind of capability within your agency requires rigorous adherence to that framework.

So you get all the steps done.

And critically, you don’t invest 10s or hundreds of 1000s of dollars and 1000s of hours to build something nobody wants.

Right, or to build something that is going to be obsolete in three months.

You know, so there’s even part of the requirements gathering is understanding how to how do you architect software that has abstraction layers in it, so that as new models and new capabilities come out, you just pick one up, put another one in.

Those are the kinds of considerations that you need to build into the project to to be thinking of in advance.

And that’s, you know, that’s what my company does, we help with the consulting on that to say, Okay, here’s, here’s all the catches that are going to happen.

Make sure you have a plan for them.

It’s a good question.

And your head is in the right place.

You are thinking along the right lines.

But it is not a person.

It’s not even a couple of people.

It is a big effort with big rewards if you do it right.

But also a lot of things that can go wrong.

If you are not diligent, if you are not prepared, if you don’t do really, really, really, really good project management.

So good question.

Thanks for asking.

If you’d like this video, go ahead and hit that subscribe button.

(upbeat music)

You Ask, I Answer: Who To Hire For Large Language Models?

Machine-Generated Transcript

Comments

Leave a Reply Cancel reply

Pin It on Pinterest