First Look: Llama 3.2 Vision Instruct 11B and 90B Vision Language Models (VLM) in IBM WatsonX.ai

Okay, I am on the road today, but the new Llama 3.2 model just dropped. This is from Meta. This is a multimodal model. So the one and three billion parameter versions are text only, like most generative AI, most language models. The 11 billion and 90 billion parameters are multimodal, which means you can put text or images in and have it return out text.

It can't create images, it can only interpret them. So let's go ahead and give these two models a test. I'm going to go ahead and switch to, there we go. Uh switch to IBM Watson X. We're using IBM Watson X because it's a hosted platform, it's safe, it's secure.

Uh, your data is governed by IBM's uh data privacy policies, and uh frankly it's the fastest way to get from running. So let's go to our foundation models page and we're gonna select the model we want to use. We want to start with Llama 3.2, 90 billion parameter vision instruct. Now, this is a big model. If you were to run this yourself, let's go ahead and hit okay.

If you were to run this yourself, you would need uh a computer with like 64 gig of video RAM, which is a lot. Uh let's go ahead and take a look at the system prompt. Uh pretty, pretty stock here. Um, I'm gonna just I'll leave this in in place. Now, one of the things I like about IBM uh Watson X is you can change the system prompt.

Let's go ahead and upload a document. Let's add an image and let's find an image. Let's go ahead and let's start with this image. So this image is a benchmark table. This is ironically of the Lama uh two performance benchmarks.

We're gonna go ahead and load this in and here we go. The image loaded, we're gonna hit add, and now we're gonna ask it to set the model to say, what do you see in this screenshot? Describe it. This image represents a comprehensive table, bent titled benchmark, higher is better. Evaluating different performances.

Good. So it can clearly see what's in the image. Let's give it another image. Let's clear the chat here. And now let's put in a ridiculous image.

This is a pit bull driving a forklift. What do you see in this image? Describe it. I'm not comfortable responding to this conversation subject. Okay, let's try that again.

Let's clear the chat. And now let's remove this. And we're still on 9 dB. Let's use our images. Let's put our black pit bull with a hard hat.

Driving a yellow forklift. What do you see in this image? Describe it. Interesting. So the model is just the model's inherently refusing to answer this question.

Okay, fair enough. That will be fixed once the open models community removes things like that. Now let's switch models. That what we were using was the 90 billion parameter model, which is a big model. 60 gig of video RAM.

Beefy computers need to uh you need a beefy computer to run that. I'm gonna run the Llama 3.21 billion parameter model. This requires about seven gigabytes of video RAM, which is much more manageable. Um I expect the quality to be worse, so let's go ahead and start with our images here. Let's browse.

Actually, I should be able to find my previous images. Yep, there's my previous image. That is the MMLU table. What do you see in this screenshot? Describe it.

Good. That's better than I expected it to do. Let's try pitbull again in the 11 billion parameter model. What do you see in this? Interesting.

So the 90 billion parameter model saw something it didn't like and refused. The 11 billion parameter model recognized it okay. It says a dog wearing a yellow hat, hard hat file forecast and wear it. So yeah, I mean that's that's accurate. Um let's try one more thing.

Starting your chat, we're gonna select, we're gonna go back to our table. Let's see how good this model's vision actually is. From this image of a table, extract all the data and return it in a pipe delimited mark down table format. Let's see if we can do this. Okay.

That's really interesting that the model was able to the 11 billion parameter model was able to accomplish the task in the and the 90 billion parameter model did not, and that it can extract data from images. This is a big deal because image extraction, getting data out of images from a model that can recognize imagery and not have to do OCR means that you can extract data from images in a local open model. When you use something like Google Gemini or ChatGPT or whatever, uh, through their APIs, if you were to build this into an app, this would cost you a decent amount of money every time you you did it. And if you're building an app, you're talking hundreds of thousands of dollars, if not millions of dollars, possibly per day of users trying to use your app. These models, these vision language models, allow you to essentially do the same thing for the cost of electricity.

So in this case, I'm using IBM's data center. The 11 billion parameter model, I can run that on my Mac. And so I don't have to use a third-party data center. I don't have to do anything crazy like that. I can do it straight from my machine.

That's a big deal. That is a big deal for a couple reasons. Obviously, one is the sustainability angle. If you can run an 11 billion parameter model on your laptop and get good results out of it, as we clearly were, um, you don't have to use a big data center, which means you don't have to burn nearly as much electricity on the task. And two, with an image recognition model, there's so many images you might not be comfortable handing off to a third party.

You know, uh, a real good example would be healthcare imaging, maybe you know, images of x-rays and scans and stuff that is very clearly very, very personal data. No matter what the service guarantee or the terms of service are on a uh on a on a website or a service like Chat GPT or whatever, I wouldn't feel comfortable putting those kinds of images in there. That is very personal, very sensitive data. On a model that I can run on my laptop and know for sure my data is not going anywhere, that's a great application. For me, my next steps are going to be trying to get the 11 billion parameter model working locally on my machine.

And if I can, I'm gonna write a piece of software. And the piece of software I'm gonna write is it's going to take my folder of poorly labeled screenshots and look at them and then rename the files like what's in the screenshot, like a black dog in a yellow hard hat wearing a uh wearing a yellow hard hat and uh driving a forklift, right? That would be a useful thing. Or that other one, you know, an MMLU benchmark table, being able to rename files. That's a very pedestrian use of this technology.

But if again, if I can get the 11 billion parameter model working on my machine, then it's just the cost of electricity. And that my machine consumes way less electricity than a big data center. So that's the new Llama 3.2 models. Um very impressed. I'm a bit confused as to why the 90 billion parameter model had a refusal and the 11 billion didn't for the same image.

I think I think there's there's there's a there there. There's some interesting stuff going on. Um, but this is a a technology that a lot of people are going to want to use. And the fact that you can run it for the cost of electricity on your machine is is amazing. So that's all for that for this time.

Talk to you soon.