So What? Six Levels of AI Proficiency

Well, hey everyone, happy Thursday. Welcome to So What, the Marketing Analytics and Insights Live Show. I'm Katie, joined by Chris and John. Oh, I was like, John, are you trying to you trying to high-five me? It was a tough position to get my armed by Chris.

I definitely had to do that. Uh this week we are talking about six levels of AI proficiency. So earlier today, um, if you're not part of our free slot community, analytics for marketers, I highly recommend you join. Again, it is free to join. We have over 4,500 other marketers, analysts, data scientists, and generally, you know, interesting, good people talking about stuff.

And today we were talking, I asked the question on a scale of one to five, where would you rate your AI proficiency? One being the lowest. Now, I purposely excuse me, did not give them what five looks like because I was interested in what the self-report of your AI proficiency was gonna be. Because this is the problem that a lot of organizations run into. And we actually, Chris ran into this uh last year when we were doing a workshop.

We asked people at the start of the workshop, honest, you know, would you rate yourself beginner, intermediate, or advanced in terms of your AI proficiency? And most people were like, Well, I'm intermediate, I'm advanced. And then we introduced Notebook LM, and everybody was like, Well, wait a second. What what is that? And so I was generally curious in our Slack community where people were rating their levels of proficiency.

And I would say most people put themselves at about a mid. The downside is that everybody, Chris, was comparing themselves to your proficiency, which I don't think is a fair comparison by any stretch of the imagination. Nobody should be comparing themselves to others, especially if there's not a level of proficiency outlined for their specific job role, which is what we want to talk about today. So with that, well, I guess first let me ask this question. John, on a scale of one to five, what would you rate your AI proficiency?

Oh, no, you're calling me out as both statistician and then uhybody knows my background in economics. I actually statistician and uh AI proficiency are two different things. So you can be a level five statistician. Oh, it's a level two. Yeah, absolutely.

Um yeah, it it really depends on like what's the sample and what's the roof, right? I mean, if you're comparing me to all of humanity right now, like, yeah, I'm at a five because I'm at a company where we're doing this every day and we're you know bringing this to business. Uh if you're comparing me to the three of us here, I'm maybe a one and a half on a good day because I'm the the local uh uh saboteur of the group here, you know, holding everything back. So yeah, it's kind of all over the place. Um, and yeah, but I have to call out Chris calling himself as a three.

Um, because you know, yeah, sure, there's the people that create these models, you know, and you're you would rank yourself below them. Uh, I'm not including them in our sample size here. So uh yeah, I'll stick with my 1.5 and let this run. And I think that that's helpful because again, I purposely did not tell people uh what a what in my question a five was. And so I do find it really interesting how people are putting themselves into these categories of their proficiency, which is where a lot of companies are struggling with people with their teams because nobody really knows where they should be.

So, Chris, where should we start today? Well, I'll I suppose I can explain why I thought I was a three, it's because there's there's knowing the technology, using the technology, modifying technology, and then building the technology. And so for me, I sort of I tap out at you know, heavily using the technology. I'm just dipping my toes now into modifying technology, creating my own versions of models, um, creating different, you know, for example, uh earlier in I think it was earlier this week in the Slack group we were talking about, uh, model quantization, like how you can modify a model to behave differently depending on your needs and stuff like that. And I'm just starting to dip my toes into that because it's a very, very computationally intense uh uh uh thing to do.

And then, you know, uh to me, the the highest level looks like someone like Andre Karpathi, who's like, oh, here, let's look, you know, in this 90-minute YouTube video, I will tell you how to build a GPT model from scratch. I'm like, whoa, I'm not doing that. Which is interesting because again, you're comparing yourself to people who are never going to work for trust insights. And when I look at the capabilities and proficiencies we need for trust insights, I don't need you at that level. And so that I think is really the core of the conversation is organizations and leadership are looking at everybody across the spectrum of you know who's in AI.

They're looking at these folks who have like started these companies who have built these things from the ground up, and they're like, why aren't you there? And that's not a fair comparison because that's not what the organization needs. I don't need you building custom GPTs from scratch. Like I just don't. That's not it's not something that the company that trust insights, like that's not our business model.

So why would you need to like if you want to personally aspire to that, that's fine. But professionally, we don't need you there. Right. But we didn't say AI proficiency professionally at the company. We said AI proficiency, period.

Correct. Was the question? And so that's and I purposely had the question phrased that way because I felt like that's the way that people are asking. They're not even saying what's your level of AI proficiency, they're asking, can you use AI or not? And they're saying, okay, you must be proficient in it because you said yes.

Well, guess what? Everybody can use it because it's built into everything already. So technically, yes, you can use it. Yeah. And and this is if you look at uh the landscape of how people talk about measuring AI proficiency, you see a lot of things like uh specific use cases.

Can you use it to do XYZ? Which I think does tend to help map towards a company's value, right? So if you say, can you're a law firm, can you use AI to process and cross-connect cases, past cases with a tool like Notebook LM? That would be a very specific proficiency use case. Not one I would say is technical proficiency, but it is it is still skillful use of the tooling.

It's like I don't need to know how to repair a blender. I do need to know how to use it safe and I do know what I should and should not try to blend in it. Those are different levels of proficiency in different domains. So this almost makes me think, you know, in in John's world of statistics and economics about things like local maxima. When you're measuring a domain, you don't have to measure everything.

You have to measure your corner of that world and say, okay, what does that look like for us here? And the analogy that you came up with, Katie, on uh it was this week's podcast, I think. Uh you that no, it's it was uh you that are on a client call, one of the two. Um, was talking about proficiency from the perspective of, well, there's there's doing stuff, there's managing stuff, and then there's this delegating stuff, which corresponds to like the the tranches or tiers in an organization, staff and individual contributors, middle management, and then leadership. And I thought that was a really cool model.

You want to talk more about that? Yeah, so this I mean, the whole idea of the levels of proficiency came about from a previous client call where they were asking us, you know, how do we train people to do better AI adoption to basically be better at their jobs? And my first question was well, who are the people, what departments are they in? Because what you need your marketing team to do is not what you need your software development team to do. And this sort of goes to the conversation we were just having about sort of the open-ended, are you AI proficient?

What we are asking John to do as the head of business development is not the same thing that we're asking you to do, Chris, as the data scientist, in terms of where your contributions to the company in terms of AI are. They're just different. And so that was where we started. And so what came from that conversation for that particular client, they had three distinct departments, three uh core competencies. They had product, marketing, and development.

And so within each of those three departments, you have basically, to your point, individual contributor all the way up to leadership, but for those specific departments. So, you know, a marketer beginner is going to be writing blog posts, editing grammar, that kind of thing, whereas a, you know, advanced leadership is going to be, you know, building channel strategies or, you know, doing attribution analysis. That's not the same thing that you would ask of your software development team or of your product managers. So you have to be thinking about roles and responsibilities and what's an appropriate level of AI proficiency. And we've since taken that and made it, I wouldn't say more generic, but more of a baseline where you can then jump off and say, okay, I know what the levels of proficiency should be.

Now let me apply my team's responsibilities to these levels. So you have levels one through six. So levels one and two, to your point, those are the individual contributors. Those are the people who are pressing the buttons, you know, they're they're getting the tasks assigned to them and they're using AI to complete those tasks. Whereas levels three and four, that's more of the management they're building.

So you have those multi-step automations, you have uh the MCPs, you're deploying custom agents, and then you have leadership who are overseeing and setting the strategy of it, and then working in things like agentic systems. So a level one is not necessarily building an agency system, but they're being given instructions on how to execute against an agentic system to complete a task. All of the levels are relevant. There's no, okay, you're only a level one, you're not good enough. That's not the case.

It's really what the needs of the business and the teams are. And it's the thing where this came about was we were talking on the podcast, because you're like, well, how do you get someone from a level one to a two to a three to a four? And my point was the same way you move people up in their career. It's this is just software. This is just another skill set.

You train them, you coach them, you give them tasks that stretch their abilities, you give them you know, courses, whatever it is you're gonna do, however, you train people today is the exact same way you train people on this piece of software. Yep. And when we look now at this sort of chart, now you can say, okay, well, where what is it that you are doing in your work? So it's not, you know, uh, as an example of building custom tuned models isn't on here, right? Because for work level proficiency for the average professional, that's not something they're gonna do unless you are a part of an AI lab.

Uh, there's a very good chance that's never a task that's gonna be necessary. Um, you can definitely see there are things where if you are building an agentic system in Python, yeah, there are definitely use cases within business to do that. Uh, but that is that is different than you know, construct this custom specific model for our environment, which really don't all very few people are going to do very few well outside of highly regulated industries, you can have a need to do that. Right. And I think that that's really where in creating these six levels, we're really asking companies to reflect inwards and stop comparing themselves to you know the creators of Anthropic Claude and you know the data scientist at Google.

You don't need that. And if you do, that's fine. But and those are your levels, but most companies don't need that person doing that kind of work. That's aspirational. That's great.

But when you look in the everyday to day, have you ever done that? Why do you need it now? Just because it's shiny. Exactly. Now, one of the interesting things here also is that this follows very much the organizational structure in terms of ROI and results, if you're doing things properly, if you're doing things well, because as you can see, as you move up these levels, you're delegating more and more to the machines, which means that on any given day, you could be doing more.

Sort of uh, there's a concept in economics called Javan's Paradox, which says that if demand is elastic for what it is that you do and you increase your capabilities, demand will increase with those capabilities in an elastic uh elastic supply, elastic demand. So as you get better at AI, this is kind of one of the funny things. As you get more proficient at AI, your workload will scale to that. So instead of working on one big project, now you have work, you're working on four big projects. There's actually uh folks on places like threads and things where they're saying, Oh, uh, are you even a coder if you don't have nine clawed code windows open at the same time, all working on nine different projects?

It's funny you say that because that's exactly where I found myself in the past few weeks is now that I feel more confident working with Claude Cowork. I'm running multiple projects at the same time. Like just before this, I just finished um a series of website updates based on one of the projects that I was working on. And then parallel, I was doing data mining for content from you know some of our other data sources. And then in parallel, I was also updating the process for the nonprofit that we work with, you know, to get, you know, notifications for the system they want to build.

And then in parallel, I was also working at this other thing for our academy. And I was like, what am I even? This is not this is not how I work, but apparently I'm just like super productive now. Could be because before it's not that AI is doing it for me, because every single one of the projects that I'm running in co-work all had to start with my knowledge base, all had to start with the work that we do at Trust Insights. That does not get replaced.

So all the years of experience, all of the work that we've done, all of the clients that we've served, that still had to happen and still has to happen. What co-work is doing is it's taking all of that stuff and just putting it together faster, like a slide deck or a PDF, than I would as the human, which is allowing me to multitask and do more things. And so it feels like my workload has grown exponentially because I'm able to move faster. But like I'm still doing all the same stuff I was doing before, just in a more efficient way. Yep.

What you're doing is uh you're increasing parallelism in the same way that it would be if you had hired three more interns, right? So you'd say, intern, go do this. Hey, intern, go do this. And the only limitations you really have is what our Claude Max subscription will allow you to do. Correct.

We are at 80% usage currently. Sorry. I'm done for the day. One of the interesting things is uh if you read and follow the folks from Anthropic as as they are working in their system, because as an employee, you do not have limits on what you're allowed to do with it because you uh yeah. Yeah, folks like Boris Cherney, who heads Claude Code, has said, yeah, I'm doing 20 to 30 major projects per day just getting things done because I work for Anthropic and I can do whatever I want and there's no meter on my console.

And it makes you it makes you think even at you know when you look at it, Claude Code, uh Claude uh Max is 200 a month. Have in just the past four weeks, Katie, have you done 200 more worth of work than you've done previously? Oh my god, yeah. If you had to put a pin on any one project that you've worked on, like websites updates and stuff. What would you have paid a person or an agency or a contractor to do the same equivalent amount of work?

At least a few thousand dollars to organize all of the content, draft new landing pages, design them, build them, uh, test them, and then keep them updated, like easily on looking at at least five grand. So when you think about that, that is astonishing that you basically had a team of three people, like three three co-work instances doing work for you. That's level five on this chart. That is on the leadership level where you are delegating large chunks of of billable work to the machines and getting very good results out. I mean, we've all, you know, behind the scenes in our company Slack, we're all like, holy crap, that it looks even this slide.

Um, this slide alone, this morning started out looking like you might use my slides usually do, which is a bit like I just dropped some pastel pastels on the floor. Um, and in the span of seven minutes, Claude is like, I have the skill that we built on a previous episode of the live stream, uh, and so I'm just gonna go ahead and and and make some changes. And it was the I prompted it to say, think about what I'm trying to accomplish. Here's the goal, here's what success looks like. How would you do this?

And said, Oh, well, you want it to look like this. I'm like, yeah, yeah, I do work. And I think that that's a huge, you know, and I want to be clear, we're not advocating for people to be put out of their jobs. We just happen to be a small company on a limited budget, and so for us, it's more efficient to get into the weeds to do these things ourselves for now. But the only reason these projects can run.

So, so you know, I'm running four projects multiple, you know, all side by side by side, but it only works if I'm organized. It only works if I start at the top with something like the 5P framework to say, what the heck am I even doing? So that when I switch context between projects, I'm not lost going, wait, what is what am I even doing? Because that's what happens with a lot of people who try to multitask is they kind of get lost switching context and it doesn't work. You have to be organized.

And the best way to do something like that is with the 5P framework. What is the question you're trying to answer? Who's involved? What's your process? What tools are you using?

And what does success look like? And so, you know, starting every engagement with the five Ps is going to allow you to do that multitasking. So, you know, I look at this, it didn't even occur to me that I was a level five. I was like, Oh, I'm probably like a level two, level three, maybe four at best. But you know, based on what we've outlined, yeah, I'm solidly a level five.

I'm not necessarily looking to get to a level six, but I don't think that's what the company needs from me either. You'll get there eventually. Um, and it you won't do it consciously. What will happen is you will come across a task, you're like, hey, I need you to do, you know, it's it's quarters and I need you to close the books, and it will say, got it, I'm going to write some Python code to do this data processing because I know that Python has this whole you know quantitative analysis finance set of libraries. It knows that if you give it the five Ps and you say your success measure is you successfully close the books and reconcile everything.

Now you're delegating in a system like a co-work, but it's built, it's going to build itself underneath the hood. So you may not see that you are building a Python based agency system because the AI may sort of just keep that behind the curtain. But it's there. When you said I wouldn't be conscious of doing it, I really thought you were going to say because you'll be asleep, and I thought that that was lovely. I mean so you joke, but for those who don't know, Claude Code has this tool called remote control that you can trigger in your clawed code session that lets you then resume that session on your phone.

So I have literally gone to bed some nights uh with remote control open on my phone saying, yeah, approve, approve. Okay, go ahead. And then as I go, I go to sleep, and then when I wake up the next morning, it says, you know, Claude has worked for two hours and 31 minutes on this, and now this is ready for you to review. So you literally you could should you be working in your sleep? No.

Could you be working in your sleep? Yes. The answer is yes. What I find very interesting about this is that, and I would love to hear both of your take on this with systems with agentic systems, because there's more than just clawed code and clawed co-work. There's uh anti gravity, there's codecs, there's a bunch of different ones.

Steps three and four seem to be vanishing because there's less of need for them. And so the question is, and here's this is this is a two part question. A, do you think that's true? And B, you've both had experience building things like gems and GPTs and stuff like that. Do you think that experience is necessary to get to level five, or can you leap from level two to level five?

What do you think, John? Because I have strong opinions on that. Yeah, this is one that we just force back into a software development box, right? Like anything worth doing, somebody will string together some tools so that you can jump three or four. You know, I think three or four is frontier work, whereas established market, everything will eventually just jump up to level five.

And as things like Cloud Cowork and Code get more powerful, they will eat, you know, four and three. There won't be need to do those things. I strongly disagree. Uh, I'm gonna be the outlier here. You know, I look at again, it's we're talking about humans.

And so I look at it the same way I look at the same way a human, like when I was managing a larger team, would I jump an individual contributor all the way up to the leadership team? Very rarely, if ever, because there's something about getting that management experience that prepares you for that leadership. Now, that doesn't mean you have to sit in that role for months or years and not move up to the leadership role. But I do think that jumping someone from like a level one to a level five, you're skipping a lot of foundational skills that they're gonna need in order to succeed at a level five. So I I disagree.

I think that the definitions as the technology evolves will also evolve. So what a level three means today will not be what a level three means, you know, three months from now, maybe even shorter, but skipping over the foundations of how these things work, how to organize them, how to communicate around what they're doing, and then you know, how to do correct project planning requirements, software development cycles, all those things that we've talked about, you know, ad nauseum. Like, you sure you can skip levels three and four, but I don't think it's a good idea. And I'm not saying you have to sit there forever to be like, oh, I'm a level three. I have like two more months at level three, and then I have to move to level four.

That's not the case. You can move through them quickly, but there's still something about mastering the foundations that's going to allow you to succeed and stay and then exceed levels five and six. It's interesting because when you we look at what's happening in the market this past week, Microsoft announced in co-pilot that they have licensed Claude Co-work to live inside Microsoft Copilot. It's going to be called rather confusingly Microsoft Co-Pilot Cowork. Thanks, Microsoft.

Uh tell me you don't have a product manager by telling me you don't have a product manager. But that does in the Microsoft Copilot ecosystem, skip three and four. Because like, well, it turns out that nobody likes those things in Copilot. And Anthropic count was something way better. So once you get into basic co-pilot, then you can say, all right, you know what?

We've gotten level one, got level two, and now here's co-pilot co-work as level five. And we're not we're gonna just discontinue levels three and four. They will go away. This is also something that I I'm starting to see, and we we've seen a lot of discussions about in the in the career field where companies, as they do layoffs, are doing a lot of mid-ma mid-management cleaning. Uh, they're they're downsizing building middle-level management.

And Katie and I, you you I talked about this on the podcast this week. When you have 18 levels of management and and stuff like that between you know the bottom and the top, and you realize nine of those 18 levels don't do anything except attend meetings and send emails, that can collapse pretty quickly with AI, because you're like, but but your job has no output, your job has no KPIs, other than how many meetings and emails did you generate this week? Right. No, and I think that you know, when we when we were talking about it, we were certainly talking about there's different versions of management. So, yes, that version of management, if we're talking about that in this context, yes, that is going to become obsolete.

There is no need to sit people in those levels. And then we were talking about co-pilot now integrating co-work by this definition here. Sure, you're going from level two to level five. But my question is how successful will those people be when they're just kind of thrown into the deep end without having any kind of sense of how to build the skills or how to build the plugins that really are what make cowork so effective. And at an organizational, particularly enterprise, I'm going to hazard a guess that's not even going to be on their radar.

They will be given the skills and plugins and say this is what you use to do your job. Your job is to be to babysit the machine, basically. Um, and and let the machine do the work because the machine will do a better job than you will. And so, and that's where I feel like you have to then rethink the definitions of these, because that's not a level five proficiency. You're back at a level one.

Because if you're just babysitting, you're not actually doing anything, how does that make you proficient? How does that make you understanding of anything that's going on behind the scenes? I feel like maybe what's missing from this is the experience and the knowledge that you have that if something goes wrong, you could fix it. And that's not in here. Like, and that, you know, that's on us to sort of like continue to refine this.

But in that example, that to me is not a level five. Just because you're pressing the button and babysitting with pre-baked tools, you're back at a level one, which is what people are doing with the chaps. You have pre-baked tools. Yep. And where this is going is what we saw with the things like open claw, where if all your job is is to hit yes, you don't need a human for that.

Open claw is perfectly capable of clicking yes by itself. Um Simpsons did an episode on that where he had the little uh bird what because he was like, Yes, yes, why? Oh, I just need why. And then the bird did it, and then the bird fell over and everything fell apart because the machine, the process, the machine, you know, stopped working in the nuclear power plant. It was a whole thing.

But to the point, you know, you if that's all you're doing, you don't need a human to do that. So that four, to me, that person doesn't even rate on the level of proficiency if they're just pressing Y over and over again. Right, which is where a lot of organizations are going to be in the next three to six months, the next year is as these tools put the the thing about co-work being introduced into Copilot inside the Microsoft ecosystem, because so many enterprises and large organizations use co-pilot for good oil. Um at that point, you have a whole bunch of essentially unnecessary babysitters because you can systematize the use of these tools, and where it we're in past you know, legacy AI stuff from a couple of years ago, yeah, you could take your content marketing team from 50 people down to 10 because they can all use chat GPT. Now you're probably down to like two.

Um, you have one person to maintain the co-work system and make sure that the skills and plugins are good, and one person to spot check outputs and and make tweaks, and that's it. But then you're sort of stuck with. I feel like no, we're now heading into a debate, but then we're sort of stuck with same. Your LinkedIn post today. My LinkedIn post today, exactly.

Because so uh I I don't have the link handy, but for those who follow me on LinkedIn at Katie Robert, I posted an article today about this exact problem. So it's essentially, you know, the you know, people were sort of mourning the billable hours. How do I prove value? And that's great, but guess what? The long-term thinking is you've cut your team down to just two resources.

What happens when they go, or what happens when they age out or they retire? Like, who's minding the shop in 2030? Which is actually not that far away from now. So, what happens when you're just AI is everything? So, yeah, this is my article.

AI ate the proven ground, the leadership crisis no one sees coming because we're only focused on oh, well, we can't do billable hours anymore. How do we justify the value? Yeah, that's a problem today, but you need to do more forward thinking of what happens when we've just given everything over to the machine and no humans are skilled to keep these things running. Sure, we can make them as autonomous as we want, but guess what? At the end of the day, it's still software, and software breaks every day.

We have a significant problem uh in the USA where we're based, uh, we have this problem right now in manufacturing. The people who you uh who make the tools to make the tools, they're called tool and die. Uh, tool and die was something that Americans had a great specialization in for a very long time, up until about the 1990s. Once global uh globalism and outsourcing became prominent, uh, and the generation of tool and die makers retired from about 1990 through about 2010, 2015, um, that whole generation of tool and die makers retired. If you want to manufacture something in the US now, you pretty much can't, because there's no one who knows how to make the tools to make the product.

Uh and so you have to outsource to a place like China because there's nobody here who knows how to do that anymore. And so this is sort of the tool and die of knowledge work. If when we talk about levels of AI proficiency, the meta skills about like knowledge, vocabulary to even just be able to describe what you want, measuring success outcomes, project management and things. If you don't have a uh a beginner's bench of of junior folks, and you've out you've baked everything into the machines, you no longer have the tool and die equivalent in your organization. You can no longer manufacture anything that isn't handed to you.

So, John, does this change your opinion of levels three and four being relevant? I'm trying to swing you over to my side. Well, the the issue is that is you know, we're anchoring the org chart to the tool stack, right? Because I totally get where you're coming from, and that you can't operate at a leadership level without management experience, right? Like if you don't know how the machines are supposed to be in the line, you can't just become CEO and have people believe you.

So that's totally true. But I'd also put out there that like you totally skipped N8N. Like there's no point in you learning or messing around with N8N because you're good enough with Claude that uh that's a waste of time for you, you know, you're going back to an old tool. So um, yeah, it that's a the big economics lesson from all this is anytime you make an abstract, simple measurement system, you give up detail and you open up the uh chance at misunderstanding and not being able to accurately talk about what's really going on under the surface. And that's what we've got.

You know, when you map org chart, which is already a mess and has all kinds of cultural issues, to a tool stack that's changing every uh 3.3 seconds, there's uh there's a lot of moving parts that we're trying to get with this model. So uh yeah, we can have differing opinions. It's because we're coming from different sides of the chart. Which I think is totally appropriate, and it's, you know, uh, so stay tuned. I do have an article coming out uh in the next week or so that sort of addresses not N8N specifically, but one of the things that N8N does, so for those who don't know, I think Chris, correct me if I'm wrong, it's an MCP server where basically at a high level, it can take all of your different systems and connect them, where something like Claude Cowork now has all those connections built in, sort of making uh MC MCP servers like an N8N or an Opal, not so much obsolete, but not as to John to your point important that you need to be able to know how to like get to the API keys, connect them, develop against them, like you know, co-work and other tools like Claude Code do that for you.

But I would argue the other side to go back to why I'm correct and you guys aren't. No, we're all right, we all have uh correct opinions in this. To your point, John, we're all coming from different places. Is if you don't even know what's in your stack to access, if you don't know the quality of the data in those systems, and you're just putting these agentic AI, you know, tools on top of really crappy data across 17 different platforms because it does all the connections for you. Guess what?

You still get terrible outputs. So you still have to go back and do those foundational pieces and understand the data that's in the system, understand the quality, understand how the data is being collected, why it's being collected, how it's going to be used. Like I will not only die on this hill, I will set it on fire and take you all down with me. That you need to understand what these things are and how they work before saying, Yep, I just I built an agent on top of our 17 different tech stack systems, you know. You know, I'm a leader.

No, what could go wrong with us just like letting the tech roads grab the whole pile and run with it? I'm sure nothing poor would happen with that. But it's a fixable problem. You can still get to that level five, level six, but you still I, you know, I again I will argue that you have to understand what's going on inside the organization enough. You don't have to be the expert in every single data point in your CRM or every single data point in your, you know, SQL databases, but you have to even know that they exist and have a good baseline of, you know what, the data overall has good data quality, it's clean.

We know what questions the data is supposed to answer. Therefore, when you're building a Gentex systems on top of these, you know, databases, you know what to expect. But if you don't know what's in those databases or that they even exist, what are you building? Yeah, and you you to have that hands on when those things were built to understand what they're actually plugged into and where that data comes from. Because yeah, otherwise you're stuck just like, oh, we got some bad results.

I have no idea why that happened, and you're just yeah, you know, nowhere to go. And then, you know, because you've put everyone at a level five, level six, nobody knows what's happening. So congratulations. You now have your C suite, and nobody has a clue how to fix anything. This is a great time to talk about trust insights, manage IT and analytics services.

You know, you're correct. You know, if you are interested in something like that, then you can reach out and contact John, who is a level six statistician, uh, and can set you up. But no, I mean, it's everyone's gonna have a different opinion on this, and that's totally valid. The opinion that I have is that you can't, you can't responsibly skip levels to get to leadership. And that's true of your professional career, that's true of using AI.

You can skip those levels. I'm using the word responsibly purposely, because in order to do it in a sustainable way, in order to do it in a way that you're going to find success, you have to have those foundational skills. If you wake up tomorrow and say, I'm gonna be the CEO of my own company and I have no experience and I've never coded, but I'm gonna start vibe coding, you could absolutely do that. I have a story about that. It's coming up in an article in a couple of weeks.

Yes, Chris. No, no, I I completely agree because the the and this is this is very quickly approaching a rat hole is the word responsibly has less and less place in business every day. And you're right, and it just breaks my heart because that's it's the short-term thinking. Again, sort of the, you know, what we were just talking about of the short-term thinking of well, we can't do billable hours, how do we do value-based? We just turn out more.

But that doesn't mean that it's good or it's better, it's just faster. In terms of AI proficiency, then, given everything we've talked about in the last 40 minutes or so, if if a company says, I want to measure AI proficiency of my staff, where do they go? What's the next step for them? You know, they start. I would my opinion is they start with what is being asked of their staff today.

What are their KPIs? What are they being measured on for success? You know, because their level of proficiency likely isn't that they are solely responsible for bringing in all the revenue and growing the company. They have to figure out what their slice looks like, and therefore what the appropriate levels are for their particular roles. So if you have your customer sales team, customer support team, and you look at that and you go, you know what, they need to be at a level five, you know, uh operating agentic systems.

My first question is, why? Doing what? What are they doing with those systems? And so, like you really I would I would push back and say you really need to help me understand what the team is responsible for today, so that we can find those appropriate levels because it's not going to look the same for everything. Our levels in Trust Insights will not translate to a different consultancy because what we do is different.

My level of proficiency is always going to look different from John's level, is always going to look different from yours, Chris, because we do different things. Yep. And I think that's where I would say, in terms of next steps, if you wanted to use the AI proficiency things, use the framework, the idea of individual contributor manager and leader, and the increasing delegation of stuff to machines as the basis on maybe even on a role level at your organization. I would fire up a copy of Claude Cowork. I would take a job description for any given position.

I would take our framework and say, what are the likely expectations for the use of AI in this position? What are the three to five use cases based on this job description and the sample data that this position works with and uh and evaluate it and say, okay, a person who is proficient at the use of AI in this role would be expected to do to supervise or do these things with it. Um so if it's a content marketer, you'd be expected to, you know, do ideation and create content. If you're finance, you would expect it to be able to do reconciliation uh and auditing to say, yeah, AI will be a second set of eyes, or maybe even it's the first set of eyes and you're the second set of eyes on the books and things. And I think that would be a great way for an organization to practically take this our framework with those job descriptions and say, yeah, this is what we expect.

And your levels now are mapped to all of those use cases in increasing levels of sophistication for that role. And if you want a copy of that framework uh today, we will share it in our free Slack community. Uh, you can go to trustinsights.ai slash analytics for marketers, and coming soon, I will add it to my list of web development things I'm apparently doing now because that is my role. Uh, I will get it up as an instant insights and we'll have a proper URL for it. Not today, though.

Not today. We have other things to just say. Uh-huh. All right. Any final thoughts about measuring AI proficiency?

I mean, I think the biggest takeaway that I always encourage people to have is the only real comparison belongs with inside your team, with inside your organization. You know, you can look at your competitors, but you really have to start with what are we doing today? What do we actually need? Versus that aspirational, you know, I need to compare myself to a Chris Penn, which is what we're seeing in our Slack community. I don't think that's a fair comparison.

I don't even compare me to me. Well, now we're getting uh existential. Or you can buy your way out of this rat hole. Trust insights AI content. That's right.

On that note, I think that's it for this week. I think everyone. Thanks for watching today. Be sure to subscribe to our show wherever you're watching it. For more resources and to learn more, check out the Trust Insights Podcast at TrustInsights.ai slash TI podcast.

At our weekly email newsletter at TrustInsights.ai slash newsletter. Got questions about what you saw in today's episode? Join our free analytics for marketers Slack group at TrustInsights.ai slash analytics for marketers. See you next time.