So What? How to Analyze AI Visibility vs Human Search

Well, hey everyone, happy Thursday. Welcome to So What the Marketing Analytics and Insights Live Show. I am Katie, joined by Chris, John, and Georgia. How's it going? Last the past couple weeks, we've been talking about your bot traffic.

So large language models like Gemini, like Claude, like Chat GPT, they are sending traffic to your website. They are sending traffic to your website because someone is putting into those systems, hey, tell me about this, or I want to know this. And so their AI assistance, their AI search is hitting your website. And so last week, we actually ran through how to set up Google Analytics for looking specifically at that traffic. And so you can catch the replay of that on our YouTube channel, the So What playlist, TrustInsights.ai slash YouTube.

So if you want to learn more about what specifically goes into it, you can go to our landing page of TrustInsights.ai slash AI bot analytics. The big takeaway is that it's not just setting up Google Analytics 4. You have to have Cloudflare as setting up a server, big query, so and so forth. So definitely check out the episode to learn more about that. Today we're going to be focusing on because one of the questions we came up with was: how do we look at AI visibility versus human search?

So right now, a lot of large language models are sitting as the intermediary between the human and the end result, which is why we're seeing a lot of the bot traffic. So it brings up the question of how much of your site and your content need to be prepped for AI visibility so that the machine can read it and send to the human. And so, do you optimize for the machine? Do you optimize for the human? Chris, where do we start?

We start waving Kaida Georgia, who's having a grand old time getting ready for uh the live stream here. Yeah, she's trying to get comfy, she doesn't know what she wants today. That's fine. The first thing we have to figure out is, and this actually came up this past week in conversation with um some folks we were talking about. This is what data is even trustworthy out there.

And the bottom line is this if the data is not coming from an AI company or search company, it's not trustworthy. So if you are looking at a third party outside of Google or Microsoft for search, you're probably getting data that at best is estimated. Um, you know, because open AI doesn't give away any data at all. Anthropic does not give away any data at all. Um we've said in the past anyone who claims that they know what people are typing into chat GPT, they're lying.

Absolutely 100% lying because that data is not available. Um, so what data is available depends on your website. Um some accounts on Google Search Console have the generative AI feature as well as regular search performance. If you have that data, that is a great way to do some apples to apples comparison to see what the data is between what AI mode and AI overviews are delivering versus what regular Google Search Console is delivering. Unfortunately, for today's live stream, none of the accounts that we have access to have that feature enabled yet.

If you actually go through Google Search Console, you will see like four different versions of the UI based on what you know what version that company is on, and you have no control over it. Believe me, I would be petitioning Google for uh to switch over to the new one if I if I had any control over it. However, the one source that we do have apples to apples for is Microsoft Bing. So before we get into that, Chris, I'm actually surprised. I figured, you know, obviously Google doesn't tell us how they're rolling these things out, but you know, there is sort of this notion of maybe it's by site size or something.

You don't have it on your personal site, because your site is a lot bigger than the Trust Insight site. No, in fact, only one of our clients has it, and then everybody else, including some sites that are bigger than that client by like an order of magnitude, do not. I was in our Search Console the other day, and I was like, oh, that's interesting. Like literally, it there's there's no rhyme or reason to it whatsoever. All right, just curious.

Yeah, I I don't I don't understand either. Um so let me show you what you what you can see in Bing Webmaster Tools, uh, because that is a good apples to apples. So there's two different tabs we're gonna pay attention to. One is the human tab, this is the search performance tab. Really important.

Make sure when anytime you're doing any of this data analysis that you do an apples to apples for the same time period. Um so if you look at six months, this is my personal website, six months um looking back, 420 clicks, 47,000 impressions. This and and you get the keywords, the pages, all the stuff that SEO folks have known forever and ever, right? So this is this is pretty stock. The the tab you're going to pay attention to really is the pages tab because that tells you what pages um in in regular search people are landing on.

That's going to be important. Um the second tab is the AI performance tab. This is set the same period time period, six months. These are the grounding queries. So these are the terms that co-pilot searched Bing for.

Uh so when it was writing queries, it would write the it would write this, and then it shows you the number of times uh your your website came up in the citation and what percentage of the results were from your website, which itself is kind of an interesting number because when you those numbers are high, it means that you potentially multiple pages from your site are showing up as the answer in Copilot, which is is a useful thing to know. And then the second is what pages themselves are are being returned in Copilot. So these two things, there's some overlap. The overlaps specifically are in the URLs, and that's where we can get some useful insight to say, okay, what if we have the URLs of our website? Which ones are the humans getting to, and which ones are the agents getting to, and and what's the difference?

Okay. I'm with you so far. Okay. So what you would need to do, and I have this pre-baked because it literally takes 11 hours to go and do this analysis. Um that sounds like a very dull live stream.

I don't think John and I have enough banter for 11 hours. No, definitely not. Um what you want to do is you want to go through and you'll notice here this download little download widths. You want to download all of your AI data first as CSV files, and then go likewise and download all of your human data as CSV files. And what you'll do is you'll you'll um depending on the system you're using, I would strongly recommend using a coding tool.

So clawed code, open code, open AI codex, Google anti-gravity. Uh, you you really, really, really want to use a coding tool for this. Put all that somewhere. Um, I have a data folder, and I have my human files for the human side of search, and I have my AI files for the AI side of search. And we just want to uh be sure to take a peek at them, make sure that they're if you're doing this with Google files, you're gonna have to do some editing because Google loves to cram five lines of comments at the top, which screws up every single importer.

Thanks, Google. Oh next, we have to think about how are we gonna do this analysis? What techniques should we use? Now you could sit here and and ruminate about them. You could have a conversation with AI about it, or you could have AI go out and do a search, take a look at the data and do a search.

Where in our case, because we pre-baked the search, um, I have a catalog here of over 1100, actually 1400 analytics techniques. Uh so these analytics techniques include lovely exciting things like cross-cutting uh analysis, time series and temporal analysis, network graphs, and relational databases, spatial and geostatistical data, and then from each of those categories, we can take something like let's take a look at causal techniques. In this huge this this analytics calculation, this is just one on causal 16,000 words of analytical techniques for establishing causation. So you'd have you give your data to your AI tool and say, take a look at the data, and then take a look at either the analytics catalog or do your web search and tell me what techniques would best fit this data, because there's a bunch that won't apply, but there's some that might apply that are outside the domain of marketing. One of the biggest risks you run into when using uh AI tools is if we say, hey, I'm doing website analysis, that limits AI's thinking to what are the ways that other people have done website analysis based on its training data.

And those algorithms may not make sense, or they may not lend any insight because A, everyone's done them, and B, they may not work with the shape of the data you've got. So for example, causal inference. We've talked in the past in the shows in the past about Granger causality. Granger causality is a very useful tool. It does not come from marketing.

If you give AI a marketing prompt, it will not think of Granger causality. It will just say I'm going to do something like audience modeling. And they're like, eh, that's not helpful. So you want to start by having AI do some exploration to say, what could we use from any domain to solve this problem? Maybe I'm oversimplifying it in my brain.

But you know, when I think of comparing what humans are searching for versus what uh machines are searching for, I'm thinking of like a V lookup table. And so, and you know, I think perhaps you know, you've obviously exported a heck of a lot more data than I was considering. I was thinking, you know, if we have the search terms from the regular human users and the search terms from the AI users, couldn't you just do a V lookup or something like that to say, like, here's the unique ones to the human, here's the unique ones to the AI, here's the overlap? Like, am I oversimplifying this? Yes.

Um that's part of it. That's that's an important part. Like, you do want to know what's uniquely human, what's uniquely AI, what's the intersection? That is absolutely important. But there's so much more data that we already have access to or that we could engineer out of this data set that could lend some insight.

So, for example, what if we were to take the contents of every one of those pages and do some quick topic modeling on it to say what is this page about? Right? What kind of page is it? Is it a blog post? Is it a podcast?

Is it a newsletter? And those categories might be important to say, does AI cite this versus that more? Um how long is it? Yes. Well, I was gonna say, as you're describing this, it strikes me as it's a good opportunity to pause and bring up the 5P framework from Trust Insights, uh, which you can get at trustinsights.ai/slash 5p hyphen framework.

So, what you're describing, Chris, is defining the purpose. Why are we looking at this data in the first place? So I had described a very simplistic use case. You're talking about we have all of these pieces of data that we could be looking at. And so before someone even starts this exercise, they should likely, uh, as you're showing on the screen, come up with a purpose of the purpose of this inquiry is to understand the difference between human search and AI search and so on and so forth.

And that's going to help you, as the person executing this exercise, figure out from your platform which data sets you want. So your purpose, as Chris has explained it here, the people, who's looking at this data? Who cares? Who's executing it? Chris is looking at it.

I'm the one who cares, and John's the one who has to go do something with it once I figure out what the service is. Process, how are we gathering this data? How are we analyzing the data? This is what Chris is going to go through. Platform, we're using Bing Webmaster tools.

We're using, I think, Claude Code, and then performance. Did we answer the question? So I just wanted to highlight that that it's a really good opportunity to bake in the five Ps into the work that you're doing. And unsurprisingly, Chris, you've already done that. I have done that.

In fact, I was going to say this recipe that I fed to Claude Code is the 5P framework by Trust Insights because it's the best way to do any kind of serious AI planning because it forces you, the human, to think this through. For example, the in performance, I have on screen here. And what concrete next steps I should take to improve the likelihood that I can attract traffic from both humans and machines. So we've talked many many times on the show about having your purpose is important and having a clear definition of success is equally important because AI needs both of them. Right.

Um, you know, so if you want again, past episodes, you can go to trustinsights.aslash YouTube, go to our so what playlist. Um, definition of done is just as important as what the heck you're doing because to Chris's point, your AI as a very helpful assistant is going to go on forever unless you stop it. And that takes up usage, that costs you money, that's resources, it's a whole list of reasons why you want to declare what done means. Exactly. So, what this is, this is me thinking out loud in a voice transcription tool.

What I wanted to do. I said I've got my data, I've got the I told it what the data is. I don't let it rely on that alone and my ex my point of view on it. Uh, I explain what the different data sources are. I explain uh how I want to think about the process, right?

I want you to set up a database, I want you to set up a web crawler. I put up uh I told it how to get past Cloudflare on my own website. I set up a special thing that basically allows uh uh an AI tool to uh not be told by Cloudflare to go away. Um how I want to process the data, what kinds of algorithms I think it should use, like Yake, which is yet another keyword extraction, is a very well known algorithm in the in the SEO community and the as well as the natural language processing community. Uh, I told it how to read the different data, how to come up with topics, what additional features should we engineer?

Uh, and I've I've given it some ideas some ideas about how to do that. Uh, I've told it how to read the analytics catalog of the 1400 different techniques for analytics to use. I've said, hey, you might want to use pausal treatments. You might want to use regression or classification. Uh recapped the five Ps and then said, hey, here's the other things you're allowed to do.

And oh, by the way, you are allowed to ask me up to 15 questions in multiple rounds until you have enough information to complete the task successfully. So even with this almost 900-word prompt, I've still said you can ask me questions and you can search the web. And it did. We went back and forth about 15 minutes last night when I was doing the prep for this, asking and answering questions about what about this? What about this?

I see this in the data. What about this? What Claude went off and did then was read that and build essentially a very, very, very large app that processed the data and then ultimately spit out uh some some conclusions about what it saw in the data. But this process, the important part of this is you have to be willing to have a conversation with the data and with your AI tool to say, well, here's the thing, let's think about this. Don't just give me the answer.

The worst thing in the world to do would be to say, uh, just tell me the answer quickly. No, absolutely not. Be curious, be thoughtful. Like, what can we do with this data? Uh, Brian, our friend Brian has a quick question.

Are your prompts typically all voice to text, or do you start there and have Claude help flesh out the rest of the prompt? Um, I always do the first draft with voice to text for myself, um, using the 5P framework by Trust Insights because it helps me think things through. And then I have Claude say, Cool, I got questions. Um, I and even after that, what I will uh uh what I do in Claude Code, which is is really nice, is I can say to Claude Code, now spin up a sub agent that uses Fable 5, because we got it for another three days. Uh that spin up a sub agent that uses Fable 5 to review our plan and then implement it.

And then one thing I probably forgot to mention is that I have a super long-ass prompt that says before you go and just run off and try and do things, I want you to think through like what's the requirements document for this? What's the uh technical spec and so on and so forth, so that we get it right, uh, and we don't end up re doing rework over and over again. Now for a live stream, maybe that maybe that's overkill, but it's a good habit to be in. I you know, for the live stream, you can't demo all of it, but we can acknowledge that it was something that we did. Um, because it is, you know, that muscle memory, you want to just always default to this is the best practice process.

Um, I sometimes do voice to text. I sometimes type it in, but I always go with the five Ps we have found it's the best way to make sure you're thinking something through from a 360 lens, not just I want to build an app, but how are people using the app? Is there privacy? Is there security? So on and so forth.

So, you know, I think to Chris's point, the prompt is a good starting place, but it shouldn't be one prompt, one answer done, move on. Absolutely not, never. So the the answer to your question, Katie, what do we come up with? Uh by the way, if you are a Google Workspace user, Google has a command line tool called the Google Workspace Command Line Interface, GWS. Claude, can you pick up that tool and use it and create Google Docs straight in the coding session?

And they they all come out very nice. So this is one that I used with the GWS client with the Trust Insights uh slide deck uh skill that Katie created. And let's take a look at what we got. So which pages appear in which both channels, 128 pages appear in both. 53 URLs appear only for AI, and 170 appear only in the human version.

This is from Big Webmaster Tools. So there is a difference between you know AI sees uh about a quarter of the data set that is not in the human data set. Hmm. Um this is interesting in terms of attention. How much more does AI hit uh certain pages than humans?

AI spends 1.8 times more um on the pages that it's it just keeps revisiting. So the the it hits fewer pages than humans do, but it comes back to them more. It keeps coming back to them over and over again. So they it it concentrates attention uh on them, and this was done with bootstrap resampling. Um a page's topic makes the difference as to is to whether it gets seen or not.

So the intent look such as informational or commercial, uh, doesn't make much of a difference. The content section, podcast, newsletter, doesn't make a difference at all. It is all about the topics that determine whether humans or AI see a page or not. So when you say topic, if you know, let's we have a bunch of pages on our website. Like, how does this analysis determine the topic?

Is it pulling it? Is it looking at the entire body of content and saying the topic of this is, or is it looking at the title and saying the topic of this is? Like, how does it know the topic? So part of the way that uh what happened in the in the behind the scenes is uh I built a website scraper that grabbed the entire post, used Travelotura to extract the main content, and then fed each post a haiku, the lightweight clawed model, and said exactly what you just said, Katie. Return the top three topics for this page.

Okay. And then how about the intent then? So is that same deal? Do you have it boiled that down to one idea or yes, exactly? We um that's exactly what it is.

Cool. And then in terms of our attention concentration, this is interesting in that um the way both human and AI search works is a power law curve. Uh, a few pages get all the attention, and then everything else is like onesie twosies along it. So, in terms of our optimization strategy, when we we start thinking about how do we optimize for AI search and human search, it's it's short head long tail, and the short head is where all the the juice is. So, one of the the prescriptive things from these reports that you should be doing is look at those top five pages and figure out what else you can do with them to either direct users elsewhere on the on your site or add more content to make them even more appealing.

But it is very much a case where the the short head gets all of the love and nothing and very little else does. And this is our good friend Jenny Dietrich is going to be so excited that she made made it on screen. Her coefficient. She's like, oh yeah, I totally did that. Downside is this one is not pronounced the way she pronounces her name.

Fair enough. Is it Gene? Or what is it? How do you say it? Is Genie, yeah, the Genie coefficient.

Oh, yeah. Shoot that in that case, she would not care for it. Yeah, yeah, no, exactly. Um get her a bottle. This is more, this is really interesting.

So this is a non-parametric correlation uh using both Kendall Town and Spearman. Uh fundamentally, we're asking the question do pages or queries matter more in terms of of rankings, right? Do humans AI humans rank the same thing similarly. And the answer to this is that AI and humans give more uh have more overlap on the search terms that they use that rather than the pages that they go to. So if we say we're trying to understand the AI visibility space, is there a is there a big difference between the way AI and humans search, or is there a big difference between the pages they end up on?

And the answer is there's more difference on the pages side than on the search side. The search side, this is very strange and kind of interesting. There's more agreement in how machines and humans search, at least from the Microsoft results, because we don't have this for Google because of the limitations I mentioned earlier. But on the Microsoft side of things, the way humans' searches work in Bing, and the way co-pilot searches are more similar than not. Hmm.

This is, I mean, this is a lot to wrap my head around. So I'm not gonna have a lot of deep insights because I'm like, what? Cool. Well, so what I would what I would interpret this to mean is that if you know what people are searching for, this the search queries, um, you're gonna have to see as much of a difference between AI and human, which means that you can then take that topic list of all the things people are searching for from the grounding queries and use that as a way to say, okay, have I answered all these questions, right? This is classic search where you take the the queries and you have the impressions and the clicks and say, What do I get impressions for?

I don't get clicks for clearly I was not the answer because nobody clicked on it. And say, Do I have content to answer these questions that I and if I don't, I need to create some. Um, our friend Olga asks, and I'm assuming this is to what you were saying before I asked, is that because it learned from search? Very likely, very likely co-pilot learned from the Bing catalog. Which would make sense.

For a long time, and we've said this on other shows, people sleep on Bing webmaster tools. Like it was something that was so deprioritized, and now because Microsoft is so integrated into organizations. So, you know, if you're using Outlook, if you're using Excel or Word, you're likely a Microsoft shop, which is why you're using Copilot. Joke's on us. If we weren't integrating Bing webmaster tools, we have missed out on a lot of really rich data.

Exactly. Now, this is interesting. Um, where for eight out of the nine topics that it it analysed my site for, uh, it came out with the same location. So basically, human and AI end up uh in the same broad categories. The one topic where there was a split is um on the LinkedIn algorithm.

Uh, what human searches went to one place, uh, link uh and AI searches went to a different place, and that is the one topic where there's a split. Otherwise, in general, for at a topic level, the machines and the humans end up in a similar place. And this is important because a lot of people are spending a lot of time right now going, oh, should I spin up you know a million pages for for AI tools to consume and make my website AI site? No. Build a site that that everybody wants because what we're seeing here is at least if in my results, um, and this is an NM1.

But what we're seeing in my results is it's more is does the content exist to answer the questions, and and which is not a new call it a golden rule. That's the way you've should have always been approaching building your content. Now, from my site, different topics get different kinds of attention, right? So I have a whole thing on because I have a whole section I did on um like business psychology, business astrology. Um, so things like MBT Myers Briggs and stuff like that, and that content gets a lot of attention from AI.

Stats and probability content gets a lot of attention from AI. Business analytics and KPI gets a lot of attention from AI's um personality types and tests gets attention from humans. Career stuff gets attention from humans, AI tools and platforms. But again, this mirrors what we know about these audiences. Going back to the the second P in the five Ps.

People are looking for stuff that people care about, right? Technology stuff is uh are things that are more granular, that a machine is more likely to query in a in a very detailed way rather than a human. That makes sense. It's interesting that it respects the power curve. That kind of throws me.

Like I would have expected the models to want to eat up everything. Do you think that the idea there is they just want to optimize spidering traffic? They're just saying, hey, we want to search like a human would, and we don't consider kind of just random data on the outliers worth even scanning, or what's the take? Um especially for Google, but also for all the AI tools, they all use some variation of what's called query fan out, where a model takes your your inquiry, your search, whatever, and basically does a whole bunch of searches on its own. Those searches are all going to be related to each other.

And so because all the searches are all related to each other, they're likely to return the same uh uh you know a similar cluster of pieces of information. So if somebody types in, oh you know, hey, co-pilot, uh recommend me some marketing podcasts to listen to. It's going to say best marketing podcast for small business, best marketing podcast for this, top marketing podcast in 2026, and so on and so forth. And marketing over coffee is going to be in there a lot, but it's probably going to be the home page. It's not going to have to dig pretty very deep because it's probably going to surface in that query fan out.

And all 80 of those queries, marketing over coffee homepage, the marketing over coffee about page, and the marketing over coffee episodes page, right? Because because those are the ones that there's that's a shallow result because the user's intent was I want, you know, I want to know specifically what Seth Godin said about marketing in 2007 on on podcasts, right? That would lead to one specific page, but that's generally not what people are asking, and that's not what triggers a search. What people ask is something like I need some new podcasts to listen to, and so it's going to come up with those very uh shallow hits, and that's how you get your power law curve, right? Yeah, yeah.

It's because it's the based on human hit. Right, it's based, it's based on fulfilling human's intent. Um thing that's interesting, there is no time lag. So uh AI citations and human searches march in lockstep, which means that what this tells me is kind of interesting is that humans using regular search and humans using AI search are start are basically asking the same things. And we've actually had this experience.

You've probably had this experience where you've talked to an AI tool, you're like, this is so freaking stupid. Why I'm just going regular search now. Just turn off the close the chat button, I'm just going to regular search and and you find your answer that way. We know from just our own experiences as humans, you may have a conversation with your AI tool and then go and Google it uh afterwards because you want to know something more specific or something that the machine didn't surface. So there is no time lag.

What this tells us also is that again that you you really aren't optimizing for humans and machines differently. You're optimizing for the same audience in the same frame of time. Okay. Um, there is no causality on that timing, right? Neither one comes uh meets the the standard there, so there's no there's no one creating the other.

Um there is from my site, there is strong season uh cyclicality for day of week. Uh the days of the week, then when people are searching that end up on my site, human or machine are all weekdays. Um, and then not nearly as much on the weekends. What is interesting here is that you see the machine stuff taking slightly uh uh greater share of Monday through Thursday and human-led stuff taking share of Friday, Saturday, Sunday. I don't know why, but that's the way it splits out, but it's not a huge difference.

That's helpful to know though, because you know, we try we tend to think of machines as they're always on 24-7, but to be able to see that, well, you know, they're always on 24/7, but this is a nice reminder that there is a human on the other side of the machine conducting the search, which you know, unscientifically, I can say that's likely why the day of week seasonality is lining up because I'm the human looking for things, sending out my AI assistant to find the thing. Exactly. Yeah, the machines are not autonomous, they are they're not your virtual chief of staff working around the clock, they are just a research tool that you are that you yourself are powering. Um there is no significant lead or lag for AI citations and human search, right? Uh again, that's the the timing.

There is no timing thing there. Um what was interesting is that for my site, humans end up and consume more of my newsletter by a significant amount and step by step tutorials and thought leadership essays, whereas machines consume more interactive QA responses, rants, um, and educational explainers. So different content types machines get to uh differently, but again, so it's I the only one there actually are a couple surprises here. The interactive QA, we know this, right? We know pretty clearly for a lot of sites, particularly sites that are doing a good job with their their general content that yeah, QA stuff is what how machines read content to understand what uh what's going on and understand the intent of a page.

Whereas a human, hopefully, someone enjoys my newsletter enough that they actually stick around to read it. And and that's what we would we would see here. And finally, are there any uh reliable traits that predict whether AI will favor page? No, there are not. Um we we did all the feature engineering, but the topics, about the types of uh content in here, and there's nothing here that is statistically significant.

This used uh elastic net logistic regression. There are things that are slightly in favor of one or the other, but there it's there's nothing we can say, yes, that is the smoking gun. We we should do more of this. There isn't uh in the data. So the skill the put our ad in my own slide.

It's like thank you. Thank you by our stuff. Um, so that's what the end result was for my site. Now, and I want to re uh reiterate that that. That was an N of one.

That was my site, and what I would encourage folks to do is take the process we used using the 5P framework by Trust Insights and do this for your own data because it's gonna be different for every site. So the executive summary for your site is essentially, if I followed correctly, you don't need to do mostly generally, you don't need to do anything different when you're breaking it down between human and machine, because it looks like for the most part, they're consuming things roughly the same way. Where they diverge was on those like more discrete topics, such as careers, personality tests versus you know, business statistics. So it really depends on what you want your site to be known for and how you want to cater to your audience. But overall, you don't have to change timing.

You really don't have to change how you approach your content. But it strikes me that you should still be following things like the hero hub help and you know, uh entertain, engage, and educate in those frameworks as you're approaching creating content because AI and humans are kind of consuming it the same way. Yeah. In in Claude's console, it said, hey, do you want if you want more attention from machines, do more analytically flavored, evidence-based, you know, studies basically. Um, which we were actually having a chat with one of our friends this week about that, and saying if you want machines to come by more often, do more stuff like that.

If you want humans to come by more often, do more of the things like uh the you know, the the super long newsletter issues and stuff that that seem to attract people rather than machines, at least on on an upper page uh type basis. Um Brian, our friend Brian says: fantastic use of search console data do you export directly from gsc or go through a look or report i have it i've had issues getting all the data for a large site from gsc so this was bing webmaster tools not google search console however with google search console yes there's um the thing that you want to do you do not want to use the web interface google search console has an API um and if you get a key uh your your api key from Google you can wire up code that will extract that data programmatically for large sites we do this for one of our clients um and every month the spreadsheet that it exports is 500 megabytes of data um which if we think about in text that is a gigantic spreadsheet that is like 18 million rows of data and and the reason it's so big is because every row is something different device type URL the number of clicks the search query and stuff like that so you get a very very fine granular data dump um it is the best source of Google Search Console data because it puts the query and the URL on the same line whereas the web interface does not do that. So you can do some really really cool analysis but you have to go through the API for that. The good news is the API's limits rate limits are very very generous. What I would suggest doing is using a coding tool like Codex or anti-gravity or clawed code give it your API key and say here's the site I want you to extract and just get me everything.

Just whatever whatever's available in the API, give it the documentation, say just if it's there, download it and then stuff it. Stuff it in a SQLite database locally and have it come up with a combination primary key. That is the uh query, URL, clicks and impressions as your primary key. That uh that will and the date uh that will help create uh an index across the database so that you can load data to it, new data to it without blowing up the existing database. You want to make it uh the key word to give your coding tool is the word idempotent.

And if none of that made sense, you can reach out to trustinsights.ai slash contact and talk to John, who will try to make some sense of that. But you know, in a that guy, in a nutshell, it's work that can be done. It's an analysis that you can do. Number one, set up Bing Webmaster Tools. Uh, take a look at your Google Search Console to see if it has that AI search component in it.

It may or may not. We have no control over when that happens. And then export both the human search and the AI search, and then follow along of what we talked through in this particular episode of the steps that you should take. Yep. Um use the 5P framework.

Use the 5P framework. Because trust insights.ai slash 5P framework. Uh you'll live it, you'll love it, you'll laugh about it. It'll be great. Um, now here's one other caution.

You might not have enough data to work with. Um, when I went to go do this for marketing over coffee, marketing over coffee for one reason or another has not gotten a lot of data in Bing specifically. Uh, I think that may have been because we've been laughing at it for 20 years. Somebody over there may have heard about it. Exactly.

So there was there wasn't enough data to work with there. Um, so when Google Search Console gives AI uh visibility data for marketing over coffee in Search Console, we'd be able to do it there. Same for Trust Insights. But uh, but yeah, that's that's what we found. Again, make sure you're using apples to apples data.

Be careful to make sure that the the data is apples to apples for time frame and things like that. Don't mix systems. You cannot use Google Search Console data with Bing Webmaster Tools data. They are different systems, they measure different things. Um, in the same way that you shouldn't conflate TikTok and Instagram.

They're they're totally different systems and totally different signals. Um, don't do that. Any final thoughts, comments, questions, barbecue recipes. I think the big thing, the big takeaway is that people are stressing about their traffic going to AI. What should we do about it?

What's our strategy? Our you know, our organic searches going down. Is it because of AI? Just create good content. Period.

Create useful content, follow the, you know, EEAT, follow Hero Hub Help, follow, you know, the three E's. Like find a framework you like and follow it, but just create good, useful content that's not just pontification. Or maybe it is, maybe that's your thing. Maybe that's your thing. But then don't get mad if you know the bots don't find it helpful.

Right. You have all these dead ends. Because that is, you know, over the past couple weeks I've been looking at all this stuff as far as hey, the volume is 10x 100x. But the punchline this week is you don't have to change the map. Like you don't need to be doing anything differently as far as your what the strategy of what your website should be.

One other thing that's worth pointing out uh in this is this you can't measure, at least not to the same level, um, what's going on on third-party platforms. So if your podcast is hosted on Libsyn, for example, and and you don't have a website there, you're not gonna be able to do this kind of analysis. If your newsletter is hosted on Substack, um, and you you can't install Webmaster Tools for your Substack, you can't do this analysis there either. So we've said for a while, don't build on rented land. Now you're starting to see why.

Um, other things, uh you as you saw, it's very difficult to engineer features out of the limited data that you're given. So for years, people said, Oh, don't put the date in your URL of your blog post, it'll date them and make them look old. Like, and okay, I get that. But then when you go to do data analysis, and the only thing you have is a URL, you no longer can do that without having to keep a separate lookup table of when you published what. If it's in the URL, it's a lot easier.

It's true. All right, folks, that's gonna do it for this week. Well, uh, stay tuned for I don't know what we're doing next week, but I'm sure it will be fun. Thanks for tuning in. We'll talk to you on the next one.

Thanks for watching today. Be sure to subscribe to our show wherever you're watching it. For more resources and to learn more. Check out the Trust Insights Podcast at TrustInsights.ai slash T I podcast at our weekly email newsletter at TrustInsights.ai slash newsletter. Got questions about what you saw in today's episode?

Join our free Analytics for Marketers Slack group at TrustInsights.ai slash analytics for marketers. See you next time.