You Ask, I Answer: Bulk Analysis of Search Terms?

In today's episode, Bill asks, if you wanted to examine search terms in bulk, how would you do so? The goal would be to determine what terms are driving traffic to a website over time and learning how those terms and their volume change over time. Easiest and best way to do this is with the data that Google already gives you. Google in particular. Now, of course, you can use other search engines, Bing, etc.

Uh, anyone that's got a webmaster tools or a search console where they're providing that data to you is a good candidate. Google Search Console, though, has the ability to give you up to 16 months of back data every single term people use to find your website and to give you several measurements like the number of impressions, which means the number of times your site showed up in a in search for a particular term, the number of clicks that your site earned for that particular term, the click-through rate, um, and then your ranking position. And you can, of course, with that data, which is really nothing more than just a big big spreadsheet, uh, download it all and then chart it out, look at you know, terms uh rising and falling. Um, we do this a ton for our customers. Uh in fact, um I've built forecasts from this that data, uh, not just only from Search Console, but also from SEO tools, because one of the limitations of Search Console is it gives you your data, which you want, and it's very accurate, but it doesn't tell you competitively what happened overall.

Right. Now obviously if you're dealing with something like uh branded search, the number of people searching for trust insights, for example, my company, that um are not searching for us specifically, I don't really care about, right? I really care about how many times did you find my company uh by searching for us by name. But for other things like Google Analytics, data analytics, uh artificial intelligence, machine learning and marketing, et cetera, those are terms where there's broad interest in the field, in the industry. Uh, and we might not have a very big share of that.

You know, we might not have that slice of the pie that's big enough to really forecast with. So I would use uh search tools as well. And with third-party search tools, there you can't get a ton of historical data out of them, and you can get some, but in bulk, what I would suggest doing, and what we do is take our data, uh, the you know, the volume data snapshot, and then use Google Trends data to essentially correlate it in bulk and say, well, you know, if this term gets, you know, 200 visits uh a month right now, and we know that it has this relative strength in Google Trends, let's extrapolate backwards to figure out what the historical volume would be. And that is a very, very effective way to build past data based on what you know the snapshot of volume is right now. And that analysis then can let you not only look backwards and see how things have changed over time, but also forecast forwards.

It's the foundation of some of the predictive analytics that we do for content marketing. Take a basket of search terms, forecast it forward for six months, nine months, twelve months, and say, okay, when is a term like data analytics uh most likely to uh be searched for in the next 52 weeks? What weeks are is it will search be highest? And then you time your marketing campaigns with that because you know that if volume is going to be highest at those times people are going to be most interested and you should be doing a campaign to show up when people are most interested in the thing. So that's how I would do so.

Now to do that um search console's data export is tricky because you get slices based on what screen you're on in Search Console. If you want to get the raw data out the best way to do it was is with Google Search Console API. So you have a piece of code or a specialized tool that can ask the API say just give me the raw data for all these search terms and the volumes and pages and queries and all that stuff. And it returns that and then you parse that into whatever format you need it to be. That's how I do it because that way you can slice and dice with the existing exports out of Search Console, they're somewhat limited.

You can get the number of queries for example that your site showed up for and you can get the number of pages but in the native interface it doesn't give you the queries per page because it's a very very large table and it's very difficult to work with if you're not working in a a more technical environment but that would be a I would say a good starting place.ai slash search console you can take a course that we have developed it is a paid course on on how to get all the juice out of Google Search Console that you could possibly get um and learn how to use all of its features. So I would very strongly suggest you take that um course and then try all the exercises try all the data export features and all the analysis tools built into it and then if you um if if you're not clear about how still how to do this then that's when you start going to the more advanced stuff like API export but that's how that's how we do it just go straight to the API uh and pull the data out and then analyze it in a the third party tool of your choice and that way you get exactly what you want from the data. So good question