You Ask, I Answer: Correlation of Visitors and Conversions by Visitor Type?

In today's episode, Andy asks, I received a request yesterday for a report that maps the correlation between the percentage of new and returning visitors on site and the number of conversions by day. My first thought was to go into Google Analytics and create a couple of different views showing new and returning visitors and conversions. What do you think? My first question is, what are we trying to prove here? Because visitors should correlate to conversions.

And so I don't know that this would be the best model to prove that, but you can do it. So here's the steps that I would take. First, uh you need to extract the data itself. So you need the number of new users by day, number of returning users by day, and the number of conversions by day. Be aware that mixing and matching different number types is a bad idea.

So instead of percentage of new and returning visitors, which isn't great, uh just use the absolute numbers. So we had 12 uh returning users and you know 44 uh new users, and then the number of conversions. Also, don't mix up data types, like don't have new users and sessions. Don't don't do that. Um, so that's step one.

Step two is get all that data out of Google Analytics and into some kind of system that can run correlations. And it has to be a system that can do different types of correlations. Generally speaking, when we talk about correlation, there's three different kinds of correlation that you can run. There's a Pearson correlation, a Spearman correlation, and a Kendall Tower correlation. And if you never heard these terms before, then you are just like me and you slept through statistics class in college, right?

Which I did. I did terribly in that class, which is ironic. I had to relearn it all later on in life. Each statistical test of correlation fits a different type of data. So Pearson correlation, which is generally what is most used in tools like Excel and stuff like that, when you type in like the correlation function in Google Sheets or in Tableau, it's probably using Pearson out of the box.

Pearson correlations are good if your data, when you plot it out, looks like a bell curve, right? Um for normal distributions, Pearson correlations are the best tool for the job. That is not most marketing data at all. If you were to take your marketing data and reorder it by you know largest to smallest, most marketing data is a power law distribution, you know, a Pareto curve. Uh 80% of your traffic comes from 20% of your of your days and so on and so forth, you know, uh long tail, you hear you'll hear that term a lot.

When you have data that is doesn't fit a normal distribution, which is called non parametric, um, spearman correlations are the best to use. They're the best technique to use. Uh instead of a P value, you get an R value. Um, and then the third one, Kendall Tau is best for ordinal data. So if you have two lists of ranked data, um, you would use Kendall Tau for that.

So get all of your data out, get it into a spreadsheet, make sure it is all ordered by day, and then run uh a spearman correlation for each. One of the things you're going to find probably is um you'll find that one of those two metrics has a stronger correlation. Um with spearman correlations, they are they're just like Pearson correlations in the sense that you know anything above 0.5 is a strong correlation, anything below uh 0.5 but above 0.25 is a weak correlation. Anything below 0.25 to zero is no correlation, and the same is true on the negative side, so you can have negative correlations as well. The question that I would ask though is so what?

Right? So if you find out more, you know, new users has a greater correlation to conversions than returning users. Okay, so what is the person who's requesting this? What are they going to do with that information? I suppose if it's new users, then you look at the channels that drive new users and say, okay, let's invest more in the channels that are driving the most new users.

Same is true for returning users. But fundamentally, I would question whether that data type is as relevant as you know the channels, the individual channels. So one thing you might want to take a look at is look at new and returning users by your more most popular channels, like search or email or social media and so on and so forth, because that might lend more granular results that you can uh save a step or two in terms of the level of analysis. That's not to say that it's not the analysis is not without value. It's the number one thing we always want to ask a stakeholder when they come to us with a very specific mathematical test is okay, what are you going to do with the information?

Right? What decisions will you make? If they're just going on a fishing trip, you might say, okay, that that's cool. And you know, still happy to run the analysis for you, but have you ever thought about what your next step is? Because most people don't.

Most people don't think about the next step. And as a result, because they don't think about the next step, they don't have a clear picture in their heads of what it is that they would do next. And that's where all the value is in analytics. Uh a lot of the time, analytics by itself is rearview mirror, right? It's what happened, it's looking backwards.

That's of limited value when you're driving. There is value in looking in the rearview mirror every now and again, but unless you're Mario Andretti, and then you just never look in the rearview mirror. Um, but if you're trying to make decisions going forward, you have to have some sense of where this person wants to go with this thing. And if they don't know, that is the opportunity for you as an analyst to add value. That's an opportunity for you to say, okay, have you also thought about this?

You know, channel level, uh, new and returning users. Um, and then help them run the test and say, okay, here's the conclusion that we reached that X or Y is a better choice uh for getting more of the result that you care about. So give that some thought. But it's a very interesting question, and I I think the analysis is worth doing. Um, let's see what uh you come up with.

Thanks for asking. If you like this video, go ahead and hit that subscribe button.