You Ask, I Answer: Incrementality Measurement in Marketing Analytics?

In today's episode, Kat asks, what's the most effective way to tackle incrementality for small businesses? If we assume incrementality in this particular instance means being able to track the impact of a campaign above and beyond business as usual, above and beyond what you would have gotten anyway. There are a couple of different approaches. The best approach for dealing with incrementality by far is simply making sure you've got really good tracking of all of your campaigns, that you've got things like UTM codes or CID codes or whatever, depending on your analytics systems, that can provide you very granular, very detailed, and very uh isolated um tracking for any campaign efforts. For example, if you've got a bus wrap, um the URL on that bus should be custom and it should be only used for that campaign.

If you've got a phone number and an ad, that phone number should have tracking and analytics uh assessed to it and it shouldn't be used anywhere else. That's where a lot of companies get into a lot of trouble when it comes to measuring incrementality. They have a tendency to reuse assets, and the moment that you use a tracking URL from one source in a different place, you're now conflating data. That means that you can no longer effectively judge are we getting the uh the leads or the traffic or whatever just from that one source. The moment you dilute it, you can't do that anymore.

So that is probably the most effective way, especially for small businesses, if where you don't necessarily have access to really big, uh really expensive uh attribution modeling software, being super disciplined about uh tracking and being very granular about it is important. Even if you do things like subdomains, for example. So let's say uh you have trust insights.ai. Maybe for a bus ad you do trustinsights.ai slash bus or bus.trust insights.ai, right? Subdomains and directories don't cost you any additional money.

It's not like buying a custom URL, where it's like you know, uh buswrap ad dot com or something, where you do have to spend some additional money. Using your existing domain with short, easy-to-remember URLs is the way to go uh for small businesses. Now, if you are a little more sophisticated when it comes to analytics, um you can do incrementality uh measurement uh with what's called uplift modeling, where you take into account the you know, sort of the the pulse of the data that you've already got, and then you assess what likely uh would have happened anyway, and then whatever the delta is between what would have happened anyway and the actuals of that day would be uh essentially the incrementality measurement. Now, there's a couple of different ways to get at that. One of the uh sort of gold standards is something called propensity score matching, where you take uh all of your data in a really big rectangular table, and for any given day that a campaign is running, uh, the software would go and try and find uh other attributes of other days in the last say year where conditions were very similar.

So let's say you had a special campaign going. Uh maybe you got a big story in the Wall Street Journal, and on the same day that happened, there was also an email marketing campaign going out, and there were some pay-per-click ads. The software would go through all of the other days of the year, find days that had similar traffic of pay-per-click ads, similar traffic of email, and then uh look at the statistical difference between the traffic you got on that campaign day versus those other days where you had every other activity happening except the big hit. And that's how you would sort of determine, okay, this is what was likely going to happen, um, and you know, sort of the uplift. So that's an example of uh uplift modeling using propensity score matching, which is a statistical technique.

The other way I've seen it done, and we've done this for some clients, and it is a little bit more speculative, it's not my favorite from a mathematical accuracy perspective, but if you've got a situation where there's a lot of noise and interference, um, and you've got uh campaigns that are just you know mixing up all the data together, you can use predictive analytics retroactively in some ways to uh take the data up to that point of the campaign, forecast it forward for the date that it would have occurred, and then again, do the delta between what actually happened and what the predictive forecast says would have happened had the campaign not been running. That technique assumes that you have strong cyclicality and strong seasonality in your data. So if you have a business where Monday through Friday your traffic is very predictable, Saturday through Sunday, it's you know the the days off, and you know that every Tuesday is a strong day for your business, and you can mathematically see that in your data, then predictive analytics would be a good choice to say, okay, well, we know what was likely to happen based on um the seasonality and the cyclicality in the existing data. So we can forecast for that period of time when we have this extra campaign, and then take the difference between the forecast and the actuals, and that would be the incrementality. That approach has trouble if your business is not highly cyclical and not highly seasonal, and it doesn't it can't disambiguate on multiple channels, right?

You can always see a gross difference or differences where the individual channels have a lot of cyclicality and seasonalities to them as well, like organic search. But again, there are situations where that can make sense. So that's how I would tackle incrementality. The number one best thing you can do is use lots and lots of discrete tracking so that you don't have to ask the question, hey, what actually happened, right? You know, where did all this extra traffic come from that we didn't know about?

By using tracking rigorously, you can substantially reduce the amount of extra guessing you have to do to figure out whether a campaign worked or not. So really good question, and it's a tough question to solve. You've got to have some stats background to do those more advanced approaches. It's not something that um you can just do off the cuff. You you you should try to learn those techniques uh and know what you're doing with them.

But if you do know them, they do work. Thanks for asking, and we'll talk to you soon.