Avoid Optimizing For Only One KPI

Feb 5, 2018

—

in analytics, Artificial Intelligence, Machine Learning, Marketing, Marketing automation, Marketing Technology, Metrics

Warning: this content is older than 365 days. It may be out of date and no longer relevant.

We love key performance indicators in marketing, because they’re often the yardstick by which our efforts are measured. We’ve discussed the difference between KPIs and metrics previously, as well as what marketing’s top KPIs are. Today, let’s explore a critical error many marketers make: optimizing for one KPI.

What is Optimizing for One KPI?

Optimizing for a single KPI is straightforward: all our efforts focus on boosting that one number, and it’s the only number we pay attention to.

In eCommerce, that might be completed shopping carts.
In B2B marketing, that might be leads generated.
In retail marketing, that might be foot traffic through the front door.

Focusing on a critical metric, a KPI, isn’t a bad thing. Focus is an essential part of measurement strategy, of deciding what to say no to. If a plan, tactic, or idea doesn’t align with our focus, we should say no to it (or not now, at least).

However, singular, exclusive optimization on one KPI is a bad idea. When we optimize for one KPI, we devote all our analysis, all our measurement, and all our efforts towards that one KPI only.

For example, at one company I worked with in the distant past, customer signups were the singular KPI that everyone in marketing and sales focused on. How many applications did we receive that day? Every activity was optimized to generate customer applications – and nothing else. Every analysis, every report began and ended with customer applications.

The Optimization Trap

When we optimize for only one KPI, we risk two dangers. The first is what Tom Webster calls the optimization trap. We progressively optimize for a smaller audience until we’re serving just a handful of customers. Mathematically, it works like this:

We run an A/B test and 60% of the customers choose A. A is the winner.
On the next test, we start with our results from A in the previous test and run another test, which we’ll call C/D. C wins in another 60/40 split.
On the third test, we start with D. Customers should be progressively happier each time, right? We run a third test, E/F, and E wins in another 60/40 split.

What’s the net effect?

optimization trap

We progressively ignore the preferences of greater portions of our audience every time we optimize. Our “winners” preferences dominate our audience and quickly no longer represent a majority of our audience.

The Attribution Trap

The second danger is the attribution trap. When we optimize for one KPI, we tend to focus solely on the KPI and the last touchpoint prior to that KPI. Last touch attribution is notoriously dangerous for marketers. By focusing only on the last thing that a customer did, we ignore the contributions of everything else that led the customer to that last thing.

For example, suppose we’re a coffee shop. A customer has these experiences:

Sees an advertisement for our shop
Talks to a friend about shops in the area
Searches on their mobile device for coffee near them
Sees a YouTube pre-roll video for our shop
Visits our shop

In last-touch attribution, we would give all the credit to YouTube, ignoring everything before; that’s the attribution trap.

We’re in greater danger than ever of the attribution trap because we have so much more data, so many more metrics we could focus on. Picking and choosing the metrics that are comprehensible or convenient is all too common in marketing organizations.

What Are the Alternatives to One KPI Optimization?

Instead of optimizing for one KPI, one outcome, we must instead use the best mathematics and tools to optimize for our KPI and everything in the value chain leading to it. Using techniques from machine learning, we have the computational power today to determine the influence and impact of every marketing activity in our portfolio, assigning weights to each and using those weights to prioritize.

Instead of binary choices like “should we do email marketing” or “should we post on Facebook”, we make more nuanced choices such as “let’s devote 5% of our time and effort to Facebook and 22% of our time to Twitter”.

By using multi-touch, integrated optimization, we avoid the attribution trap; we also avoid the optimization trap by no longer focusing just on a subset of our audience. We optimize for the behavior of every member of our audience, and we optimize the complete audience every time. Every new behavior adds to our model, adds to our understanding, rather than reducing our view of the audience.

By optimizing for the entire chain of value, taking into account all variables and all behaviors, we will produce more robust, better marketing.

Ask your marketing analytics software vendors what kind of optimization they do under the hood, and what percentage of behaviors they use in their models. The best vendors will provide robust explanations of how their technology works and whether it conforms to the best practices above.

Comments

2 responses to “Avoid Optimizing For Only One KPI”

joenunziante

February 5, 2018

Interesting and often overlooked topic. But, I think there’s a difference between what you optimize against (your KPI(s)), and how you apportion credit to the touchpoints leading up to a conversion. The former is a strategy question; the latter is more of a technology & methodology question. Even with multi-touch attribution (MTA), you still need to ‘point’ the analysis at a given KPI — you’d still need to identify the action it is that you want to begin splitting credit from…which brings us back to your original question.

I think it’s a topic worth more discussion: even with today’s MTA tools which are often implemented with a focus on splitting credit for one specific action (could be Adds to a Shopping Cart or Store Locator usage, etc.), we still need to think about whether or not the single KPI which we’re pointing our MTA tools at is adequate for properly understanding the real value of all the media we have in market.

To an extent, if you point your MTA platform at the most ‘complete’ action you can drive (e.g. a final purchase), your MTA platform should still be capable of properly determining the contributory value of all your media, including higher-funnel brand-oriented tactics (video, for example) because the MTA tool should be looking far enough back to understand if the sheer presence of any touchpoints (e.g. a video ad impression) did materially increase the likelihood for downstream conversion. It might’ve happened three weeks ago, but if it drove an increased likelihood to convert, we should see that come through via MTA and that video will get some credit.

That leads us to the next discussion topics of 1) whether we loosen the CPA thresholds for certain media tactics because we know a video ad will likely cost us more to drive a conversion (even if we are at a point where we can quantify the quantity of conversions it helped drive), and 2) what sort of secondary KPIs can we add to the mix that will help provide a clearer picture of performance and contribution…

…and THAT leads us back to your original premise — should we optimize against just one KPI?

Reply
1. Christopher S. Penn
  
  February 5, 2018
  
  The big thing is path integrity and path data. The more of it we have, the more successfully we can point ML and AI solutions at it. Most companies have woefully little path data, unfortunately.
  
  Reply