Calculating Non-Response Rate (RR1) in Google Analytics

Tom Webster recently wrote a terrific piece on boiler room-style content marketing. He was 100% right about everything until the last paragraph.

“And here is what I think these content “boiler rooms” don’t consider fully: the non-response bias of the people this content doesn’t convert. … Ultimately what I am getting at, as I often do, is the limits of clickstream data for things like this. Boiler Room content is very easy to quantify if you look at leads per post. But damage per post? Google Analytics won’t tell you that.”

Actually, Google Analytics can tell you that – sort of. Google Analytics + some top of funnel data (which you can inject through Measurement Protocol) can yield a reasonable approximation of what, in Tom’s world, would be a non-response rate. Then a firm like Edison Research could go research, survey, and determine that crappy content marketing was the cause of the non-response rate, which is the non-response bias.

I’ll reinforce what Tom said, that the clickstream (digital marketing data) has limitations. We can’t see people who swear aloud to their coworkers about yet another terrible blog post from a terrible marketing company. What can we see, however?

We’ll use the AAPOR RR1 definition as our starting point to model our content marketing response rate and non-response rate. Feel free to download the full Excel spreadsheet from AAPOR to see the whole model.

Consider the official pieces of RR1:

I/(I+P) + (R+NC+O) + (UH+UO)

This formula is completed telephone or mail interviews – people who did what you wanted them to do – divided by everyone else.

Consider what data our digital marketing tools deliver to us. Social media, earned media, email marketing, search marketing, and advertising data can provide a reasonably complete analogue of a respondent pool. We have the total number of people who have seen our content or an invitation to view our content, such as:

• Social media post impressions
• Email messages delivered
• Display ad viewability
• Appearances in organic search
• Search ads displayed
• Earned media coverage/appearances
• Native content ad displayed
• Storefront passersby (via a Placemeter or foot traffic counter)

While not in Google Analytics natively, we can connect to most of these data sources via API and push the data into Google Analytics with the Measurement Protocol.

The people who click through to read our content at least once are an analogue of a partial respondent. They’ve engaged with us at least once. This is data available in Google Analytics as new website visitors.

The people who come back – our returning website visitors – are our completions. Our content was not offensive enough to prevent them from coming back for more. This is data available in Google Analytics.

Even if we just examined total new visitors to our website and total returning visitors, we’d have a sense of how good or bad our content marketing is. If returning visitors are on the decline, our content marketing isn’t working for us. We’re alienating our audience.

If we put all three series of data together in a spreadsheet, we have a reasonably complete picture of our digital effectiveness: total number of people reached (including non-responses from ads, social posts, etc.), people who have engaged once (new website visitors), and people who have returned. A boiler room focus on content marketing only looks at the first and second series. A focus on quality content marketing focuses on all three, but weights returning visitors most heavily.

And with enough effort, focus, time, and resources, all this data can be made available in Google Analytics.

You might also enjoy:

• You Ask, I Answer: AI Music Collaborations and Copyright?
• Almost Timely News, February 11, 2024: How To Evaluate a Generative AI System
• Almost Timely News, February 4, 2024: What AI Has Made Scarce
• Fireside Chat: Geraldine Deruiter on Food, Feminism, and Fury
• Mind Readings: What Makes A Good Conference/Event?

Take my Generative AI for Marketers course!

Join my Analytics for Marketers Slack Group!