Insidious misuses of statistics

Warning: this content is older than 365 days. It may be out of date and no longer relevant.

One of the most insidious mis-uses of statistics I’ve seen recently was a citation about the effectiveness of video in marketing. The original statistic in question read along the lines of:

“73% of marketers surveyed believe that video was more impactful than other marketing methods for lead conversion”

When recited, the marketer I was speaking to interpreted that to mean that video would improve their conversion rates by 73%.

Re-read the citation above.

What it says is if you asked 100 marketers, 73 of them would tell you that they think video is more impactful than other marketing methods for lead conversion. The citation does not say in any way how much more effective video is. If you asked those 73 marketers, they might say that video was only 5% more effective on average than other methods. 5% is a far cry from 73%, but people misinterpret the above number to mean they should see a 73% improvement in lead conversion.

There’s an even more insidious problem with the citation that can lead you astray.

If you asked 100 horse and buggy manufacturers in 1905, they would have told you that Henry Ford was an idiot. 100% of horse and buggy manufacturers would have believed that horses and buggies were more impactful and more effective for transportation in 1905 than the horseless carriage.

Asking for the opinions of fellow marketers gives you insight only into fellow marketers. It doesn’t provide you with usable data about your audience, unless your audience is composed of marketers.

What should you do with the original citation? Despite appearing quantitative (objective data), it’s really qualitative in nature (anecdote and opinion). That means it’s a great place to start asking more questions, rather than start accepting answers. The question that should form in your mind is, “Should I be experimenting with video in my lead conversion processes?” and the answer is, yes, if you have the resources and capabilities to do so.

Whenever you see any statistic cited in the media, question whether it is truly quantitative or if it’s qualitative. If it’s the latter, use it to ask more interesting questions and get your own answers. Most of all, do what you can to educate your stakeholders about the difference so that you’re not constantly fighting airplane magazine syndrome/shiny object syndrome.