When bad data can be okay

Warning: this content is older than 365 days. It may be out of date and no longer relevant.

As marketers, and especially as marketing technologists, correct data, correct metrics, correct information is prized by us (or should be). Incorrect data, faulty data, and misleading data are anathema to our profession and our ability to do our jobs. So it might seem absolutely absurd, even heretical, to make the statement that sometimes, bad data can be okay.

When could wrong data, bad data ever be okay? Here’s a thought exercise for you.

Imagine for a moment you wanted to know what time it was. On the wall was a clock that was clearly the wrong time. You know for sure it’s noon but the clock says 3. When you check the clock later that day as you head home, you know it’s 5 o’clock but the clock says 8.

Is the data bad? Yes. Can you still use it? Yes, as long as you know the clock is three hours ahead. The data is bad but predictably and reliably bad. You can develop a mental model (just subtract three) to compensate for the error.

Now imagine the clock shows 3 when you know it’s 12. In an hour, it shows 5. Then it goes backwards and shows 11 within a few minutes. Is the data bad? Yes. Can you still use it? No. In this case the data is bad and unreliably, unpredictable, seemingly random. It’s not something you can develop a model on, and thus it’s totally useless.

Many of the measures we use in marketing come from other derived sources, such as Klout scores and other social influence measures or estimated web traffic. When you’re looking at metrics and tools, the question you should be asking yourself isn’t necessarily whether the data is right (though that’s an important question) but whether it’s reliable.

You can model reliably wrong data that you understand. You cannot model correct data with surety if you don’t know what it’s made of, because things could be changing behind the scenes that you can’t see or compensate for. One day you wake up and what seemed like right data became wrong data overnight.

No better example of this exists than Google’s algorithm. No one knows what’s in it, and thus trying to “win at SEO” is an impossible task because what you think is right today may be wrong tomorrow, but you have no way of knowing it until you lose search rankings. Even worse, because you don’t know what’s in it, you don’t know how to fix what’s wrong except by random experimentation.

Ask how reliable your data is!

Comments

Leave a Reply Cancel reply

Pin It on Pinterest