One of the biggest hanging questions from my previous post on the algorithm change to Klout scores was: does the new Klout score do a better or worse job of predicting influence? Let’s attempt to answer that together today.
Before we begin, the disclosures and disclaimers. This set of tests was done with two datasets from my audience on Twitter. It’s a niche audience of folks largely focused on digital marketing, which means that it’s not representative of the general public. I also interact with my audience in peculiar ways, including using a variety of tools to do some funky automated stuff. Thus, my audience should not be interpreted to be representative of the general public and certainly not representative of your audience.
First things first. Let’s see if we can ascertain what the new Klout score uses as its basis for making influence decisions. In the past, Klout scores relied heavily on activity, meaning that if you tweeted a lot, you’d get a halfway decent score. I pulled a random sample of 2,516 Twitter IDs from my followers and grabbed their followers, following, tweets, and lists counts.
Second, the usual warning applies. Correlation is not causation!
Is there a correlation between followers and Klout score? Yes, a relatively weak one:
It’s weak enough that I wouldn’t rely on it, but not weak enough that it’s statistically insignificant.
How about the people you follow and Klout score?
Weaker than followers but still not insignificant.
What about being listed? After all, if someone puts you on a Twitter list, they must want to follow you in some sense.
Also weak, though stronger than following count.
Finally, what about being just flat out noisy?
Weak, but stronger than following and listed.
What does this tell us? No one factor is dominant in the new Klout algorithm, though if you had to pick something to focus on for activity, getting new followers is the best of a bad lot. There’s another possibility as well: Klout may be giving more weight to other social networks, which means that Twitter (which this data set is based off of) may have less impact on your influence score overall.
Now, let’s get to the meat of the question: do people with higher Klout scores do what I want them to do? That, after all, is the definition of influence, the ability to change an outcome or cause an action to be taken. As you know from many past posts, I use an open source package called TwapperKeeper to keep a log of all my tweets and mentions. I pulled out everyone who has ever retweeted me since I installed the software, which was about a year ago, and then did a count of how many times they’d retweeted me. After all, if I’m influential to you, chances are you’ll retweet me more than once over the span of a year, right? It also follows logically that if you retweet me, chances are you retweet other people too, which should in turn make you influential and as a result you should have a higher Klout score.
So, to answer the question whether a Klout score is an accurate predictor of whether you’ll do what I want you to do (in this case, retweet), let’s run the numbers:
Uh oh. It turns out that Klout score is a horrible predictor of whether someone will retweet me. The Pearson R score is so low that it effectively says there’s no relationship between the Klout score and the likelihood that you’ll retweet me frequently.
The bottom line is this: if you are using or want to use Klout scores to determine who to follow for the purposes of getting them to retweet you, Klout is a useless metric for this purpose, at least with my digital marketing crowd.
As always, I believe strongly in peer review, so I’m including the anonymized data sets for the information shown above so that you can run your own tests on it. I’m not a statistician by any stretch of the imagination, so I would encourage you to do your own study using my methodology or at least download my data sets and slice & dice ’em for yourself.
Download the random sample of Klout scores vs. followers and other general measures in a CSV.
Download the people who retweeted me vs. Klout scores in a CSV.
What’s your take on this Klout data?
You might also enjoy:
- You Ask, I Answer: Retrieval Augmented Generation vs Fine-Tuning?
- Almost Timely News: Recipes vs. Principles in Generative AI (2024-03-03)
- Almost Timely News, Febuary 18, 2024: From Comment to Content
- You Ask, I Answer: AI Works And Copyright?
- Mind Readings: You Need Passwords for Life in the Age of Generative AI Fraud
Want to read more like this from Christopher Penn? Get updates here:
 Take my Generative AI for Marketers course! |
Leave a Reply