Do New Klout Scores Predict Influence?

One of the biggest hanging questions from my previous post on the algorithm change to Klout scores was: does the new Klout score do a better or worse job of predicting influence? Let’s attempt to answer that together today.

Before we begin, the disclosures and disclaimers. This set of tests was done with two datasets from my audience on Twitter. It’s a niche audience of folks largely focused on digital marketing, which means that it’s not representative of the general public. I also interact with my audience in peculiar ways, including using a variety of tools to do some funky automated stuff. Thus, my audience should not be interpreted to be representative of the general public and certainly not representative of your audience.

First things first. Let’s see if we can ascertain what the new Klout score uses as its basis for making influence decisions. In the past, Klout scores relied heavily on activity, meaning that if you tweeted a lot, you’d get a halfway decent score. I pulled a random sample of 2,516 Twitter IDs from my followers and grabbed their followers, following, tweets, and lists counts.

Second, the usual warning applies. Correlation is not causation!

Is there a correlation between followers and Klout score? Yes, a relatively weak one:

SOFA Statistics Report 2011-11-01_09:10:58

It’s weak enough that I wouldn’t rely on it, but not weak enough that it’s statistically insignificant.

How about the people you follow and Klout score?

SOFA Statistics Report 2011-11-01_09:10:58

Weaker than followers but still not insignificant.

What about being listed? After all, if someone puts you on a Twitter list, they must want to follow you in some sense.

SOFA Statistics Report 2011-11-01_09:10:58

Also weak, though stronger than following count.

Finally, what about being just flat out noisy?

SOFA Statistics Report 2011-11-01_09:10:58

Weak, but stronger than following and listed.

What does this tell us? No one factor is dominant in the new Klout algorithm, though if you had to pick something to focus on for activity, getting new followers is the best of a bad lot. There’s another possibility as well: Klout may be giving more weight to other social networks, which means that Twitter (which this data set is based off of) may have less impact on your influence score overall.

Now, let’s get to the meat of the question: do people with higher Klout scores do what I want them to do? That, after all, is the definition of influence, the ability to change an outcome or cause an action to be taken. As you know from many past posts, I use an open source package called TwapperKeeper to keep a log of all my tweets and mentions. I pulled out everyone who has ever retweeted me since I installed the software, which was about a year ago, and then did a count of how many times they’d retweeted me. After all, if I’m influential to you, chances are you’ll retweet me more than once over the span of a year, right? It also follows logically that if you retweet me, chances are you retweet other people too, which should in turn make you influential and as a result you should have a higher Klout score.

So, to answer the question whether a Klout score is an accurate predictor of whether you’ll do what I want you to do (in this case, retweet), let’s run the numbers:

SOFA Statistics Report 2011-11-01_09:21:38

Uh oh. It turns out that Klout score is a horrible predictor of whether someone will retweet me. The Pearson R score is so low that it effectively says there’s no relationship between the Klout score and the likelihood that you’ll retweet me frequently.

The bottom line is this: if you are using or want to use Klout scores to determine who to follow for the purposes of getting them to retweet you, Klout is a useless metric for this purpose, at least with my digital marketing crowd.

As always, I believe strongly in peer review, so I’m including the anonymized data sets for the information shown above so that you can run your own tests on it. I’m not a statistician by any stretch of the imagination, so I would encourage you to do your own study using my methodology or at least download my data sets and slice & dice ‘em for yourself.

Download the random sample of Klout scores vs. followers and other general measures in a CSV.

Download the people who retweeted me vs. Klout scores in a CSV.

What’s your take on this Klout data?


If you enjoyed this, please share it with your network!


Want to read more like this from ? Get daily updates now:


Get my book!

Subscribe to my free newsletter!


  • http://www.convinceandconvert.com jaybaer

    Good analysis Chris, although the biggest flaw I see is using retweets as the basis for whether or not your Twitter behavior is a success. Ultimately, we almost always want a click, which is perhaps a better measure of influence and its sources.

    • http://www.ChristopherSPenn.com Christopher S. Penn

      I totally agree, Jay. The next test would be to set up an auto-DM campaign to your top 500 followers and assign them each an individual, personalized, trackable URL that they could then share to their followers. You’d have to collate all that data and correlate it to the Klout score to get an idea of how much of the desired behavior their audiences would generate.

      • http://twitter.com/webby2001 Tom Webster

        I once contemplated that *exact* same test, Chris (you’ve set it up perfectly). But I decided it wasn’t worth it. And my Klout isn’t high enough to get 500 people to do what I want :)

    • http://twitter.com/webby2001 Tom Webster

      But it’s that “flaw” that actually makes this bit of work sound, Jay – he confined it to retweet behavior from the get-go. If you want retweets, Klout can’t predict them, plain and simple. 

      • http://www.convinceandconvert.com jaybaer

        Sorry for the confusion. I didn’t mean it was a flaw in the research, but rather a strategic flaw. Engaging in Twitter for the purposes of RT generation alone is half-baked at best. In terms of Klout specifically, I don’t believe they ever purported to predict RTs. That’s not to say they can predict clicks either, but to suggest that RTs are the end product of “Klout” is probably a stretch, especially since they are now including FB, Linkedin, Foursquare and a bunch of other online behaviors in the data pool.

        • http://twitter.com/webby2001 Tom Webster

          Well, I can’t argue with that. Still, if Klout were a proxy for “some” form of influence, you’d expect retweets to at least be a trailing variable, no? Not the end goal, but some kind of activity diagnostic that correlates to other behaviors? I just find it remarkable that there is essentially NO correlation between Klout score and retweet behavior, as limited as that might be.

          • http://www.convinceandconvert.com jaybaer

            You would think, especially since they started out as Twitter-only. It would have been awesome to run this same analysis on Klout before the recent algorithm overhaul, as I suspect it may have been more Twitter-centric with a greater correlation then. Or perhaps not at all, which is both the interesting and aggravating thing about all of these “influence” measures. Hard to take it at face value when you don’t know what the face represents below the water line.

  • http://limecubemarketing.com/blog Simon Mason

    Interesting set of figures Chris, as someone who actively avoids Klout as it seems to be a completely backward way of looking at social media to me, this post shows a better insight into where the value of Klout data may lay than most.

    I have a friend who is a data wizard, gets visibly excited at the thought of a large data set and knows more about analysing data from loyalty schemes etc than you would believe possible – I’ll throw your data over to him and see if he comes up with anything interesting.

  • Anonymous

    Hi Chris – enjoyed your post; one of my big take aways from Podcamp Boston was the notion that influence is getting someone to “so something you want them to do.”  If you or @jasonbaer:disqus or @twitter-755294:disqus want to try a DM test, @medialabrat:twitter has a new tool that makes sending and tracking really easy – it is called Chirpaloo.  (I let him know :)  Annie

    • http://twitter.com/MediaLabRat Robert Moore

      Chris, being a massive data junkie and some that is coming into social media with a heavy data and direct response background, I really liked the way you approached this issue.  @AnneWeiskopf , thanks for the intro!

  • Anonymous

    I agree with this entire analysis, otherwise, I’d be able to get a penny per other people’s followers to create my next 168 Project film. http://my168project.co for more info….

  • http://ariherzog.com Ari Herzog

    “Klout is a useless metric.”

    Sums it up for me. Thanks.

  • http://twitter.com/JoeFernandez Joe Fernandez

    Hey Chris,

    Just catching up on this. I’ve enjoyed your analysis of the new Klout algorithm. The idea that someones Klout score should in any way be correlated to their likelihood of retweeting you doesn’t make sense to me though.

    Joe

    • http://www.ChristopherSPenn.com Christopher S. Penn

      Joe – it’s not statistically sensible, but it’s the way a lot of marketers are using the score. There’s going to be a followup study to measure the impact of people and how the score indicates or doesn’t indicate the node impact down the tree.