Do New Klout Scores Predict Influence?
One of the biggest hanging questions from my previous post on the algorithm change to Klout scores was: does the new Klout score do a better or worse job of predicting influence? Let’s attempt to answer that together today.
Before we begin, the disclosures and disclaimers. This set of tests was done with two datasets from my audience on Twitter. It’s a niche audience of folks largely focused on digital marketing, which means that it’s not representative of the general public. I also interact with my audience in peculiar ways, including using a variety of tools to do some funky automated stuff. Thus, my audience should not be interpreted to be representative of the general public and certainly not representative of your audience.
First things first. Let’s see if we can ascertain what the new Klout score uses as its basis for making influence decisions. In the past, Klout scores relied heavily on activity, meaning that if you tweeted a lot, you’d get a halfway decent score. I pulled a random sample of 2,516 Twitter IDs from my followers and grabbed their followers, following, tweets, and lists counts.
Second, the usual warning applies. Correlation is not causation!
Is there a correlation between followers and Klout score? Yes, a relatively weak one:
It’s weak enough that I wouldn’t rely on it, but not weak enough that it’s statistically insignificant.
How about the people you follow and Klout score?
Weaker than followers but still not insignificant.
What about being listed? After all, if someone puts you on a Twitter list, they must want to follow you in some sense.
Also weak, though stronger than following count.
Finally, what about being just flat out noisy?
Weak, but stronger than following and listed.
What does this tell us? No one factor is dominant in the new Klout algorithm, though if you had to pick something to focus on for activity, getting new followers is the best of a bad lot. There’s another possibility as well: Klout may be giving more weight to other social networks, which means that Twitter (which this data set is based off of) may have less impact on your influence score overall.
Now, let’s get to the meat of the question: do people with higher Klout scores do what I want them to do? That, after all, is the definition of influence, the ability to change an outcome or cause an action to be taken. As you know from many past posts, I use an open source package called TwapperKeeper to keep a log of all my tweets and mentions. I pulled out everyone who has ever retweeted me since I installed the software, which was about a year ago, and then did a count of how many times they’d retweeted me. After all, if I’m influential to you, chances are you’ll retweet me more than once over the span of a year, right? It also follows logically that if you retweet me, chances are you retweet other people too, which should in turn make you influential and as a result you should have a higher Klout score.
So, to answer the question whether a Klout score is an accurate predictor of whether you’ll do what I want you to do (in this case, retweet), let’s run the numbers:
Uh oh. It turns out that Klout score is a horrible predictor of whether someone will retweet me. The Pearson R score is so low that it effectively says there’s no relationship between the Klout score and the likelihood that you’ll retweet me frequently.
The bottom line is this: if you are using or want to use Klout scores to determine who to follow for the purposes of getting them to retweet you, Klout is a useless metric for this purpose, at least with my digital marketing crowd.
As always, I believe strongly in peer review, so I’m including the anonymized data sets for the information shown above so that you can run your own tests on it. I’m not a statistician by any stretch of the imagination, so I would encourage you to do your own study using my methodology or at least download my data sets and slice & dice ‘em for yourself.
Download the random sample of Klout scores vs. followers and other general measures in a CSV.
Download the people who retweeted me vs. Klout scores in a CSV.
What’s your take on this Klout data?
If you enjoyed this, please click here and share it with your network!
Want to read more like this from Christopher Penn? If so, please subscribe right now!
Click here to read my blog on Google Currents on your mobile!
Marketing White Belt |
Watch me speak:
Attend virtually! |
I recommend: for Twitter audience building. |
I’m rolling out an auto-DM campaign!
Consider this your fair warning: I’m rolling out an auto-DM campaign to my Twitter followers.
/waits out the inevitable fit of rage
You’re probably wondering why. I’m testing a belief that many people on Twitter would engage more, would get more value, would be happier followers, if they actually saw half of what I published on Twitter. I suspect that people miss just about everything because it’s very noisy. My audience consists of many folks who are marketing professionals. They in turn follow and subscribe to lots of people, which means that even blocks of updates like #the5 are gone within minutes of them logging into Twitter, so they miss the good stuff.
I firmly believe that things like newsletters are the antidote to this. Newsletters are a better archive than hitting the favorites button, they’re a more lasting archive, and they’re a more convenient archive that’s portable and self-contained.
Here’s the campaign details and how I’ve set it up to work. Everyone following me should get one and only one auto-DM. Each day, my TweetAdder software will send the maximum allowed number of daily DMs (250) out to everyone who is following me with this tweet:
Thanks for following. May I please ask you to subscribe to my newsletter so you don’t miss useful marketing news? http://bit.ly/twadm
It will get you to this Twitter landing page, which I just wrote. If you inspect the URL in the tweet closely, you’ll find that it contains a referrer field that will flow into my CRM, which will give me an idea of what percentage of the 43,000+ people following me have decided the DM was of enough value to subscribe. Of course, it also contains the usual Google Analytics tracking codes too.
Naturally, I’ll be able to track analytics as well, following down the chain of actions:
- How many DMs did I send?
- How many were clicked on? (bit.ly data and GA data)
- How many “converted”? (newsletter subscriber data)
For those who do subscribe and fill out the form completely, I’ll also be able to cross reference Twitter handles and when you started following me; this should give me an idea whether newer followers are more interested in engaging in this way than older followers.
Stay tuned in! I will publish semi-regular updates about the experiment, which according to my math, should conclude in 175 days or roughly on April 16, 2012. At or after that time, I’ll share some rollup statistics on how it went. If you’re a data junkie who likes to crunch this sort of information, please check back in around mid-April and I’ll gladly share an anonymized data set with you if I can.
As I said at the beginning of the post, consider this your fair warning. If getting a single auto-DM really, really upsets you (and it honestly does to some people), please take a few moments to unfollow me now. I won’t be offended, since that’s effectively the equivalent of opting out. I’ll tweet out this post, too. However, if you miss the tweet, you definitely prove the point that the auto-DM campaign is trying to make, yes?
If you enjoyed this, please click here and share it with your network!
Want to read more like this from Christopher Penn? If so, please subscribe right now!
Click here to read my blog on Google Currents on your mobile!
Marketing White Belt |
Watch me speak:
Attend virtually! |
I recommend: for Twitter audience building. |
How to archive your tweets permanently
A question frequently asked of me is, how do you archive your tweets? The answer is a free, open-source package called YourTwapperKeeper, from the folks who make the service of the same name. In order to use it, you must be familiar with how to install a server-side application, including setting up a database, editing a configuration file, running a MySQL script, and setting up a Twitter application using the developer’s interface. If you’re not able to do this, you may want to look for other options, but this solution is quite powerful and you can’t beat the cost.
The one thing that will trip up folks when installing is that unlike other PHP-based open-source packages, you must actually copy/paste or run the included MySQL configuration script separately. It’s a very manual install, which can be intimidating for novice developers. Follow the directions in the README file to the letter and you should be okay. If you’re not sure how to run a MySQL script, you may want to have someone else do it for you.
Once you’ve gotten the package installed, configured, and operational, you authenticate with Twitter and can begin to archive your tweets by any text string, including your username, hashtags, and more.
Here I’ve set up a couple of searches, for myself and for the recent Blue Sky Factory conference.
Once the searches pull in some results, you can do an incredible amount of slicing and dicing of the archive, excluding retweets, filtering based on users or text, looking at specific time periods, and more. Unlike regular Twitter search, you can dig back into the archives for as long as you have data collected, which can be handy for analyzing Twitter patterns over longer periods of time or in month over month/year over year periods, far more than the two weeks of history that Twitter provides.
The most powerful feature, however, is the ability to export to a variety of data formats, including XML, JSON, and CSV/Excel. This is valuable if you want to provide, say, a tweet-based event recap, or you want to do analysis of timestamps, user interactions, and networks in third party data tools.
Here’s a partial example from the Excel export. I can, for example, sort out what percentage of tweets and people are using what clients in my audience, get geographic coordinates for use in services like BatchGeo, and do time-based analysis of how tweets and retweets flow (hat tip to Gilad Lotan @gilgul for the idea).
YourTwapperKeeper isn’t for everyone. It’s decidedly unfriendly to install. In fact, for most marketers, you’ll want to have the IT guys do the heavy lifting for you entirely, but once you’ve got it up and running, you’ll wonder how you ever recorded Twitter data without it.
If you enjoyed this, please click here and share it with your network!
Want to read more like this from Christopher Penn? If so, please subscribe right now!
Click here to read my blog on Google Currents on your mobile!
Marketing White Belt |
Watch me speak:
Attend virtually! |
I recommend: for Twitter audience building. |
Do Auto-DMs Work?
Some people love them, some people hate them (rather like popups, right?). But the question isn’t whether or not you like them, the question is, do they work for generating any kind of traffic or attention? The answer is… let’s test!
You see, as much as I may have an opinion on any given marketing practice, I frequently need to remind myself that I am not my audience. I am not my customers. I am no one except myself, one anecdotal bit of evidence in a sea of data waiting to be explored. I routinely rail against the HIPPO problem – the highest individually paid person’s opinion – as being the cause of many marketing failures, so it’d be absurd for me to let my personal preferences dictate whether or not something is worth trying.
So let’s look at a few days’ worth of data from my analytics. The goal in this case was pure traffic alone, not conversion. I wanted to know if auto-DMs would have any kind of real effect on traffic to my website, since traffic is the easiest commitment to get out of someone – just show up. On June 6, I kept doing all of my normal Twitter practices (welcome message, #the5, etc.) plus turned on the auto-DM feature in TweetAdder (affiliate link) with a simple message and a custom, trackable hyperlink.
Before I show the data, I want to ask you this as a fellow marketer: do you have an opinion of auto-DMs? Is that opinion founded on data you collected or the HIPPO problem in your head? If the latter, there’s a good chance you’re not making the most of many different marketing tactics and strategies because you’re letting your judgement cloud opportunities.
Ready for the data?
Here’s the contents of the daily welcome message I send, usually first thing in the morning:
Good morning friends from (wherever I am). (something relevant daily). New friend? Welcome aboard: http://cspenn.com/w
This message shows up in my analytics as cspenn.com welcome message / linkshortener.
Here’s the auto-DM I was sending:
Thank you for following me. If you’re interested in learning more, here’s a quick summary: http://cspenn.com/dm
cspenn.com welcome DM direct message / linkshortener
A key part of testing is making sure you have tracking turned on and made as granular as practical in order to get reliable data. Doing this experiment without correct link tracking would lead to bad conclusions or no conclusion at all. In this case I set up custom URLs on my site and tagged them with the Google Analytics URL builder.
In the period between June 6 and June 10, I picked up 86 new followers. (thank you and welcome aboard, folks)
In that time period, we can see that 45 of them responded to the welcome message (since very few followers who’ve been around for more than a day or two click through on it – they’ve seen it). 3 responded to the auto-DM.
Proportionally:
- The daily welcome message engaged 52% of new followers.
- The auto-DM engaged 3.5% of new followers.
The winner is clearly the daily welcome message and not the auto-DM. Once I had a few days’ worth of data, I turned it off. With as large a Twitter audience as I have and as busy a frequency I have, I’m confident enough in my own data to say that it’s not working for my audience.
Does this mean you shouldn’t use auto-DMs? NO! No, it doesn’t, because your audience may be different from mine. The people interested in you may respond differently. If you don’t test it, you’ll never know. If you let your opinion in advance of data cloud your decisions, you’ll never know and you may be losing money, opportunities, or other things you value. Test it. Test everything, and only after you have reliable data that you sourced yourself should you pass judgement. This is true of every marketing method you have available to you. Write off methods without testing at your peril.
If you enjoyed this, please click here and share it with your network!
Want to read more like this from Christopher Penn? If so, please subscribe right now!
Click here to read my blog on Google Currents on your mobile!
Marketing White Belt |
Watch me speak:
Attend virtually! |
I recommend: for Twitter audience building. |
Who are you reading?
I’m always on the lookout for new sources, new points of interest, new things to research, new ideas. I would imagine you are, too. I’d like you to leave suggestions for who you’re reading in the comments below. Here’s the criteria for who I’m looking for you to share:
- The original content test. As much as I love Techcrunch, Mashable, ReadWriteWeb, etc., I’d much rather subscribe to the sources they read. Who is writing original stuff? Share them below!
- The first test. When you open your blog reader, whose blog do you go to first? Share them below!
- The blind retweet test. Who are some people who are so worth reading that you automatically retweet their stuff first, then read it, because you know they are always providing ridiculously good value? Share them below!
- The dark horse test. There are some folks who are pretty well known, like Chris Brogan, Mitch Joel, Avinash Kaushik, Jason Falls, etc. that most people know. Who don’t we know but is writing at the same level of value and quality? Share them below!
If you include a link (please do), there’s a good chance the comment will be moderated by Disqus, so I’ll approve it manually – thus, don’t hit submit comment a whole bunch of times if nothing appears. I’m out of the office today, so come back at the end of the day to see all the comments, or if you folks leave a phenomenal list, I may take the time to publish it as an OPML file for everyone.
If you enjoyed this, please click here and share it with your network!
Want to read more like this from Christopher Penn? If so, please subscribe right now!
Click here to read my blog on Google Currents on your mobile!
Marketing White Belt |
Watch me speak:
Attend virtually! |
I recommend: for Twitter audience building. |
Who you want to be
Take a few moments now to look at the following digital properties.
- Your blog
- Google (for your name)
Do you see the person you are?
Or do you see the person you want to be?
If you don’t like what you see, if you see someone who isn’t inspiring to you or others, if you see someone who complains too much or doesn’t say enough of value, then make the conscious choice to change this now.
Who do you want to be? Who are you supposed to be?
Imagine a Future You, the person you want to be, the person you are supposed to be. What would you find in their digital channels? Would Future You be tweeting about a lukewarm burger or a cranky flight attendant to an audience of 50,000? (would anyone care?) Would Future You be relentlessly spamming your Fortune 500 executive connections on LinkedIn with random, pointless quotes or repetitive book pitches? Would Future You be waxing poetic about your intestinal bug and its vivid consequences for 2,000 words on your blog?
Probably not. Future You would probably be a great deal funnier, more noble, more insightful, more helpful, more kind, more gracious, more powerful, more connected, more wealthy, more happy. Take another few moments and decide what things Future You would think, say, write, and do. Make a list of things that Future You would be retweeted for, recommended about, or blogged about, and post that near your workstation, on the back of your phone, on your iPad case, or wherever you do your communicating with the world.
Here’s the good news. Future You is within your reach right now, beginning the moment you stop reading this post and put your fingers to the keyboard to communicate again. Put away Present You and start communicating with the world as close to Future You as you possibly can each day, and sooner than you think, they’ll be one and the same.
If you enjoyed this, please click here and share it with your network!
Want to read more like this from Christopher Penn? If so, please subscribe right now!
Click here to read my blog on Google Currents on your mobile!
Marketing White Belt |
Watch me speak:
Attend virtually! |
I recommend: for Twitter audience building. |
Book Me to Speak
Got an event coming up? Learn why you should book me to speak!
• Download my 2012 speaking kit
•Book me to speak now!
Get Marketing White Belt
Grab my new book, Marketing White Belt: Basics for the Digital Marketer, and get better at marketing right now!
Blog Delivery
Things I recommend:
I recommend:
for Wordpress themes like this one.
I recommend:
for Twitter audience building.