How to set up a social media split test

Warning: this content is older than 365 days. It may be out of date and no longer relevant.

A split test is any situation in which you want to measure the effect of your content. Services such as, Argyle Social, and many other shorteners provide you with nearly everything you need to conduct an effective, statistically valid test. Let’s look at how you might deploy one using Twitter.

First, design your links. We’re going to use my blog’s Twitter welcome page as an example. I need to have four different tweetable links that track separately. I’ll start by feeding the welcome page to Google’s URL builder to ensure correct GA tracking.

Tool: URL Builder - Analytics Help

This first tweet will be tagged dmwelcome1. I’ll set up four of these URLs:


Next, let’s feed each to so that we get a nice clean link for tweeting.

bitly statistics for

For this experiment, I’m going to set up a series of four direct messages on Twitter. You don’t have to use automatically sent direct messages. This methodology works for Facebook pages, public tweets, Google+ posts – anywhere that you want to test the same destination content with different hooks to try to bring in new eyeballs. I am using auto-DMs mainly because I can get a fair number of responses very quickly. Think outside the box!

Here’s the four tweets I’ll load up:

  • Thanks for following me. If you’d like to get to know me better, visit:
  • Thanks for following me. If you’d like to learn more about me, visit:
  • Thanks for following! Get to know me better here:
  • Thanks for following me. Learn more about me here:

Now we’re ready to test out this four way split. Obviously, you can substitute any content you like, such as calls to action to sales and other things of interest. We’ll load each of the tweets into TweetAdder:

TweetAdder 3.0 Build#110811

And we’ll dispatch 200 tweets (Twitter wisely imposes a hard limit of 250 DMs a day).

Now we watch and wait as the test goes out, looking at two things:

  1. Which of the four tweets was most appealing to people as measured by clickthroughs using
  2. Which of the four tweets was most in sync with my site’s content as measured by Google Analytics conversions to newsletter subscribers?

Setting up your own four way, eight way, or however many way test using these freely available tools is just as straightforward. One suggestion I’d offer is to add in a fifth message, a control, that you tweet out in public and compare it to see how private versus public messages works for your audience.

Take this recipe, this methodology, and apply it to your own marketing to see how impactful it is.

You might also enjoy:

Want to read more like this from Christopher Penn? Get updates here:

subscribe to my newsletter here

AI for Marketers Book
Take my Generative AI for Marketers course!

Analytics for Marketers Discussion Group
Join my Analytics for Marketers Slack Group!


One response to “How to set up a social media split test”

  1. Brilliant. And trying it out now.

    We ran a simple test on LinkedIn a couple weeks ago, testing a variety of headlines, subject lines and links. Still tracking…test and learn, test and learn.

Leave a Reply

Your email address will not be published. Required fields are marked *

Pin It on Pinterest

Share This