How to set up a social media split test

A split test is any situation in which you want to measure the effect of your content. Services such as Bit.ly, Argyle Social, and many other shorteners provide you with nearly everything you need to conduct an effective, statistically valid test. Let’s look at how you might deploy one using Twitter.

First, design your links. We’re going to use my blog’s Twitter welcome page as an example. I need to have four different tweetable links that track separately. I’ll start by feeding the welcome page to Google’s URL builder to ensure correct GA tracking.

This first tweet will be tagged dmwelcome1. I’ll set up four of these URLs:

• https://www.christopherspenn.com/welcome-aboard/?utm_source=socialmedia&utm_medium=twitter&utm_campaign=dmwelcome1
• https://www.christopherspenn.com/welcome-aboard/?utm_source=socialmedia&utm_medium=twitter&utm_campaign=dmwelcome2
• https://www.christopherspenn.com/welcome-aboard/?utm_source=socialmedia&utm_medium=twitter&utm_campaign=dmwelcome3
• https://www.christopherspenn.com/welcome-aboard/?utm_source=socialmedia&utm_medium=twitter&utm_campaign=dmwelcome4

Next, let’s feed each to Bit.ly so that we get a nice clean link for tweeting.

For this experiment, I’m going to set up a series of four direct messages on Twitter. You don’t have to use automatically sent direct messages. This methodology works for Facebook pages, public tweets, Google+ posts – anywhere that you want to test the same destination content with different hooks to try to bring in new eyeballs. I am using auto-DMs mainly because I can get a fair number of responses very quickly. Think outside the box!

Here’s the four tweets I’ll load up:

• Thanks for following me. If you’d like to get to know me better, visit: https://bit.ly/cpdm-1
• Thanks for following me. If you’d like to learn more about me, visit: https://bit.ly/cpdm-2
• Thanks for following! Get to know me better here: https://bit.ly/cpdm-3
• Thanks for following me. Learn more about me here: https://bit.ly/cpdm-4

Now we’re ready to test out this four way split. Obviously, you can substitute any content you like, such as calls to action to sales and other things of interest. We’ll load each of the tweets into TweetAdder:

And we’ll dispatch 200 tweets (Twitter wisely imposes a hard limit of 250 DMs a day).

Now we watch and wait as the test goes out, looking at two things:

1. Which of the four tweets was most appealing to people as measured by clickthroughs using Bit.ly?
2. Which of the four tweets was most in sync with my site’s content as measured by Google Analytics conversions to newsletter subscribers?

Setting up your own four way, eight way, or however many way test using these freely available tools is just as straightforward. One suggestion I’d offer is to add in a fifth message, a control, that you tweet out in public and compare it to see how private versus public messages works for your audience.

Take this recipe, this methodology, and apply it to your own marketing to see how impactful it is.

You might also enjoy:

• You Ask, I Answer: Retrieval Augmented Generation for Tax Law?
• You Ask, I Answer: Reliability of LLMs vs Other Software?
• Fireside Chat: Geraldine Deruiter on Food, Feminism, and Fury
• Almost Timely News, January 28, 2024: Copyright Must NEVER Apply to AI-Made Works
• Mind Readings: Generative AI and Addition vs Substitution of Jobs

Take my Generative AI for Marketers course!

Join my Analytics for Marketers Slack Group!