Why Facebook’s Emotional Testing Isn’t Just A/B Testing


Much ink has rightly been spilled about the revelation that Facebook wanted to judge whether emotional states were affected and contagious in their most recent research on 600,000+ people. You can read more of the details here.

On social media, reactions have been mixed, from people calling it an outrage to people saying, “What’s the big deal, it’s just A/B testing, get over it”. This latter point is one worth digging into. A/B testing normally is used to optimize conversions and provide a way of understanding how your content performs. What’s different about what Facebook was doing deals more with professional ethics in research. As both Tom Webster and I have pointed out, many organizations in the research space have codes of ethics that give researchers guidelines about what they should and should not do. Here’s one from AAPOR, the American Association of Public Opinion Researchers, from section I A:

1. We shall avoid practices or methods that may harm, endanger, humiliate, or seriously mislead survey respondents or prospective respondents.

2. We shall respect respondents’ desires, when expressed, not to answer specific survey questions or provide other information to the researcher. We shall be responsive to their questions about how their contact information was secured.

3. Participation in surveys and other forms of public opinion research is voluntary, except for the decennial census and a few other government surveys as specified by law. We shall provide all persons selected for inclusion with a description of the research study sufficient to permit them to make an informed and free decision about their participation. We shall make no false or misleading claims as to a study’s sponsorship or purpose, and we shall provide truthful answers to direct questions about the research. If disclosure could substantially bias responses or endanger interviewers, it is sufficient to indicate that some information cannot be revealed or will not be revealed until the study is concluded.

Where Facebook fell down is on points 1 and 3. On point 3, yes, the Terms of Service permit them to legally do anything they want to their data and their users, but there’s a difference between implied consent buried in the Terms of Service and informed participation in a research study. All Facebook had to do would have been to put up a little header at the top of the News Feed to say, “Facebook would like you to participate in an emotional research study (click here for details), are you willing to participate? If so, click the Like button on this banner.”

The biggest part where Facebook fell down was on point 1. The difference between A/B testing the conversion rate of your website and intentionally altering peoples’ emotions positively or negatively is the impact of the potential outcome. If I succeed in manipulating your behavior to get you to buy X% more stuff, there’s moderate to low risk of me causing serious permanent harm to your life beyond financial impact. If I succeed in manipulating your emotions to make you sad and depressed, there’s a certain percentage of people – small, but non-zero – who will amplify that to the extreme of harming themselves or others.

That’s the difference between regular A/B testing and what Facebook’s experiment did wrong. I would wager a class action lawsuit will be on its way in no short order, and it’s deserved for an ethics violation that has had realistic potential to cause serious harm to participants of the study.

If you enjoyed this, please share it with your network!

Want to read more like this from ? Get updates here:

Marketing Blue Belt Preorder

Order your 2016 Marketing Planning Framework

Timeframes, analytics, and objectives

One of the most misused parts of marketing analytics is the timeframe. Whether it’s daily/weekly/monthly views in Google Analytics or People Talking About This in Facebook or the metric of your choice, we tend as marketers to use the timeframe (no surprise, we’re marketers) that makes things look the best, or at least look meaningful. Most of the time, this is unintentional and not malicious. We look for patterns, trends, and information that is meaningful. Sometimes it’s just what we’re given by the tools we use.

Here’s why timeframes matter in analytics. The timeframe tells you what results you are capable of generating using any given marketing method. For example, let’s say your focus is on audience and awareness building, core functions of things like advertising and public relations. Monthly or even quarterly metrics timeframes are perfectly okay to work with because you’re looking more at the cumulative effect of all of the communications with your prospective audiences. You don’t necessarily need to be top of mind 24×7, just enough that you maintain share of mind.

However, if your focus is on something like direct response, you might want to work in a weekly or daily timeframe. Direct response marketing and lead generation typically have much shorter timeframes, timeframes in which you must meet certain numbers. You might, for example, need to generate a certain number of leads before the end of the month to meet a quota. Working in weekly or daily timeframes in your metrics will tell you how likely you are to achieve your goals.

Here’s an example using Facebook’s People Talking About This. By default, Facebook reports PTAT as a weekly timeframe metric. In your Page’s insights, you can also get daily and 28-day PTAT:


If you’re in charge of growing audience and growing awareness, looking at the monthly PTAT vs. Total Likes is a reasonable thing to do. What chunk of your audience did you reach in the last 28 days?

If you’re in charge of lead generation, looking at the daily PTAT vs. Total Likes is important, because it will help guide your expectations about how many people today will see your offer in a very short period of time.

When you mix the two is when disaster can strike. If you’re a direct response marketer and you see the monthly reach numbers, you might expect that up to 100 people could respond to your offer on any given day, when the reality is that at best, 20 would be the maximum number of people in a given day. Conversely, if you’re a PR professional, you might be distraught at the idea that 3 out of 1,300 people are seeing your work today, when the reality is that your content is being seen by a hundred in the month. Today’s post might be invisible to an audience member, but tomorrow’s might be quite prominent.

Know the timeframes that your marketing methods operate in, and measure accordingly!

If you enjoyed this, please share it with your network!

Want to read more like this from ? Get updates here:

Marketing Blue Belt Preorder

Order your 2016 Marketing Planning Framework

The simplest Facebook metric to remember

Quick, what Facebook metrics actually matter? Likes? Comments? Shares? The answer is… all of the above, and yet none of the above. What actually matters is engagement, which is a composite number of likes, comments, shares, re-shares, etc. Facebook sums all of this up in a number in your Page Insights called People Talking About This. Here’s what the official text says:

“Daily: The number of people sharing stories about your page. These stories include liking your Page, posting to your Page’s timeline, liking, commenting on or sharing one of your Page posts, answering a question you posted, responding to one of your events, mentioning your Page, tagging your Page in a photo or checking in at your location. (Unique Users)”

Facebook’s algorithms pay close attention to these behaviors, these activities. If you’re a Page manager, you’ll find this in the Insights download:


Download your spreadsheet of Page metrics and open it up in the spreadsheet software of your choice. Look for two columns, Daily People Talking About This (PTAT) and Lifetime Likes:


Think of these two columns as the number of people you did reach and the number of people you could have reached. Divide Daily PTAT by Lifetime Likes and you get a sense of how much engagement you’re actually getting on a day to day basis…


Take a look at those numbers. On my best day, my Facebook page is getting 1.47% engagement. Now you might say, well, that’s because maybe I just suck at Facebook. I did a bit of digging, though, and looked at a well-respected non-profit: 0.14% engagement on day to day engagement vs. total potential audience. Major consumer brand with big audience and a beloved product? 0.95% engagement. Super-hot consumer startup with a product that’s on fire and getting major coverage? 1.65% engagement.

Is it any wonder that brands simply have resorted to getting out the credit card and paying to play?

If Facebook isn’t delivering results for your brand any more, if your numbers look like these, you have two basic choices: you can either reduce the resources allocated to it, or you can pay to play. Either way, what you’re doing right now probably isn’t working as well as you’d like. Run this simple engagement math on the Facebook Pages you manage and see how your day to day engagement is really doing.

If you enjoyed this, please share it with your network!

Want to read more like this from ? Get updates here:

Marketing Blue Belt Preorder

Order your 2016 Marketing Planning Framework