Why Facebook’s Emotional Testing Isn’t Just A/B Testing

Warning: this content is older than 365 days. It may be out of date and no longer relevant.

etymology_of_ethics_-_Google_Search

Much ink has rightly been spilled about the revelation that Facebook wanted to judge whether emotional states were affected and contagious in their most recent research on 600,000+ people. You can read more of the details here.

On social media, reactions have been mixed, from people calling it an outrage to people saying, “What’s the big deal, it’s just A/B testing, get over it”. This latter point is one worth digging into. A/B testing normally is used to optimize conversions and provide a way of understanding how your content performs. What’s different about what Facebook was doing deals more with professional ethics in research. As both Tom Webster and I have pointed out, many organizations in the research space have codes of ethics that give researchers guidelines about what they should and should not do. Here’s one from AAPOR, the American Association of Public Opinion Researchers, from section I A:

1. We shall avoid practices or methods that may harm, endanger, humiliate, or seriously mislead survey respondents or prospective respondents.

2. We shall respect respondents’ desires, when expressed, not to answer specific survey questions or provide other information to the researcher. We shall be responsive to their questions about how their contact information was secured.

3. Participation in surveys and other forms of public opinion research is voluntary, except for the decennial census and a few other government surveys as specified by law. We shall provide all persons selected for inclusion with a description of the research study sufficient to permit them to make an informed and free decision about their participation. We shall make no false or misleading claims as to a study’s sponsorship or purpose, and we shall provide truthful answers to direct questions about the research. If disclosure could substantially bias responses or endanger interviewers, it is sufficient to indicate that some information cannot be revealed or will not be revealed until the study is concluded.

Where Facebook fell down is on points 1 and 3. On point 3, yes, the Terms of Service permit them to legally do anything they want to their data and their users, but there’s a difference between implied consent buried in the Terms of Service and informed participation in a research study. All Facebook had to do would have been to put up a little header at the top of the News Feed to say, “Facebook would like you to participate in an emotional research study (click here for details), are you willing to participate? If so, click the Like button on this banner.”

The biggest part where Facebook fell down was on point 1. The difference between A/B testing the conversion rate of your website and intentionally altering peoples’ emotions positively or negatively is the impact of the potential outcome. If I succeed in manipulating your behavior to get you to buy X% more stuff, there’s moderate to low risk of me causing serious permanent harm to your life beyond financial impact. If I succeed in manipulating your emotions to make you sad and depressed, there’s a certain percentage of people – small, but non-zero – who will amplify that to the extreme of harming themselves or others.

That’s the difference between regular A/B testing and what Facebook’s experiment did wrong. I would wager a class action lawsuit will be on its way in no short order, and it’s deserved for an ethics violation that has had realistic potential to cause serious harm to participants of the study.


You might also enjoy:


Want to read more like this from Christopher Penn? Get updates here:

subscribe to my newsletter here


AI for Marketers Book
Take my Generative AI for Marketers course!

Analytics for Marketers Discussion Group
Join my Analytics for Marketers Slack Group!



Comments

8 responses to “Why Facebook’s Emotional Testing Isn’t Just A/B Testing”

  1. Shel Holtz Avatar
    Shel Holtz

    Nice post, Christopher, but I’m struggling to fit what Facebook did into the realm of public opinion research. For me, it’s closer to Upworthy’s testing of 25 different headlines for every post to see which one provokes the best emotional response. I’m not saying it was right. I am saying if it was wrong, it’s wrong for everybody who engages in emotional manipulation, which would include any publisher who tries different approaches to content in order to elicit different behaviors from their target audiences.

    1. This specific instance was completely unethical–no gray line. This was a psychological experiment on humans, not A/B testing of a headline. The APA code of ethics calls for informed consent. My research code of ethics calls for informed consent. There was no informed consent here. There was also no independent review of the primary data collection method–ANY academic or psychological study requires that. And this was submitted as academic research. I hold it to those standards. It’s not an ethical gray area. Period.

      1. Tom, I have a question about this. The study was published in the Proceedings of the National Academy of Science. They say they require authors to obtain informed consent from study participants. The study author, an employee of Facebook, claims that the Data Use Policy constitutes informed consent, and it seems PNAS accepted that argument.

        So if PNAS has accepted the fact that the TOS allow consent, and two universities have said the study was conducted appropriately, where is the ethics line? Is it unethical because we the people think it’s horrible to be manipulated this way or is it unethical because they didn’t follow the correct psychology research protocol? And, if it’s the latter, why did PNAS, UCSF, and Stanford all agree to it?

        1. Two things, Gini: first, the universities in question accepted the data, but did not review the data collection method. The data collection methodology was never reviewed, and it never would have been approved by an IRB because of the second issue: there is no WAY that sticking the word “research” into a TOS constitutes informed consent for a study. Informed consent means exactly what it implies, according to the Federal “Common Rule” for experimenting on humans. We may have given some kind of blanket consent, but certainly not informed consent.

          This got published in PNAS because hey, Facebook. We can argue about the ethics of the action, but the ethics of the *study* are inarguable. If my company did such a study without *informed* consent of participants we would unquestionably be in violation of the Common Rule.

          1. That’s helpful. Thanks!

  2. I enjoyed reading this and think it’s a worthwhile thought exercise. And I agree that they probably crossed an ethical line with regard to #3. However, I agree with Shel that their version of A/B testing is no different than running two different headlines on Google AdWords, for example. All decisions are emotional (that’s a biological fact). Therefore, sales persuasion is all about emotional manipulation. I think it’s debatable which is worse: Making me feel said or manipulating me into spending an extra $20k for a BMW instead of a Toyota.

  3. Unfortunately, this is the kind of behavior foreshadowed by dozens of other FB decisions. Can anyone really claim to be “Shocked, shocked! There’s a lack of ethics there!”?

  4. mlvlatina Avatar
    mlvlatina

    No integrity.

Leave a Reply

Your email address will not be published. Required fields are marked *

Pin It on Pinterest

Shares
Share This