the ethics of a/b test
TRANSCRIPT
![Page 1: The ethics of A/B test](https://reader036.vdocuments.us/reader036/viewer/2022062503/58f16e691a28abce4e8b45ed/html5/thumbnails/1.jpg)
The Ethics Of A/B Testing
![Page 2: The ethics of A/B test](https://reader036.vdocuments.us/reader036/viewer/2022062503/58f16e691a28abce4e8b45ed/html5/thumbnails/2.jpg)
What is A/B Testing
Also named randomized controlled experiments, online controlled experiments, and split testing.
In marketing, A/B testing is a simple randomized experiment with two variants, A and B.
It sees if different versions of website with little changes make us use more, visit more, click more or buy more.
1
![Page 3: The ethics of A/B test](https://reader036.vdocuments.us/reader036/viewer/2022062503/58f16e691a28abce4e8b45ed/html5/thumbnails/3.jpg)
Example
This example for a website that make two version with little changes
2
![Page 4: The ethics of A/B test](https://reader036.vdocuments.us/reader036/viewer/2022062503/58f16e691a28abce4e8b45ed/html5/thumbnails/4.jpg)
Facebook feeling A/B test
PNAS magazine published results of an experiment on 689,003 users to see if showing them more positive or negative sentiment posts in the News Feed would affect their happiness levels as deduced by what they posted.
3
![Page 5: The ethics of A/B test](https://reader036.vdocuments.us/reader036/viewer/2022062503/58f16e691a28abce4e8b45ed/html5/thumbnails/5.jpg)
The Experiment Had Almost No Effect
Lets take a look on this experiment. Experiment duration : 1 Week
From January 11–18, 2012 People shown fewer positive posts (a
more depressing feed) posted 0.1% fewer positive words in their posts.
People shown fewer negative posts (a happier feed) post 0.07% fewer negative words.
4
![Page 6: The ethics of A/B test](https://reader036.vdocuments.us/reader036/viewer/2022062503/58f16e691a28abce4e8b45ed/html5/thumbnails/6.jpg)
Experiment risk and fails
the National Institute Of Mental Health says 9.5% of Americans have mood disorders, which can often lead to depression.
Some people who are at risk of depression were almost surely part of Facebook’s study group.
Status update isn’t the real emotion, may this people just following the trends they see on Facebook.
Another research says that some people feel sad when they see lots of happy status, because they feels that them life isn’t happy as they see on Facebook.
6
![Page 7: The ethics of A/B test](https://reader036.vdocuments.us/reader036/viewer/2022062503/58f16e691a28abce4e8b45ed/html5/thumbnails/7.jpg)
Facebook Didn’t Get Ethics Board Approval
Facebook only did an internal review to decide if the study was ethical.
the study would likely fail to meet many of the pre-requisites. But actually Facebook manipulates the News Feed all the time to
test what types of stories and designs makes you spend time, money like ,comment or generally more active.
Facebook technically has consent from all users, as its Data Use Policy people automatically agree to when they sign up says
“we may use the information we receive about you…for…data
analysis, testing, research and service improvement.”
5
![Page 8: The ethics of A/B test](https://reader036.vdocuments.us/reader036/viewer/2022062503/58f16e691a28abce4e8b45ed/html5/thumbnails/8.jpg)
Everyone Is A/B Testing
there are lots of A/B Tests but most are pushing for more business-oriented results like increasing usage or clicks or purchases.
A lot of times we don’t use the “real” Facebook. Or Twitter. Or Google, Yahoo, or LinkedIn. We are almost all part of experiments they quietly.
7
![Page 9: The ethics of A/B test](https://reader036.vdocuments.us/reader036/viewer/2022062503/58f16e691a28abce4e8b45ed/html5/thumbnails/9.jpg)
Regulation, Or At Least Safeguards
Offering an opt out Not for every test, but ones with some real risk like showing users a more depressing feed.
8
![Page 10: The ethics of A/B test](https://reader036.vdocuments.us/reader036/viewer/2022062503/58f16e691a28abce4e8b45ed/html5/thumbnails/10.jpg)
Regulation, Or At Least Safeguards (cont.)
Independent protection against harmful emotional manipulation on a grand scale.
At the very least, the tech companies should educate their data scientists and others designing A/B tests about the ethical research methods associated with having experiments approved by the IRB
9
![Page 11: The ethics of A/B test](https://reader036.vdocuments.us/reader036/viewer/2022062503/58f16e691a28abce4e8b45ed/html5/thumbnails/11.jpg)
Till this happen
Till this regulations happen, all of us are Laboratory rats
10
![Page 12: The ethics of A/B test](https://reader036.vdocuments.us/reader036/viewer/2022062503/58f16e691a28abce4e8b45ed/html5/thumbnails/12.jpg)
Thank you :D
11