can anyone become an internet troll? - web science · 2016-06-07 · complementing observational...
TRANSCRIPT
![Page 1: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/1.jpg)
Can Anyone Become an Internet Troll?
Jure Leskovec (@jure)Including joint work with Justin Cheng,M. Bernstein, C. Danescu-Niculescu-Mizil
![Page 2: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/2.jpg)
Two Metaphors for the WebWeb as a library Web as a social place
+–
You?
Me! No, me!
![Page 3: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/3.jpg)
Web as a Social Place
3
Crucial for the success of the Web is the ability of
users to contribute content and provide feedback
![Page 4: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/4.jpg)
![Page 5: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/5.jpg)
![Page 6: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/6.jpg)
![Page 7: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/7.jpg)
Two Metaphors for the WebWeb as a library§ One static
version§ Everyone’s
experience is fixed
§ No user interactions
Web as a social place§ Complex dynamic
feedback effects§ One user’s
experience is a function of other users’ experiences
§ User interactions
![Page 8: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/8.jpg)
ChallengesWant users to engage and contribute (good) content (and make others’ experiences better)§ How do we keep users engaged?§ How do we promote good behavior?§ How do we ensure health of the
community and the content ecosystem?
![Page 9: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/9.jpg)
Challenges§ We know a lot about overall average user
behavior in online communities§ But many times few users can have a
disproportionate (negative) effect on the entire community
§ How do we study and model behavior of such users?
§ Have do we use the insights to improve online communities?
![Page 10: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/10.jpg)
10
![Page 11: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/11.jpg)
Antisocial BehaviorInternet Troll is a person who sows discord on the Internet§ by starting arguments and upsetting
people§ by posting inflammatory, extraneous,
or off-topic messages in an online community
11
[wikipedia]
![Page 12: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/12.jpg)
12
![Page 13: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/13.jpg)
13
Girls are focused on their image way too much. This isn't new. And when they talk about other girls 'fat', 'ugly', and 'b*tch' are recurring words.
![Page 14: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/14.jpg)
14
It also shows that Islam and Christianity teaching women to dress modest could be right afterall.
![Page 15: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/15.jpg)
15
Religious nut alert
Clearly that is the only logical conclusion to this article. Now if you'll excuse me, I need to iron my tarp. I have work on Monday, and I want to appear 'modest'.
fail at life. go bomb yourself.
It also shows that Islam and Christianity teaching women to dress modest could be right afterall.
![Page 16: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/16.jpg)
16
Both of these little skanks are ugly to the bone.
![Page 17: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/17.jpg)
17
Both of these little skanks are ugly to the bone.
Troll
![Page 18: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/18.jpg)
PROBLEM: Trolls disrupt online discussions
Baker, P. (2001); Donath, J. S. (1999); Herring, S., et al. (2011); Shachaf, P. and Hara, N. (2010)
18
![Page 19: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/19.jpg)
Facebook is for losers with no friends in real life.
Less positive
19
Facebook:Theencyclopediaofbeauty?
![Page 20: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/20.jpg)
Suck a f****ng c@ck u d@uchebag
More profanity
20
SoccerstarMuamba still'critical'afteron-pitchcollapse
![Page 21: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/21.jpg)
I have a question for CNN.......why as a
Penny, once again you show why you are one of the best of the league. Always a class-act. Always a good example.
Fantastic story. Kudos to Penny Hardaway. This is what you call giving back.
*sniff* This had me tearing up in the office. Great job Penny! You were always a class act on and off the court. Your Grandmother is applauding you again in heaven!
21
Ex-NBAstarreturnstoinnercity,bringshoopdreams
![Page 22: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/22.jpg)
stop reading then. Just sayin.. CNN not making you read...you chose to.
How many white NBA players grew up
I have a question for CNN.......why as a reader do I not see any articles similar to this about white NBA basketball players?......every single touchy feely story is about a black ball player and then all you do is blast Tim Tebow............YOU GUYS MAKE ME SICK AS A READER !
Penny, once again you show why you are one of the best of the league. Always a class-act. Always a good example.
This is what you call giving back.
Off-topic: Less similar to previous posts
22
![Page 23: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/23.jpg)
many stories of charitable efforts by white NBA players, but this isn't a story about a Black NBA player...it's a story about someone who came back to their roots to contribute.
b\c young black men need to see examples from their own race. They need to see that even minorities can succeed and give back instead. They connect with people of their own race. It allows for mentorship and guidance.
A story about a millionaire helping kids in his poor neighborhood personally, and being a positive role model makes you sick as a reader. Gotta love conservatives.
Gets more replies from other users
23
![Page 24: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/24.jpg)
If y ou clai m yo u w an t le ss go ve rn me nt bu t w an t t o c on tr ol t he b ed ro om ,yo u'r e a R ep ublic an ; I f y ou wa nt tocut Ed uca tio n, yo u're a R ep ublic an ; I f y ou w an t t o cut So cial Sec uri ty, yo u'r e a Re pu blica n; If yo u wa nt tocut M edic ar e an d Me dic aid, yo u'r e a R ep ublic an If yo u clai m y ou wa nt les s g ov er nm en t but wa nt to co nt ro lthe be dr oo m,y ou' re a Re pu blic an; I f yo u wa nt t o cu t Ed uca tio n, yo u'r e a R ep ubli ca n; If y ou w an t to c u tSocial Sec uri ty, yo u'r e a Re pu blic an; If yo u wa nt t o c ut M edi ca re an d Me dic aid, y ou'r e a R ep ubli ca n y ou' rea Re pu blica n; I f yo u wa nt t o c ut So cial Se cu rity, yo u're a R ep ublic an ; If y ou w an t to cut Me dic ar e a ndMedicaid, Republican.
If y ou clai m yo u w an t le ss go ve rn me nt bu t w an t t o c on tr ol t he b ed ro om ,yo u'r e a R ep ublic an ; I f y ou wa nt tocut Ed uca tio n, yo u're a R ep ublic an ; I f y ou w an t t o cut So cial Sec uri ty, yo u'r e a Re pu blica n; If yo u wa nt tocut M edic ar e an d Me dic aid, yo u'r e a R ep ublic an If yo u clai m y ou wa nt les s g ov er nm en t but wa nt to co nt ro lthe be dr oo m,y ou' re a Re pu blic an; I f yo u wa nt t o cu t Ed uca tio n, yo u'r e a R ep ubli ca n; If y ou w an t to c u tSocial Sec uri ty, yo u'r e a Re pu blic an; If yo u wa nt t o c ut M edi ca re an d Me dic aid, y ou'r e a R ep ubli ca n y ou' rea Re pu blica n; I f yo u wa nt t o c ut So cial Se cu rity, yo u're a R ep ublic an ; If y ou w an t to cut Me dic ar e a ndMedicaid, Republican.
If y ou clai m yo u w an t le ss go ve rn me nt bu t w an t t o c on tr ol t he b ed ro om ,yo u'r e a R ep ublic an ; I f y ou wa nt tocut Ed uca tio n, yo u're a R ep ublic an ; I f y ou w an t t o cut So cial Sec uri ty, yo u'r e a Re pu blica n; If yo u wa nt tocut M edic ar e an d Me dic aid, yo u'r e a R ep ublic an If yo u clai m y ou wa nt les s g ov er nm en t but wa nt to co nt ro lthe be dr oo m,y ou' re a Re pu blic an; I f yo u wa nt t o cu t Ed uca tio n, yo u'r e a R ep ubli ca n; If y ou w an t to c u tSocial Sec uri ty, yo u'r e a Re pu blic an; If yo u wa nt t o c ut M edi ca re an d Me dic aid, y ou'r e a R ep ubli ca n y ou' rea Re pu blica n; I f yo u wa nt t o c ut So cial Se cu rity, yo u're a R ep ublic an ; If y ou w an t to cut Me dic ar e a ndMedicaid, Republican.
If yo u cl aim yo u wa nt l ess go ve rn me nt but wa nt t o c ont rol th e b ed ro om ,yo u'r e aRep ubli ca n; If y ou w an t to c ut Ed uc atio n, y ou' re a Re pu blica n; If y ou wa nt to cut S ocialSecu rity , yo u're a Re pu blic an; I f yo u wa nt t o cut Me dic ar e a nd Me dicai d, y ou' re aRep ubli ca n I f y ou cl aim y ou wa nt les s gov er n me nt bu t w an t t o co ntr ol th e be dr oo m, yo u'rea R ep ubli ca n; If yo u wa nt to cu t E du cati on , y ou' re a R ep ublic an ; I f yo u w an t t o cu t S ocialSecu rity , yo u're a Re pu blic an; I f yo u wa nt t o cut Me dic ar e a nd Me dicai d, y ou' re aRep ubli ca n y ou' re a Re pu blic an; If y ou wa nt to c ut S oci al S ecu rity , y ou' re a Re pu blic an; Ifyou want to cut Medicare and Medicaid, Republican.If yo u cl aim yo u wa nt l ess go ve rn me nt but wa nt t o c ont rol th e b ed ro om ,yo u'r e aRep ubli ca n; If y ou w an t to c ut Ed uc atio n, y ou' re a Re pu blica n; If y ou wa nt to cut S ocialSecu rity , yo u're a Re pu blic an; I f yo u wa nt t o cut Me dic ar e a nd Me dicai d, y ou' re aRep ubli ca n I f y ou cl aim y ou wa nt les s gov er n me nt bu t w an t t o co ntr ol th e be dr oo m, yo u'rea R ep ubli ca n; If yo u wa nt to cu t E du cati on , y ou' re a R ep ublic an ; I f yo u w an t t o cu t S ocialSecu rity , yo u're a Re pu blic an; I f yo u wa nt t o cut Me dic ar e a nd Me dicai d, y ou' re aRep ubli ca n y ou' re a Re pu blic an; If y ou wa nt to c ut S oci al S ecu rity , y ou' re a Re pu blic an; Ifyou want to cut Medicare and Medicaid, Republican.If yo u cl aim yo u wa nt l ess go ve rn me nt but wa nt t o c ont rol th e b ed ro om ,yo u'r e aRep ubli ca n; If y ou w an t to c ut Ed uc atio n, y ou' re a Re pu blica n; If y ou wa nt to cut S ocialSecu rity , yo u're a Re pu blic an; I f yo u wa nt t o cut Me dic ar e a nd Me dicai d, y ou' re aRep ubli ca n I f y ou cl aim y ou wa nt les s gov er n me nt bu t w an t t o co ntr ol th e be dr oo m, yo u'rea R ep ubli ca n; If yo u wa nt to cu t E du cati on , y ou' re a R ep ublic an ; I f yo u w an t t o cu t S ocialSecu rity , yo u're a Re pu blic an; I f yo u wa nt t o cut Me dic ar e a nd Me dicai d, y ou' re aRep ubli ca n y ou' re a Re pu blic an; If y ou wa nt to c ut S oci al S ecu rity , y ou' re a Re pu blic an; Ifyou want to cut Medicare and Medicaid, Republican.
Posts more frequently
Posts more in the same thread
If y ou clai m yo u w an t le ss go ve rn me nt bu t w an t t o c on tr ol t he b ed ro om ,yo u'r e a R ep ublic an ; I f y ou wa nt tocut Ed uca tio n, yo u're a R ep ublic an ; I f y ou w an t t o cut So cial Sec uri ty, yo u'r e a Re pu blica n; If yo u wa nt tocut M edic ar e an d Me dic aid, yo u'r e a R ep ublic an If yo u clai m y ou wa nt les s g ov er nm en t but wa nt to co nt ro lthe be dr oo m,y ou' re a Re pu blic an; I f yo u wa nt t o cu t Ed uca tio n, yo u'r e a R ep ubli ca n; If y ou w an t to c u tSocial Sec uri ty, yo u'r e a Re pu blic an; If yo u wa nt t o c ut M edi ca re an d Me dic aid, y ou'r e a R ep ubli ca n y ou' rea Re pu blica n; I f yo u wa nt t o c ut So cial Se cu rity, yo u're a R ep ublic an ; If y ou w an t to cut Me dic ar e a ndMedicaid, Republican.
If y ou clai m yo u w an t le ss go ve rn me nt bu t w an t t o c on tr ol t he b ed ro om ,yo u'r e a R ep ublic an ; I f y ou wa nt tocut Ed uca tio n, yo u're a R ep ublic an ; I f y ou w an t t o cut So cial Sec uri ty, yo u'r e a Re pu blica n; If yo u wa nt tocut M edic ar e an d Me dic aid, yo u'r e a R ep ublic an If yo u clai m y ou wa nt les s g ov er nm en t but wa nt to co nt ro lthe be dr oo m,y ou' re a Re pu blic an; I f yo u wa nt t o cu t Ed uca tio n, yo u'r e a R ep ubli ca n; If y ou w an t to c u tSocial Sec uri ty, yo u'r e a Re pu blic an; If yo u wa nt t o c ut M edi ca re an d Me dic aid, y ou'r e a R ep ubli ca n y ou' rea Re pu blica n; I f yo u wa nt t o c ut So cial Se cu rity, yo u're a R ep ublic an ; If y ou w an t to cut Me dic ar e a ndMedicaid, Republican.
If y ou clai m yo u w an t le ss go ve rn me nt bu t w an t t o c on tr ol t he b ed ro om ,yo u'r e a R ep ublic an ; I f y ou wa nt tocut Ed uca tio n, yo u're a R ep ublic an ; I f y ou w an t t o cut So cial Sec uri ty, yo u'r e a Re pu blica n; If yo u wa nt tocut M edic ar e an d Me dic aid, yo u'r e a R ep ublic an If yo u clai m y ou wa nt les s g ov er nm en t but wa nt to co nt ro l 24
![Page 25: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/25.jpg)
Discussion Communities
Complete data over 18 months, ~1.7 million users generated ~40 million posts and ~100 million votes
25
![Page 26: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/26.jpg)
Post Deletion Rate 21.4% 2.3% 2.7%
User Ban Rate 3.3% 1.7% 2.2%
How Common is Bad Behavior?
26
![Page 27: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/27.jpg)
This Talk: Trolling§ Large-scale investigation of
trolling in online discussion communities
§ Common theme:Complementing observational data with experiments
27
![Page 28: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/28.jpg)
This Talk: Quesitons1) Who are trolls?
§ Mechanisms behind trolling behaviors: Mood and discussion context
2) Does feedback mitigate trolling?§ Votes lead to cascading of bad behaviors
3) Can we detect trolling early?§ Predicting whether a given user will be
blocked from the community
![Page 29: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/29.jpg)
Who are these people?What makes them troll?Why is trolling so common?
29
![Page 30: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/30.jpg)
30
124
Donate (1999), Baker (2001), Schwartz (2008)
124
Donate (1999), Baker (2001), Schwartz (2008)
124
Donate (1999), Baker (2001), Schwartz (2008)
![Page 31: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/31.jpg)
Studies on TrollingPrior research suggests: § Trolls are born and not made
[Baker ‘01; Herring et al. ‘11; Shachaf-Hara ‘10]§ Trolls have particular personality
[Buckels et al. ’14; Raine ‘02]§ Trolls are sociopathic individuals
[Rensin ’14; Schwartz ‘08]
§ Q: Are trolls really special people or can anyone become a troll?
![Page 32: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/32.jpg)
32
![Page 33: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/33.jpg)
Hypotheses: Mood & Context
33
How can we study these?
130
Mood Discussion ContextH1: Negative mood increases a user’s likelihood of trolling
H2: The discussion context affects a user’s
susceptibility to troll.
![Page 34: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/34.jpg)
Designing an Experiment
MTukers only care about being paid.How do we design an experiment?
34131
![Page 35: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/35.jpg)
Designing an ExperimentTwo-by-two experimental design:§ Step 1: Mood – Take a test (5min)
§ Easy test & tell people they did well§ Hard test & tell people they did poorly
§ Step 2: Context – Be in a discussion§ Neutral context of neutral comments§ Negative context of negative comments
People do not know the real purpose. Are debriefed at the end.
35
![Page 36: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/36.jpg)
Controlling Mood: POSMOOD
36
Controlling for Mood
132
![Page 37: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/37.jpg)
Controlling Mood: NEGMOOD
![Page 38: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/38.jpg)
Context: NEUTRALCONTEXT
![Page 39: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/39.jpg)
Context: NEGCONTEXT
39
Controlling for Discussion Context
133
![Page 40: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/40.jpg)
Mechanical Turk Experiment
667 participants, 40% female, age 34.2, 54% Democrat, 25% Moderate, 21% Republican
Contributed 791 posts (37.8 words/post), 1392 votes40131
![Page 41: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/41.jpg)
Results: Test Performance§ POSMOOD (easy test):
11 out of 15 questions correct§ Outperforming stated avg. score of 8
§ NEGMOOD (hard test):1.9 out of 15 questions correct§ After below stated avg. score of 8
Does the test score affect user’s mood?
![Page 42: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/42.jpg)
Results: Does it affect mood?§ After the test: 65 questions on how
people were feeling § Based on the Profile of Mood States
§ People in NEGMOOD felt worse:§ Higher anger, confusion, depression,
fatigue, and tension scores
§ How good were the comments?42
![Page 43: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/43.jpg)
Comments: MOOD
§ Users in the negative mood condition are more likely to troll (~80% higher odds, p < 0.05)
43
Prop. Troll Posts Negative Affect (LIWC)POSMOOD NEGMOOD POSMOOD NEGMOOD
NEUTRALCONTEXT 0.35 0.49 0.011 0.014NEGCONTEXT 0.47 0.68 0.023 0.029
Table 1: Both the proportion of user-written posts that werelabeled as trolling and negative affect are lowest in the(POSMOOD, NEUTRALCONTEXT) condition, and highest inthe (NEGMOOD, NEGCONTEXT) condition.
NEGCONTEXT condition, the first three comments were trollposts:
Oh yes. By all means, vote for a Wall Street sellout – alying, abuse-enabling, soon-to-be felon as our next Pres-ident. And do it for your daughter. You’re quite the rolemodel.
In the NEUTRALCONTEXT, they were more innocuous:
I’m a woman, and I don’t think you should vote for awoman just because she is a woman. Vote for her be-cause you believe she deserves it.
These comments were slightly edited from real commentsposted by users in comments in the original article, as wellas other online discussion forums discussing the issue (e.g.,Reddit).
Does this explain universes well? To ensure that the effectswe observed were not path-dependent (i.e., if a discussionbreaks down by chance because of a single user), we createdeight separate “universes” for each condition [71], for a totalof 32 universes. To replicate the interactions that may takeplace in actual online discussions, multiple participants wereassigned to the same “universe” within each condition. Par-ticipants assigned to the same universe could see and respondto other participants who had commented prior, but not inter-act with participants from other universes.
Measuring discussion quality. We evaluated discussionquality in two ways: whether subsequent posts written weretroll posts, and the difference in negative affect of these posts.To evaluate whether a post was a troll post or not, two ex-perts independently labeled posts as being troll or non-trollposts, with disagreements resolved through discussion. Bothexperts reviewed CNN.com’s community guidelines [21] forcommenting – at a high level, posts that were offensive, irrel-evant, or designed to elicit an angry response, whether inten-tional or not, were flagged as trolling. To measure the neg-ative affect of a post, we used LIWC [65] (Vader [39] givesempirically similar results).
[MSB: I’m increasingly a fan of describing the statistical testsup in the method section. What tests did you run on the data?]Like every single statistical test?
Results667 participants (40% female, mean age of 34.2, 54% Demo-crat, 25% Moderate, 21% Republican) completed the experi-ment, with an average of 21 participants in each universe. In
Fixed Effects Coef. SE z(Intercept) �0.70⇤⇤⇤ 0.17 �4.23NEGMOOD 0.64⇤⇤ 0.24 2.66NEGCONTEXT 0.52⇤ 0.23 2.38NEGMOOD ⇥ NEGCONTEXT 0.41 0.33 1.23Random Effects Var. SEUser 0.41 0.64
Table 2: A mixed effects logistic regression reveals a signif-icant effect of both NEGMOOD and NEGCONTEXT on trollposts (⇤: p<0.05, ⇤⇤: p<0.01, ⇤⇤⇤: p<0.001). In other words,both negative mood and the presence of initial troll posts in-creases the probability of trolling.
aggregate, these workers contributed 791 posts (with an aver-age of 37.8 words written per post) and 1392 votes.
Manipulation checks. On average, participants in the POS-MOOD condition obtained an average of 11.2 out of 15 ques-tions correct, performing above the stated “average” score of8. In contrast, participants in the NEGMOOD condition an-swered only an average of 1.9 questions correctly, perform-ing significantly worse (t(594)=63.2, p<0.001), and belowthe stated “average”. Correspondingly, the post-test POMSquestionnaire confirmed that participants in the NEGMOODcondition experienced higher mood disturbance on all axes,with higher anger, confusion, depression, fatigue, and tensionscores, and a lower vigor score (t>7.0, p<0.001). Total mooddisturbance, where a higher scores correspond to more nega-tive mood, was 12.2 for participants in the POSMOOD condi-tion (comparable to a baseline level of disturbance measuredamong athletes [81]), and 40.8 in the NEGMOOD condition.
Similarly, the initial posts in the NEGCONTEXT conditionobtained a significantly lower proportion of up-votes (0.36)as compared to those in the NEUTRALCONTEXT condition(0.90) (t(507)=15.7, p<0.001), demonstrating that partici-pants did perceive these initial troll posts worse.
Negative mood and prior troll posts reduce discussionquality. Table 1 demonstrates how the proportion of trollposts and negative affect (or the proportion of negative words)differ in each condition. The proportion of troll posts washighest in the (POSMOOD, NEUTRALCONTEXT) condition,drops in both the (NEGMOOD, NEUTRALCONTEXT) and(POSMOOD, NEGCONTEXT) conditions, and is lowest in the(NEGMOOD, NEGCONTEXT) condition, which suggests amain effect of either condition. For negative affect, we ob-serve similar, but smaller differences.
Fitting a mixed effects logistic regression model, with thetwo conditions as fixed effects, user as a random effect, andwhether a post was trolling or not as the outcome variable,we do observe a significant effect of both NEGMOOD andNEGCONTEXT in this model (p < 0.05) (Table 2), confirm-ing both H1 and H2. In fact, the presence of prior troll postsincreases the odds of trolling by 64%, and negative mood in-creases the odds of trolling by 79%. An ANOVA comparingmodels without these effects also revealed significant differ-
5
![Page 44: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/44.jpg)
Comments: CONTEXT
§ Users in the negative context condition are more likely to troll (~60% higher odds, p < 0.05)
44
Prop. Troll Posts Negative Affect (LIWC)POSMOOD NEGMOOD POSMOOD NEGMOOD
NEUTRALCONTEXT 0.35 0.49 0.011 0.014NEGCONTEXT 0.47 0.68 0.023 0.029
Table 1: Both the proportion of user-written posts that werelabeled as trolling and negative affect are lowest in the(POSMOOD, NEUTRALCONTEXT) condition, and highest inthe (NEGMOOD, NEGCONTEXT) condition.
NEGCONTEXT condition, the first three comments were trollposts:
Oh yes. By all means, vote for a Wall Street sellout – alying, abuse-enabling, soon-to-be felon as our next Pres-ident. And do it for your daughter. You’re quite the rolemodel.
In the NEUTRALCONTEXT, they were more innocuous:
I’m a woman, and I don’t think you should vote for awoman just because she is a woman. Vote for her be-cause you believe she deserves it.
These comments were slightly edited from real commentsposted by users in comments in the original article, as wellas other online discussion forums discussing the issue (e.g.,Reddit).
Does this explain universes well? To ensure that the effectswe observed were not path-dependent (i.e., if a discussionbreaks down by chance because of a single user), we createdeight separate “universes” for each condition [71], for a totalof 32 universes. To replicate the interactions that may takeplace in actual online discussions, multiple participants wereassigned to the same “universe” within each condition. Par-ticipants assigned to the same universe could see and respondto other participants who had commented prior, but not inter-act with participants from other universes.
Measuring discussion quality. We evaluated discussionquality in two ways: whether subsequent posts written weretroll posts, and the difference in negative affect of these posts.To evaluate whether a post was a troll post or not, two ex-perts independently labeled posts as being troll or non-trollposts, with disagreements resolved through discussion. Bothexperts reviewed CNN.com’s community guidelines [21] forcommenting – at a high level, posts that were offensive, irrel-evant, or designed to elicit an angry response, whether inten-tional or not, were flagged as trolling. To measure the neg-ative affect of a post, we used LIWC [65] (Vader [39] givesempirically similar results).
[MSB: I’m increasingly a fan of describing the statistical testsup in the method section. What tests did you run on the data?]Like every single statistical test?
Results667 participants (40% female, mean age of 34.2, 54% Demo-crat, 25% Moderate, 21% Republican) completed the experi-ment, with an average of 21 participants in each universe. In
Fixed Effects Coef. SE z(Intercept) �0.70⇤⇤⇤ 0.17 �4.23NEGMOOD 0.64⇤⇤ 0.24 2.66NEGCONTEXT 0.52⇤ 0.23 2.38NEGMOOD ⇥ NEGCONTEXT 0.41 0.33 1.23Random Effects Var. SEUser 0.41 0.64
Table 2: A mixed effects logistic regression reveals a signif-icant effect of both NEGMOOD and NEGCONTEXT on trollposts (⇤: p<0.05, ⇤⇤: p<0.01, ⇤⇤⇤: p<0.001). In other words,both negative mood and the presence of initial troll posts in-creases the probability of trolling.
aggregate, these workers contributed 791 posts (with an aver-age of 37.8 words written per post) and 1392 votes.
Manipulation checks. On average, participants in the POS-MOOD condition obtained an average of 11.2 out of 15 ques-tions correct, performing above the stated “average” score of8. In contrast, participants in the NEGMOOD condition an-swered only an average of 1.9 questions correctly, perform-ing significantly worse (t(594)=63.2, p<0.001), and belowthe stated “average”. Correspondingly, the post-test POMSquestionnaire confirmed that participants in the NEGMOODcondition experienced higher mood disturbance on all axes,with higher anger, confusion, depression, fatigue, and tensionscores, and a lower vigor score (t>7.0, p<0.001). Total mooddisturbance, where a higher scores correspond to more nega-tive mood, was 12.2 for participants in the POSMOOD condi-tion (comparable to a baseline level of disturbance measuredamong athletes [81]), and 40.8 in the NEGMOOD condition.
Similarly, the initial posts in the NEGCONTEXT conditionobtained a significantly lower proportion of up-votes (0.36)as compared to those in the NEUTRALCONTEXT condition(0.90) (t(507)=15.7, p<0.001), demonstrating that partici-pants did perceive these initial troll posts worse.
Negative mood and prior troll posts reduce discussionquality. Table 1 demonstrates how the proportion of trollposts and negative affect (or the proportion of negative words)differ in each condition. The proportion of troll posts washighest in the (POSMOOD, NEUTRALCONTEXT) condition,drops in both the (NEGMOOD, NEUTRALCONTEXT) and(POSMOOD, NEGCONTEXT) conditions, and is lowest in the(NEGMOOD, NEGCONTEXT) condition, which suggests amain effect of either condition. For negative affect, we ob-serve similar, but smaller differences.
Fitting a mixed effects logistic regression model, with thetwo conditions as fixed effects, user as a random effect, andwhether a post was trolling or not as the outcome variable,we do observe a significant effect of both NEGMOOD andNEGCONTEXT in this model (p < 0.05) (Table 2), confirm-ing both H1 and H2. In fact, the presence of prior troll postsincreases the odds of trolling by 64%, and negative mood in-creases the odds of trolling by 79%. An ANOVA comparingmodels without these effects also revealed significant differ-
5
![Page 45: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/45.jpg)
Comments: Negativity
§ Negative mood and context lead to more negative posts
45
Prop. Troll Posts Negative Affect (LIWC)POSMOOD NEGMOOD POSMOOD NEGMOOD
NEUTRALCONTEXT 0.35 0.49 0.011 0.014NEGCONTEXT 0.47 0.68 0.023 0.029
Table 1: Both the proportion of user-written posts that werelabeled as trolling and negative affect are lowest in the(POSMOOD, NEUTRALCONTEXT) condition, and highest inthe (NEGMOOD, NEGCONTEXT) condition.
NEGCONTEXT condition, the first three comments were trollposts:
Oh yes. By all means, vote for a Wall Street sellout – alying, abuse-enabling, soon-to-be felon as our next Pres-ident. And do it for your daughter. You’re quite the rolemodel.
In the NEUTRALCONTEXT, they were more innocuous:
I’m a woman, and I don’t think you should vote for awoman just because she is a woman. Vote for her be-cause you believe she deserves it.
These comments were slightly edited from real commentsposted by users in comments in the original article, as wellas other online discussion forums discussing the issue (e.g.,Reddit).
Does this explain universes well? To ensure that the effectswe observed were not path-dependent (i.e., if a discussionbreaks down by chance because of a single user), we createdeight separate “universes” for each condition [71], for a totalof 32 universes. To replicate the interactions that may takeplace in actual online discussions, multiple participants wereassigned to the same “universe” within each condition. Par-ticipants assigned to the same universe could see and respondto other participants who had commented prior, but not inter-act with participants from other universes.
Measuring discussion quality. We evaluated discussionquality in two ways: whether subsequent posts written weretroll posts, and the difference in negative affect of these posts.To evaluate whether a post was a troll post or not, two ex-perts independently labeled posts as being troll or non-trollposts, with disagreements resolved through discussion. Bothexperts reviewed CNN.com’s community guidelines [21] forcommenting – at a high level, posts that were offensive, irrel-evant, or designed to elicit an angry response, whether inten-tional or not, were flagged as trolling. To measure the neg-ative affect of a post, we used LIWC [65] (Vader [39] givesempirically similar results).
[MSB: I’m increasingly a fan of describing the statistical testsup in the method section. What tests did you run on the data?]Like every single statistical test?
Results667 participants (40% female, mean age of 34.2, 54% Demo-crat, 25% Moderate, 21% Republican) completed the experi-ment, with an average of 21 participants in each universe. In
Fixed Effects Coef. SE z(Intercept) �0.70⇤⇤⇤ 0.17 �4.23NEGMOOD 0.64⇤⇤ 0.24 2.66NEGCONTEXT 0.52⇤ 0.23 2.38NEGMOOD ⇥ NEGCONTEXT 0.41 0.33 1.23Random Effects Var. SEUser 0.41 0.64
Table 2: A mixed effects logistic regression reveals a signif-icant effect of both NEGMOOD and NEGCONTEXT on trollposts (⇤: p<0.05, ⇤⇤: p<0.01, ⇤⇤⇤: p<0.001). In other words,both negative mood and the presence of initial troll posts in-creases the probability of trolling.
aggregate, these workers contributed 791 posts (with an aver-age of 37.8 words written per post) and 1392 votes.
Manipulation checks. On average, participants in the POS-MOOD condition obtained an average of 11.2 out of 15 ques-tions correct, performing above the stated “average” score of8. In contrast, participants in the NEGMOOD condition an-swered only an average of 1.9 questions correctly, perform-ing significantly worse (t(594)=63.2, p<0.001), and belowthe stated “average”. Correspondingly, the post-test POMSquestionnaire confirmed that participants in the NEGMOODcondition experienced higher mood disturbance on all axes,with higher anger, confusion, depression, fatigue, and tensionscores, and a lower vigor score (t>7.0, p<0.001). Total mooddisturbance, where a higher scores correspond to more nega-tive mood, was 12.2 for participants in the POSMOOD condi-tion (comparable to a baseline level of disturbance measuredamong athletes [81]), and 40.8 in the NEGMOOD condition.
Similarly, the initial posts in the NEGCONTEXT conditionobtained a significantly lower proportion of up-votes (0.36)as compared to those in the NEUTRALCONTEXT condition(0.90) (t(507)=15.7, p<0.001), demonstrating that partici-pants did perceive these initial troll posts worse.
Negative mood and prior troll posts reduce discussionquality. Table 1 demonstrates how the proportion of trollposts and negative affect (or the proportion of negative words)differ in each condition. The proportion of troll posts washighest in the (POSMOOD, NEUTRALCONTEXT) condition,drops in both the (NEGMOOD, NEUTRALCONTEXT) and(POSMOOD, NEGCONTEXT) conditions, and is lowest in the(NEGMOOD, NEGCONTEXT) condition, which suggests amain effect of either condition. For negative affect, we ob-serve similar, but smaller differences.
Fitting a mixed effects logistic regression model, with thetwo conditions as fixed effects, user as a random effect, andwhether a post was trolling or not as the outcome variable,we do observe a significant effect of both NEGMOOD andNEGCONTEXT in this model (p < 0.05) (Table 2), confirm-ing both H1 and H2. In fact, the presence of prior troll postsincreases the odds of trolling by 64%, and negative mood in-creases the odds of trolling by 79%. An ANOVA comparingmodels without these effects also revealed significant differ-
5
Prop. Troll Posts Negative Affect (LIWC)POSMOOD NEGMOOD POSMOOD NEGMOOD
NEUTRALCONTEXT 0.35 0.49 0.011 0.014NEGCONTEXT 0.47 0.68 0.023 0.029
Table 1: Both the proportion of user-written posts that werelabeled as trolling and negative affect are lowest in the(POSMOOD, NEUTRALCONTEXT) condition, and highest inthe (NEGMOOD, NEGCONTEXT) condition.
NEGCONTEXT condition, the first three comments were trollposts:
Oh yes. By all means, vote for a Wall Street sellout – alying, abuse-enabling, soon-to-be felon as our next Pres-ident. And do it for your daughter. You’re quite the rolemodel.
In the NEUTRALCONTEXT, they were more innocuous:
I’m a woman, and I don’t think you should vote for awoman just because she is a woman. Vote for her be-cause you believe she deserves it.
These comments were slightly edited from real commentsposted by users in comments in the original article, as wellas other online discussion forums discussing the issue (e.g.,Reddit).
Does this explain universes well? To ensure that the effectswe observed were not path-dependent (i.e., if a discussionbreaks down by chance because of a single user), we createdeight separate “universes” for each condition [71], for a totalof 32 universes. To replicate the interactions that may takeplace in actual online discussions, multiple participants wereassigned to the same “universe” within each condition. Par-ticipants assigned to the same universe could see and respondto other participants who had commented prior, but not inter-act with participants from other universes.
Measuring discussion quality. We evaluated discussionquality in two ways: whether subsequent posts written weretroll posts, and the difference in negative affect of these posts.To evaluate whether a post was a troll post or not, two ex-perts independently labeled posts as being troll or non-trollposts, with disagreements resolved through discussion. Bothexperts reviewed CNN.com’s community guidelines [21] forcommenting – at a high level, posts that were offensive, irrel-evant, or designed to elicit an angry response, whether inten-tional or not, were flagged as trolling. To measure the neg-ative affect of a post, we used LIWC [65] (Vader [39] givesempirically similar results).
[MSB: I’m increasingly a fan of describing the statistical testsup in the method section. What tests did you run on the data?]Like every single statistical test?
Results667 participants (40% female, mean age of 34.2, 54% Demo-crat, 25% Moderate, 21% Republican) completed the experi-ment, with an average of 21 participants in each universe. In
Fixed Effects Coef. SE z(Intercept) �0.70⇤⇤⇤ 0.17 �4.23NEGMOOD 0.64⇤⇤ 0.24 2.66NEGCONTEXT 0.52⇤ 0.23 2.38NEGMOOD ⇥ NEGCONTEXT 0.41 0.33 1.23Random Effects Var. SEUser 0.41 0.64
Table 2: A mixed effects logistic regression reveals a signif-icant effect of both NEGMOOD and NEGCONTEXT on trollposts (⇤: p<0.05, ⇤⇤: p<0.01, ⇤⇤⇤: p<0.001). In other words,both negative mood and the presence of initial troll posts in-creases the probability of trolling.
aggregate, these workers contributed 791 posts (with an aver-age of 37.8 words written per post) and 1392 votes.
Manipulation checks. On average, participants in the POS-MOOD condition obtained an average of 11.2 out of 15 ques-tions correct, performing above the stated “average” score of8. In contrast, participants in the NEGMOOD condition an-swered only an average of 1.9 questions correctly, perform-ing significantly worse (t(594)=63.2, p<0.001), and belowthe stated “average”. Correspondingly, the post-test POMSquestionnaire confirmed that participants in the NEGMOODcondition experienced higher mood disturbance on all axes,with higher anger, confusion, depression, fatigue, and tensionscores, and a lower vigor score (t>7.0, p<0.001). Total mooddisturbance, where a higher scores correspond to more nega-tive mood, was 12.2 for participants in the POSMOOD condi-tion (comparable to a baseline level of disturbance measuredamong athletes [81]), and 40.8 in the NEGMOOD condition.
Similarly, the initial posts in the NEGCONTEXT conditionobtained a significantly lower proportion of up-votes (0.36)as compared to those in the NEUTRALCONTEXT condition(0.90) (t(507)=15.7, p<0.001), demonstrating that partici-pants did perceive these initial troll posts worse.
Negative mood and prior troll posts reduce discussionquality. Table 1 demonstrates how the proportion of trollposts and negative affect (or the proportion of negative words)differ in each condition. The proportion of troll posts washighest in the (POSMOOD, NEUTRALCONTEXT) condition,drops in both the (NEGMOOD, NEUTRALCONTEXT) and(POSMOOD, NEGCONTEXT) conditions, and is lowest in the(NEGMOOD, NEGCONTEXT) condition, which suggests amain effect of either condition. For negative affect, we ob-serve similar, but smaller differences.
Fitting a mixed effects logistic regression model, with thetwo conditions as fixed effects, user as a random effect, andwhether a post was trolling or not as the outcome variable,we do observe a significant effect of both NEGMOOD andNEGCONTEXT in this model (p < 0.05) (Table 2), confirm-ing both H1 and H2. In fact, the presence of prior troll postsincreases the odds of trolling by 64%, and negative mood in-creases the odds of trolling by 79%. An ANOVA comparingmodels without these effects also revealed significant differ-
5
![Page 46: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/46.jpg)
Bad mood and context make people write bad posts.
How do we combat that?
46
![Page 47: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/47.jpg)
Sites allow users to vote on the content and hopefully bad posts get voted down
47
![Page 48: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/48.jpg)
![Page 49: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/49.jpg)
![Page 50: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/50.jpg)
![Page 51: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/51.jpg)
51
When a user evaluatesa comment, she is indirectly
evaluating the author
Do such evaluations help authors contribute better
content?How Community Feedback Shapes User BehaviorJ. Cheng, C. Danescu-Niculescu-Mizil, J. LeskovecAAAI ICWSM, 2014
![Page 52: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/52.jpg)
PositiveVote
NegativeVote
?
?
52
![Page 53: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/53.jpg)
§ Number of up-votes?§ Up-votes minus Down-votes?§ Fraction of up-votes?
How to Summarize Feedback?
53
![Page 54: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/54.jpg)
Doesn’t account for down-votes
15
P: Up-Votes
54
Number of up-votes:
0 15 15vs
![Page 55: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/55.jpg)
Doesn’t account for proportion of up-votes
5
P–N : Up – Down Votes
55
Difference in up-votes and down-votes:
0 50 45vs
![Page 56: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/56.jpg)
Doesn’t account for total number of votes
4
P/(P+N): Fraction Upvotes
56
Proportion of up-votes:
1 40 10vs
![Page 57: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/57.jpg)
How is Feedback Perceived?Measure how users perceive votes
§ Crowdsourcing task:Workers rate, on a 7 point scale, if they felt more positive, or negative about receiving a certain number of up/down-votes.
57
![Page 58: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/58.jpg)
7 = Positive
1 = Negative
0 20
20
# of up-votes
# of
dow
n-vo
tes
10
10
Results: Perceiving Votes
User ratings are independent of the total number of votes
58
![Page 59: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/59.jpg)
Number of up-votes
P-NDifference in up/down-votes
P/(P+N)Proportion of up-votes
0.410
0.879
0.920
R2
59
P
![Page 60: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/60.jpg)
Evaluation of a Post§ A post is positively/negatively
evaluated when the proportion of up-votes is higher/lower than a given threshold.
60
P/(P+N) = 9/(9+1) = 0.9 ≥ High Threshold
![Page 61: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/61.jpg)
How does positive/negativefeedback influencesubsequent behavior?
defined by the proportion of up-votes
61
![Page 62: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/62.jpg)
Effects of Evaluations§ Post quality
§ How well you write§ Community bias
§ How people evaluate you§ Posting frequency
§ How regularly you post§ Voting behavior
§ How you vote on others62
![Page 63: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/63.jpg)
Skinner, B. F. (1938). The behavior of organisms: An experimental analysis.
Do Users Improve?Operant conditioning predicts that feedback would guide authors towards better behavior: § up-votes are “reward” stimuli§ down-votes are “punishment” stimuli
63
![Page 64: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/64.jpg)
Brinko, K. T. (1993). The practice of giving feedback to improve teaching: what is effective?Baumeister, R. F., Bratslavsky, E., Finkenauer, C., & Vohs, K. D. (2001). Bad is stronger than good.
Or Do They Get Worse?Feedback can have negative effects:§ People given only positive feedback
tend to become complacent
§ Also, bad impressions are quicker to form and more resistant to disconfirmation
64
![Page 65: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/65.jpg)
…
…≈ ≈
…
…
Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects.
3 posts before 3 posts after
Matching Similar UsersCompare pairs of users who were evaluated differently on similar content
65
![Page 66: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/66.jpg)
Details of MatchingMatch pairs of users where one got positive and one got negative evaluation§ Match based on
§ history text quality§ number of posts§ post length§ overall proportion of up-votes
66
Text quality determined by training a machine learning model using text features, validated using crowd workers.
![Page 67: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/67.jpg)
0.45
0.55
0.65
0.75
0.85
Prop
ortio
n of
up
-vot
es
{ {
Positiveevaluation
Negativeevaluation
*
Before After
Results: Neg-Votes Increase
67
![Page 68: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/68.jpg)
Down-voting because of post content
I hate …
Down-voting because the community dislikes the author
We don’t like you.
Content vs. CommunityCan valuations be explained by textual or community effects?
68
![Page 69: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/69.jpg)
Results: Negativity Bias§ Text quality drops significantly
after a negative evaluationbut does not change after a positive evaluation
p < 0.05 in all communities
To learn more about these types of effects, see Kanouse, D. E., & Hanson Jr, L. R. (1987). Negativity in evaluations.
![Page 70: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/70.jpg)
Evaluations can Affect…
…Community Bias (How people perceive you)
§ How does community perception of a user change after an evaluation?
70
![Page 71: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/71.jpg)
Community Bias
Actual Evaluation P/(P+N)
Text Quality
Text QualityUp-votesDown-votes
0.9
0.8
0.9-0.8= +0.1
Community Bias?
71
![Page 72: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/72.jpg)
User A
User B
…
…
Comparecommunity bias before and afterthe evaluation
}72
![Page 73: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/73.jpg)
Communities DiscriminatePosts made after a negative evaluation were perceived worse than those made after a positive evaluation
§ On CNN, on average, your posts are being evaluated 48% worse than would be predicted by text quality
p < 0.05 in all communities
![Page 74: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/74.jpg)
Before
Mor
e Pos
itive
After
SimilarText Quality
Similar History
Positive Eval.
Negative Eval.
Worse Perception
Worse Text
Summary of Effects
74
![Page 75: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/75.jpg)
Evaluations can Affect…
…Posting Frequency (How often you post)
§ Does feedback regulate posting quantity?
75
![Page 76: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/76.jpg)
Feedback and Posting Activity§ Do users post more after a
positive/negative evaluation?
76
……Before After
Time
![Page 77: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/77.jpg)
Negativity Increases Activity
Users who receive negative feedback post more frequently
77
![Page 78: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/78.jpg)
Evaluations can Affect…
…Voting Behavior (How you vote on others)
§ Does feedback result in subsequent backlash of negative behavior?
78
![Page 79: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/79.jpg)
0.60.625
0.650.675
0.7U
p-vo
tes
give
n
Before After
Positive
Negative*
More Down-Votes Given
Users who receive negative feedback are more likely to down-vote others
79
![Page 80: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/80.jpg)
Summary so far…§ Negatively-evaluated users
§ Write worse (and more!), § Are evaluated worse by the community§ And evaluate others worse
§ Positively-evaluated usersdon’t do any better
80
![Page 81: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/81.jpg)
Is there a downward spiral in communities?
81
![Page 82: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/82.jpg)
82
124
Donate (1999), Baker (2001), Schwartz (2008)
![Page 83: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/83.jpg)
83
124
Donate (1999), Baker (2001), Schwartz (2008)
124
Donate (1999), Baker (2001), Schwartz (2008)
124
Donate (1999), Baker (2001), Schwartz (2008)
124
Donate (1999), Baker (2001), Schwartz (2008)
124
Donate (1999), Baker (2001), Schwartz (2008)
124
Donate (1999), Baker (2001), Schwartz (2008)
124
Donate (1999), Baker (2001), Schwartz (2008)
124
Donate (1999), Baker (2001), Schwartz (2008)
![Page 84: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/84.jpg)
84
125
Donate (1999), Baker (2001), Schwartz (2008)
![Page 85: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/85.jpg)
Downward Spiral in CNN
Proportion of down-votes is increasing over time
85
0.16
0.18
0.2
0.22
0.24
Jan Mar May Jul
Prop
.dow
n-votes
0.8m down-votes
1.7m down-votes
![Page 86: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/86.jpg)
0.18
0.23
Jan Mar May Jul0.10
0.20
Jan Mar May Jul
0
0.12
Jan Mar May Jul0.05
0.11
Nov Jan Mar May
Breitbart allkpop
CNN IGN
86
![Page 87: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/87.jpg)
87
![Page 88: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/88.jpg)
Can we identify antisocial behavior quickly and prevent downward spirals?
88
Antisocial Behavior in Online Discussion CommunitiesJ. Cheng, C. Danescu-Niculescu-Mizil, J. Leskovec.AAAI ICWSM, 2015.
![Page 89: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/89.jpg)
Post# of words…
Activityposts/day…
Community% upvoted…
Moderatordeletions…
Prediction Task§ Task: Detect which users will be
banned in the future
§ ~19k users CNN, balanced dataset, first 10 posts
89
![Page 90: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/90.jpg)
Prediction Results
90
0.70
0.74
0.5 0.6 0.7 0.8 0.9
BagofWords
DeletionRate
ROCAUC
![Page 91: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/91.jpg)
Prediction Results
91
0.70
0.74
0.62
0.5 0.6 0.7 0.8 0.9
BagofWords
DeletionRate
Post
ROCAUC
![Page 92: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/92.jpg)
Prediction Results
92
0.700.74
0.620.73
0.830.84
0.5 0.6 0.7 0.8 0.9
BagofWordsDeletionRate
Post+Activity
+Community+Moderator
ROCAUC
![Page 93: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/93.jpg)
Time to Detection
93
0.70
0.75
0.80
0.85
0.90
0 10 20 30 40Using First N Posts
AUC
>= N5−10
10−2020−40
40−8080+
(a) Performance against # posts
●●
●
●●●
●
●
●
●●●●
●●
●
●●●
●
●
●
●
●
●
●●●●●●●
●
●●●●
●
●●●
●●●●
●●●●●
●●●●
●●●
●
0.825
0.850
0.875
0.900
0 10 20 30Starting from Nth Last Post (Window=5)
AUC
● ● ●10−20 20−40 40−80
(b) Performance against time
Figure 7: (a) Prediction performance increases as more postsare observed. Additionally, users who live longer are moredifficult to identify early on. (b) Using a sliding window offive posts, performance decreases with increasing temporaldistance from the time of deletion.
To understand the relative importance of these features,we computed the classification performance of each in-dividual feature using logistic regression. Unsurprisingly,moderator features are the strongest predictors of beingsubsequently banned (individual feature set AUC=0.75 forCNN), with the most performant feature being the propor-tion of posts deleted (0.73). Community features were nextstrongest (0.75), with a lower proportion of up-votes re-ceived (0.67) and a higher number of reported posts (0.66)both indicators of antisocial behavior. The text similarity ofa post with previous posts in a thread, while correlated withpost deletion, does not improve classifier performance. Ac-tivty features follow (0.66), with the number of posts perday the most indicative (0.64) of being subsequently banned.Post features were collectively the weakest predictors ofwhether a user will ultimately get banned (0.62). Futurework could involve identifying better textual features (e.g.,phrase structure), and take the context of the post (i.e., thesurrounding posts) into account.
How does prediction performance change with the num-ber of posts observed? If the classifier only has access tothe first five posts, it can still predict whether a user will getbanned with an AUC of 0.80 (across all communities). Moregenerally, performance seems to peak near ten posts (Figure7a). The same figure also shows how classifier performancechanges for users with different post counts: the more postsa user eventually makes, the more difficult it is to predictwhether they will get eventually banned later on.
How does prediction performance change with “dis-tance” from when a user gets banned? Instead of lookingat a user’s initial posts, we now consider sliding windowsfive posts in width, starting from the last five posts of a user,to understand if posts made further in the past are as effectiveas predicting whether a user will get banned. As Figure 7bshows, it becomes increasingly difficult to predict whethera user will subsequently get banned the further in time theexamined posts are from when the user gets banned. Thissuggests that changes in both user or community behaviordo occur leading up to a ban.
How does the classifier perform on different types ofusers? We previously identified two types of FBUs: those
Trained onCNN IGN Breitbart
Test
edon CNN 0.84 0.74 0.76
IGN 0.69 0.83 0.74Breitbart 0.74 0.75 0.78
Table 5: Cross-domain classifier performance (AUC) is rela-tively high, suggesting that these learned models generalizeto multiple communities.
with high deletion rates (Hi-FBUs), and those with lowdeletion rates (Lo-FBUs). Overall, the classifier identifiesHi-FBUs (mean recall=0.99) more reliably than Lo-FBUs(mean recall=0.41). As Hi-FBUs exhibit high deletion ratesfrom the beginning of their life while Lo-FBUs do not, fea-tures such as the proportion of deleted posts are highly in-formative in the former case but not the latter. Further, Lo-FBUs write more posts in total than Hi-FBUs, so overall weare examining a smaller fraction of a Hi-FBU’s total life ina community. Still, by assigning higher weights to instancesof Lo-FBUs during training, we can maintain a similar AUC,while increasing recall of Lo-FBUs to 0.57 (but decreasingthat of Hi-Del users to 0.97). If we are only interested dif-ferentiating Lo-FBUs from other users, a classifier trainedon the same features obtains a mean AUC of 0.79 across allcommunities (recall of Lo-FBUs=0.83). In this case, we mayuse other mechanisms to identify Hi-FBUs separately.
How generalizable are these classifiers? Using a modelthat uses all four feature sets, we find that cross-domainperformance is high (mean AUC=0.74) relative to within-domain performance (Table 5), suggesting not only the ap-plicability of these features, but also the generalizability ofmodels learned on single communities. Most striking is thata classifier trained on the Breitbart community only per-forms slightly worse if tested on CNN (0.76) or IGN (0.74)than on Breitbart itself (0.78). We further note that bag-of-words classifiers do not generalize as well to other commu-nities (mean AUC=0.58).
Discussion & ConclusionThis paper presents a data-driven study of antisocial behav-ior in online discussion communities by analyzing users thatare eventually banned from a community. This leads to acharacterization of antisocial users and to an investigation ofthe evolution of their behavior and of community response:users that will eventually be banned not only write worseposts over time, but the community becomes less tolerant ofthem. Next, it proposes a typology of antisocial users basedon post deletion rates. Finally, introduces a system for iden-tifying undesired users early on in their community life.
By using explicit signals of undesirability (i.e., perma-nent banning), we are able to study users engaged in awide variety of antisocial behavior. While scalable, our ap-porach has several limitations. A more fine-grained label-ing of users (perhaps through crowdsourcing), may reveal agreater range of behavior. Similarly, covert instances of an-tisocial behavior (e.g., through deception) might be signif-
User lifetime:
![Page 94: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/94.jpg)
Results: Cross-Domain
94
ROC AUC
0.84 0.74 0.76
0.69 0.83 0.74
0.74 0.75 0.78
Trained on
Test
ed o
n
![Page 95: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/95.jpg)
Conclusion§ Web is a social place!
But people are less inhibited online
§ Important to understand interaction mechanisms and behavioral patterns
§ Towards healthier and more welcoming online communities
95
![Page 96: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/96.jpg)
SummaryLarge-scale longitudinal analysis of antisocial behavior in discussion forums
Combination of data analysis and human experiments
Trolls are not “special” people§ Mood and discussion context
increase the probability of trolling96
![Page 97: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/97.jpg)
SummaryBad behavior can spread§ The effects of negative feedback
are more pronounced§ Write worse and more§ Are evaluated worse by others§ Evaluate others worse
§ Positively-evaluated users, on the other hand, don’t do any better
97
![Page 98: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/98.jpg)
Summary§ Prediction task: Predicting whether a
user will be banned in the future
§ After observing just 10 posts, we can accurately predict whether the user will be banned in the future
§ Generalization performance across communities
98
![Page 99: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/99.jpg)
Further Questions§ Social feedback loops: Are trolls
socially created by harsh feedback?§ Reputation: Effects of user reputation§ Role of moderators: Are moderators
effectively helping and moderating§ Linguistic norms: Linguistic analysis§ Feedback mechanism: Communities
with only positive feedback (likes)99
![Page 100: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/100.jpg)
100
![Page 101: Can Anyone Become an Internet Troll? - Web Science · 2016-06-07 · Complementing observational data with experiments 27. This Talk: Quesitons 1) Who are trolls? § Mechanisms behind](https://reader033.vdocuments.us/reader033/viewer/2022043001/5f78b8bef8d40b7aea338549/html5/thumbnails/101.jpg)
References§ How Community Feedback Shapes User Behavior
J. Cheng, C. Danescu-Niculescu-Mizil, J. Leskovec. AAAI International Conference on Weblogs and Social Media (ICWSM), 2014.
§ Antisocial Behavior in Online Discussion Communities by J. Cheng, C. Danescu-Niculescu-Mizil, J. Leskovec. AAAI International Conference on Weblogs and Social Media (ICWSM), 2015.
101