FromBrusselstoBrexit:Islamophobia,Xenophobia,RacismandReportsofHatefulIncidentsonTwitter

ResearchPreparedforChannel4Dispatches-‘RacistBritain’

CarlMillerFrancescaArcostanzoJoshSmithAlexKrasodomski-JonesSusannWiedlitzkaRoohamJamaliJackDale

CentrefortheAnalysisofSocialMedia,Demos

OVERVIEWTheCentrefortheAnalysisofSocialMedia(CASM)atDemosconductedresearchfeaturedintheChannel4DispatchesDocumentaryRacistBritain,Firstshownon11thJuly,2016.TheresearchfocussedonthereactiononTwittertotwosigniFicantevents:theFirstwastheterroristattacksonZaventemAirportandMaalbeekMetrostationinBrusselsonMarch22nd,2016,andthesecondwastheannouncementofthedecisionoftheUKbyreferendumtoleavetheEuropeanUniononJune24th,2016.

Theoverallobjectiveoftheresearchwas,againstthebackgroundofthegeneralreactiononTwittertothesetwoimportantevents,(a)tounderstandtheamountofIslamophobic,racist,xenophobicorhatefulviewsexpressedonTwitter;and(b)understandtheroleofTwitterasaforumpromotingsolidarityandsupportformigrants,andethnicandreligiousminorities,andasaplatformforthosethemselvesthevictimsofhatecrime.ThespeciFicresearchquestionswere:

FortheEUReferendum:

• towhatextentdidimmigrationasaissuefeatureinthepro-leaveandpro-remaincampaigningonTwitterintherun-uptothereferendum?

• Overtheperiodoftheannouncementofthereferendumresult,whatwasthebroadvolumeofUK-baseddiscussionofmigrationandmigrantsonTwitter?TowhatextentwasthisdiscussionexplicitlylinkedtotheEUreferendum,andhowweremigrantsandmigrationdiscussed?

• WhatwasthevolumeofexplicitexpressionsofxenophobicviewsonTwittersentfromtheUKovertheperiodofthereferendum?TowhatextentdidtheannouncementofaBrexitresultcoincidewithanincreaseinIslamophobicmessagesthatcouldbemeasured?

• HowmanyIslamophobicviewsweresentonTwittersentfromtheUKovertheperiodofthereferendum?TowhatextentdidtheannouncementofaBrexitresultcoincidewithanincreaseIslamophobicmessagesthatcouldbemeasured?

• TowhatextentwasTwitteralsousedasaforumtoshowsolidaritywithandexpresssupportformigrantgroups,andracial,religiousandethnicminoritygroups?

• TowhatextentwasTwitterusedbythepossiblevictimsofxenophobic,Islamophobicorracistabusetodrawattentiontotheirexperiences?

FortheterroristattacksinBrussels:

• WhatwasthebroadvolumeofreactiontotheBrusselsterroristattacksonTwitter?

• Ofthebroadreaction,whatwerethedifferentwaysthatpeopleusedTwittertoreacttotheBrusselsattacks?Towhatextentdidthisreactionincludediscussionofimmigration,integrationandIslam?

• TowhatextentdidthereactiontotheBrusselsattacksonTwitterincludeadiscussionlinkingwiderissuesrelatedtoaperceivedlackofintegration,assimilation,orissueswithinIslamitself?

• HowmanyIslamophobicviewsweresentonTwittersentfromtheUKovertheperiodoftheBrusselsattacks?TowhatextentdidtheBrusselsattackscoincidewithanincreaseIslamophobicmessagesthatcouldbemeasured?

• WhatwasthenatureoftheIslamophobicmessagesthatcouldbeidentiFied?

RESULTSFortheEUReferendum:

Theoverallreaction• ImmigrationwasoneofthekeythemesmadebyBrexitsupportersonTwitterintherun-uptothevote.Between27thMayand2ndJune,28%ofTweetsrelatedtotheEUcampaign,andsentbyTwitteraccountsjudgedtobepro-Brexit,wereaboutimmigration.

• Between22ndJuneand30thJune,258,553TweetsweresentfromtheUKcontainingthewords‘migrant’,‘migrants’,‘immigrant’,‘immigrants’,‘refugee’,and‘refugees’.GeneraldiscussionaboutmigrantsandmigrationsharplyincreasedoverthedayoftheBrexitannouncement.

Xenophobia• Between19thJuneand1stJuly,16,151TweetsweresentfromtheUKcontainingawordorhashtagrelatedtoxenophobia(seeannex).13,236oftheseTweetsweresentbetween24thJuneand1stJuly.ThedayoftheBrexitannouncementitselfsawthehighestvolumeoftheseTweetswithinthisperiod.

• NotallTweetscontainingawordrelatedtoxenophobiaexpressaxenophobicview,ofcourse.Manypeopleusedthesewordsandhashtagstocriticisexenophobia.Ofthe16,151TweetsusingaxenophobictermsentfromtheUK,5,484wereclassiFiedasderogatoryandxenophobic.10,671wereclassiFiedasnotxenophobic(andoftenangrilydenouncedxenophobia). 1

• OnJune24th,707TweetswereclassiFiedasxenophobic,and3,549assupportive.• OnJune25th,502TweetswereclassiFiedasxenophobic,and2,225assupportive.

Islamophobia• Between18Marchand30June,2016,4,123,705TweetsweresentaroundtheworldcontainingawordthatcanbeusedinanIslamophobicway(seeannex). 2

• Ofthese4,123,705Tweets,28,034werejudgedtofromtheUKandwereclassiFiedasexplicitlyanti-Islamicandderogatory.

• Asmallpeakofthevolumeofderogatoryanti-IslamicTweetsoccurredbetween8pmonJune23rdandmidnightonJune25th.• 479derogatoryanti-IslamicTweetsweresentonJune24th• 146derogatoryanti-IslamicTweetsweresentonJune25th

Reportsofof6lineabuseandhatecrimeonTwitter• BetweenJune25thandJuly4th,98,948Tweetsweresentcontainingeitherthehashtag#postrefracismor#safetypin.

• Ofthese98,948Tweets,81,688wereclassiFiedas‘generalawareness’-Tweetsgenerallyexpressingsolidaritywithmigrantgroups,orexpressingageneralconcernwithapossibleincreaseofhatecrimeinthepost-Brexitperiod.

• However,theremaining17,260Tweetsusingeither#postrefracismor#safetypinwereclassiFiedtobeeither(a)providingdirectaccountsofspeciFicincidentsofhatecrimeorhatefulabuse;or(b)relayingotheraccountsofspeciFicincidentsofhatecrimeorhatefulabuse. 3

N.B. Four Tweets were doubly classified as borderline cases. 1

Most of these words were Islamophobic slurs. It is of course possible to express an Islamphobic view without using an 2

Islamophobic slur; so the numbers presented here are not claimed to be exhaustive of the total amount of xenophobic views expressed on Twitter over this time period.

It should be noted, of course, that none of the accounts of specific incidents of hate crime or hateful abuse could be 3

verified.

• Ofthese17,260Tweetsprovidingeitheradirect,orrelayed,accountofspeciFicincidentsofhatecrimeorhatefulabuse,14,847wereRe-tweets,and2,413wereoriginalmessages.

• Ofthese17,260Tweetsprovidingeitheradirect,orrelayed,accountofspeciFicincidentsofhatecrimeorhatefulabuse,itwasjudged(seeannex)that5,690camefromanothercountry,6,720camefromtheUK.

FortheBrusselsAttacks:

TheOverallReaction• FromthestartoftheBrusselsattacksandoverthenextFiveweeks,around13,800,000TweetsweresentaboutBrusselsandinreactiontotheattacks. Around7,000,000weresentonthedayofthe4

attackitself.

• The8,452,661Tweetsthatcontainedthewords‘Brussels’or‘Bruxelles’werealgorithmicallyclassiFiedintothefollowingcategories:• 32.2%ofTweetsexpressingsolidaritywithmigrantgroups,underliningthestrengthofcommonhumanity,andrejectingtheabilityofterroriststodividecommunities.

• 24.1%ofTweetssharingbreakingnewsoftheattackitself.• 18.2%ofTweetsexpressingcondolenceswiththevictimsoftheattacks,theirfamilyandfriends,andthepeopleofBrussels.

• 8%ofTweetssharingongoingnewscoverageoftheattacks.• 5.8%ofTweetsthatlinkedtheattackswiththebroaderissuesofimmigration,integrationandIslam.

(11.7%ofTweetscouldnotbeplacedintoanyofthesecategories).

GeneralDiscussionofIntegrationandIslam• The13,800,000TweetssentinreactiontoBrusselsusedaverylargenumberofhashtags.Ofthe500mostshared,50explicitlyreferredtoIslam.

• Themostpopularhashtagwas#stopislam.AcrossMarch22ndand23rd,327,391Tweetsusedit.However,thehashtagwasusedtohostbothmessagesdefendingMuslimcommunitiesandIslaminthewakeoftheattacks,andalsothosecriticalofthem.OfTweetsusingthehashtag:• 180,000wereclassiFiedasbeingdefensiveofMuslimsandIslam• 146,000wereclassiFiedasexpressingaviewthatbroadlypointedtoissueswithinIslam,migrationoralackofassimilationasunderlyingcontributorstotheBrusselsattacks. 5

• 157,000TweetsweresentcontainingahashtagthatwasexplicitlysupportiveofIslam.• 49,000TweetsweresentcontainingahashtagthatwasexplicitlycriticalofIslam.

Islamophobia• FromMarch22ndtoMarch30th,58,074Tweetsweresentcontainingawordthatcanbeusedasananti-Islamicslur(seeAnnex).

• Ofthese58,074Tweetscontainingawordthatcanbeusedasananti-Islamicslur,4,798wereclassiFiedasangry,severelyderogatoryandexplicitlyanti-Islamic.• OverthetwodaysbeforetheBrusselsattacks,anaverageof216derogatoryanti-IslamicTweetsweresentperday.Inthedaysaftertheattacks,anaverageof680derogatory,anti-IslamicTweetsweresentperday.

• Arandomsampleof100TweetsclassiFiedasangry,severelyderogatoryandexplicitlyanti-Islamicwerequalitativelyanalysed,andFivecategoriesofIslamophobiawereidentiFied:

This included data from a number of different collections: (1) Tweets containing the words ‘Brussels’ or Bruxelles’, (2) 4

Tweets sent on a number of related hashtags created in reaction to the attacks, and Tweets containing Islamophobic slurs. (see annex)

N.B. There is no suggestion that these views are necessarily anti-Islamic or derogatory. 5

• ‘IslamistheEnemy’-ViewsthatconsideredIslamasinherentlypermissiveofviolenceandfundamentallyhostiletotheWest.

• ‘MuslimsarePedophiles’-ViewsinsinuatingthatMuslimsaremorelikelytocommitsexcrimes.• ‘They’reallterrorists’-ViewssuggestingthatthewiderMuslimpopulationsarepermissiveandsupportiveofterrorism.

• ‘CallsforAction-AsmallnumberofTweetsseekingtoorganiseofFlineactionagainstMuslimcommunitiesinthewakeoftheBrusselsattacks.

• ‘AngerandAbuse’-Abusive,derogatoryanti-islamiccomments,somesentdirectlytootherTwitteraccounts(insomecasesusersintendedtobethetargetsofthisabuse).

OVERALLMETHODOLOGYTwitterdataisoftenchallengingtoanalyse.Datadrawnfromsocialmediaareoftentoolargetofullyanalysemanually,andalsooftennotamenabletotheconventionalresearchmethodsofsocialscience.TheresearchteamusedatechnologyplatformcalledMethod52,developedbyCASMtechnologistsbasedattheTextAnalyticsGroupattheUniversityofSussex. Itisdesignedtoallownon-technical6

researcherstoanalyseverylargedatasetslikeTwitter.

DataCollectionMethod52wasusedtodirectlycollectTweetsfromTwitter’sStreamandSearch‘ApplicationProgrammingInterfaces’(orAPIs).TheyallowallTweetstobecollectedthatcontainoneofanumberofspeciFiedkeywords.Thekeywordsusedinthevariouscollectionsusedinthisresearcharedetailedintheannex.

DataAnalysisMethod52allowsresearcherstotrainalgorithmstosplitapart(‘toclassify’)Tweetsintocategories,accordingtothemeaningoftheTweet,andonthebasisofthetexttheycontain.Todothis,itusesatechnologycallednaturallanguageprocessing.NaturallanguageprocessingisabranchofartiFicialintelligenceresearch,andcombinesapproachesdevelopedintheFieldsofcomputerscience,appliedmathematics,andlinguistics.Ananalyst‘marksup’whichcategoryheorsheconsidersatweettofallinto,andthis‘teaches’thealgorithmtospotpatternsinthelanguageuseassociatedwitheachcategorychosen.Thealgorithmlooksforstatisticalcorrelationsbetweenthelanguageusedandthecategoriesassignedtodeterminetheextenttowhichwordsandbigramsareindicativeofthepre-deFinedcategories.Detailsabouthowthesealgorithmswereused,andhowwelltheyworked,areprovidedbelow. 7

TheAccuracyofAlgorithmsTomeasuretheaccuracyofalgorithmsintothecategorieschosenbytheanalyst,weuseda‘goldstandard’approach.Foreach,around100tweetswererandomlyselectedfromtherelevantdatasettoformagoldstandardtestsetforeachclassiFier.TheseweremanuallycodedintothecategoriesdeFinedabove.ThesetweetswerethenremovedfromthemaindatasetandsowerenotusedtotraintheclassiFier.

AstheanalysttrainedtheclassiFier,thesoftwarereportedbackonhowaccuratetheclassiFierwasatcategorisingthegoldstandard,ascomparedtotheanalyst’sdecisions.Onthebasisofthiscomparison,classiFierperformancestatistics–‘recall’,‘precision’,and‘F-score’arecreatedandappraisedbyahumananalyst.EachmeasurestheabilityoftheclassiFiertomakethesamedecisionsasahumaninadifferentway:

Overallaccuracy:ThisrepresentsthepercentagelikelihoodofanyrandomlyselectedTweetwithinthedatasetbeingplacedintotheappropriatecategorybythealgorithm.Itisbasedonthreeothermeasures(below).

This group is led by Professor David Weir and Dr Jeremy Reffin. More information is available about their work at: 6

http://users.sussex.ac.uk/~davidw/styled-3/

For a more detailed description of this methodology, see the Demos paper Vox Digitas, 7

Recall:ThenumberofcorrectselectionsthattheclassiFiermakesasaproportionofthetotalcorrectselectionsitcouldhavemade.Iftherewere10relevanttweetsinadataset,andarelevancyclassiFiersuccessfullypicks8ofthem,ithasarecallscoreof80percent.

Precision:ThisisthenumberofcorrectselectionstheclassiFiersmakesasaproportionofalltheselectionsithasmade.IfarelevancyclassiFierselects10tweetsasrelevant,and8ofthemactuallyareindeedrelevant,ithasaprecisionscoreof80percent.

F-Score:AllclassiFiersareatrade-offbetweenrecallandprecision.ClassiFierswithahighrecallscoretendtobelessprecise,andviceversa.The‘overall’scorereconcilesprecisionandrecalltocreateone,overallmeasurementofperformanceforeachdecisionbranchoftheclassiFier.

N.B.thevaluesforeachalgorithm(calledaclassiFier)arepresentedwithinthedetailedmethodologyofthisreport.Thevaluesareexpressedasvalueupto1:avalueof0.76,forinstance,indicatesa76%accuracy.

CAVEATS

Theresearchoflargesocialmediadatasetsisareasonablynewundertaking.Itisimportanttosetoutaseriesofcaveatsrelatedtotheresearchmethodologythattheresultsmustbeunderstoodinthelightof:

• Thealgorithmsusedarenotperfect:throughoutthereport,someofthedatawillbemis-classiFied.Thetechnologyusedtoanalysetweetsisinherentlyprobabilistic,andnoneofthealgorithmstrainedandusedtoproducetheFindingsforthispaperwere100%accurate.Theaccuracyofallalgorithmsusedinthereportareclearlysetoutinthisreport.

• Somedatawillbemissed:AcquiringTweetsonthebasisofthekeywordsthattheycontainpresentstwopossibleproblems.First,theinitialdatasetmaycontaintweetsthatareirrelevanttothethingbeingstudied.Secondly,itmaymisstweetsthatarerelevanttothethingbeingstudied.Researchersworkedtoconstructascomprehensivealistofkeywordsaspossible(thesearedetailedinthereport,below),howeveritislikelysomeweremissed,andthenumberspresentedinthisreportarelikelyasubsetofthetotal.

• TwitterisnotarepresentativewindowintoBritishsociety:TwitterisnotevenlyusedbyallpartsofBritishsociety.Ittendstobeusedbygroupsthatareyounger,moresocio-economicallyprivilegedandmoreurban.Additionally,thepoorest,mostmarginalisedandmostvulnerablegroupsofsocietyareleastrepresentedonTwitter;anissueespeciallyimportantwhenstudyingtheprevalenceofxenophobia,Islamophobiaandthereportingofhateincidents. 8

• Overall,thisresearchisintendedtobeanindicative,First-takeofthereactiononTwittertotheseimportantevents.ItisnotpresentedaseitherexhaustiveordeFinitive;anditisverymuchhopedthatitwillstimulatefurtherresearchonthisvitaltopicsinthefuture.

For a longer discussion of this issue, see the Demos paper The Road to Representivity 8

DETAILEDMETHODOLOGY

TheEUReferendum

ThissectiondetailsthemethodusedtoresearchTwitterovertheperiodoftheannouncementoftheEUreferendumresult.

PARTI:TheReferendumCampaign

FromMay20thtoJune2ndwecollectedalltweetssenttoBritishMPscontainingakeywordorahashtagrelatedtotheEUReferendum(seetheappendix).AsaFirststep,wehavecreatedacampaignclassiFierinordertodistinguishbetweenpro-leaveandpro-remaintweets.

(hence,theoveralllikelihoodofanygivenTweetbeingcorrectlycategorieswas69.3%.Thealgorithmwasbetteratclassifyingpro-leaveandpro-remaintweets;lessgoodatclassifying‘other’tweets).

Asasecondstep,wehavecreatedanissueclassiFierinordertodistinguish,forbothpro-Brexitandpro-RemainTweeters,whichwerethemostrelevantissuesintothedebate.

Togetherwitheconomyandsovereignty,immigrationresultedtobeoneofthethreeissuesmostlyassociatedwiththereferendumdebatebothforpro-Remainandpro-BrexitTweeters.

Last,allTweetswerecollectedsentfromtheUKthatcontainedoneofthefollowingwords:‘migrant’,‘migrants’,immigrant’,immigrants’,‘refugee’orrefugees’.Thisresultedinacollectionof258,553tweets.Fromthisdataset,wehavebeenabletoidentifyasub-sampleof40,255tweetscontainingakeywordlinkedtoBrexit(seetheannexforalistofthesekeywords).

PARTII:XenophobiaonTwitter

InordertomeasurethenumberofTweetscontainingxenophobicviewsfromtheUKovertheannouncementoftheBrexitresult,FirstallTweetswerecollectedbetween19thJuneand1stJulythatcontainedaterm,includinghashtags,thatcouldberelatedtoxenophobia.Thisresultedinadatasetof16,151tweetsbasedintheUK.Seetheannexforalistofthesekeywords.N.B.thisisnotcomprehensive,andmaybeonlyasmallamountofthetotalnumberofxenophobicTweetsthatweresentoverthisperiod.

However,notallTweetsthatcontainaxenophobictermexpressaxenophobicattitude.Xenophobicwordsandhashtagscanbeusedinabroadnumberofdifferentways,includingtodisputexenophobicviewsandexpresssupportformigrants.InordertomeasuretheextentofTweetsexpressingxenophobicviews,a‘xenophobiaclassiFier’wastrainedtodistinguishbetweentweetsthatexpressedderogatoryandxenophobicviewstowardsmigrants,andthosethatdidn’t.

(Theclassi6ierwasbetteratidentifyingderogatoryTweetsthansupportiveones.The'other'categorywasnotusedforclassi6icationThecapacityoftheclassi6iertoidentifyderogatoryTweetswasitsmostimportantfunction.Overall,anygivenTweethada76%chanceofbeingcorrectlyclassi6ied)

TheoutcomeoftheclassiFiershowedthat,ofTweetsusingxenophobicterms,roughlytwiceasmanyTweetsweresupportiveofmigrantsanddisputingxenophobicattitudesaswerexenophobic.

PARTIII:IslamophobiaonTwitter

TheidentiFicationofIslamophobicTweetsinthewakeoftheBrusselsattacksformedpartofalonger-termresearchprocess.Itisdetailedwithinthisreport,below,withinthemethodologicaldescriptionoftheidentiFicationofIslamophobicTweetsinthewakeoftheBrusselsattacks.

PARTIV:Reportsofhatefulincidents

IntheaftermathoftheEUreferendum,wecollectedalltweetscontainingthehashtags#SafetypPinand/or#PostRefRacism.Thisresultedinadatasetof98.948tweetssentfrom25thJunetoJuly4th.Thefunctionofthesehashtagsweretoshowsolidaritywithmigrantsinthewakeofthebrexitdecision,andalsotoraiseawarenessofhatefulincidents(whetherxenophobic,Islamophobic,orracist)takingplace.

InordertounderstandthenumberofaccountsofspeciFicandhatefulincidents,aclassiFierwastrainedtodistinguishbetweentweetsgenerallyraisingawarenessabouthatecrime,fromtweetscontainingspeciFicaccountsofhatefulincidents.

(the‘recall'ofthealgorithmforspeci6icstoriesisreasonablylow.Thiswasacceptedasreasonablebecauseitmeantthattheclassi6ierwasunlikelytoexaggeratethenumberofspeci6icstoriesidenti6ied).

ThealgorithmclassiFied17,260TweetsascontaininganaccountofaspeciFicincident,and81,688asgenerallyraisingawarenessofhatecrimeandhatefulincidents.Ofthese17,260tweetsthatcontainedanaccountofaspeciFicincident,2413wereoriginaltweets,andtherest-14,847-wereretweetsoftheseoriginaltweets.

N.B.Uponqualitativeevaluation,theTweetswithinthespeciFicstoriesdatasetincludedbothFirst-handaccountsofhatefulincidents,butalsoaccountssecond-hand,orthattheTweeterhadheardaboutfromanothersource.

TheBrusselsAttacks

ThissectiondetailsthemethodusedtoresearchTwitterovertheperiodoftheannouncementoftheBrusselsattacks.

PARTI-TheWiderReaction

TounderstandtheoverallreactiontotheBrusselsattacks,allTweetswerecollectedthatcontainedthewords‘Brussels’or‘Bruxelles’sentbetween14Marchand31March2016.Thisreturned8,452,661Tweets.Thislargedatasetwasanalysedusingaseriesofalgorithms.

First,aclassiFierwastrainedtoidentifyTweetssimplysharingnewsconcerningtheattacks.ItsplitTweetsintothreecategories:• ‘News’:thosesharingnewstheattackshadtakenplace;• ‘Condolence’:thoseexpressingcondolenceforthoseaffectedbythem;• ‘Other’:allotherTweets.

Second,aclassiFierwastrainedtosortallTweetsidentiFiedas‘Other’(seeabove)intothefollowingcategories:

• ‘Solidarity’:Tweetsthatcalledforreligioustolerance,thatexpressedsolidaritywithMuslimcommunities,andexpressedtheviewthattheterroristattackswereinnowaylinkedtoIslamormigration.

• ‘Criticismandwiderissues’:TweetsthatlinkedtheterroristattackstowiderissueswithinIslam,migrationorintegration,thatcalledforMuslimstobemoreactiveincombattingterrorism,andthatexpressedsupportforDonaldTrump’sviewsonhaltingimmigrationofMuslimsinthewakeoftheattacks.

• News:Tweetscoveringnewdetailsoftheattacksandattackersastheybecameknown.• ‘Rest’:Tweetsthatdidnotfallintoanyoftheothercategories

PARTII-#StopIslam

Inordertoanalysetheimportanthashtag#stopislam,thatwasusedbeforetheBrusselsattacks,butbecauseimportantinthereactiontothem,allTweetswerecollectedthatcontained‘#stopislam’between14thMarchand31stofMarch.Thisreturned697,836Tweets.420,635oftheseTweetswerewritteninEnglish.

Aclassi6ierwastrainedtoseparatealltheEnglish-languageTweetsusing#stopislaminthisdatasetintothosethatwere:

• ‘Critical’ofIslam:thatpointedtowideraspectsofIslamorMuslimcommunities,theirfaithorculture,asbeinginsomewayimplicatedintheattacks.N.B.bynomeanswerealltheseviewsderogatory,hatefuloranti-Islamic.

• ‘Supportive’ofIslam:thatdefendedIslamandMuslims,anddisputedthecriticalviewsexpressedinTweetsusingthehashtag.

• ‘Other‘-TweetsthatdidnotFitwithineitherofthetwocategories,above.

Part3-Islamophobia

IdentifyingTweetsthatwerehateful,derogatoryandanti-Islamicwasaformidableanalyticalchallenge.

First,allTweetswerecollectedthatcontainedoneofanextensivelistoftermsthatcouldbeusedinananti-Islamicway(seeannex).Thiscollectionbeganonthe18thMarchandcontinueduntilthe30thofJune.ItreturnedaverylargenumberofTweetsoverthisperiod:4,123,705.TheverylargemajorityoftheseTweetswerenotanti-Islamicorhateful.Aseriesofalgorithmswerebuilttorespondtothedifferentchallengesthatthisdatasetposed.EachwasdesignedtoremoveTweetswhichwerenotIslamophobicfromthedataset:

• AlargenumberofTweetscontainedtheword‘Paki’. AclassiFierwasusedtoseparatederogatory9

usesofthiswordfromnon-derogatoryuses.• AlargenumberofTweetsalsocontainedtheword‘terrorist’.Ofcourse,manyTweetscontainingthiswordwereinnowwayderogatoryoranti-Islamic.TwoclassiFierswerebuilttoanalysetweetscontainingthesewords:• First,aclassiFierwastrainedtoseparateTweetsreferringtoIslamistterrorismfromotherformsofterrorism.

• Second,oftheTweetsreferringtoIslamistterrorism,aclassiFiertodistinguishviewsbroadlyattackingMuslimcommunitiesinthecontextofterrorism,fromthosebroadlydefendingMuslimcommunities.

N.B. whilst this word refers to an ethnic rather than religious group, it was found that it was often used interchangeably 9

to refer to Muslim communities

• AclassiFierwastrainedtoseparateallotherTweetsinthedatasetintothosethatwerederogatoryandanti-Islamicfromthosewhichwerenot.

• Last,theTweetsthat,basedontheabove,(a)usedtheterm‘Paki’inaderogatoryway,(b)thatusedtheterm‘terrorist’tobroadlyattackMuslimsorMuslimcommunities,(c)thatusedtheotherpossibleslurtermsinthecollectioninawaythatwasanti-Islamicwerecombined.ThesewerethenFilteredtoincludeonlyTweetssentfromtheUK.ThisresultedintheFinaltotalofIslamophobicTweets.

Theaccuracyofthesealgorithmsareasfollows: 10

Due to the large number of classifiers used, the accuracy was checked by taking a random sample of Tweets that - 10

according to the system of algorithms described above - were classified as derogatory anti-Islamic. 78 of these 100 were identified by an analyst as derogatory and anti-Islamic.

Thesealgorithmswereconnectedtogetherintoan‘architecture’,shownbelow.EachTweetcollectedpassedthroughthearchitectureonthebasisofhowitwasclassiFied.Overall,thissystemofalgorithmssucceededinFilteringtheverylarge(4.12million)numberofTweetsintoamuchsmaller(28,034)subsetthatweremuchmorelikelytobehateful,derogatoryandanti-Islamic.

Annex-DataCollectionKeywordsTheannexcontainsthekeywordsusedtocollectTweetsanalysedthroughoutthisreport.

I.Thereferendumcampaign.ListofusedhashtagsrelatedtotheEUreferendum:

• #euref• #eureferendum• #brexit• #voteleave• #strongerin• #strongerout• #voteremain• #votestay• #takecontrol• #remain• #Leave

II.UK-basedgeneraldiscussionofmigration

• migrant• migrants• immigrant• immigrants• refugee• refugees

III.Words/HashtagsusedtoidentifyTweetswithinthegeneraldiscussionofmigration(collectedusingtermsdescribedabove,inAnnexII)explicitlylinkedtotheEUReferendum

• brexit• EuropeanUnion• EU• Ukip• @UKIP• #strongerin• #remain• #voteremain• #votein• #bremain• #intogether• #voteleave• #leaveeu• #takecontrol• #go• #leave• #betteroffout• Farage• BorisJohnson• @Nigel_Farage• @BorisJohnson• #safetypin• #PostRefRacism• #PolesinUK• #londonstays

• #EUref

IV.Words/HashtagsusedtocollectTweetsthatcouldberelatedtoXenophobia

• #refugeesnotwelcome• #defendEurope• #Whitegenocide• #whitepower• #whitepride• Illegals• #whiteresistance• #whiterevolution• #sendthemhome• Refugeesnotwelcome• #MakeBritainwhiteagain• #sendthemback• #Getoutwevotedleave• #Stopimmigration• #DeportallMuslims• #NeverIslam• Rapefugee• #Polesgohome• #NojobsinUKforEU• golliwog• Rapeugee• #fuckislam• #PolishVermin• #NoIslam• #BanIslam• muzzrats• muzzle• Londonistan• muscat• muzrat• muzzrat• musrats• immigrantsgohome• refugeesgohome• migrantsgohome• currymunching• dirtypack• anti-immigrant• anti-immigration• #NoMoreRefugees• #StopTheInvasion• #NoMoreMigrants• #EndIslam• #IslamIsTheProblem• Raghead

V.Words/HashtagsusedtocollectTweetsthatcouldbederogatoryandanti-Islamic

• Jihad• Jihadi• SandFlea• Terrorist• hijab• CamelFucker

• CarpetPilot• Clitless• DerkaDerka• Diaper-Head• DiaperHead• DuneCoon• DuneNigger• Durka-durka• Jig-Abdul• Muzzie• Q-TipHead• Rab• Racoon• Rag-head• RugPilot• Rug-Rider• SandMonkey• SandMoolie• SandNigger• SandRat• SlurpeeNigger• Towel-head• MuslimPaedos• Muslimpigs• Muslimscum• Muslimterrorists• Muzrats• muzzies• Paki• Pakis• Pisslam• raghead• ragheads• Towelhead• FuckMuslims• WhiteGenocide• Pegida• EDL• BNP• Rapefugee• Rapeugee• mudshark• kuffar• kafFir

VI.Searchtermsforpost-referendumhashtagsusedtoreportpossiblehatefulincidents

• #SafetyPin• #PostRefRacism