bagging the bull
TRANSCRIPT
BaggingtheBull:Skepticalskillsinanageofmisinformation
WarrenThorngateEmeritusprofessor,PsychologyDepartment,CarletonUniversity
Lecture5,13February2018
Coursetopicreminder
1. Informationgrowthandtheeconomyofattention2. Attitudeformationandchange3. Advertising4. Propagandaandeducation5. Scienceandsciencejournalism6. Skepticalskills
Themes
• Thepurposeofadvertising:toinduceyoutobuyproductsorservicesregardlessoftheirquality• Thepurposeofpropaganda:toinduceyoutobelievethingsregardlessoftheirtruth• Thepurposeofscience andjournalism:toinduceyoutounderstandthingsregardlessoftheirconsequences• Politicalconsequences• Economicconsequences• Personalconsequences• Etc.
Today’sadventure• Mostpeopleconsiderscienceasasourceofobjectivefact– somethingsolidtobelievein
• Butfewpeoplereadwhatscientistswrite• Toohardtoobtain• Toodifficulttounderstand• Tooboring
• Thuscreatingamarketforjournalistswhoselectandsummarizescientificstudies• PopularScience(begun1872)• ScientificAmerican• PsychologyToday• Reportsofselectedstudiesindailynews
• BBC“Science”• CBC“Technology”• Economist“Scienceandtechnology”
• Manyofthesesummariesaredistorted– onesourceofbull• Why?• Howcanwetellthegoodstufffromthebad?
Somearticletitles:• Thestudyofsociology• Therecenteclipseofthesun• Scienceandimmortality• Earlysuperstitionsofmedicine• Womanandpoliticalpower
Warmup!
Theproliferationofsecondarysources
• Asinformationproliferatesbeyondoursuppliesofattention,wearemorelikelytoingestselectionsandsynopsessuppliedbyothers=secondarysources• Liketheshiftfromhome-cookedtopreparedmeals
• Today,almostallinformationweingestabouttheworldissecond-handorgreater– fromsecondarysources• Newsprogrammes,magazinecolumns,• Websites,blogs,tweets,Facebook,etc.• Workplacechats,dinnergossip,etc.• Teachers,books,etc.• Thiscourse!
• Mostsecondarysourcespackagetheirinformationinreports orstories writtenorspokenbypeoplewedonotknow(“Anon”)• Howcanwebeconfidentthatthesereportsweingestarecorrect/valid/true?
• Howdowejudge?• Whomcanwetrust?
Asampleofcriteriaforattendingtosecondarysources• Credibilityofcommunicator
• Aretheytrainedtobeskeptical?• Aretheyeducatedaboutthetopicsofreports?• Whopaysthemtosummarizeandreport?
• Reputationofmedium• Havetheybeentruthful/accurateinthepast?• Hastheirorganizationbeenaccurateinthepast?
• Also…• Convenience
• Aretheirreportseasytoaccess?• Aretheirreportcomprehensible?
• Popularity• Dofriendsandstrangersreadthesereports?• Dofriendsandstrangerstrustthereports?
• Interestandbeliefconfirmation• Dothestoriesmatchmyinterests?• Dothestoriessupportoropposemyattitudes?
Somewidely-trustedanddistrustedoccupations[“Scientists”notincludedinsurvey]
“Veryorextremelytrustworthy?”
• Firefighters(77%)• Paramedics(74%)• Nurses(69%)• Doctors(65%)• Farmers(58%)• Teachers(52%)• Judges(42%)• Psychologists(35%)• Journalists(18%)
“Veryorextremelytrustworthy?”
• Lawyers(16%)• Automechanics(16%)• CEOs(11%)• Localpolitician(6%)• Bloggers(6%)• Nationalpoliticians(6%)• Carsalespeople(5%)• Telemarketers(4%)
Somewidely-trustedanddistrustednewsmedia(2014Pewpollof3,000Americans)
Mosttrusted• Economist• BBC• NPR,PBS,ABC,CBS,NBC• WallStreetJournal• WashingtonPost• NewYorkTimes• TheGuardian• GoogleNews
Leasttrusted• Breitbart• HuffingtonPost• DailyShow• SeanHannityShow• AlJazeeraAmerica• GlennBeckProgram• RushLimbaughShow• Buzzfeed
Soifscientistsaretrusted,whyaretheyoftenignored?
• Appliedscienceisassociatedwithtechnologythatmakesushappy!• “Deliverables”=Smartphones,laptops,driverlesscars,Internet,ducttape,medicalmiracles,InstantPot,etc.
• Butthescientificmethodwasbuiltonskepticismthatregularlychallengednon-scientificbeliefs• TheChurchcirca1500:“Wemustbelieveinordertounderstand.”• Scientificchallengecirca1500:“Wemustunderstandinordertobelieve.”• Scientificmethodsinclude
• observationandmeasurement(empiricism);logicalthoughtandobjectivity(rationalism)• disprovingincorrecthypotheses;replicatingfindings,peerreview
• Andsciencehasbecomeincreasinglydifficulttounderstand• Obscuretopics,infestedwithjargon;exoticanalyses;boringreports,difficulttoacquire
• Inaddition,scientistsoftendonothaveanswerstoquestionsthepublicasks(especiallymoralquestions)• Whattodo?
Enterthejournalists…
• Task1:Searchforthescientificstuffthatmightinterestreaders• Task2:Translatethescience-babbleintoplain-speak;spoon-feedandentertainthereaders• Tellthestorythatmakesitmostlikelytocapturelimitedattention
• Example:EarlyTEDtalks
Journalistsasjargon-to-plain-languagetranslators:thechainofcommunication
Scientificidea Lab,
funding,… ExperimentsManuscript(asynopsis) Journal,
Conference PeerreviewManuscript
published!Journalist’sreview
Storypublished!
Journalist’sstory(synopsisofsynopsis) Editor’s
review
Reader’sreview
Reader’ssynopsisofsynopsisofsynopsis
Listener’ssynopsisofsynopsisofsynopsisofsynopsis
Scientistsversusjournalists:TheTwoCultures
• Audienceassumedforsciencecommunication• Educated“peers”,withsharedknowledgeofscientificmethod• Specificknowledgeofauthors’areaofspecialization• Skepticaltraining• Beliefthat“scientificknowledge”isadurablegood• Anacquiredtastefortechnicaldetails
• (4-yeardegreetolearnwhatthetermsmean!)
• Audienceassumedfornews• Littleknowledgeofscientificmethod• Littletimetoread,watchoflisten• Thirstforbizarre,“Geewhiz!”andpracticalinformation
• Sciencepackagedasentertainmentincompetitionforattention• Interestin“scientificknowledge”asaperishablecommodity (entertainment)
StoryA:“Let’sinterviewanexpert…”[Recentemail]
11February2018HiProfThorngate,I'magraduatejournalismstudentatCarleton,andI'mmakingashortradiodoconthepsychologyofescaperooms,whythey'resofun,andhowthey'rebeneficialforourcognitiveskills.HereareafewexamplesofescaperoomsinOttawa:https://www.escapemanor.com/ottawahttp://amnesiaescape.com/IwashopingIcouldmeetyouinpersonforashortinterviewatsomepointearlythisweek,ifyouwerewillingandknowledgeableonthetopicatall.Pleaseletmeknowifyou'reinterested!
StoryB:PaulBindrim,therapist
1920-1997• MAfromDukeUniversitywithJ.B.Rhine(ESPguy)• OrdainedinChurchofDivineMetaphysics1958• ObtainedCaliforniaPsychologylicencein1967• Attended1969WesternPsychologicalAssociationconferenceinHotelVancouver.• Reportedonhisrecentresearch
• Iwasastudentvolunteerattheconference,fetchingpeoplewhomreporterswantedtointerview,basedonthelistoftopicsandpresentersintheconferenceprogramme
Pretendyouareajournalisttoldbyyoureditortoreportonnewsworthypresentationsatalocalpsychologyconference.
Whichofthefollowingsixconferencepresentationtitleswouldyouchoosetoreport?
Classexercise
Thelist…
1. Goalversustraitexplanations:Causalattributionsbeyondthetrait-situationdichotomy.
2. Theeffectsofpaternaldisengagementonwomen’sperceptionsofmalematingintent.
3. Thebipolarityofaffectanddepressivesymptoms.4. Identity-specificmotivation:Howdistinctidentitiesdirectself-
regulationacrossdistinctsituations.5. Someevidencefortheusefulnessofanoptimalforagingtheory
perspectiveongoalconflictandgoalfacilitation.6. Recentdevelopmentsinnude grouptherapy:Theoryandpractice.
Why?
• Whatmakesastorynewsworthy?• Importance(impact,relevance)• Timeliness(novelty,newness)• Humaninterest(emotionaleffect)• Proximity(nearness)• Celebrity• Conflict• Thebizarre• AndforTV:Goodvisuals
• Compareonthesecriteria:• “NewtrendsinBayesianstatisticalinference:beyondaflatapriori”• Versus
• “Crotcheyeballing:Thebarefacts”
Individualdifferencesinjournalistsandtheiremployers
• Someperiodicalswitharticleswrittenbysciencejournaliststryingtosummarizeandcommunicateaccurately• ScientificAmerican,ScientificAmericanMind
• Note:ScientificAmericancirculationabout463,000;Peopleabout3,419,000• Science,NewScientist,Nature,Discover,someofPsychologyToday• ConsumerReports• BBC,Economist,CBC(QuirksandQuarks;BobMcDonald)
• Periodicalswrittenbyhacksthatbendoriginalsourcestosensationalize,prescribe,titillate,entertain• Tabloidseverywhere(examplesfromhttps://gizmodo.com/tag/tabloid-science )
• ElectricityinBrainCellsStrongerThanLightning• GrowingMicewithHumanLivers• HeliumLeakingOutoftheGroundinNevada• LadyAntelopesKillforSex• BreastStereography— Nowin3D!• PollutionisaGreatSourceofCleanEnergy
Somecommonreportingerrors
• Misleadingtitles• Originaljournaltitle:“AssociationofDietary,Circulating,andSupplementFattyAcidsWith CoronaryRisk:ASystematicReviewandMeta-analysis.”
• NewYorkTimesheadline:“Butterisback”• Gee-whizstatistics;nocomparisongroups
• Originalresult:“65%ofrecentrefugeesreportsleepingdifficulties”[hypotheticalreport]• Skepticalquestion:Isthisanygreaterorlessthannon-refugees?• Minoreffectsizes(ES),Smallsamplesizes
• Unspecifiedorinflatedrisk• “Coffeeincreasesriskofdepression”[hypotheticaltitle]
• ScientistsfoundasignificantrelationshipbetweenreportofcoffeeconsumptionandBeckDepressionscores• Buthowmuch?Depressionincreasefrom5%to75%?Orfrom1%to1.05?
• Causalinferencefromcorrelation(“Correlationisnotcausation!”andconfounds)• “Valentine'sDaywarningfromscientistssaysstopsexting”(CBCnews10February2018)
• AdamGalovan andhiscolleaguessurveyed615adults.(HalfwereCanadian).Allwereincommittedrelationships,bothheterosexualandsamesex.Theyfoundthatthosewhosextedthemost— categorizedas"frequent"or"hyper"sexters — reportedthattheprovocativetextingspiceduptheirsexlives.But here'sthedownside: Sextingunderminedotheraspectsofahealthyrelationship.Thesesupersexters feltlesssecureintheirrelationship,hadcommitmentissues andweremorelikelytowatchporn.
Afewcasehistories
• Mercuryinpermafrost• Alzheimer’sdisease• DietandIQ• Cognitivebenefitsofexerciseinolderadults
Example:MercuryinpermafrostOriginaltitle:“PermafrostStoresaGloballySignificantAmountofMercury”
• GeophysicalResearchLetters,5February2018http://onlinelibrary.wiley.com/doi/10.1002/2017GL075571/full
• Quote:“ThismakesthereservoirofHginpermafrostsoilsvulnerabletoreleaseoverthenextcentury,withunknownconsequencestotheenvironment.”
• Headline:“TheArcticisfulloftoxicmercury,andclimatechangeisgoingtoreleaseit.”(OttawaCitizenviaWashingtonPost7Feb2018)• Quote:“WealreadyknewthatthawingArcticpermafrostwouldreleasepowerfulgreenhousegases.OnMonday,scientistsrevealeditcouldalsoreleasemassiveamountsofmercury– apotentneurotoxinandseriousthreattohumanhealth.”
• Headline:“ClimateChangeCouldUnleashHugeAmountsofMercuryFromPermafrost”(Motherboardnewsservice)• Quotes• “Exposuretomercurycanbefatal,sotherealitythatourwarmingplanetmightunleashuntoldamountsofithasseriousmedicalandenvironmentalimplications.”
• “ButbeforeyouintegratethisfrighteningnewrevelationintoyourDoomsdayprepspreadsheet,takesometemporarysolaceinthefactthatalotofresearchstillneedstobedone.”
Example:Alzheimer’sdisease
• Originaltitle:“ThinkingOutsidetheBoxinAlzheimer’sDisease:CouldInfectionbetheAnswer?”BrianBalin,ChristineHammond,C.ScottLittle,SusanHingley,Denah Appelt”• (IAGGWorldCongressofGerontology&Geriatrics,2011/2017)• https://www.intechopen.com/books/advanced-understanding-of-neurodegenerative-diseases/evidence-for-an-infectious-etiology-in-alzheimer-s-disease
• NewsHeadline:“CouldavaccinesomedaypreventAlzheimer's?Someresearcherssayit'spossible”(OttawaCitizen,25July2017)• Excerpt:“Thatisthetantalizingpromiseofabodyofscientificresearchthatpointstomicrobes,includingtheubiquitousherpesvirus,asapossiblecauseofthedisease.ThelinkbetweenmicrobesandAlzheimer’scouldpavethewayforeventualtreatmentsoracure.”
Example:ResearcherslinkprocessedfoodtolowerIQinkidsOriginal Are dietary patterns in childhood associated with IQ at 8 years of age? A population-based cohort study• Methods Thecurrentstudy,basedontheAvonLongitudinalStudyofParentsandChildren,usesdataonchildren'sdietreported
byparentsinfood-frequencyquestionnairesat3,4,7and8.5 yearsofage.Dietarypatternswereidentifiedusingprincipal-componentsanalysisandscorescomputedateachage.IQwasassessedusingtheWechslerIntelligenceScaleforChildrenat8.5 years.Dataonanumberofconfounderswerecollected,andcompletedatawereavailablefor3966children.
• Results Afteradjustment,the‘processed’(highfatandsugarcontent)patternofdietat3 yearsofagewasnegativelyassociatedwithIQassessedat8.5 yearsofage—a1SDincreaseindietarypatternscorewasassociatedwitha1.67pointdecreaseinIQ(95%CI−2.34to−1.00;p<0.0001).The‘health-conscious’(salad,rice,pasta,fish,fruit)patternat8.5 yearswaspositivelyassociatedwithIQ:a1SDincreaseinpatternscoreledtoa1.20pointincreaseinIQ(95%CI0.52to1.88;p=0.001).
• Conclusion Thereisevidencethatapoordietassociatedwithhighfat,sugarandprocessedfoodcontentinearlychildhoodmaybeassociatedwithsmallreductionsinIQinlaterchildhood,whileahealthydiet,associatedwithhighintakesofnutrientrichfoodsdescribedataboutthetimeofIQassessmentmaybeassociatedwithsmallincreasesinIQ.
CNN report http://thechart.blogs.cnn.com/2011/02/07/processed-food-linked-to-lower-kids-iqs/?hpt=T2/• Headline:Processedfoodlinkedtolowerkids'IQs• Ifa3-year-oldeatstoomuchprocessedfood,itmightlowerhisorher IQbytheageof8,anewstudysuggests.Researchersin
Britaintrackedwhat14,000childrenateanddrankattheagesof3,4,7,and8.5yearsofage,byaskingparentstocompletequestionnairesdetailingtheirchild'sdiet.
• Thestudyauthorssuggesttheirstudyfoundsomeevidencethatwhen3-year-oldchildreneatadietrichinfoodsthatarehighinfat,highinsugarandareprocessed,theirIQmayfindasmalldecreaseintheirIQfiveyearslater. Ontheflipside,thisnewstudysuggestseatingahealthy,nutrientrichdietmaybeassociatedwithasmallincreaseinIQ.
• Thestudyauthorsnotethatinthispaper"wereportweakbutnovelassociationsbetweendietarypatternsinearlychildhood...withgeneralintelligenceassessedat8.5yearsofage."Theirresearchalsosuggeststhatwhatachildeatsinthefirstthreeyearsoflifeisassociatedwithamodestdecreaseinintelligence,butwhatachildateatage4and7didnot.
Originaltitle:PhysicalActivityandHippocampalSub-RegionStructureinOlderAdultswithMemoryComplaintsDr.Prabha Siddarth,UCLA,JournalofAlzheimer'sDisease,vol.61,no.3,pp.1089-1096,2018
Abstract: Background:Physicalactivity(PA)playsamajorroleinmaintainingcognitioninolderadults.PAhasbeenshowntobecorrelatedwithtotalhippocampalvolume,amemory-criticalregionwithinthemedialtemporallobe(MTL).However,researchonassociationsbetweenPAandMTLsub-regionintegrity islimited.Objective:ToexaminetherelationshipbetweenPA,MTLthickness,anditssub-regions,andcognitivefunctioninnon-dementedolderadultswithmemorycomplaints.Methods:Twenty-ninesubjectsaged≥60years,withmemorycomplaintswererecruitedforthiscross-sectionalstudy.PAwastrackedfor7daysusingaccelerometers,andaveragenumberofsteps/daydetermined.Subjectswerecategorizedintotwogroups:thosewhowalked≤4000steps/day(lowerPA)andthosewith>4000steps/day(higherPA).Subjectsreceivedneuropsychologicaltestingand3TMRIscans.NonparametricANCOVAscontrollingforageexamineddifferencesbetweenthetwogroups.Results:Twenty-sixsubjectsaged72.7(8.1)yearscompletedthestudy.ThehigherPAgroup(n = 13)hadthickerfusiformgyrus(mediandifference = 0.11 mm,effect size(ES) = 1.43,p = 0.001)andparahippocampal cortex(mediandifference = 0.12 mm,ES = 0.93,p = 0.04)comparedtothelowerPAgroup.ThehigherPAgroupalsoexhibitedsuperiorperformanceinattentionandinformation-processingspeed(mediandifference = 0.90,ES = 1.61,p = 0.003)andexecutivefunctioning(mediandifference = 0.97,ES = 1.24,p = 0.05).Memoryrecallwasnotsignificantlydifferentbetweenthetwogroups.Conclusion:Oldernon-dementedindividualscomplainingofmemorylosswhowalked>4000stepseachdayhadthickerMTLsub-regionsandbettercognitivefunctioningthanthosewhowalked≤4000steps.Futurestudiesshouldincludelongitudinalanalysesandexploremechanismsmediatinghippocampalrelatedatrophy.---------------- [note:ESmeans“effectsize”]----------------SummarizedinMedicalNewsToday,https://www.medicalnewstoday.com/articles/320463.php?iacp
• Walk4,000stepseverydaytoboostbrainfunction,PublishedThursday28December2017
• By MariaCohut,Factchecked byJasminCollier• “RecentresearchledbytheUniversityofCalifornia,LosAngelesshowsthattakingashortwalkeachdaycanhelptokeepthebrain
healthy,supportingtheoverallresilienceofcognitivefunctioning….Thiscouldboostyourattention,theefficiencywithwhich youprocessinformation,andothercognitiveskills,sayfirststudyauthorPrabha Siddarth andcolleagues.”
Butthereishope…Sampletitlesfrom“Journalist’sresource”• Howtofindanacademicresearchpaper
• https://journalistsresource.org/tip-sheets/find-academic-research-paper-for-journalists
• 4tipsforunderstandingstatistics• https://journalistsresource.org/tip-sheets/statistics-research-math-reporting
• Reportingconflictingresearchresults• https://journalistsresource.org/tip-sheets/research/reproducibility-disparities-research-cancer-reporting?utm_source=JR-email&utm_medium=email&utm_campaign=JR-email%22%20target=%22_self%22
• Alsosee:Howtointerviewscientists• https://www.centerforhealthjournalism.org/blogs/interviewing-skills-how-get-great-stories-scientists-researchers-and-health-care-professionals
• https://www.theguardian.com/science/2014/apr/03/top-tips-conducting-interviews-scientists-science-writing-prize