evaluating research as a communicator
TRANSCRIPT
Practical Tips For Evaluating Research as a CommunicatorQ. Settle
The Problem
Tips
Practice
But this is just body language.
So what?
“Listen, mistakes happen. Especially in Excel. But hopefully they don’t happen in papers that provide intellectual edifice for an economic experiment – austerity – that has kept millions out of work.”Matthew O’BrienThe Atlantic
Research impacts• Education• Economy• Relationships• Public Health
How do we know something went wrong before we publish it?
Tips & Red Flags
Does it smell funny?
Bias
Researcher personal biasDid someone fund the research?
He has accepted more than $1.2 million in money from the fossil-fuel industry over the last decade while failing to disclose that conflict of interest in most of his scientific papers. At least 11 papers he has published since 2008 omitted such a disclosure, and in at least eight of those cases, he appears to have violated ethical guidelines of the journals that published his work.
But does funding source really matter?
How is the research being shared?JournalsConferenceWhite PapersOther nonformal outreach
Good Questions
Types of MethodsNo gold standard
Population
Sample
How were Q’s and A’s phrased?
Correlation≠Causation
Statistical Significance≠Practical Significance
Overstated Findings
Questionable Logic
Do They State Alternative Points of View?
Whole Picture vs. Small Details
Is there general consensus?People with PhDs like to disagree with each other
Let’s Play a Game
What did the abstract say?Never mention Alzheimer’s in results/conclusionsSitting once per week = better on learning testSitting 5 times per week = worse on learning test
What went wrong?Research addressed a cognitive test, not dementiaOther factors could have affected performance• SES may impact babysitting patterns• Those who have dementia may not be
able to babysit
Let’s Talk About Brand MavensWhat questions do you have about the research?
Brand mavens are active in social media
Prettier than the Best FriendWhat questions do you have about the research?
The Cynic’s Bank AccountWhat questions do you have about the research?
Sleep Secrets
What questions do you have about the research?
Final Thought
We can’t catch everythingBut that doesn’t mean we should stop tryingResearch is just a logic puzzle
Extra Resource
8 ways to be a more savvy science readers-Vox Card Stacks
