evaluating research as a communicator

Practical Tips For Evaluating Research as a CommunicatorQ. Settle

The Problem

Tips

Practice

But this is just body language.

So what?

“Listen, mistakes happen. Especially in Excel. But hopefully they don’t happen in papers that provide intellectual edifice for an economic experiment – austerity – that has kept millions out of work.”Matthew O’BrienThe Atlantic

Research impacts• Education• Economy• Relationships• Public Health

How do we know something went wrong before we publish it?

Tips & Red Flags

Does it smell funny?

Bias

Researcher personal biasDid someone fund the research?

He has accepted more than $1.2 million in money from the fossil-fuel industry over the last decade while failing to disclose that conflict of interest in most of his scientific papers. At least 11 papers he has published since 2008 omitted such a disclosure, and in at least eight of those cases, he appears to have violated ethical guidelines of the journals that published his work.

But does funding source really matter?

How is the research being shared?JournalsConferenceWhite PapersOther nonformal outreach

Good Questions

Types of MethodsNo gold standard

Population

Sample

How were Q’s and A’s phrased?

Correlation≠Causation

Statistical Significance≠Practical Significance

Overstated Findings

Questionable Logic

Do They State Alternative Points of View?

Whole Picture vs. Small Details

Is there general consensus?People with PhDs like to disagree with each other

Let’s Play a Game

What did the abstract say?Never mention Alzheimer’s in results/conclusionsSitting once per week = better on learning testSitting 5 times per week = worse on learning test

What went wrong?Research addressed a cognitive test, not dementiaOther factors could have affected performance• SES may impact babysitting patterns• Those who have dementia may not be

able to babysit

Let’s Talk About Brand MavensWhat questions do you have about the research?

Brand mavens are active in social media

Prettier than the Best FriendWhat questions do you have about the research?

The Cynic’s Bank AccountWhat questions do you have about the research?

Sleep Secrets

What questions do you have about the research?

Final Thought

We can’t catch everythingBut that doesn’t mean we should stop tryingResearch is just a logic puzzle

Extra Resource

8 ways to be a more savvy science readers-Vox Card Stacks

Questions?

quisto.settle@msstate.edu

@QuistoSettle