lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years
DESCRIPTION
My talk to MIT Medical Evidence boot camp of Knight Science Journalism Program, December 8, 2011TRANSCRIPT
![Page 1: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/1.jpg)
Lessons to improve our reporting on evidence – analysis of 1,600 stories
over 6 years
MIT Medical Evidence Boot CampDecember 8, 2011
Gary SchwitzerPublisher
HealthNewsReview.org
![Page 2: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/2.jpg)
![Page 3: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/3.jpg)
![Page 4: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/4.jpg)
![Page 5: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/5.jpg)
![Page 6: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/6.jpg)
Site stats
28 reviewers: 19 clinicians or researchers, 7 journalists, 2 breast cancer survivors
Site launched April 2006
1,655 stories reviewed
Blog launched in 2004 - >2,500 posts
Sole funding from the Foundation for Informed Medical Decision Making here in Boston (Co-founders Jack Wennberg, Al Mulley, etc.)
![Page 7: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/7.jpg)
![Page 8: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/8.jpg)
After 5.5 years and 1665 stories
71% fail to adequately discuss costs.
66% fail to quantify benefit - often how small is potential benefit
65% fail to quantify harm - often how large is potential harm
63% fail to evaluate the quality of the evidence
57% fail to compare new idea with existing options
![Page 9: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/9.jpg)
Kid-in-candy-store picture of U.S. health
care
70% of ideas are terrific 70% are risk-free 70% - cost not an issue
![Page 10: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/10.jpg)
A clear pattern
News stories tend to emphasize or exaggerate the benefits of treatments, tests, products and procedures.
And they tend to minimize or ignore the harms.
![Page 11: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/11.jpg)
Other themes:
Failure to convey limits of observational studies
Rush to report on talks at scientific meetings
Common flaws in health business storiesSingle source storiesNo independent perspectives
Coverage of screening issues
![Page 12: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/12.jpg)
Daily drumbeat of news Steady diet of news from journals - disregard
for publication bias for positive findingsAdvertising Web health infoNews releases from industry, academic medical
centers, even from journalsMisleading Twitter messages
Concerns about cumulative effect of health messages
![Page 13: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/13.jpg)
Billboard Bombardm
ent
![Page 14: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/14.jpg)
Prizes for Prostates
![Page 15: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/15.jpg)
Bats & balls & urinals
![Page 16: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/16.jpg)
![Page 17: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/17.jpg)
Actual newspaper headlines on DaVinci Robotic Surgery
Cancer survivors meet lifesaving surgical robot
Da Vinci puts magical touch on the prostateRobotic surgeon's hands never trembleDa Vinci is code for faster recoveryHospital hopes robot surgery will lure
patientsWHAT’S THE HARM OF THIS?
![Page 18: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/18.jpg)
Stories about wonderful health care gifts are NOT what they
appear to be.
![Page 19: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/19.jpg)
Stories of robotic regret, complications, costs starting to
pile up Long learning curve for surgeons – one analysis showed that 3
experienced surgeons needed more than 1,600 cases to achieve acceptable outcomes with robotic-assisted laparoscopic prostatectomy
With 2,000 of these $ million machines in place, how many surgeons have done 1,600 prostates?
A 2010 study showed robot-assisted hysterectomies were associated with longer surgical times and cost an average of $2,600 more.
Bloomberg: A urologist confesses he was seduced by the robot. But now rails against hospitals “tawdry marketing more familiar on late-night infomercials by using patient testimonials. "I cannot believe how quickly I recovered," a vigorous-looking patient is quoted as saying. As a surgeon I have to ask: Where is the data?”
![Page 20: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/20.jpg)
Story in Family Practice News 2 weeks ago…
A recent explosion of interest in robotic surgery for routine hysterectomies and treatment of other non-oncologic gynecologic conditions is raising concern about the cost and comparative value of the robot over conventional laparoscopy.
In 2010, for the first time, more hysterectomies were performed with the da Vinci Surgical System than any other procedure, including prostatectomy.
There have not been any large-scale randomized trials of robot-assisted surgery, and the "limited observational evidence fails to show that the long-term outcomes of robot-assisted surgery are superior to those of conventional procedures."
![Page 21: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/21.jpg)
The high-speed train of expensive, unproven technologies has left the station and it can’t be stopped.
Robotic surgical systemsProton beam radiation facilities Intensity-modulated radiation therapy
facilities
![Page 22: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/22.jpg)
![Page 23: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/23.jpg)
Two stories that used drug company VPs as main sources, one predicting a “breakthrough” without giving any evidence!
One that was based solely on a news release on mouse research to tout a new approach to fight bone loss!
One proclaimed that a "treatment may work" after it was tested on 9 pieces of tissue outside the body!
Another anointed a cancer gene therapy as a "breakthrough" after it had been tested on just three tumor samples!
In print journalism in one week we saw.....
![Page 24: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/24.jpg)
Were these in a student newspaper?
New York Times
Los Angeles Times
Reuters wire service
HealthDay wire service
![Page 25: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/25.jpg)
Costs
![Page 26: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/26.jpg)
From BreastCancer.org discussion board
![Page 27: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/27.jpg)
![Page 28: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/28.jpg)
To prevent a single prostate cancer (that may not even be a killer):
71 men
x 365 days/yr.
x 7 yrs.
x $3/pill =
$544,215 And with prostate cancer, there’s a good chance
it wouldn’t have been a deadly cancer anyway!
![Page 29: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/29.jpg)
Excellent job by Duff Wilson/NYT
![Page 30: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/30.jpg)
Disease-mongering
MSNBC – Oct. 4, 2011 – Plastic surgeon wants to fix your 'runner's face’
Runners, beware. A New Jersey cosmetic surgeon has pinpointed something more nefarious than shin splints, stress fractures and even dead butt syndrome: "runner's face."
…that skinny, Skeletor-esque look some dedicated runners may unwittingly develop, as they're likely distracted with all the miles they have to cover to prepare for their next race. (The surgeon) describes the horrors thusly, in a press release issued today:
Runner’s face generally occurs in both men and women ages 40+ who exercise to improve their body, and in doing so end up with a skeletal and bony face.
He suggests a Botox-Restylane (or other injectable filler) combo, which will smooth wrinkles and plump that gaunt face right back up.
![Page 31: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/31.jpg)
Which “benefits” matter?
• Framing surrogate markers or intermediate endpoints (test scores, blood values, etc.) as if they were the outcomes that people should really care about to establish improved quality of life or longevity.
• Perhaps the Avastin story is best example where limitations of progression-free survival were often not adequately explained.
“will not make you live longer, won't make you feel better, and may have life-threatening side effects, but it will keep your cancer from worsening by an average of 1 to 2 months."
![Page 32: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/32.jpg)
Observational studies: "Women who drank >3 cups of coffee per day had a 20% lower risk of developing basal cell carcinoma than women who drank <1 cup/month"
WebMD: “Coffee Fights Common Skin Cancer” ”3 cups of coffee a day to help keep skin cancer away?"
asks MSNBC. ”Coffee Lowers Risk of Skin Cancer" announced Medical
News Today. ”Coffee cuts skin cancer risk,” stated the Washington
Post The American Association for Cancer Research news
release said the only thing you really can say: “Coffee consumption associated with decreased risk for basal cell carcinoma”
If that's too boring, then don't report the story because otherwise you're simply wrong.
![Page 33: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/33.jpg)
Rush to report on talks at scientific meetings
Last month’s American Heart Assoc. Scientific Sessions: experimental stem cell intervention in 14 people - only 8 of whom were followed for a year.
Tyranny of the anecdote: All 3 TV networks & WebMD rode the same glowing patient anecdote
Money quote keeps paying dividends: The same researcher quote - "biggest advance in cardiovascular medicine in my lifetime" - appeared in the ABC and NBC stories.
EXCITEMENT! WebMD reported that one researcher they interviewed could barely contain his
excitement. ABC anchor Diane Sawyer discussed on the air how "excited" correspondent
Richard Besser was. Is that the job of journalism to convey how excited a reporter is?
Anchor Sawyer also gushed this question to Besser, "Is this the real prospect that the nation's number one killer could be cured?"
CBS used cure and breakthrough in the same headline.
![Page 34: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/34.jpg)
Health business storySingle source
No independent perspective
Reuters: Medtronic tests stent for erectile dysfunction October 20, 2011
Is “an initial feasibility study in 30 men” worth reporting?
With a company-sponsored investigator as the only source?
“30 million US men and 300 million worldwide with erectile dysfunction”
• That counts ALL degrees of erectile dysfunction but not all = a total inability to achieve an erection.
• Many different causes and vascular causes might not be safely amenable to stenting.
• So, even in a business story, the "market" of "a real unmet clinical need" for a surgical procedure may have been inflated in this story.
![Page 35: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/35.jpg)
News coverage & poor public discussion of screening issues should be most concerning.
Worst, most biased coverage I’ve seen in 37 years
![Page 36: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/36.jpg)
10 claims that many stories didn’t challenge
1. This is all about saving money.
2. This is about rationing.
3. It’s the first sign of Obamacare.
4. Government is deciding some lives don’t matter.
5. Early detection always saves lives.
6. The fact that I or someone I know was saved by a mammogram proves that more testing is better.
7. The shifting recommendations prove that scientists are clueless.
8. This was surprising and came out of nowhere.
9. The task force members must be uninformed idiots - especially since none was an oncologist or radiologist.
10. American Cancer Society disagrees so the task force must be wrong.
![Page 37: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/37.jpg)
“The decision to start regular, biennial screening mammography before the age of 50 years should be an individual one and take patient context into account, including the patient's values regarding specific benefits and harms.”
What the Task Force actually wrote:
![Page 38: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/38.jpg)
Annals of Internal Medicine editors called it a “media
cacophony”
“Confusion, politics, conflicted experts, anecdote, and emotion ruled front pages, airwaves, the Internet, and dinner-table conversations.”
![Page 39: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/39.jpg)
The Public's Response to the U.S. Preventive Services Task Force's 2009 Recommendations on Mammography Screening
“These results are consistent with previous studies that suggest a media bias in favor of mammography screening."
![Page 40: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/40.jpg)
37-year Chicago Tribune & NY Times veteran John Crewdson
wrote:
“There are multiple reasons women are ill-informed about breast cancer. The fault lies primarily with their physicians, the cancer establishment, and the news media--especially the news media. Until coverage of breast cancer rises above the level of scary warnings mixed with heartwarming stories of cancer survivors, women are likely to go on being perplexed."
![Page 41: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/41.jpg)
A national poll suggests lingering impact of imbalanced
news• 11 percent of women surveyed said they believe mammograms should start in the 20s, even for women with no risk factors
• 29 percent believe mammograms should start in their 30s.
• 45 percent said the guidelines were an attempt to reduce health-care costs
![Page 42: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/42.jpg)
Other examples:
Chicago Sun-Times
Wall Street Journal
Washington Post
New York Daily News
Minneapolis Star Tribune
All TV networks
![Page 43: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/43.jpg)
The ink isn’t even
dry on studies before
marketin
g begins
![Page 44: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/44.jpg)
On Twitter
Men’s Health magazine:
If you're a smoker, you NEED to get a CT scan. Body of story: “But don't run out and ask for one.” (as it discusses issues of costs and false positive test results)
American Association of Retired Persons:
Are you a smoker? CT scan those lungs Body of story: 371 words of caveats.
![Page 45: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/45.jpg)
HealthDay wire service didn’t challenge researcher promoting universal pancreatic cancer screening for everyone over 50.
After study of tissue from 7 people!
![Page 46: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/46.jpg)
Journalists’ Screening Bias
Crusading
one-sided
advocacy
![Page 47: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/47.jpg)
Why don’t we deliver this
message?
“All screening tests cause harm;
some may do good.”But much health journalism consistently emphasizes benefits & minimizes harms
![Page 48: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/48.jpg)
![Page 49: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/49.jpg)
And that’s the message I think we’re missing
That there’s a harm behind screening promotions that are not based on evidence and that do not fully inform readers/consumers/would-be patients
That there’s a harm in selling sickness…in selling the search for weapons of mass destruction inside all of us
Communication of the evidence about tradeoffs – about harms - is a key health policy issue
![Page 50: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/50.jpg)
2007 Cochrane Review of randomized clinical trials of
screened versus not screened – only small benefits from screening
mammography
![Page 51: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/51.jpg)
Breast Self Exam
2 randomized controlled trials have shown no benefit from breast self exam in decreasing mortality
Breast self exam does lead to the discovery of more benign masses
![Page 52: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/52.jpg)
Screening - Myth and Truth
Myth: if women go for their recommended screening, their breast cancer will be caught early when it is 98% curable
Truth: Many breast cancers are not picked up on screeningScreening decreases the risk of dying of breast cancer
by only 15-30% Screening best finds the slower growing less aggressive
cancers with better prognosisFaster growing, more aggressive tumors (biology!) are
more likely to be found between screens and higher numbers have metastasized prior to diagnosis
![Page 53: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/53.jpg)
What pinkwashing leaves out
![Page 54: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/54.jpg)
The National Breast Cancer Coalition talks about “changing the
conversation”
Measure real progress, not by counting how many women have been screened or what the five-year survival rates are but by seeing significant reductions in incidence and mortality rates.
![Page 55: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/55.jpg)
With screening messages,
we see a clash between…Science
Evidence
Data
Recommendations for entire population
Grasping uncertainty
What we can prove
Intuition
Emotion
Anecdote
Decision-making by an individual
Promoting false certainty
What we believe, wish, or hope
![Page 56: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/56.jpg)
“I honestly believe it is better to know nothing than to know what ain’t so.”
Josh Billings (pen name of humorist Henry Wheeler Shaw, 1818 – 1885)
![Page 57: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/57.jpg)
Pitfalls of a steady diet of journal
stories
PLoS Med 2005; 2(8): e124
![Page 58: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/58.jpg)
The problem begins with the public’s rising expectations of science. Being human, scientists are tempted to show that they know more than they do. The number of investigators—and the number of experiments, observations and analyses they produce—has also increased exponentially in many fields, but adequate safeguards against bias are lacking. Research is fragmented, competition is fierce and emphasis is often given to single studies instead of the big picture. Much research is conducted for reasons other than the pursuit of truth. Conflicts of interest abound, and they influence outcomes.
![Page 59: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/59.jpg)
![Page 60: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/60.jpg)
Spinning results of randomized clinical
trialsBoultron et al, JAMA May 26, 2010, 303 (20): 2058
In this representative sample of RCTs published in 2006 with statistically non-significant primary outcomes, the reporting and interpretation of findings was frequently inconsistent with the results.
![Page 61: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/61.jpg)
Reporting bias in medical research - a narrative
review
McGauran et al. Trials 2010, 11:37
We identified reporting bias in 40 indications comprising around 50 different pharmacological, diagnostic, and preventive interventions. Many cases involved the withholding of study data by manufacturers and regulatory agencies or the active attempt by manufacturers to suppress publication. The ascertained effects of reporting bias included the overestimation of efficacy and the underestimation of safety risks of interventions.
![Page 62: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/62.jpg)
Reason for optimism
New models such as ProPublica, Kaiser Health News
Smart bloggers – including physicians, scientists, hospital administrator
Training This boot camp NIH Medicine in the Media workshops Association of Health Care Journalists USC Annenberg California Endowment fellowships
![Page 63: Lessons to improve our reporting on evidence - analysis of 1,600 stories over 6 years](https://reader036.vdocuments.us/reader036/viewer/2022081602/554d6891b4c90578428b518d/html5/thumbnails/63.jpg)
Regular columns analyzing evidence
LA Times’ monthly “Healthy Skeptic” column (Seeing Through Skin Supplements’ Claims)
New York Times “Really?” weekly column (The Claim: Chewing Gum Can Prevent Ear Infections)
WSJ “Research Report” biweekly column – 6 studies each time
Washington Post “Quick Study” weekly column (Vibrating machines seem to offer no benefit for older women’s bones )