“Much of the scientific literature, perhaps half, may simply be

untrue. (...) Science has taken a turn towards darkness.”

R.Horton, Lancet, 2015

Lancet, 2014

It is estimated that about 85% of research investment—totaling $200 billion of the investment in 2010—is wasted.

What will you do if the results of the trial are not statistically significant?

Pharmaceutical company

Academic investigator

Peer-reviewer

Journal editor

7

Results publication

• Are all study results published?• How do you know?• Does it matter?• What could we do about it?

8

Are all study results published?

How do we know?

9

Are all study results published?

NO

10

Are all study results published?

NO

• 646 essais complétés entre 2007 et 2010

• 44% des résultats non publiés 3 ans après la fin de l’essai

• 646 completed trials between 2007 and 2010

• 44% still unpublished 3 years after the end of the trial

Unpublished trials

151 Chan AW, Lancet 20142 Anderson NEJM 2015

• Trials registration and legal requirements insufficient2

– 13,327 completed trials subject to FDAAA provisions from 2008-2012

– 13% posted results within 12 months after trial completion

Reporting bias. A frequent cause of waste in research

• Publication bias

– 50% of completed clinical trials are published1

– Same proportion for 30 years

16

1 Chan AW, Song F, Vickers A, Jefferson T, Dickersin K, Gøtzsche PC, Krumholz HM, Ghersi D, van der Worp HB, Lancet 2014

17

Reporting bias. A frequent cause of waste in research

• Selective reporting of outcomes– 62% of trials reports had at least one primary outcome that was

changed, introduced, or omitted compared with the protocol2– A comparison of the primary outcomes registered and

published showed some evidence of discrepancies in 31% of RCTs. These discrepancies favoured statistically significant results3.

18

2 Chan AW, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. JAMA. 20043 Mathieu S, Boutron I, Moher D, Altman DG, Ravaud P. JAMA. 2009

19

In EM, 29% of RCTS

• En médecine d’urgence, en 2014: 29%

20

• En médecine d’urgence, en 2014: 29%

21

Does it matter?

22

• 74 FDA-registered antidepressant trials– 31%, were not published

• Meta-analyses of journal data sets compared to FDA dataset showed an increase in effect size of 32% overall– from 11% to 69% for

individual drugs

Turner, NEJM, 2008 24

Selective publication

Impact of publication bias

25

Turner, NEJM, 2008

2006

27

Abstract: « Oseltamivir 150 mg daily prevented lower respiratory tract complications (OR 0.32, 95% CI 0.18 to 0.57). «

2009

UPDATE

28

29

2014

30

31

« Many trials were unpublished, data weren’t shared, and decisions on purchasing, stockpiling, and using the drugs

were based on a slim and skewed representation of the total evidence base. »

« Oseltamivir (Tamiflu) has generated sales in excess of $18bn(£11bn; €13bn) for Roche since 1999, something more than the “nice little earner” …United States stockpiled 65 million treatments for a cost of $1.3bn. The United Kingdom spent

£424m on a stockpile of 40 million doses. By 2009, 96 countries possessed enough osteltamivir for 350 million

people. GlaxoSmithKline’s drug zanamivir (Relenza) was less successful but still generated sales in the region of $2bn. »

32

Does it matter?

Does it matter?

34

Does it matter?

• Conclusions: Paroxetine is generally well tolerated and effectivefor major depression in adolescents.

• Conclusion: Neither paroxetine nor high dose imipramine showed efficacy for major depression in adolescents, and there was an increase in harms with both drugs.

35

How could we improve it?

36

Trial registration

37

Posting results

• The 2007 US FDA Amendments Act requires that basic results from clinical trials of FDA–approved drugs and devices conducted in the United States must be– posted at ClinicalTrials.gov (CT.gov)– within 1 y of the completion of the trial, – whether the results are published or not

• Not complying with this requirement could result in civil monetary penalties (up to $10 000 a day), and for federally funded studies the withholding of grant funds.

38

379 phase IV trials ✓meeting the FDAAA801 requirements ✓basic results were not posted on ClinicalTrials.gov one year after completion of the trial

Reminders of the FDAAA 801 requirement

No contact

Results posted3 Months: 36/190 (19%)6 months: 46/190 (24%)

Results posted3 Months: 24/189 (13%)6 months: 27/189 (14%)

Randomisation

• 3 months: RR=1.5, [0.9 to 2.4], p=0.096• 6 months : RR= 9.9, [2.1 to 17.8] p=0.014

Maruani, Boutron, Baron, Ravaud BMJ 201439

Are reports of RCTs published in peer-review journal describing all important information?

40

41Hopewell S BMJ

42

No implementation of CONSORT abstract extension (NEJM, JAMA)

Level change: p = 0.78Trend change: p = 0.44

Level change: p = 0.0035Trend change: p = 0.0154

Active implémentation (Lancet, Ann Intern Med)

Hopewell S, Ravaud P, Baron G, Boutron I. BMJ. 2012

Editors955 abstracts of randomized trials

EQUATOR Network

Enhancing the QUAlity and Transparency Of health Research

EQUATOR Network

• Since 2008• International initiative that seeks to improve the

reliability and value of published health research literature by promoting transparent and accurate reporting and wider use of robust reporting guidelines.

• First coordinated attempt to tackle the problems of inadequate reporting systematically and on a global scale; it advances the work done by individual groups over the last 15 years.

44

• Oxford (DG Altman)– Developing « reporting guidelines »– «Reporting guidelines» library– Trainings (workshop, seminar)

• Paris (Ph Ravaud), – Implementation des « reporting guidelines »

• Developing interventions to enhance reporting and validating them in RCTS

– « Conducting guidelines »• Developing et validating tools to help researchers improve their

protocoles• Ottawa (D Moher)

– Journalology / editors– Training of peer reviewers

45

3 centers

Site Web EQUATOR

46

Reporting guidelines

47

Consolidated Standards of Reporting Trials : CONSORT statement et extensions

ARRIVE guidelines

48

Nature 2013Science 2013

Nature 2012

Editors

49

Peer- reviewersAuthors

One more thing

50

From «Reporting Guidelines» to«conduct guidelines »

• Willingness to improve not only "reporting" but also the planning, implementation and analysis of health research projects

• Need for new recommendations that can be used in the planning and analysis phase of studies (linked to recommendations in the reporting phase)

Easy methodological adjustmentsInitially

Yordanov et al., BMJ 2015

ONLY 16% of RCTs included in Cochrane reviewsare at low risk of bias

Precise reporting IS necessary

Transparency and honesty

Control mechanisms exist

Perfectible

CONCLUSION