Finding out what’s already known and what’s already happening

before planning additional research

Iain Chalmerson behalf of

Mike Clarke, Sally Hopewelland Mona Nasser

Evidence-Based Research NetworkBergen, 3 December 2014

Mike ClarkeBelfast

Sally HopewellOxford/Paris

Mona NasserPlymouth

Embarking on research without reviewing systematically what is

already known - particularly when the research involves people or animals - is

unethical, unscientific, and wasteful.

JAMA 1992;268:240-248.

The human costs of failing to cumulate evidence from research scientifically

“Advice on some life-saving therapies has been delayed for more than a decade, while other treatments have been recommended long after controlled research has shown them to be harmful.”

Antman et al. JAMA 1992;268:240-8.

Data from Clarke and Hopewell, 2013, cited in Chalmers et al., 2014

20 animal studies: “The results of this review did not show convincing evidence to substantiate the decision to perform trials with nimodipine in large numbers of patients.

Stroke 2001;32:2433-8.

STUDIES IN ANIMALS

Horn J, Limburg M. Calcium antagonists for acute ischemic stroke. The Cochrane Database of Systematic Reviews, 2000

“46 trials were identified of which 28 were included (7521 patients). No effect of calcium antagonists on poor outcome at the end of follow-up (OR 1.07, 95% CI 0.97/1.18), or on death at end of follow-up (OR 1.10, 95% CI 0.98/1.24) was found.”

STUDIES IN HUMANS

[4][2]

[24][19]

Findings

Some illustrative examples of waste from redundant research

Sena et al. 2010

Redundant animal research

Redundant clinical research…

…leaving key questions unaddressed

Cumulative odds ratios for front versus non-front sleeping position of sudden infant deaths versus controls. Gilbert et al. 2005.

Redundant epidemiological research

“Systematic review of preventable risk factors for SIDS from 1970 would have led to earlier recognition of the risks of sleeping on the front and might have prevented over 10 000 infant deaths in the UK and at least 50 000 in Europe the USA and Australasia.”

Consequences of failure to analyse epidemiological research cumulatively

What should research funders, research regulators, researchers

and journals do to reduce this sometimes lethal waste?

Research funders

NIHREngland

Yes – Applications to fund primary research have to be supported with systematic reviews of existing evidence. For commissioned calls, this will have been done by the funder.

NHMRCAustralia

No

CIHRCanada

Partially - Systematic reviews of prior clinical trials are encouraged but not required. There is a knowledge synthesis scheme but this does not seem to be related to funding for primary research.

NIHUSA

No – A ‘check of literature’ is encouraged but there is no clear guidance covering all projects or clinical trials.

MRCUK

No, in general – Only the Global Health Clinical Trial Unit requires systematic reviews, but none of the other major grant opportunities.

Do funders require applicants seeking support for primary research to refer to systematic reviews of existing evidence?

(Mona Nasser, personal communication)

The Wellcome Trust

2013

2003

MS Society supports systematic reviews

http://www.nets.nihr.ac.uk/about/adding-value-in-research

Research regulators

Research ethics committees/IRBs

Inappropriate continued use of placebo controls in clinical trials assessing the effects on death of antibiotic prophylaxis for colorectal surgery

Health Research Authority

Researchers

? First use of the termEvidence-Based Research

Karen Robinson (2009). Use of prior research in the justification and interpretation of clinical trials. The Johns Hopkins University, ProQuest, UMI Dissertations Publishing, 2009. 3392375, p 123.

Conceptualising an Evidence-Based Research Network

Tidsskrift for Den norske Legeforening, 29.11.2013, pp 2-3.

Lund H (2014). From evidenced-based practice to evidence-based research - Reaching research-worthy problems by applying an evidenced-based approach. Eur J Physioth 16:65-6.

Announcement of An Evidence-Based Research Network

Journals

Low priority questions addressed

Important outcomes not assessed

Over 50% studies designed without reference to systematic reviews of existing evidence

Questions relevant

to users of research?

Over 50% of studies fail to take adequate steps to reduce biases

Studies with inadequate statistical power

Inadequate replication of initial observations

Appropriate research

design, conduct and analysis?

Over 50% of studies never published in full

Biased under-reporting of studies with disappointing results

Biased reporting of data within studies

Accessible, fullresearch reports?

Over 30% of trial interventions not sufficiently described

Over 50% of planned study outcomes not reported

Most new research not interpreted in the context of systematic assessment of other relevant evidence

Unbiased and usable reports?

Research waste

Increasing value and reducing waste in biomedical research. Lancet series, 8 Jan 2014

Hyper-regulation of research

Inefficient delivery of research

Poor re-use of data

Failure to promote evaluative research as an integral element of good clinical practice

Efficient researchregulation

and delivery?

www.researchwaste.net

Iain Chalmers, health services researcher

Michael B Bracken, epidemiologist

Benjamin Djulbegovic, oncologist, methodologist

Silvio Garattini, clinical pharmacologist

Jonathan Grant, science policy analyst

Metin Gulmezoglu, clinical trialist

David Howells, preclinical animal researcher

John PA Ioannidis, methodologist, bibliometrician

Sandy Oliver, social scientist

Issue 1

Issue 2

Issue 3

Recommendation 3

Recommendation 4

Chalmers I. Lancet 2000;356:774

BMJ 2010;340:456-7

Alessandro Liberati

Patients have suffered and died unnecessarily and resources for research have been wasted because the research community has failed to review existing evidence systematically when planning new research.

Why should patients and the public trust us if we and our professional institutions fail to make systematic, efficient use of the results of the research that the public has funded?

2006 2011

Promote research on the effects of treatments…

…but only if it meets scientific and ethical principles

Promote research on the effects of treatments…

53

“This is an important, scary and encouraging book.”

“I think all doctors and medical students will benefit from this little book, which can be read in a few hours. It opens the eyes and gives medicine the critical foundation it needs.”

Editor: Kjetil Olsen

“Bad Science introduces the basic scientific principles to help everyone become a more effective bullshit

detector.”

Embarking on research without reviewing systematically what is

already known - particularly when the research involves people or animals - is

unethical, unscientific, and wasteful.