Avoiding waste in research: the role of public involvement

Iain ChalmersCoordinator

James Lind Initiative

NCRN Consumer Liaison Group MeetingLeeds, 25 June 2013

better, more equal

‘The Liberati Manifesto’, 2011

Reasons for wanting to participate in clinical trials

Self-interest, albeit with some altruism thrown in

(2012)

7

2001-2005

1977-1985

Patients

1 Demand clinical trials addressing unanswered questions important to you

2 Demand and help to reduce waste in research

Patients

1 Demand clinical trials addressing unanswered questions that are important

2 Demand reduced wastein research

1 Demand clinical trials addressing unanswered questions that are important

2 Demand reduced wastein research

To promote Priority Setting Partnerships involving patients and clinicians to identify and promote their shared priorities for therapeutic

research

The UK Database of Uncertainties about the Effects of Treatments

To publish uncertainties about the effects of treatments which cannot currently be answered by referring to relevant and reliable, up-to-date

systematic reviews of existing research evidence

JLA Priority Setting Partnerships

Completed Current •Asthma•Urinary incontinence•Vitiligo•Prostate cancer•Schizophrenia•Type 1 diabetes•ENT aspects of balance•Life after stroke•Eczema•Tinnitus•Cleft lip and palate•Lyme disease

•Acne•Childhood disability •Dementia•Dialysis•Head and neck cancer•Inflammatory bowel disease•Multiple sclerosis •Pressure ulcers•Pre-term birth•Sight loss and vision •Skin - Hidradenitis Suppurative•Hips and Knees - surgical interventions•Spinal Cord Injury•Palliative Care

Involving patients, carers and clinicians in research priority setting

The JLA’s principles•Inclusive

•Balance of perspectives•Accessible to all

•Supportive•Recognising a range of capacities and skills

•Transparent and democratic•Data sharing•Agreed protocol•Declaration of interests•Neutral facilitation •Communication and feedback

Research priority themes [across asthma,

incontinence, vitiligo, eczema, stroke, prostate cancer, schizophrenia, aspects of balance, and type 1 diabetes]

• Assessment of long-term effects (wanted and unwanted) of treatments

• Assessment of safety and adverse effects of treatments

• Assessment of complementary and non-prescribed treatments

• Assessment of strategies to improve early diagnosis and treatments, and harmonisation of practice

• Assessment of the effectiveness and safety of self-care

23

397

689

29

332

89

69

307

20

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

JLA patient-clinicianPriority Setting

Partnerships

Registered non-commercial trials

Registeredcommercial trials

Education and training, servicedelivery, psychological, physical,exercise, complementary, diet,other

Radiotherapy, surgery andperioperative, devices, anddiagnostic

Drugs, vaccines and biologicals

Interventions mentioned in research priorities identified byJames Lind Alliance patient-clinician Priority Setting Partnerships,and among registered trials, 2003-2012. (Crowe et al. forthcoming)

www.jla.nihr.ac.uk

Measuring outcomes that matter to patients

Which outcomes of the effects of treatments do cancer patients rate as most important to them?

• Cure• Life expectancy• Freedom from symptoms• Quality-Adjusted Life Years (QALYs)• Tumour size• X-rays/MRI images• Blood/urine test results

1 Demand clinical trials addressing unanswered questions that are important

2 Demand reduced wastein research

Low priority questions addressed

Important outcomes not assessed

Clinicians and patients not involved in setting research agendas

Questions relevantto clinicians &

patients?

Over 50% studies designed without reference to systematic reviews of existing evidence

Over 50% of studies fail to take adequate steps to reduce biases, e.g. unconcealed treatment allocation

Appropriate design and methods?

Over 50% of studies never published in full

Biased under-reporting of studies with disappointing results

Accessible full publication?

Over 30% of trial interventions not sufficiently described

Over 50% of planned study outcomes not reported

Most new research not interpreted in the context of systematic assessment of other relevant evidence

Unbiased and usable report?

50%

85% Research waste = over $85 Billion / year

50%

50%

Overall conclusion:

Patients are suffering and dying unnecessarily because of avoidable

waste in research.

Low priority questions addressed

Important outcomes not assessed

Clinicians and patients not involved in setting research agendas

Questions relevantto clinicians &

patients?

Over 50% studies designed without reference to systematic reviews of existing evidence

Over 50% of studies fail to take adequate steps to reduce biases, e.g. unconcealed treatment allocation

Appropriate design and methods?

Over 50% of studies never published in full

Biased under-reporting of studies with disappointing results

Accessible full publication?

Over 30% of trial interventions not sufficiently described

Over 50% of planned study outcomes not reported

Most new research not interpreted in the context of systematic assessment of other relevant evidence

Unbiased and usable report?

50%

85% Research waste = over $85 Billion / year

50%

50%

Sources of waste in deciding what research to do

Studies designed without reference to systematic

reviews of existing evidence, published and unpublished

Analysis of Introduction sections of all reports of controlled trials in May 2009 and May 2012 in Lancet, New Eng J Med, BMJ, JAMA, & Ann Int Med (Clarke et al. forthcoming)

2009

(n=29)2012

(n=35)First trial addressing the question 5 5

Contains an updated systematic review, which was used to inform the design of a new trial

1 1

Discusses a relevant systematic review, which was not used to inform the design of a new trial

10 13

Refers to other randomized trials 4 10

Although not first trial, does not refer to other randomized trials

9 6

"It is essential that existing sources of evidence, especially systematic reviews, are considered carefully prior to undertaking research…

Research which duplicates other work unnecessarily, or which is not of sufficient quality to contribute something useful to existing knowledge, is in itself unethical."

(Department of Health 2001)

Sometimes the most important advances in knowledge come from analyses of existing evidence

Low priority questions addressed

Important outcomes not assessed

Clinicians and patients not involved in setting research agendas

Questions relevantto clinicians &

patients?

Over 50% studies designed without reference to systematic reviews of existing evidence

Over 50% of studies fail to take adequate steps to reduce biases, e.g. unconcealed treatment allocation

Appropriate design and methods?

Over 50% of studies never published in full

Biased under-reporting of studies with disappointing results

Accessible full publication?

Over 30% of trial interventions not sufficiently described

Over 50% of planned study outcomes not reported

Most new research not interpreted in the context of systematic assessment of other relevant evidence

Unbiased and usable report?

50%

85% Research waste = over $85 Billion / year

50%

50%

Alessandro Liberati

Proportion (%) of clinical trials registered by 1999 and published by 2007

(from Ross et al. PLoS Med 2009;6(9): e1000144).

Country

Size

Phase

Funder

UK HTA program

www.alltrials.net

“(It) is my belief that by taking part in research, I will help other patients in future. For that to happen, the results of the research must be made available. Even if the research isn't finished or the results aren't as expected, the data and information are still of value and should be made available…. Patients become participants to add to knowledge and to eliminate uncertainties. Hiding results, no matter what the reason, isn't in that spirit at all.”

Richard Stephens, 12 June 2013

In summary, patients should require the research community:

• to seek help from patients in selecting questions to address in research

• to review existing evidence systematically before planning new research

• to publish the results of research, whatever their direction or strength

What are patient groups doing to address these problems?

Be more discriminating about what research and

which clinical trials to support

BMJ 2010;340:c725

Chalmers I. Lancet 2000;356:774

www.testingtreatments.org

www.testingtreatments.org

Promote research on the effects of treatments…

…but only if it meets scientific and ethical principles

Promote research on the effects of treatments…

better, more equal