Understanding the evidence base: the need to critically examine all the studies

and evidence on drug related interventions as collected and analysed

by the Cochrane Collaboration

Marina DavoliCo-ordinating Editor Cochrane Drugs and Alcohol

Review Group

Lisbon, 6-8 May 2009

SUMMARY

• Why the evidence base

• Where is the evidence

• Future challenges and perspectives

A reminder of the responsibilities of practitioners, policy makers and

researchers to the public

Good intentions and plausible theories

are no substitute for reliable evidence from empirical

research about the effects of social and healthcare

interventions

Iain Chalmers, 8° CC 2000

Example I: SIDS

…could it be achieved earlier?

Example 1: SIDS

Jilbert, IJE 2005

Systematic review of effectiveness: RR of death by sleep position

Example 1: SIDS

• Jilbert suggests that the delay is attributable to a greater “believe” in theoretical mechanisms (cardiac, cerebral, etc) of death than in the evidence of effectiveness

• From 1970 to 1992 10000 US 10000 US and 5000050000 european children could have survived if guidelines would have issued earlier

Jilbert, IJE 2005

Example 2

Interventions intended to prevent

delinquent adolescents from becoming career

criminals

The likelihood of a prison sentence for a17-20 year old convicted of criminal

damage

• Powys 5.1%• Surrey 5.9%• Suffolk 8.1%• Norfolk 8.3%• Gloucestershire 9.8%• Nottinghamshire 21.1%• South Yorkshire 21.9%• Northumbria 25.3%• Kent 26.1%• Hertfordshire 31.6%

Home Office statistics for Crown Courts in 1988, Guardian 23 Jan 90

International variation in sentencing policy

• Sentences for a 19-year old armed robber who stole £800,000 with four others in a bank raid

• Nigeria Death N Zealand 9 years• H Kong Life India 7 years• Texas99 years Denmark 6 years• Greece 20 years Canada 5 years• England 14 years Norway 2 years• Scotland 10 years

• Survey of the International Bar Association (1990): based on judges’ and legal experts’ responses to hypothetical cases

RCTs of ‘Scared Straight’ programmes

Trial Change in criminal behaviour

Michigan 1967 26 % increase

Greater Egypt 1979 5 % increase

Yarborough 1979 1 % increase

Orchowsky 1981 2 % increase

Vreeland 1981 11 % increase

Finckenauer 1982 30 % increase

Lewis 1983 14 % increase

Petrosino et al 2000

ConclusionsResponsible professionals in the health and social services need

more thangood intentions and plausible

theoriesto guide their practice.

Their prescriptions and proscriptions for others should be

informed by reliable research evidence showing that they are

more likely to do good than harm.

The Cochrane CollaborationThe Cochrane Collaboration

An international network of people who conduct, update and disseminate systematic reviews of the effect of health care interventions. There are 51 Review Groups in the Cochrane Collaboration

Marina Davoli: Co-ordinating EditorLaura Amato: Managing Editor Simona Vecchi:Trial search Co.Zuzana Mitrova: Assistant Co.

Robert Ali (Australia)Zhao Chengzheng (China)Fabrizio Faggiano (Italy)Michael Farrell (UK) David Foxcroft (UK)Walter Ling (USA)

The editorial base

The Editors

Cochrane Review Group on Drugs and Alcohol

RCTs on interventions for drug addiction (n=6530)

Specialized Register Drugs and Alcohol Cochrane Group. CL 2.2009

0

100

200

300

400

500

600

1956-1975

1976-1980

1981-1985

1986-1990

1991-1995

1996-2000

2001-2005

2006-marzo2009years

n°

of

refe

ren

ce

s

Alcohol

Opiates

Not specified substance

Cocaine

cannabis

amphetamine

benzodiazepines

Other substances

Cochrane Systematic reviewsCochrane Systematic reviews

• Are the result of a complex process :– Formulate a proper question– Comprehensive study search– Objective selection and data

extraction– Critical evaluation of primary studies– Synthesis– Update

         

State of the art CLIB 3.2009http://www.cdag.cochrane.org

Substance N° Reviews N° ProtocolsN° Titles

registeredTotal

Alcohol 8 10 3 23

Opiate 20 4 3 27

Psychostimulants 9 2 1 12

Other 3 1  / 4

Poly drugs 4 2 6 12

Prevention 3 /   / 3

Total 47* 19 13 79

*629 studies included/1824 considered for inclusion

Effectiveness of all interventions considered Effectiveness of all interventions considered in CRs published in CLIB 3.2009in CRs published in CLIB 3.2009

26%

2%

34%

33%

5%

beneficial or lilely to be beneficial trade off between benefits and harms

unknown effectiveness unlikely to be beneficial

likely to be ineffective or harmful

Beneficial or likely to be beneficial

Trade off between benefits and harms Unknown effectiveness

Unlikely to be beneficialLikely to be ineffective or harmful

OTHER

100%

PREVENTION

20%

70%

10%

OPIATES

43%

17%

27%

13%

POLY DRUGS

7%

20%

46%

27%

PSYCHOSTIMULANTS

16,6%

83,3

ALCOHOL

27%

13%60%

N= 15 N= 8 N= 25

N= 48 N= 20 N= 15

http://www.who.int/substance_abuse/publications/opioid_dependence_guidelines.pdf

CONCLUSIONS• Cochrane reviews provide the “best available

evidence” answering the questions they are planned to answer

• Cochrane reviews sometimes provide useful results to identify effective interventions

• They also identify a wide range of interventions unlikely to be beneficial and even likely to be ineffective or harmful

• Cochrane reviews also help in identifying areas where more systematic reviews are necessary and areas where more primary research is required

BUT….

THE EMCDDA AND THE EU SCIENTIFIC COMMUNITY PLAY A KEY ROLE

• Ask the question:– Help in prioritizing questions to be addressed by systematic

reviews in order to be relevant for:• Patients and their families• Care givers• Policy maker

– Less than 1/3 of published studies considered for inclusion

• Support in making the evidence available and accessible

• Use the evidence to inform guidelines

• The Cochrane Collaboration is a “virtual” organisation open to all possible contributions

Country of origin of the studies Country of origin of the studies included in the Specialized Registerincluded in the Specialized Register

Asia Australia/New Zealand Europe Middle East

North America South America South Africa

68.3%

21.7%4.4%4.1%

0.7%

0.5%

0.3%