Why bother with Evidence-Based Practice?

Professor Paul GlasziouBond Universitywww.crebp.net.au/

www.testingtreatments.orgFree pdf: Ch 5, 6, 7

Introductory Lecture: Objectives

What is Evidence-Based Medicine?

The steps of doing EBM1. Formulate Clinical Questions2. Search for Evidence3. Appraisal of research4. Apply to clinical problem

What is Evidence-Based Medicine?

“Evidence-based medicine is the integration of best research evidence with clinical expertise and patient values”

- Dave Sackett

Patient Concerns

Clinical Expertise

Best ResearchSackett DL, Rosenberg WMC, Gray JAM, Haynes RB, Richardson WS: Evidence based medicine: what it is and what it isn’t. BMJ 1996;312:71-2.

“EBM” - birth of a term

Update of CMAJ series on how to read a paper

JAMA User guides 1991authors seek a new term

Clinical epidemiology?

Scientific medicine?

Evidence-based medicine!

The need for EvidenceVertebroplasty for osteoporotic fractures

Gray et al Spine 2008. Nation-wide and State-specific primary vertebroplasty rates per 100,000 Part B fee-for-service

Steady rise in use of vertebroplastyfor a decade

2009: two large trials publishshowing no effect over placebo

Slides courtesy R Buchbinder

Unnecessary Procedures

http://www.cbsnews.com/8301-18563_162-5217954.html

Please click on the black screen to view the video

Some popular treatment innovations that did not work

Vertebroplasty for osteoporotic fractures (NEJM, 2010)

Arthoscopic knee lavage (Moseley, NEJM, 2002) Blood glucose monitoring for non-insulin

dependent diabetes (DiGEM trial, BMJ 2007) Tight control of diabetes (ACCORD, NEJM, 2010) Prostate cancer screening (Djulbegovic, BMJ

2010) Ovarian cancer screening (JAMA, 2011)

Most medical innovations don’t workan analysis of 136 trials in myeloma

New TreatmentBetter

New TreatmentWorse

How can we find the research that will improve the care of our patients?

0

500000

1000000

1500000

2000000

2500000

Biomedical MEDLINE Trials Diagnostic?

Med

ical

Art

icle

s p

er Y

ear

5,000?per day

1,500 per day

95 per day

Medic

al A

rtic

les

Per

Year

19 of 20

What should you believe?

Keeping up to dateWhat is your JASPA* score?* (Journal Associated Score of Personal Angst)

J: Are you ambivalent about renewing your JOURNAL subscriptions?A: Do you feel ANGER towards prolific authors?S: Do you ever use journals to help you SLEEP?P: Are you surrounded by PILES of PERIODICALS?A: Do you feel ANXIOUS when journals arrive?

*Modified from: BMJ 1995;311:1666-1668

0 (?liar) 1-3 (normal range) >3 (sick; at risk for polythenia gravis and related conditions)

Coping with the overload: three possible things you might try

A. Read an evidence-based abstraction journal (and cancel other journals)

B. Keep a logbook of your own clinical questions

C. Run a case-discussion journal club with your practice

Part 2: The 4 steps of “pull” EBM

1. Formulate an answerable question2. Track down the best evidence 3. Critically appraise the evidence4. Individualise, based clinical expertise and patient concerns

Step 1Formulate an answerable clinical questionStructure of researchable

questions – PICO-T

Population/Patients

Intervention

Comparison

Outcome

Time

Formulating answerable clinical questionsStructure of researchable

questions – PICO-T

Population/Patients

Outcome

What are your clinical questions?

A 35 year old man says his brother

recently died of a ruptured

cerebral aneurysm.

He is worried about whether he

might have one and what the

chances are that it would rupture.

-> PICO Table

Risk Factors

Cause(s)

SymptomsSigns, Tests

Prognosis

Treatment Effect

Past Current Future

Types of question: stroke

Frequency

Cohort Study SurveyInception Cohort Study

Treatments

Randomised Trial

CT Scan

Cross Sectional Study

What are the … outcomes (PO?)

Outcomes ?

Patients

Qualitative Research

The “best” evidence depends on the type of questionLevel Treatment Prognosis Diagnosis

I

II Randomised trial

Inception Cohort

Cross sectional

III

Level Treatment Prognosis Diagnosis

I Systematic Review of …

Systematic Review of …

Systematic Review of …

II Randomised trial

Inception Cohort

Cross sectional

III

The “best” evidence depends on the type of question

What do you do about conflictingclaims?

2. Searching: finding good answers?

Impact of searching on correctness of answers to clinical questions

Right to Right

Wrong to Right

Right to Wrong

Wrong to Wrong

McKibbon(GP or IM)

28% 13% 11% 48%

Impact of searching on correctness of answers to clinical questions

Right to Right

Wrong to Right

Right to Wrong

Wrong to Wrong

McKibbon(GP or IM)

28% 13% 11% 48%

Quick Clinical(GPs)

21% 32% 7% 40%

Hersh(Med students)

20% 31% 12% 36%

Hersh(Nursing)

18% 17% 14% 52%

3. Rapid Critical Appraisal

It’s peer-reviewed, therefore it must be OK?

Is the PICO a POEM?

Patient

Oriented

Evidence that

Matters

Critical Appraisal Steps

Did you find good quality studies?

Two steps1.What is the PICO (Question)2.Is the potential bias low?• “RAMMbo” (Valid Study?)• “FAITH” (Valid Review?)

Use the RAMMbo to check validity

Was the Study valid?1. Representativeness

Who did the subjects represent?2. Allocation

Was the assignment to treatments randomised?

3. Maintainence Were the groups treated equally?

4. Measurements blinded OR objective

Were patients and clinicians “blinded” to treatment? OR

Were measurements objective & standardised?

Modified from: User Guide. JAMA, 1993

Fundamental Equation of Error

Measure = Truth + Bias + Random Error

Use good study

design

Use large

numbers

Researcher

Critically AppraiseDesign

ConfidenceIntervals

andP-values

Reader

Two methods of assessing the role of chance

P-values (Hypothesis Testing) use statistical test to examine the ‘null’

hypothesis associated with “p values” - if p<0.05 then

result is statistically significant Confidence Intervals (Estimation)

estimates the range of values that is likely to include the true value

Relationship between p-values and confidence intervals - if the value corresponding to ‘no effect’ (RR of 1 or treatment difference of 0) falls outside the CI then the result is statistically significant

Step 4: Applying to the individual

What do the results mean on average?

What do they mean for this individual?

Applying research requires both “Whether to” and “How to”

“Whether to” Evidence quality Individual applicability

“How to” What & where? How long & how often?

BMJ 2003; 327 : 135