medical evidence increasing at epidemic rates: we all need ebp skills to keep up-to-date bastian,...

Medical evidence increasing at epidemic rates: we all need EBP skills to keep up-to-date

Bastian, Glasziou, Chalmers (2010) 75 Trials and 11Systematic Reviews a Day: How Will We Ever Keep Up? PLoS Med 7(9)

Medical evidence increasing at epidemic rates: we all need EBP skills to keep up-to-date

approx 75 new

trials published

every day

Bastian, Glasziou, Chalmers (2010) 75 Trials and 11Systematic Reviews a Day: How Will We Ever Keep Up? PLoS Med 7(9)

Medical evidence increasing at epidemic rates: we all need EBP skills to keep up-to-date

MEDLINE 20102,000 articles / day

approx 75 new

trials published

every day

Bastian, Glasziou, Chalmers (2010) 75 Trials and 11Systematic Reviews a Day: How Will We Ever Keep Up? PLoS Med 7(9)

About 1/3 of worthwhile evidence is

eventually refuted or attenuated

About 10% of published evidence

is worth reading

About 1/2 of relevant evidence is

not implemented

6

Rapid critical appraisal

using GATE

Rod Jackson

University of Auckland, NZ

August 2011

Graphic Approach To Evidence Based Practice

Graphic Approach To Epidemiology

Graphic Appraisal Tool for Epidemiological studies

8

the GATE frame

the shape of every epidemiological study8

9

British doctors

non-smokerssmokers

Lung canceryes

no 5 years

smoking status measured

Longitudinal (cohort) study

10

British doctors

non-smokerssmokers

Lung functionnormal

abnormal

smoking status measured

Cross-sectional study

11

British doctors

placeboaspirin

Myocardial infarctionyes

no 5 years

Randomised to aspirin or placebo

RCT

12

Middle-aged US women

Breast cancer

Mammogram negative

yes

no

Test applied

Clinical use of a diagnostic test

Mammogram positive

13

Middle-aged US women

no Breast cancerBreast cancer

mammogrampositive

negative

Diagnostic test accuracy study

14

GATE: Graphic Appraisal Tool for Epidemiological studies

1 picture, 2 formulas & 3 acronyms14

15

One picture: the GATE frame

every epidemiological study hangs on the GATE frame

16

Participants

Exposure Group Comparison Group

OutcomesTime

P

E C

O

T

The 1st acronym = PECOT : the 5 parts of every epidemiological study

17

Lewis RT et al. Should antibiotic prophylaxis be used routinely in clean surgical procedures: A

tentative yes. Surgery 1995;118:742-7.

18

Background. The incidence of surgical site infection (SSI) after clean surgical procedure is regarded as too low for routine antibiotic prophylaxis. But risk of SSI can be as high as 20%. We assessed the value of prophylactic cefotaxime in patients stratified for risk of SSI in a double-blind RCT.Methods. Patients having clean elective operations were stratified for risk & randomized to receive IV cefotaxime 2 gm or placebo before operation & followed for 4-6 weeks for SSI.Results. The 378 of 775 patients who received cefotaxime had 70% fewer SSIs than those who did not --Mantel-Haenszel risk ratio (MH-RR) 0.31; 95 % CI 0.11 to 0.83. Benefit was clear in the 616 low risk patients--0.97% versus 3.9% SSI (MH-RR 0.25, CI 0.07 to 0.87, p = 0.018), but only a trend was seen in 136 high risk patients--2.8% versus 6.1% SSI (MH-RR 0.48, CI 0.09 to 2.5).Conclusions. The results indicate clear benefit for routine antibiotic prophylaxis in clean surgical procedures. High risk patients need more study. 1

8

19

20

P

E C

O

T

1st critical appraisal task: describe study’s design by hanging on GATE frame using PECOT acronym

21

Participants

Study Setting

Eligible Participants

ParticipantsP

22

ParticipantsStudy Setting: patients admitted to QE Hospital, Montreal, Canada (1992-5?)

Eligible Participants: undergoing clean surgery or simple cholecystectomy

Participants: 633 (?consecutive eligible patients)

P

Lewis Trial

23

Exposure & Comparison Groups

Exposure or Intervention Group

(EG)

Comparison or Control Group

(CG)EG CG

24

Exposure & Comparison Groups:low risk group

Exposure or Intervention Group

(EG):2g cefotaxime IV pre-

op

Comparison or Control Group

(CG):Identical placebo

IV

316 317

633

25

Exposure & Comparison Groups:low risk group

Exposure or Intervention Group

(EG):2mg cefotaxime IV

pre-op

Comparison or Control Group

(CG):Identical placebo

IV

316 317

633

308* 308*

* With complete follow-up

26

Outcomes (O)

Outcomes (O)Oa b

c d

yes

no

‘Dis-ease’

27

Outcomes (O)

Primary Outcome (O)

O

a= 3 b= 12

c d

yes

no

Surgical site infection (SSI)

316 317

633

28

Time (T)

T

incidence

prevalence

29

Time (T)

T= time from initiation of treatment

to end of follow-up

incidence

Outcome: SSI316 317

30

Study design: GATE frame & PECOT

Participants

Exposure Group Comparison Group

OutcomesTime

P

E C

O

T

31

Participants

Exposure Group:IV cefotaxime

Comparison Group:IV placebo

Outcomes:SSI

Time:Up to 6 wks post-op

633

Lewis

316 317

Setting: QE Hospital, MontrealEligible: clean surgery or cholecystectomy

3 12

308 308

32

Questions?

33

The 1st formula: study analyses

Occurrence (risk) of disease

= Numerator ÷ Denominator

D

N

34

Denominator (Participants)D

N Numerator (Outcomes)

Occ = N÷D

All epidemiological studies involve measuring the OCCURRENCE of ‘outcomes’

35

Denominator (Participants)D

N Numerator (Outcomes)

Occ = N÷D (T?)

All epidemiological studies involve measuring the OCCURRENCE of ‘outcomes’

T During what period of time (T) was N measured? (incidence)

36

Denominator (Participants)D

N Numerator (Outcomes)

Occ = N÷D (T?)

All epidemiological studies involve measuring the OCCURRENCE of ‘outcomes’

TAt what point in time (T) was N measured?

(prevalence)

37

The 1st formula:Occurrence (risk) = Numerator ÷ Denominator

O

DE DC

NE NC

P

T T

ExposedGroup

ComparisonGroup

38

P

EG CG

O

Exposure Group (EG)

Numerator 1: a

Comparison Group (CG)

a b

c d

Numerator 2: b

2nd appraisal task: describe analyses by hanging numbers on the GATE frame and calculating

occurrences in exposure & comparison groups

Denominator 1: Denominator 2:

39

Occurrence = N ÷ D

P

EG CG

O

Denominator 1:Exposure Group

EG

Numerator 1:a

Denominator 2:Comparison Group

CG

a b

c d

Numerator 2:b

Exposure Group Occurrence:EGO = a ÷ EG

Comparison Group Occurrence:CGO = b ÷ CG

40

Occurrence = N ÷ D

P

EG CG

O

Denominator 1:Exposure Group

EG

Numerator 1:a

Denominator 2:Comparison Group

CG

a b

c d

Numerator 2:b

Exposure Group Occurrence:EGO = a ÷ EG

Comparison Group Occurrence:CGO = b ÷ CG

41

Occurrence = N ÷ D

P

EG CG

O

Denominator 1:Exposure Group

EG

Numerator 1:a

Denominator 2:Comparison Group

CG

a b

c d

Numerator 2:b

Exposure Group Occurrence:EGO = a ÷ EG

Comparison Group Occurrence:CGO = b ÷ CG

42

Calculate EGO & CGO for SSI in low risk group

P

EG CG

Oa b

c d

Denominator 1:Exposure Group

EG = 316

Numerator 1:a = 3

Denominator 2:Comparison Group

CG = 317

Numerator 2:b = 12

EGO = 3/316= 9.5/1000 at 6 weeks

CGO = 12/317 = 37.9/1000 at 6 weeks

ITT (intention to treat) analysis

43

Calculate EGO & CGO for SSI in low risk group

P

EG CG

Oa b

c d

Denominator 1:Exposure Group

EG = 308

Numerator 1:a = 3

Denominator 2:Comparison Group

CG = 308

Numerator 2:b = 12

EGO = 3/308= 9.7/1000 at 6 weeks

CGO = 12/308 =39/1000 at 6 weeks

OT (on treatment) or per-protocol analysis

44

Describing differences between occurrences

Relative difference or Relative Risk = EGO ÷ CGO

Absolute Difference or Risk Difference = EGO - CGO

Number Needed To Treat (NNT) = 1 ÷ RD

45

Describing differences between occurrences (SSI in low risk patients)

Relative difference or Relative Risk = EGO ÷ CGO

Absolute Difference or Risk Difference = EGO - CGO

Number Needed To Treat (NNT) = 1 ÷ RD

= 9.5/1000 ÷ 37.9/1000 = 0.25

= 9.5/1000 - 37.9/1000 = -28.4/1000

= 1 ÷ (- 28.4 /1000) = - 1000/28.4 = 35

‘if 35 patients were given IV cefotaxime pre-op, there would be 1 fewer SSI up to 6 weeks post-op’

46

Study analyses

it’s all about EGO & CGO

47

Questions?

48

P

E C

OT

P

E

C

O

T

* Paul Glasziou

The 2nd acronym = RAMBO* : assessing bias

‘strength of study’

Recruitment

Allocation

Maintenance

Blind or

Objective outcomes measurement

48

49

P

E C

OT

P

E

C

O

T

* Paul Glasziou

The 2nd acronym = RAMBO* : assessing non random error (i.e. bias)

Recruitment

Allocation

Maintenance

Blind or

Objective outcomes measurement

49

50

P

E C

OT

P

E

C

O

T

3rd appraisal task: assess the degree of bias by applying the RAMBO acronym

Recruitment

Allocation

Maintenance

Blind or

Objective outcomes measurement

51

RAMBO

E C

OT

were Recruitment processes appropriate to study goals?P

• Study setting & eligibility criteria well described?e.g. Recruit random/representative sample OR consecutive eligibles OR volunteers from advertisements• Participants representative of eligibles?• Prognostic/risk profile appropriate to study question?

P

Study setting

Eligible people

52

EG CG

OT

RCT: Allocate randomly by investigators (e.g drugs)

EG CG

OT

Cohort: Allocate by measurement (e.g. smoking)

RAMBO: A is for Allocation

were EG & CG similar at baseline?

Was Allocation to EG & CG

successful?

53

RAMBO

EG CG

OT

were Participants Maintained as allocated?

did most participants remain in allocated groups (EG & CG)

P

Participants &/or investigators blind to exposure (and comparison exposure)?

Compliance high & similar in EG & CG?Contamination low & similar in EG & CG?Co-interventions low & similar in EG & CG?

Completeness of follow-up high & similar in EG & CG?

54

RAMBO

EG CG

OT

Were outcomes measuredBlind or Objectively?

P

If outcome measurements not Objective (eg. automated or definitive) were investigators Blind to exposure (and comparison exposure)

55

The 4 (GATE) study biases

P

E C

OT

Recruitment bias

Allocation bias

Maintenance bias

Outcomes Measurement bias

56

Questions?

57

The 2nd formula: assessing random error

Random error = 95% Confidence Interval(1.96 x Standard Error)

57

58

4th appraisal task: assess degree of random error in study findings using the 2nd formula

Random error = 95% Confidence Interval

For the Outcome SSI (low risk group) :

EGO = 9.5/1000; (95% CI = 3.2 to 27.5)

CGO = 37.9/1000; (95% CI = 21.8 to 65)

EGO÷CGO = 0.25 (0.07 to 0.88)

EGO-CGO = -28.4 (-52 to -4.8)

NNT = -35 (-19 to -211)58

59

Excel CATs & paper Gate-lites

There is a GATE for every study designwww.epiq.co.nz

59

60

Final appraisal task: search for & appraise SRs / meta-analyses using 3rd acronym

(FAITH)

• Find appropriate studies?

• Appraise selected studies?

• Include only valid studies?

• Total-up (synthesise) appropriately?

• Heterogeneity adequately addressed?

60

61

Systematic Reviews

There are 4 Cochrane SRs on this topic and the findings are not consistent

Using GATE as a framework for evidence

based practice

The first 4 steps of EBP1. Ask a focused question.

2. Access (systematically search for) epidemiological evidence to help answer question.

3. Appraise evidence found for its validity, effect size, precision (ideally all the relevant evidence)

4. Apply the evidence:

a. amalgamate the valid evidence with other relevant information (patient/community values, clinical/health issues, & policy context) and make an evidence-based decision; and

b. act (implement) the decision in practice

EBP Step 1: Ask- turn your question into a 5-part PECOT question

Participants (the patient problem)

Exposure (e.g. a therapy)

Comparison (there is always an alternative! - another therapy or no treatment…

Outcome (e.g. a disease you want to prevent or manage)

Time frame (over which you expect a result)

65

EBP Step 2: Access the evidence – use PECOT to choose search terms

Participants (the patient problem)

Exposure (e.g. a therapy)

Comparison (there is always an alternative! - another therapy or no treatment…

Outcome (e.g. a disease you want to prevent or manage)

Time frame (over which you expect a result)

65

EBP Step 3: Appraise the evidence

‘using the best evidence from epidemiology to help inform decisions’

• more critically (using GATE)

• more systematically (using FAITH)

EBP Step 4: APPLY the evidence by: a. AMALGAMATING the relevant information & making

an evidence-based decision:’ the X-factor

©

Evidence

Clinical / health considerations

Policy issues

Patient / community preferences

X-factor: making evidence-based decisions

Xpertise: ‘putting it all together’ the art of practice