medical evidence increasing at epidemic rates: we all need ebp skills to keep up-to-date bastian,...
TRANSCRIPT
Medical evidence increasing at epidemic rates: we all need EBP skills to keep up-to-date
Bastian, Glasziou, Chalmers (2010) 75 Trials and 11Systematic Reviews a Day: How Will We Ever Keep Up? PLoS Med 7(9)
Medical evidence increasing at epidemic rates: we all need EBP skills to keep up-to-date
approx 75 new
trials published
every day
Bastian, Glasziou, Chalmers (2010) 75 Trials and 11Systematic Reviews a Day: How Will We Ever Keep Up? PLoS Med 7(9)
Medical evidence increasing at epidemic rates: we all need EBP skills to keep up-to-date
MEDLINE 20102,000 articles / day
approx 75 new
trials published
every day
Bastian, Glasziou, Chalmers (2010) 75 Trials and 11Systematic Reviews a Day: How Will We Ever Keep Up? PLoS Med 7(9)
About 1/3 of worthwhile evidence is
eventually refuted or attenuated
About 10% of published evidence
is worth reading
About 1/2 of relevant evidence is
not implemented
6
Rapid critical appraisal
using GATE
Rod Jackson
University of Auckland, NZ
August 2011
Graphic Approach To Evidence Based Practice
Graphic Approach To Epidemiology
Graphic Appraisal Tool for Epidemiological studies
8
the GATE frame
the shape of every epidemiological study8
9
British doctors
non-smokerssmokers
Lung canceryes
no 5 years
smoking status measured
Longitudinal (cohort) study
10
British doctors
non-smokerssmokers
Lung functionnormal
abnormal
smoking status measured
Cross-sectional study
11
British doctors
placeboaspirin
Myocardial infarctionyes
no 5 years
Randomised to aspirin or placebo
RCT
12
Middle-aged US women
Breast cancer
Mammogram negative
yes
no
Test applied
Clinical use of a diagnostic test
Mammogram positive
13
Middle-aged US women
no Breast cancerBreast cancer
mammogrampositive
negative
Diagnostic test accuracy study
14
GATE: Graphic Appraisal Tool for Epidemiological studies
1 picture, 2 formulas & 3 acronyms14
15
One picture: the GATE frame
every epidemiological study hangs on the GATE frame
16
Participants
Exposure Group Comparison Group
OutcomesTime
P
E C
O
T
The 1st acronym = PECOT : the 5 parts of every epidemiological study
17
Lewis RT et al. Should antibiotic prophylaxis be used routinely in clean surgical procedures: A
tentative yes. Surgery 1995;118:742-7.
18
Background. The incidence of surgical site infection (SSI) after clean surgical procedure is regarded as too low for routine antibiotic prophylaxis. But risk of SSI can be as high as 20%. We assessed the value of prophylactic cefotaxime in patients stratified for risk of SSI in a double-blind RCT.Methods. Patients having clean elective operations were stratified for risk & randomized to receive IV cefotaxime 2 gm or placebo before operation & followed for 4-6 weeks for SSI.Results. The 378 of 775 patients who received cefotaxime had 70% fewer SSIs than those who did not --Mantel-Haenszel risk ratio (MH-RR) 0.31; 95 % CI 0.11 to 0.83. Benefit was clear in the 616 low risk patients--0.97% versus 3.9% SSI (MH-RR 0.25, CI 0.07 to 0.87, p = 0.018), but only a trend was seen in 136 high risk patients--2.8% versus 6.1% SSI (MH-RR 0.48, CI 0.09 to 2.5).Conclusions. The results indicate clear benefit for routine antibiotic prophylaxis in clean surgical procedures. High risk patients need more study. 1
8
19
20
P
E C
O
T
1st critical appraisal task: describe study’s design by hanging on GATE frame using PECOT acronym
21
Participants
Study Setting
Eligible Participants
ParticipantsP
22
ParticipantsStudy Setting: patients admitted to QE Hospital, Montreal, Canada (1992-5?)
Eligible Participants: undergoing clean surgery or simple cholecystectomy
Participants: 633 (?consecutive eligible patients)
P
Lewis Trial
23
Exposure & Comparison Groups
Exposure or Intervention Group
(EG)
Comparison or Control Group
(CG)EG CG
24
Exposure & Comparison Groups:low risk group
Exposure or Intervention Group
(EG):2g cefotaxime IV pre-
op
Comparison or Control Group
(CG):Identical placebo
IV
316 317
633
25
Exposure & Comparison Groups:low risk group
Exposure or Intervention Group
(EG):2mg cefotaxime IV
pre-op
Comparison or Control Group
(CG):Identical placebo
IV
316 317
633
308* 308*
* With complete follow-up
26
Outcomes (O)
Outcomes (O)Oa b
c d
yes
no
‘Dis-ease’
27
Outcomes (O)
Primary Outcome (O)
O
a= 3 b= 12
c d
yes
no
Surgical site infection (SSI)
316 317
633
28
Time (T)
T
incidence
prevalence
29
Time (T)
T= time from initiation of treatment
to end of follow-up
incidence
Outcome: SSI316 317
30
Study design: GATE frame & PECOT
Participants
Exposure Group Comparison Group
OutcomesTime
P
E C
O
T
31
Participants
Exposure Group:IV cefotaxime
Comparison Group:IV placebo
Outcomes:SSI
Time:Up to 6 wks post-op
633
Lewis
316 317
Setting: QE Hospital, MontrealEligible: clean surgery or cholecystectomy
3 12
308 308
32
Questions?
33
The 1st formula: study analyses
Occurrence (risk) of disease
= Numerator ÷ Denominator
D
N
34
Denominator (Participants)D
N Numerator (Outcomes)
Occ = N÷D
All epidemiological studies involve measuring the OCCURRENCE of ‘outcomes’
35
Denominator (Participants)D
N Numerator (Outcomes)
Occ = N÷D (T?)
All epidemiological studies involve measuring the OCCURRENCE of ‘outcomes’
T During what period of time (T) was N measured? (incidence)
36
Denominator (Participants)D
N Numerator (Outcomes)
Occ = N÷D (T?)
All epidemiological studies involve measuring the OCCURRENCE of ‘outcomes’
TAt what point in time (T) was N measured?
(prevalence)
37
The 1st formula:Occurrence (risk) = Numerator ÷ Denominator
O
DE DC
NE NC
P
T T
ExposedGroup
ComparisonGroup
38
P
EG CG
O
Exposure Group (EG)
Numerator 1: a
Comparison Group (CG)
a b
c d
Numerator 2: b
2nd appraisal task: describe analyses by hanging numbers on the GATE frame and calculating
occurrences in exposure & comparison groups
Denominator 1: Denominator 2:
39
Occurrence = N ÷ D
P
EG CG
O
Denominator 1:Exposure Group
EG
Numerator 1:a
Denominator 2:Comparison Group
CG
a b
c d
Numerator 2:b
Exposure Group Occurrence:EGO = a ÷ EG
Comparison Group Occurrence:CGO = b ÷ CG
40
Occurrence = N ÷ D
P
EG CG
O
Denominator 1:Exposure Group
EG
Numerator 1:a
Denominator 2:Comparison Group
CG
a b
c d
Numerator 2:b
Exposure Group Occurrence:EGO = a ÷ EG
Comparison Group Occurrence:CGO = b ÷ CG
41
Occurrence = N ÷ D
P
EG CG
O
Denominator 1:Exposure Group
EG
Numerator 1:a
Denominator 2:Comparison Group
CG
a b
c d
Numerator 2:b
Exposure Group Occurrence:EGO = a ÷ EG
Comparison Group Occurrence:CGO = b ÷ CG
42
Calculate EGO & CGO for SSI in low risk group
P
EG CG
Oa b
c d
Denominator 1:Exposure Group
EG = 316
Numerator 1:a = 3
Denominator 2:Comparison Group
CG = 317
Numerator 2:b = 12
EGO = 3/316= 9.5/1000 at 6 weeks
CGO = 12/317 = 37.9/1000 at 6 weeks
ITT (intention to treat) analysis
43
Calculate EGO & CGO for SSI in low risk group
P
EG CG
Oa b
c d
Denominator 1:Exposure Group
EG = 308
Numerator 1:a = 3
Denominator 2:Comparison Group
CG = 308
Numerator 2:b = 12
EGO = 3/308= 9.7/1000 at 6 weeks
CGO = 12/308 =39/1000 at 6 weeks
OT (on treatment) or per-protocol analysis
44
Describing differences between occurrences
Relative difference or Relative Risk = EGO ÷ CGO
Absolute Difference or Risk Difference = EGO - CGO
Number Needed To Treat (NNT) = 1 ÷ RD
45
Describing differences between occurrences (SSI in low risk patients)
Relative difference or Relative Risk = EGO ÷ CGO
Absolute Difference or Risk Difference = EGO - CGO
Number Needed To Treat (NNT) = 1 ÷ RD
= 9.5/1000 ÷ 37.9/1000 = 0.25
= 9.5/1000 - 37.9/1000 = -28.4/1000
= 1 ÷ (- 28.4 /1000) = - 1000/28.4 = 35
‘if 35 patients were given IV cefotaxime pre-op, there would be 1 fewer SSI up to 6 weeks post-op’
46
Study analyses
it’s all about EGO & CGO
47
Questions?
48
P
E C
OT
P
E
C
O
T
* Paul Glasziou
The 2nd acronym = RAMBO* : assessing bias
‘strength of study’
Recruitment
Allocation
Maintenance
Blind or
Objective outcomes measurement
48
49
P
E C
OT
P
E
C
O
T
* Paul Glasziou
The 2nd acronym = RAMBO* : assessing non random error (i.e. bias)
Recruitment
Allocation
Maintenance
Blind or
Objective outcomes measurement
49
50
P
E C
OT
P
E
C
O
T
3rd appraisal task: assess the degree of bias by applying the RAMBO acronym
Recruitment
Allocation
Maintenance
Blind or
Objective outcomes measurement
51
RAMBO
E C
OT
were Recruitment processes appropriate to study goals?P
• Study setting & eligibility criteria well described?e.g. Recruit random/representative sample OR consecutive eligibles OR volunteers from advertisements• Participants representative of eligibles?• Prognostic/risk profile appropriate to study question?
P
Study setting
Eligible people
52
EG CG
OT
RCT: Allocate randomly by investigators (e.g drugs)
EG CG
OT
Cohort: Allocate by measurement (e.g. smoking)
RAMBO: A is for Allocation
were EG & CG similar at baseline?
Was Allocation to EG & CG
successful?
53
RAMBO
EG CG
OT
were Participants Maintained as allocated?
did most participants remain in allocated groups (EG & CG)
P
Participants &/or investigators blind to exposure (and comparison exposure)?
Compliance high & similar in EG & CG?Contamination low & similar in EG & CG?Co-interventions low & similar in EG & CG?
Completeness of follow-up high & similar in EG & CG?
54
RAMBO
EG CG
OT
Were outcomes measuredBlind or Objectively?
P
If outcome measurements not Objective (eg. automated or definitive) were investigators Blind to exposure (and comparison exposure)
55
The 4 (GATE) study biases
P
E C
OT
Recruitment bias
Allocation bias
Maintenance bias
Outcomes Measurement bias
56
Questions?
57
The 2nd formula: assessing random error
Random error = 95% Confidence Interval(1.96 x Standard Error)
57
58
4th appraisal task: assess degree of random error in study findings using the 2nd formula
Random error = 95% Confidence Interval
For the Outcome SSI (low risk group) :
EGO = 9.5/1000; (95% CI = 3.2 to 27.5)
CGO = 37.9/1000; (95% CI = 21.8 to 65)
EGO÷CGO = 0.25 (0.07 to 0.88)
EGO-CGO = -28.4 (-52 to -4.8)
NNT = -35 (-19 to -211)58
59
Excel CATs & paper Gate-lites
There is a GATE for every study designwww.epiq.co.nz
59
60
Final appraisal task: search for & appraise SRs / meta-analyses using 3rd acronym
(FAITH)
• Find appropriate studies?
• Appraise selected studies?
• Include only valid studies?
• Total-up (synthesise) appropriately?
• Heterogeneity adequately addressed?
60
61
Systematic Reviews
There are 4 Cochrane SRs on this topic and the findings are not consistent
Using GATE as a framework for evidence
based practice
The first 4 steps of EBP1. Ask a focused question.
2. Access (systematically search for) epidemiological evidence to help answer question.
3. Appraise evidence found for its validity, effect size, precision (ideally all the relevant evidence)
4. Apply the evidence:
a. amalgamate the valid evidence with other relevant information (patient/community values, clinical/health issues, & policy context) and make an evidence-based decision; and
b. act (implement) the decision in practice
EBP Step 1: Ask- turn your question into a 5-part PECOT question
Participants (the patient problem)
Exposure (e.g. a therapy)
Comparison (there is always an alternative! - another therapy or no treatment…
Outcome (e.g. a disease you want to prevent or manage)
Time frame (over which you expect a result)
65
EBP Step 2: Access the evidence – use PECOT to choose search terms
Participants (the patient problem)
Exposure (e.g. a therapy)
Comparison (there is always an alternative! - another therapy or no treatment…
Outcome (e.g. a disease you want to prevent or manage)
Time frame (over which you expect a result)
65
EBP Step 3: Appraise the evidence
‘using the best evidence from epidemiology to help inform decisions’
• more critically (using GATE)
• more systematically (using FAITH)
EBP Step 4: APPLY the evidence by: a. AMALGAMATING the relevant information & making
an evidence-based decision:’ the X-factor
©
Evidence
Clinical / health considerations
Policy issues
Patient / community preferences
X-factor: making evidence-based decisions
Xpertise: ‘putting it all together’ the art of practice