jüni clinical evidence · evidence evidence evidencerandomized trial meta-analysis ... tavi more...

1

Peter Jüni MD [email protected]

Director, Applied Health Research Centre (AHRC)Li Ka Shing Knowledge Institute

Professor, Department of MedicineUniversity of Toronto

Clinical evidence: separating the wheat from the chaff

Hierarchy of evidence


evidenceevidence

evidence

Randomized trial

Meta-analysisNot all meta-analyses

are the same …

Heterogeneity

Jüni et al, Lancet 2004

2

Lindholm et al, Lancet 2005

• Level A– Multiple RCTs or meta-analyses

• Level B– Single RCT or large non-randomised

study• Level C

– Consensus or small studies, retrospective studies or registries

Levels of Evidence

Higgins et al, BMJ 2003

Heterogeneity: I2 Statistic

0% No heterogeneity

25% Low heterogeneity

50% Moderate heterogeneity

75% High heterogeneity


I2=0%I2=0%

Lindholm et al, Lancet 2005

I2=52%I2=52%

Size

Egger et al, BMJ 1998

3

Comparison of meta-analyses with large trials

Requirements for a meta-analysis to be helpful

No or little heterogeneity between trialsAt least one moderately sized trial 50 to 75 events (binary outcome)150 to 200 patients (cont. outcome)

Symmetrical funnel plot

✔

✔

✔


evidenceevidence

evidence

Randomized trial

Meta-analysis✔

4

Requirements for a meta-analysis to be helpful

No or little heterogeneity between trialsAt least one moderately sized trial 50 to 75 events (binary outcome)150 to 200 patients (cont. outcome)

Symmetrical funnel plot

✗

✗

✗


evidenceevidence

evidenceRandomized trialMeta-analysis✗

Observational studies of any help?

Papanikolaou et al, CMAJ 2006

Papanikolaou et al, CMAJ 2006

Relative risk

Confounding factors

Unmeasured ... Imperfectlymeasured ... Unadjusted for ...

5

Propensity scores

Consecutive patients with aortic valve replacement

Transcatheter implantation: n= 114

Surgical replacement: n=1008

Piazza et al, EuroIntervention 2009

Crude 4.57 (2.17 to 9.65)4.57 (2.17 to 9.65)

TAVI more beneficial SAVR more beneficial

10.1 0.2 0.5 1 2 5 10

Odds ratio (95% CI)


(n=114) (n=1008)Age, y (SD) 82.8 (5.5) 69.9 (11.4) 12.9 (10.8 to 15.1) < 0.001Female 64 (56.1%) 408 (41.5%) 15.7% (6.1 to 25.3%) 0.001EuroSCORE (SD) 20.1 (13.4) 9.1 (10.2) 11.0 (9.0 to 13.1) < 0.001NYHA class <0.001 I 1 (0.9%) 147 (14.6%) II 14 (12.3%) 403 (40.0%) III 78 (68.4%) 356 (35.3%) IV 21 (18.4%) 102 (10.1%)Previous coronary bypass surgery 28 (24.6%) 42 (4.2%) 20.4% (12.4 to 28.4%) < 0.001LVEF < 0.001 > 50% 67 (58.8%) 834 (82.7%) 30-50% 40 (35.1%) 135 (13.4%) < 30% 7 (6.1%) 39 (3.9%)Atrial fibrillation 22 (19.3%) 90 (8.9%) 10.4% (2.9 to 17.8%) <0.001Cereberovascular disease 20 (17.5%) 50 (5.0) 12.6% (5.5 to 19.7%) < 0.001Peripheral vascular disease 21 (18.4%) 47 (4.7%) 13.8% (6.5 to 21.0%) < 0.001COPD 24 (21.1%) 134 (13.3%) 7.8% (0.0 to 15.5%) 0.024

Pulmonary hypertension 34 (29.8%) 86 (8.5%) 21.3% (12.7 to 29.9%) < 0.001Creatinine, umol/L (SD) 114 (92.9) 98.6 (61.5) 15.7 (3.0 to 28.3) 0.016

PDifferenceTranscatheter implantation

Surgical implantation

Crude

Multivariable

4.57 (2.17 to 9.65)

3.05 (1.09 to 8.51)

4.57 (2.17 to 9.65)

3.05 (1.09 to 8.51)


10.1 0.2 0.5 1 2 5 10

Odds ratio (95% CI)


0

5

10

15

20

Pro

babi

lity

dens

ity

0 0.2 0.4 0.6 0.8 1

Propensity score

TAVI

SAVR


6

Crude

Multivariable

PS adjusted

PS adjusted, multivariable

PS matched

IPT weighted

IPT weighted, multivariable

4.57 (2.17 to 9.65)

3.05 (1.09 to 8.51)

3.02 (1.01 to 9.04)

2.76 (0.88 to 8.69)

3.38 (0.90 to 12.7)

0.60 (0.11 to 3.36)

1.25 (0.42 to 3.72)

4.57 (2.17 to 9.65)

3.05 (1.09 to 8.51)

3.02 (1.01 to 9.04)

2.76 (0.88 to 8.69)

0.60 (0.11 to 3.36)

1.25 (0.42 to 3.72)


10.1 0.2 0.5 1 2 5 10

Odds ratio (95% CI)


0

5

10

15

20

Pro

babi

lity

dens

ity

0 0.2 0.4 0.6 0.8 1

Propensity score

TAVI

SAVR


0

5

10

15

20

Pro

babi

lity

dens

ity

0 0.2 0.4 0.6 0.8 1

Propensity score

PS ≤ 0.325 PS > 0.325

TAVI

SAVR


As good (or bad) as multivariable adjustment

Da Costa et al, EuroIntervention 2014

7

Observational studies

Not to reliably estimate treatment effects

in the absence of RCTs …

Vandenbroucke & Psaty, JAMA 2008

To understand the full spectrum of adverse effects — those that occur late, that werenot known beforehand, and that are rarebut nevertheless serious — and to be able

to investigate the true incidence of knownadverse effects in circumstances ofactual prescribing, well-designed observational studies will always be

necessary.

To understand the full spectrum of adverse effects — those that occur late, that werenot known beforehand, and that are rarebut nevertheless serious — and to be able

to investigate the true incidence of knownadverse effects in circumstances ofactual prescribing, well-designed observational studies will always be

necessary.Daemen et al, Lancet 2007

8

Incidence of stent thrombosis with DES

Daemen et al, Lancet 2007

NNH & NNT

Myocardial infarction, versus

control: NNH ~610

Ulcer complications, versus

naproxen: NNT ~130

Ray et al, Lancet 2002

Major cardiovascular disease 42%

Peptic ulcer or 3%gastrointestinal bleeding

Baseline Risk and NNT/NNH

0

200

400

600

800

1000

1 10 100 500

Events per 1000 PY

Num

ber

need

ed t

o ha

rm

Extrapolation to routine settings: rofecoxib & myocardial infarction

0

10

20

30

Eve

nts

per

1000

PY

x 2.24 x 2.24

Trials Routine

NNH610 NNH70

ControlRofecoxib


9

Advantages of (properlyconducted & large) randomised trials

No confounding

No fishing

A drug simply cannot be declared “safe”without measuring the balance of benefits

and risks in a randomized controlled trialover an appropriate period of time in a large population representing those who will use

the treatment in practice.

A drug simply cannot be declared “safe”without measuring the balance of benefits

and risks in a randomized controlled trialover an appropriate period of time in a large population representing those who will use

the treatment in practice.

Califf, JAMA 2005

Thank you

jüni clinical evidence · evidence evidence evidencerandomized trial meta-analysis ... tavi more...

Documents