network meta-analyses in a sparse evidence bases · background •sparse networks are relatively...

How informative priors can help model heterogeneity

Network meta-analyses in a

sparse evidence bases

April 11, 2016

Background

• Sparse networks are relatively common when dealing with network meta-analyses (NMA) • For example, biologics trials can be very expensive

leading to few trials describing each comparison

• When a network is too sparse it may become infeasible to conduct random-effects (RE) NMA • If conducted, credible intervals can “explode”

• Random effects estimates not significant when the pairwise meta-analysis is

2

Psoriatic arthritis example

• As a motivating example, consider a network of biologics for the treatment of psoriatic arthritis (PsA)

• The principal outcome was the American College of Rheumatology responder criteria (ACR) 20 (20 percent improvement in tender or swollen joint counts as well as 20 percent improvement in three of the other five criteria)

• Also of interest was Psoriasis Area and Severity Index (PASI) 50 (reduction in PASI score of at least 50%)

3

placebo

Etanercept 25mg biw

ACR 20 at 12 weeks – Biologics-naïve

Ustekinumab 90mg

RAPID-PSA

ADEPT Genovese 2007

PSUMMIT1 PSUMMIT2

PSUMMIT1 PSUMMIT2

PSUMMIT1 PSUMMIT2

Mease 2004

GO-REVEAL

GO-REVEAL

GO-REVEAL

Golimumab

100mg

Golimumab

50mg

Ustekinumab 45mg

Adalimumab

40mg

Certolizumab

400 mg

Relative risk of ACR 20 at 12 weeks

5

Placebo 0.29

(0.22, 0.53)

0.40

(0.23, 1.04)

0.29

(0.20, 0.64)

0.26

(0.19, 0.52)

0.26

(0.19, 0.52)

0.49

(0.30, 1.12)

0.51

(0.30, 1.18)

3.41

(1.88, 4.64) ADA40

1.36

(0.63, 3.43)

0.99

(0.52, 2.13)

0.90

(0.48, 1.73)

0.89

(0.49, 1.78)

1.68

(0.78, 3.79)

1.74

(0.82, 3.95)

2.51

(0.96, 4.26)

0.74

(0.29, 1.60) CZP

0.73

(0.27, 1.83)

0.66

(0.25, 1.52)

0.66

(0.25, 1.49)

1.24

(0.43, 3.16)

1.28

(0.45, 3.26)

3.45

(1.56, 5.05)

1.01

(0.47, 1.92)

1.37

(0.55, 3.69) ETN25BIW

0.91

(0.42, 1.84)

0.90

(0.41, 1.81)

1.69

(0.71, 3.87)

1.75

(0.73, 4.18)

3.80

(1.92, 5.29)

1.11

(0.58, 2.08)

1.51

(0.66, 3.95)

1.10

(0.54, 2.38) GOL100

0.99

(0.62, 1.56)

1.87

(0.82, 4.35)

1.93

(0.85, 4.44)

3.84

(1.94, 5.32)

1.12

(0.56, 2.05)

1.52

(0.67, 3.93)

1.11

(0.55, 2.42)

1.01

(0.64, 1.60) GOL50

1.88

(0.85, 4.41)

1.94

(0.86, 4.45)

2.03

(0.89, 3.36)

0.59

(0.26, 1.28)

0.81

(0.32, 2.31)

0.59

(0.26, 1.42)

0.53

(0.23, 1.22)

0.53

(0.23, 1.18) UST45

1.03

(0.54, 1.99)

1.96

(0.85, 3.28)

0.58

(0.25, 1.22)

0.78

(0.31, 2.22)

0.57

(0.24, 1.36)

0.52

(0.23, 1.18)

0.51

(0.22, 1.16)

0.97

(0.50, 1.84) UST90

Pair wise MA RRs: 2.03 (1.15-3.29) and 1.97 (1.12-3.00)

PASI 50 at 12 weeks – Biologics-naïve

placebo Adalimumab

40mg ADEPT SPIRIT-P1

Infliximab 5mg/kg

IMPACT2

GO-REVEAL

GO-REVEAL GO-REVEAL

Golimumab

50mg

Golimumab

100mg

Relative risk of PASI 50 at 12 weeks

7

Placebo 0.18

(0.09, 188.34)

0.15

(0.08, 5362.44)

0.17

(0.09, 17609.73)

0.13

(0.08, 813.65)

5.51

(0.01, 11.14) ADA40

0.85

(0.00, 9871.41)

0.98

(0.00, 38643.48)

0.73

(0.00, 1793.35)

6.84

(0.00, 11.87)

1.18

(0.00, 367.10) GOL100

1.13

(0.00, 2770.24)

0.89

(0.00, 1238.09)

5.75

(0.00, 11.62)

1.02

(0.00, 322.66)

0.89

(0.00, 647.70) GOL50

0.76

(0.00, 1176.73)

7.91

(0.00, 12.32)

1.38

(0.00, 506.41)

1.12

(0.00, 12822.37)

1.32

(0.00, 42370.61) INF5

QUICK REVIEW

8

Bayesian parameter estimation

Pr(|Y) Pr(Y|) x Pr()

“Likelihood”

Suppose we wish to draw inference on , a parameter or set of parameters of interest (e.g., treatment effects), therefore

= parameter of interest

Y represents the observed data

We begin with a “prior” probability distribution for the parameters

• Typically a non-informative prior (blank slate)

• Full Bayesian framework allows for external information to inform our prior belief

“Prior”

Then use the data to determine the likelihood

• “How likely parameter values are given the observed data”

• The basis of frequentist statistics

Parameter estimation is based on the posterior distribution

• Bayesian thinking: given my prior knowledge and data likelihood, what is the probability of a parameter estimate being true

“Posterior”

NMA can be conducted in either the frequentist of Bayesian framework

• Majority of NMA are conducted in the Bayesian framework

Arm-based fixed & random effects network meta-analysis model

Random effects

Fixed effects

10

PROBLEM AND SOLUTION

11

Likelihood

Pro

bab

ilit

y

Prior

Posterior

Can result in unrealistically wide 95% credible intervals

0 1 2 3 4

Between-trial variance (2)

Posterior heterogeneity distributions

Informative priors

• If there is too little information to overcome vague, non-informative priors, what are we to do? • Informative priors can be used to integrate scientific knowledge

external to the evidence base

• Turner et al reviewed 14,886 meta-analyses to help inform

the distribution of τ when working with binary data • This information can be used to construct informative priors on

the heterogeneity variance parameter

• How it is done: • Use a log-Normal prior on the heterogeneity variance τ2 with

mean and precision based on decades of scientific work

13

Informative heterogeneity priors

10

15

Changes to the BUGS/JAGS model

• Effectively, this implies changing only two lines to most NMA code • From

• To

• Why a log-Normal distribution? • Best fitting distribution among a variety of

candidates

16

PsA example revisited

17

Placebo 0.18

(0.11, 0.33)

0.14

(0.09, 0.30)

0.17

(0.10, 0.42)

0.12

(0.08, 0.20)

5.60

(2.99, 8.98) ADA40

0.81

(0.43, 1.68)

0.97

(0.49, 2.36)

0.69

(0.38, 1.16)

6.93

(3.39, 10.71)

1.23

(0.60, 2.33) GOL100

1.19

(0.74, 2.25)

0.85

(0.44, 1.40)

5.76

(2.40, 9.58)

1.03

(0.42, 2.05)

0.84

(0.44, 1.36) GOL50

0.71

(0.31, 1.24)

8.18

(4.98, 12.00)

1.46

(0.86, 2.62)

1.17

(0.71, 2.29)

1.41

(0.80, 3.25) INF5

Placebo ADA40 CZP ETN25BIW GOL100 GOL50

UST45 2.03

(1.20, 3.05)

0.59

(0.34, 1.02)

0.81

(0.42, 1.63)

0.59

(0.33, 1.09)

0.54

(0.30, 0.96)

0.53

(0.30, 0.94)

UST90 1.96

(1.12, 2.98)

0.58

(0.32, 0.99)

0.78

(0.40, 1.58)

0.57

(0.31, 1.06)

0.52

(0.28, 0.93)

0.52

(0.28, 0.92)

Relative risk of ACR 20

Relative risk of PASI 50

Application to other data

• Dichotomous data are very popular

• A more recent study has conducted the same exercise for continuous data modeled with a Normal likelihood • It requires that the data first be transformed as

standardized mean differences • And be back-transformed after completing the NMA

• As of yet, there is no such evidence for models based on Poisson or Multinomial likelihoods

18

Placebo

LAMA LABA+LAMA

ICS+LABA+LAMA

LAMA+PDE-4

LABA PDE-4 LABA+PDE-4

ICS+LABA

ICS 3 trials

Pair wise MA RR: 0.84 (0.71-0.97)

Network MA RR: 0.85 (0.66-1.03)

Example - COPD

A B

C

A B

C

Scenario #1 Scenario #2

I2=25%

I2=35% I2=45%

I2=5%

I2=35% I2=65%

How else might this be used?

• Most NMA assumes the degree of heterogeneity is equal in each comparison

k=3

k=4 k=5

k=12

k=12 k=12

• If we relax this assumption, we may have to borrow strength estimation power from somewhere else

Conclusions

• Sparse networks and heterogeneity are two distinct concepts

• Informative priors can be used to ensure the correct model is used (i.e., random-effects) when it is otherwise infeasible

• Use of informative heterogeneity priors are becoming widely endorsed (used within NICE, CADTH, and academic experts)

• There are limitations to the use of informative priors: • It can only be used with Binomial and Normal likelihoods • It can only be used when all treatment comparisons fall

within the same category

21

22

Thank you!

E-mail

steve.kanters@precisionhealtheconomics.com

www.PHEconomics.com