estimation and adjustment of bias in randomised evidence using mixed treatment comparison...
TRANSCRIPT
![Page 1: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/1.jpg)
Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis
Sofia Dias, NJ Welton, AE Ades
with Valeria Marinho, Georgia Salanti, Julian Higgins
Avon RSS, May 2010
Department of Community Based Medicine
![Page 2: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/2.jpg)
2
Overview
• Motivation• Treatment networks and MTC
• Adjusting for Bias in Mixed Treatment Comparisons Meta-analysis (MTC)• The MTC model• Example: Fluoride dataset• Probability of bias model
• Results and Conclusions
![Page 3: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/3.jpg)
3
Mixed Treatment Comparisons
• Often more than two treatments for a given condition• Network of trials comparing different interventions
for a condition• Direct and indirect evidence available on treatment effects
• Because of the network structure, there is enough information to estimate and adjust for bias within the network
• For bias adjustment, there is no need to rely on exchangeability assumption between meta-analyses in different fields
![Page 4: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/4.jpg)
4
Example: The Fluoride Data
• 6 different interventions for preventing dental caries in children and adolescents
1. No Treatment
2. Placebo
3. Fluoride in Toothpaste
4. Fluoride in Rinse
5. Fluoride in Gel
6. Fluoride in Varnish
• From 6 Cochrane Reviews*
Active Treatments
*Marinho et al., 2002; 2003; 2004 (Cochrane Library)
![Page 5: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/5.jpg)
5
Network and Number of trials
Pl
NT
G
RV
T
691
31
133
1
4
9
46
31
4
1
• 130 trials • eight 3-arm trials • one 4-arm trial
• 150 pairwise comparisons
![Page 6: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/6.jpg)
6
Introduction to MTC
1. Six treatments 1,2,3,4,5,6
2. Take treatment 1 (No Treatment) as reference
3. Then the treatment effects d1k of all other treatments relative to 1 are the basic parameters
4. Given them priors:
d1,2, d1,3,…, d1,6~ N(0,1002)
![Page 7: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/7.jpg)
7
Functional parameters in MTC• The remaining contrasts are functional parameters
d2,3 = d1,3 – d1,2
d2,4 = d1,4 – d1,2
…
d4,6 = d1,6 – d1,4
d5,6 = d1,6 – d1,5
• Any information on functional parameters tells us indirectly about basic parameters
• Either FE or RE model satisfying these conditions
CONSISTENCY assumptions
1 2 3
![Page 8: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/8.jpg)
Notation• Data
i = 1,…,130 study index
k = 1, 2, 3,…,6 treatment index
rik – number of caries occurring in trial i, treatment k, during the trial follow-up period
Eik – exposure time in arm k of trial i
(in person years)
![Page 9: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/9.jpg)
9
Fluoride: Poisson MTC RE model
rate at which events occur in arm k of trial i
~ Poisson( )ik ik ikr E
Exposure time in person years
1
21, 1,~ ,
ik iik t tN d d
MTC consistency equations
21,
2
~ (0,10)
~ (0,100 ) 2,...,6
~ (0,100 ) 1,...,130
j
i
U
d N j
N i
Priors
1log( )ik i ik kI
i = 1,…,130
![Page 10: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/10.jpg)
10
MTC results: LHR relative to No Treatment
Pl
T
R
G
V
-.8 -.6 -.4 -.2 0
Residual deviance is 278.6 (270 data points)
![Page 11: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/11.jpg)
11
Posterior mean of residual deviances for each point
20 40 60 80 100 120
01
23
45
6
study number
MT
C
102
42
42
63
![Page 12: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/12.jpg)
Check how evidence is combined in the network
• Poor fit can indicate inconsistency in the network
• For each pair, separate direct evidence from indirect evidence implied by the rest of the network*
• Can see how evidence is combined in the network to give overall MTC estimate
• Helpful to locate pairs of comparisons where there may be problems
12*Dias et al., Stats in Med. 2010
![Page 13: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/13.jpg)
LHR for Placebo v Toothpaste
Bayesian p-value = 0.32
-1.0 -0.5 0.0 0.5
05
10
15
(Pl,T) is split
log-hazard ratio
De
nsi
ty
Direct
Indirect
MTC
13
![Page 14: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/14.jpg)
LHR for Placebo v Varnish
-1.0 -0.5 0.0 0.5
05
10
15
(Pl,V) is split
log-hazard ratio
De
nsi
ty
DirectIndirect
MTC
Bayesian p-value = 0.04
14
![Page 15: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/15.jpg)
LHR for Rinse v Varnish
-1.0 -0.5 0.0 0.5
05
10
15
(R,V) is split
log-hazard ratio
De
nsi
ty
DirectIndirect
MTC
Bayesian p-value = 0.02
15
![Page 16: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/16.jpg)
BIAS MODELS
But we have additional information on the risk of bias of all included studies
16
![Page 17: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/17.jpg)
17
TreatmentsNo of
studies
Allocation
concealmentBlinding
NT P T R G V adequateunclea
rinadequate Double
Single
?
1 0 1 0 0 1 0
4 1 3 0 3 1 0
3 0 3 0 1 0 2
1 0 1 0 1 0 0
3 0 2 1 0 2 1
9 0 5 4 0 6 3
4 0 3 1 0 3 1
61 8 46 7 61 0 0
25 2 20 3 22 0 3
9 0 6 3 9 0 0
3 0 3 0 3 0 0
1 0 1 0 1 0 0
1 0 0 1 0 1 0
4 0 3 1 2 2 0
1 0 1 0 0 1 0
Total 130 11 98 21 103 17 10
![Page 18: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/18.jpg)
18
MTC RE model with bias
1log( )ik i k i kiik X I
1
21, 1,~ ,
ik iik t tN d d
MTC consistency equations
21,
2
~ (0,10)
~ (0,100 ) 2,...,6
~ (0,100 ) 1,...,130
k
i
U
d N k
N i
Priors
1 if study at risk of bias
0 otherwiseiX
~ ( , )ik iN b
![Page 19: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/19.jpg)
19
MTC Bias Model
• Assume non-zero mean bias, bi = b ≠ 0, in comparisons of NT or Pl with Active treatments
• For Active-Active comparisons assume mean bias is zero
• Expect bias to increase size of treatment effect: b < 0
![Page 20: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/20.jpg)
20
Fluoride: Risk of Bias indicators• Allocation concealment
• Best empirical evidence of bias• But… 98/130 studies ‘unclear’• Only 11/130 studies ‘adequate’• Some comparisons have no adequately concealed
trials
• Blinding also available to inform risk of bias status• Used “Any bias” as a composite indicator of bias:
54/130 studies at risk of bias.
![Page 21: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/21.jpg)
21
Probability of Bias Model• Any study with unclear allocation
concealment has a probability p of being at risk of bias
• Adequately concealed trials are not at risk of bias
• Inadequately concealed trials are at risk of bias• Use only allocation concealment as bias
indicator• Bias terms identifiable in this rich network
![Page 22: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/22.jpg)
22
Probability of Bias Model
1ik i ik ik i kX I
1 if allocation concealment is inadequate
if allocation concealment is unclear
0 if allocation concealment is adequatei iX B
~ Bernoulli( ) and ~ (1,1)iB p p Beta
![Page 23: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/23.jpg)
23
Comparing Model Fit
ResDev* pD DICBetween trial heterogeneity
MTC with no bias adjustment 278.6 259.3 537.9 0.22 (0.19, 0.26)
Bias adjustment
AnyBias 277.6 257.9 535.5 0.15 (0.12, 0.18)
Probability of bias 274.6 253.0 527.6 0.12 (0.10, 0.15)
* Compare with 270 data points
![Page 24: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/24.jpg)
24
Posterior mean of residual deviances for each point: MTC and Prob of bias models
0 1 2 3 4 5 6
01
23
45
6
MTC
Pro
ba
b o
f bia
s
10242
4263
Study 42: Placebo v Toothpaste (1 of 69 trials)Allocation concealment unclearStudy 63: No Treat v Varnish (1 of 4 trials)Allocation concealment unclear and not “double blind”Study 102: Placebo v Varnish (1 of 3 trials)Allocation concealment unclear
![Page 25: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/25.jpg)
25
Treatment effects relative to No Treatment (LHR)Unadjusted MTC (solid) and Probability of Bias model (dashed)
Pl
T
R
G
V
-.8 -.6 -.4 -.2 0
![Page 26: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/26.jpg)
Varnish effects
• Cochrane Review to assess efficacy of Fluoride Varnish (Marinho et al, 2004)
• Noted that the small number and poor methodological quality of varnish trials might be overestimating the true effect of this intervention.
• The results of the bias-adjusted analysis support this hypothesis.
26
![Page 27: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/27.jpg)
27
Which treatment is best?Unadjusted MTC Bias-adjusted MTC
Probability Best (%) Rank
Probability Best (%) Rank
No Treatment 0 6 0 6
Placebo 0 5 0 5
Toothpaste 3.6 2.9 9.3 2.7
Rinse 4.1 2.8 53.8 1.6
Gel 3.7 3.2 12.4 2.9
Varnish 88.5 1.2 24.6 2.8
![Page 28: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/28.jpg)
28
Results: Probability of Bias• Bias
• posterior mean = -0.19, CrI (-0.36, -0.02)
• posterior sd = 0.40, CrI (0.29, 0.55)
• Trials with unclear allocation concealment are at risk of bias with probability p• Posterior mean of p = 0.13
• Model identified 5 trials (with unclear allocation concealment) as having a high probability of bias
![Page 29: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/29.jpg)
Prob of bias for studies with unclear allocation concealment
0 20 40 60 80 100 120
0.0
0.2
0.4
0.6
0.8
1.0
Study
Pro
po
rtio
n
1721 42 63 102 142143144145146147148149150151
o – unclear allocation concealment+ – unclear allocation concealment and single blind∆ – unclear allocation concealment and unclear blinding status
29
![Page 30: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/30.jpg)
30
Other findings
• Between trial heterogeneity in treatment effects reduced in bias-adjusted model
• Model with Active-Active bias was also fitted with similar results: Active-Active bias had posterior mean of zero• But assumptions on direction of bias…• Assumed bias would favour the newest treatment
(also the most intensive)
![Page 31: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/31.jpg)
31
Conclusions• Bias estimation and adjustment possible within
MTC because there is a degree of redundancy in the network
• Assumption that study specific biases are exchangeable within the network• Uses only internal evidence• Weaker than required from using external evidence
• Ideas extend to multiple bias indicators• But will need a very rich evidence structure
![Page 32: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/32.jpg)
32
Consequences for Decision Modelling
• Uses only internal evidence • May be more acceptable to patient groups,
pharmaceutical industry…• Risk of bias indicator chosen based on empirical
research• Results may change if different bias indicators
chosenAgain:• Assessment of model fit & sensitivity analysis
crucial if decisions based on these models are to have credence
![Page 33: Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis Sofia Dias, NJ Welton, AE Ades with Valeria Marinho,](https://reader036.vdocuments.us/reader036/viewer/2022062619/5515de16550346dd6f8b4bc7/html5/thumbnails/33.jpg)
33
References• Our website: http://bristol.ac.uk/cobm/research/mpes
• Dias S, Welton NJ, Marinho VCC, Salanti G, Higgins JPT and Ades AE (2010) Estimation and adjustment of Bias in randomised evidence using Mixed Treatment Comparison Meta-analysis. Journal of the Royal Statistical Society A, to appear Vol 173 issue 4 (available online).
• Dias S, Welton NJ, Caldwell DM and Ades AE (2010) Checking consistency in mixed treatment comparison meta-analysis. Statistics in Medicine, 29, 945-955.
• Schulz KF, Chalmers I, Hayes RJ and Altman DG (1995) Empirical Evidence of Bias. Dimensions of Methodological Quality Associated With Estimates of Treatment Effects in Controlled Trials. JAMA, 273, 408-412.