introduction to meta-analysis - aureliotobias.com · introduction to meta-analysis aurelio tobías...
TRANSCRIPT
Introduction to Meta-analysis
Aurelio Tobí[email protected]
Aims of the session
• At the end of this session you should be able to understand1) The basic principles for meta-analysis of interventions
2) How to do meta-analysis in practice using OpenMetaAnalyst
Introduction
• Meta-analysis is “… the statistical analysis of a large collection of analysis results from individual studies for the purpose of integrating the findings” (Glass 1976)
• A meta-analysis is basically a way of calculating an average effect
Where to use meta-analysis?
• Systematic reviews– Identification and evaluation of all the available scientific
evidence– Meta-analysis may be appropriate to combine the
estimates of studies included in the systematic review
Systematic review
• PICO question• Literature search• Critical appraisal • Meta-analysis• Reporting results and
conclusions (PRISMA)
Systematic review
Systematicreview
Meta-analysis
• Statistical methods (meta-analysis) can be used or not to summarize the results of the included studies included in a systematic review
Where to use meta-analysis?
• Systematic reviews– Identification and evaluation of all the available scientific
evidence– Meta-analysis may be appropriate to combine the
estimates of studies included in the systematic review• Multi-centre studies – Each participating centre analyses its own individual data
under the same (or not) protocol– Meta-analysis may be appropriate to combine the
estimates reported from each study-centre
Why do a meta-analysis?
• Objectives– To increase statistical power and precision– To assess consistency of results– To answer questions not comprised by the individual
studies • Views– Synthetic view summarizes study results– Analytic view identifies differences between study results
Usual meta-analysis ‘slang’
Some tips
• All studies included in the meta-analysis must answer the same research question
• Only include studies with the same design• Same effect size and measure of variability for all
studies included in the meta-analysis• Meta-analysis techniques allow to combine effect
sizes in additive (symmetric) scale• Other measures of frequency of the disease
(incidence, prevalence) or effect sizes (correlation) can also be used
Data to be collected
• Effect sizei) Binary outcomes
– Risk Difference (RD)– Risk Ratio (RR)– Odds Ratio (OR)
ii) Continuous outcomes– Mean difference (D)
• Variabilitya) From confidence interval
– 95% CI: (lcli to ucli)– SE: si = (ucli – lcli)/2�1.96– Variance: si
2 = si�si
b) to variance– Variance: si
2
– SE: si = √si2
– 95% CI: yi� 1.96�si
Row data to be collected
• Binary outcome– 2�2 table
– Effect size in additive (symmetric) scale• RD = (a/(a+b)) − (c/(c+d))
– Effect sizes in multiplicative (asymmetric) scale• RR = (a/(a+b)) / (c/(c+d))• OR= (a/b) / (c/d)
Outcome (y)Treat. (x) Yes NoYes a bNo c d
Row data to be collected
• Binary outcome– 2�2 table
– Effect size in additive scale and its variance• DR with s2
DR = (ab/(a+b)3)-(cd/(c+d)3)– Effect sizes to additive scale and their variances
• log(RR) with s2log(RR) = (1/a)-(1/(a+b))+(1/c)-(1/(c+d))
• log(OR) with s2log(OR) = (1/a)+(1/b)+(1/c)+(1/d)
Outcome (y)Treat. (x) Yes NoYes a bNo c d
Row data to be collected
• Continuous outcome– Table of means
– Mean difference and its variance• D = m1 – m2 with s2
D = ((n1-1)s12+(n2-1)s2
2)/(n1+n2-2) – Standardised mean difference and its variance
• d = (m1 – m2)/s with s2d = (n1+n2)/(n1n2)+d2/2(n1+n2-2)
with s = √((n1-1)s12+(n2-1)s2
2)/(n1+n2-2)
Outcome (y)Treat. (x) n mean (sd)Yes n1 m1 (s1)No n2 m2 (s2)
Fixed effects model
Favours to treatment Favours to control
Common fixed effect (q)
yi yi = q
Fixed effects model
Favours to treatment Favours to control
(ei)
Common fixed effect (q)
yi yi = q + ei
assumes ei ~ N(0, si2)
Fixed effects model• Homogeneity of effects must be assumed
H0: y1 = y2 = … = yk
• Meta-analysis pooled estimate
– Weights defined as, wi = 1/si2
– Inverse variance weight method (Cochran 1937)
!!!!
€
⌢ θ = wiyi∑ / wi∑ !!
€
with!s⌢ θ 2 =1/ wi∑
º
∑
�
1/si2
Borenstein et al. "Introduction to Meta-analysis” (Capítulo 19, página 151)
Heterogeneity• Testing for heterogeneity
– H0: y1 = y2 = … = yk
– The test has low power when there are few studies, but high when there are many, so the p-value is difficult to interpret
!!!!
€
Q = wi yi −⌢ θ ( )
2
∑ ~ χk−12
• Quantifying heterogeneity– Based on Q’s
– Proportion of total variability explained by heterogeneity
– Cut-off values of 25%, 50% and 75% might be considered as low, moderate, high and very high heterogeneity, respectively (Higgins and Thompson 2003)
!!
€
I2 =Q −(k −1)
Q×100%
Borenstein et al. "Introduction to Meta-analysis” (Capítulo 19, página 151)
Borenstein et al. "Introduction to Meta-analysis” (Capítulo 19, página 151)
Random effects models
Favours to treatment A favor del control
Study-specific effect (qi)
yi yi = µ + di + ei
assumes di~N(0, t2) ei~N(0, si
2)
t2Random effects distribution
(ei)
1/(si2+t2)
∑
X
Borenstein et al. "Introduction to Meta-analysis” (Capítulo 19, página 151)
Random effects model• Assumes that the true effect in each study is
randomly distributed across studies with a fixed mean and variance (t2)
• Meta-analysis pooled estimate
– Weights defined as, wi = 1/(si2+t2)
– DerSimonian & Laird’s method (1986)
!!!!
€
⌢ µ = wi
*yi∑ / wi*∑ !!
€
with!s⌢ µ 2 =1/ wi
*∑
Fixed vs. random effects model
Fixed effects• Interpretation
– All studies share an identical true treatment effectxxxxxxxxxxxxxxxx
– The meta-analysis estimates this effect
• Strength– Easy to understand and simple
to carry out• Limitation
– Homogeneity of effects must be assumed
Random effects• Interpretation
– The true treatment effect in each study is randomly distributed across studies
– The meta-analysis estimates an average of these effects
• Strength– It is not limited by a restrictive
assumption • Limitation
– Does not answer why results between studies are different?
What is heterogeneity?
• Heterogeneity is the variation in the true effect which may be shown in more observed variation than expected by chance
• Individual studies are conducted in different places, times, populations and conditions leading to different between-study estimates
• Heterogeneity should not be ignored, must be explained
Causes of heterogeneity
1) Study characteristics– Variations in the study
designs– Development of studies– Attrition
• Methodological (or statistical) heterogeneity due to bias
2) Population characteristics− Type of participants− Temporal/geographical
settings− Treatment (or exposures) and
outcome measures
• Clinical heterogeneity due to biological diversity
Other data to be collected
• Study design characteristics– Method of randomization, blinding, type of comparison
and analysis, loss to follow up, etc. • Population characteristics– Conditions under investigation, eligibility criteria,
geographical, information, etc. • Details of treatment (or exposure)– Dose, duration, type of treatment (or exposure), type of
control, etc.
Subgroup meta-analysis
• Synthetic vs. analytic views - Stratified meta-analysis by study/population characteristics
- Clear definitions of subgroups is essential
- Identify homogeneity between studies within each subgroup, but heterogeneity between subgroups
Analytic
Effect modifier (xi)
Synthetic
Effe
ct si
ze (y
i)
1.83/1.49=1.23
Borenstein et al. "Introduction to Meta-analysis” (Capítulo 19, página 151)
Limitations
• Small number of studies (low statistical power)• Observational relationship between studies
(confounding bias)• Definition of sub-groups and use of aggregated
covariates (information bias)• Too many sources of methodological and clinical
heterogeneity (interpretation bias)
Summary
• Simple statistical basis for meta-analysis– Homogeneity of effects assumption– Weighted mean
• In case of heterogeneity between studies– The fixed effects model is clearly inappropriate … but the
random effects models is inappropriate too – Heterogeneity should not be ignored, must be explained.
But avoid over-interpretation of findings
Further readings
• Borestein M, Hedges LV, Higgins JPT, Rothstein HR. Introduction to Meta-analysis (mainly Chapters 4, 5, 11, 12, 13, 16, 19 and 20). Wiley: Chichester, 2009
• Egger M, Davey-Smith G, Altman D (Eds.). Systematic Reviews in Health Care: Meta-Analysis in Context 2nd Edition. BMJ Books, 2001
• Higgins JPT, Green S (Eds.). Cochrane Handbook for Systematic Reviews of Interventions. The Cochrane Collaboration, 2011 [http://handbook-5-1.cochrane.org]
• http://www.cebm.brown.edu/openmeta/
OpenMetaAnalyst
OpenMetaAnalyst
OpenMetaAnalyst
Meta-analysis Sub-groupMeta-analysis
Import dataOpen dataset
Hands on!
• gastric – Meta-analysis of 13 randomized clinical trials to evaluate the effectiveness of the use of adjuvant chemotherapy in gastric cancer remission.
• biomtx – Meta-analysis of 20 randomized clinical trials to evaluate the effectiveness of the use of biological agents for patients with rheumatoid arthritis.
• depression – Meta-analysis of 10 clinical randomized clinical trials to evaluate the effectiveness of exercise as an intervention for the treatment of depression.