JSM 2008JSM 2008 11

Comparison of Individual Patient-Level and Study-Level Meta-Analyses Using Time to Event Analysis in Drug-Eluting Stent Data

Hsini Liao, Yun Lu, and Hong WangPresented to JSM 2008

Boston Scientific Corporation

JSM 2008JSM 2008 22

Conflict of Interest DisclosuresConflict of Interest Disclosures

DISCLOSURE INFORMATION:DISCLOSURE INFORMATION:The following relationships exist relatedThe following relationships exist relatedto this presentation:to this presentation:

Hsini Liao, Yun Lu, Hong WangHsini Liao, Yun Lu, Hong WangFull time employees of Full time employees of Boston Scientific CorporationBoston Scientific Corporation

JSM 2008JSM 2008 33

OutlinesOutlines

•• MotivationMotivation•• MetaMeta--AnalysisAnalysis•• Time to Event AnalysisTime to Event Analysis•• StentStent DataData•• ReferencesReferences

JSM 2008JSM 2008 44

MotivationMotivation

•• MetaMeta--analysis provides a structure of consolidating analysis provides a structure of consolidating the outcomes from several studies and deriving the outcomes from several studies and deriving statistical inferences of the outcomes. statistical inferences of the outcomes.

•• MetaMeta--analysis of timeanalysis of time--toto--event data is less common event data is less common than of binary or continuous data.than of binary or continuous data.

•• Aggregate Data (AD) Aggregate Data (AD) –– TwoTwo--Step ApproachStep Approach•• Individual Patient Data (IPD) Individual Patient Data (IPD) –– OneOne--Step Approach Step Approach •• Comparison of the analysis of AD and IPD should be Comparison of the analysis of AD and IPD should be

found discrepant results, but no clear general found discrepant results, but no clear general systematic differences, especially when homogeneity systematic differences, especially when homogeneity is assumed.is assumed.

•• AD AD vsvs IPD in TTE DES DataIPD in TTE DES Data

JSM 2008JSM 2008 55

MetaMeta--AnalysisAnalysis

•• A Systematic Review of Literature to Measure the A Systematic Review of Literature to Measure the Effect SizeEffect Size

•• Single Study/EffectSingle Study/Effect•• Many Studies/Narrative ReviewMany Studies/Narrative Review•• Effect Magnitude/Adequate PrecisionEffect Magnitude/Adequate Precision•• Combine the Effects to Give Overall Mean EffectCombine the Effects to Give Overall Mean Effect•• A Recent Survey in Practice (A Recent Survey in Practice (SimmondsSimmonds et al, et al,

2005): Majority used simple fixed2005): Majority used simple fixed--effect model; effect model; only small proportion considered amongonly small proportion considered among--study study heterogeneity heterogeneity

JSM 2008JSM 2008 66

MetaMeta--AnalysisAnalysis(AD (AD vsvs IPD)IPD)

•• Effect Size: Event Rate, Effect Size: Event Rate, OR, RR, HR, OR, RR, HR, CorrCorr, etc., etc.

•• Sample Size/Standard Sample Size/Standard Error to Assign WeightError to Assign Weight

•• Limit Analyses (e.g. Limit Analyses (e.g. Fixed/Random Effect, Fixed/Random Effect, MetaMeta--Regression)Regression)

•• Less Time and CostsLess Time and Costs

•• Ensuring Data Quality Ensuring Data Quality (e.g. Date of Outcome)(e.g. Date of Outcome)

•• Detailed Data Checking Detailed Data Checking (e.g. Randomization)(e.g. Randomization)

•• Ensuring the Ensuring the Appropriateness of the Appropriateness of the Analyses (e.g. KM)Analyses (e.g. KM)

•• More Time and costsMore Time and costs

JSM 2008JSM 2008 77

Fixed Effect Model Fixed Effect Model (AD: FEM)(AD: FEM)

•• The FEM assumes that all studies in the The FEM assumes that all studies in the metameta--analysis are drawn from a common analysis are drawn from a common population.population.

•• The observed effect size varies from one The observed effect size varies from one study to the next only because of the study to the next only because of the random error inherent in each study.random error inherent in each study.

•• Under the FEM there is one true effect Under the FEM there is one true effect size. The combined effect is an estimate size. The combined effect is an estimate of this value.of this value.

JSM 2008JSM 2008 88

FEM (ContFEM (Cont’’d)d)

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2

εA

εB

εC

TA

TB

TC

Study A

Study B

Study C

FEM with sampling error (real world)

e.g. TA= 0.6 – 0.1 = 0.5

JSM 2008JSM 2008 99

FEM (ContFEM (Cont’’d)d)

•• More generally, the observed effect More generally, the observed effect TTjjfor any study is given by the for any study is given by the population mean plus the sampling population mean plus the sampling error in that study. That is,error in that study. That is,

Tj = µ + εjεj~N(0,σ2)

j = 1,…,k

JSM 2008JSM 2008 1010

Random Effects ModelRandom Effects Model(AD: REM)(AD: REM)•• The REM assumes that the studies are drawn from The REM assumes that the studies are drawn from

populations that differ from each other in ways that populations that differ from each other in ways that could impact on the treatment effect. could impact on the treatment effect.

•• The observed effect size varies from one study to The observed effect size varies from one study to the next for two reasons. The first is random error the next for two reasons. The first is random error within studies, as in the FEM. The second is true within studies, as in the FEM. The second is true variation in effect size from one study to the next. variation in effect size from one study to the next.

•• Under the REM there is not one true effect size, but Under the REM there is not one true effect size, but a distribution of effect sizes. The combined effect is a distribution of effect sizes. The combined effect is not an estimate of one value, but is meant to be not an estimate of one value, but is meant to be the mean of a distribution of values. the mean of a distribution of values.

JSM 2008JSM 2008 1111

REM (ContREM (Cont’’d)d)

0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 1.10 1.20

TA

µAεA

Study A

ξA

REM, true effect and observed effect in one study (real world)

e.g. TA= 0.60 – 0.05 – 0.15 = 0.40

µ

JSM 2008JSM 2008 1212

REM (ContREM (Cont’’d)d)

•• More generally, the observed effect More generally, the observed effect TTjj for any study for any study is given by the grand mean is given by the grand mean µµ, the deviation of this , the deviation of this studystudy’’s true effect from the grand mean, and the s true effect from the grand mean, and the deviation of this studydeviation of this study’’s observed effect from this s observed effect from this studystudy’’s true effect.s true effect.

Tj = µ + ξj + εjξj ~ N(0,τ2); εj~N(0,σj

2)

j = 1,…,k

JSM 2008JSM 2008 1313

HeterogeneityHeterogeneity

•• Using FEM/REM is based on assumption.Using FEM/REM is based on assumption.•• FEM FEM

•• Ignore heterogeneityIgnore heterogeneity•• Removing outliers causes biasness.Removing outliers causes biasness.

•• REMREM•• Q Statistic is defined as total sum of squares.Q Statistic is defined as total sum of squares.•• II22 Statistics to describe the ratio of true/total varianceStatistics to describe the ratio of true/total variance•• Subgroup/Sensitivity Analysis, ANOVA, MetaSubgroup/Sensitivity Analysis, ANOVA, Meta--

RegressionRegression

JSM 2008JSM 2008 1414

Heterogeneity (ContHeterogeneity (Cont’’d)d)

∑=

•−=k

iii TTwQ

1

2)(∑

∑∑

=

=

=

−= k

ii

k

iiik

iii

w

TwTwQ

1

2

1

1

2)(

⎪⎩

⎪⎨⎧

≤>−

=dfQifdfQif

CdfQ

,,

02τ

• Total Sums of Squares:

• Heterogeneity (Between-Studies Variance):

• Test of Heterogeneity ( )

21~ −kQ χ

kH θθθ === ...: 210

JSM 2008JSM 2008 1515

Time to Event AnalysisTime to Event Analysis(AD (AD vsvs IPD)IPD)•• HR derived from HR derived from

published data.published data.•• Always compare 2 Always compare 2

groupsgroups•• Estimate HR from KMEstimate HR from KM•• Not able to adjust for Not able to adjust for

covariates, but able to covariates, but able to select HR being select HR being adjusted for same adjusted for same covariates.covariates.

•• Model time to eventModel time to event•• Account for censoringAccount for censoring•• Compare survival curve Compare survival curve

between 2+ groupsbetween 2+ groups•• Assess relationship Assess relationship

between survival time and between survival time and covariates.covariates.

•• UnivariateUnivariate Method: Method: KaplanKaplan--Meier (KM) CurvesMeier (KM) Curves

•• Multivariate Method: CoxMultivariate Method: Cox--Proportional Hazards Proportional Hazards ModelModel

h(t) = λ(t)*exp(Xiβi) i = 1,…,k

JSM 2008JSM 2008 1616

StentStent DataData

•• Study Outcome: Target Vessel RevascularizationStudy Outcome: Target Vessel Revascularization•• Treatment: DrugTreatment: Drug--Eluting Eluting StentStent vs. Barevs. Bare--Metal Metal StentStent•• Data: total 825 diabetic randomized patients over 5 Data: total 825 diabetic randomized patients over 5

studiesstudies•• Study A: (n=11) 5 yearsStudy A: (n=11) 5 years•• Study B: (n=51) 5 yearsStudy B: (n=51) 5 years•• Study C: (n=318) 5 years Study C: (n=318) 5 years •• Study D: (n=356) 3 yearsStudy D: (n=356) 3 years•• Study E: (n=89) 5 yearsStudy E: (n=89) 5 years

•• The hazard ratio estimate of study outcomes from The hazard ratio estimate of study outcomes from pooled IPD is compared with that from the AD to pooled IPD is compared with that from the AD to assess the treatment effect in diabetic patients. assess the treatment effect in diabetic patients.

JSM 2008JSM 2008 1717

Study Outcome: TVRStudy Outcome: TVR

In-segment

In-stent5 mmdistaledgeCovers entire stented length

5 mm

proximaledge

JSM 2008JSM 2008 1818

AngioplastyAngioplasty

A B

DC

JSM 2008JSM 2008 1919

KM Estimate for TVR (IPD)KM Estimate for TVR (IPD)

JSM 2008JSM 2008 2020

HR for Pooled IPDHR for Pooled IPD

Treatment Effect without Adjustment

JSM 2008JSM 2008 2121

Study name Statistics for each study Hazard ratio and 95% CI

Hazard Lower Upper Relative ratio limit limit p-Value weight

Study A 1.172 0.106 12.978 0.897 1.00Study B 0.768 0.357 1.651 0.499 9.88Study C 0.689 0.488 0.971 0.034 49.01Study D 0.818 0.538 1.242 0.345 33.09Study E 0.553 0.223 1.372 0.202 7.02

0.729 0.573 0.928 0.010

0.1 0.2 0.5 1 2 5 10

DES Better BMS Better

Meta-Analysis of TVR for Diabetic Patients (AD)

Fixed Effect Model

JSM 2008JSM 2008 2222

Test of HeterogeneityTest of Heterogeneity

0.0000.0000.9230.923440.9140.914

II--squaredsquaredPP--valuevalueDF (Q)DF (Q)QQ--valuevalue

Heterogeneity (AD)Heterogeneity (AD)

JSM 2008JSM 2008 2323

AD AD vsvs IPD in DES DataIPD in DES Data

Treatment Effect without AdjustmentTreatment Effect without Adjustment

.1387.1387

.1370.1370

StdErrStdErr

[.4581, .7891][.4581, .7891]--.5087.5087.6013.6013ADAD[.4592, .7857][.4592, .7857]--.5098.5098.6006.6006IPDIPD

95% CI95% CILog(HRLog(HR))Hazard Hazard RatioRatio

JSM 2008JSM 2008 2424

AD AD vsvs IPDIPD

No by Model AssumptionNo by Model AssumptionNo by Test of No by Test of HeterogeneityHeterogeneity

Heterogeneity Heterogeneity FoundFound

Cox Regression Model Cox Regression Model with Fixed Treatment with Fixed Treatment EffectEffect

Fixed/Random Effects Fixed/Random Effects ModelModel

Model UsedModel Used

KaplanKaplan--MeierMeierForest Plot, Sensitivity Forest Plot, Sensitivity AnalysisAnalysis

Visual Visual Presentation Presentation for TTE Datafor TTE Data

Relatively LargeRelatively LargeAlways SmallAlways SmallSample SizeSample Size

PatientPatient--LevelLevelStudyStudy--LevelLevel

JSM 2008JSM 2008 2525

AD AD vsvs IPD (ContIPD (Cont’’d)d)

Should check Should check poolabilitypoolabilitybetween study and between study and study.study.

Overall may not be Overall may not be consistent with each consistent with each individual study.individual study.

Individual Individual vsvsOverallOverall

Subgroups are able to be Subgroups are able to be freely defined. The freely defined. The power for each subgroup power for each subgroup can be calculated.can be calculated.

Subgroup of studies is Subgroup of studies is available. Outcome for available. Outcome for subgroup of baseline subgroup of baseline may not be published.may not be published.

Subgroup Subgroup AnalysisAnalysis

PatientPatient--LevelLevelStudyStudy--LevelLevel

JSM 2008JSM 2008 2626

Other TopicsOther Topics

•• Extracting information from survival Extracting information from survival curvecurve

•• Published with insufficient detailsPublished with insufficient details

JSM 2008JSM 2008 2727

ReferencesReferences

•• Riley RD, Lambert PC, Riley RD, Lambert PC, StaessenStaessen JA, Wang J, JA, Wang J, GueyffierGueyffier F, F, ThijsThijs L, L, BoutitieBoutitie, , F. F. ““MetaMeta--Analysis of Continuous Outcomes Combining Individual Patient Analysis of Continuous Outcomes Combining Individual Patient Data and Aggregate DataData and Aggregate Data””, Stat in Med. 2008; 27:1870, Stat in Med. 2008; 27:1870--18931893

•• SimmondsSimmonds MC, Higgins JPT, Stewart LA, Tierney JF, Clarke MJ, MC, Higgins JPT, Stewart LA, Tierney JF, Clarke MJ, Thompson SG. Thompson SG. ““MetaMeta--Analysis of Individual Patient Data from Analysis of Individual Patient Data from Randomized Trials: A Review of Methods Used in PracticeRandomized Trials: A Review of Methods Used in Practice””, , ClinClin Tri. Tri. 2005; 2:2092005; 2:209--217217

•• Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. ““Introduction to Introduction to MetaMeta--AnalysisAnalysis”” (Draft), 2007(Draft), 2007

•• Sutton AJ, Abrams KR, Jones DR, Sheldon TA, Song F. Sutton AJ, Abrams KR, Jones DR, Sheldon TA, Song F. ““Methods for Methods for MetaMeta--Analysis in Medical ResearchAnalysis in Medical Research”” (Reprint with Correction), 2004(Reprint with Correction), 2004

•• ParmarParmar MKB, MKB, TorriTorri V, Stewart L. V, Stewart L. ““Extracting Summary Statistics to Extracting Summary Statistics to Perform MetaPerform Meta--Analyses to the Published Literature for Survival Analyses to the Published Literature for Survival EndpointsEndpoints””, Stat in Med. 1998; 17:2815, Stat in Med. 1998; 17:2815--28342834

•• Smith CT, Williamson PR, Smith CT, Williamson PR, MarsonMarson AG. AG. ““Investigating Heterogeneity in an Investigating Heterogeneity in an Individual Patient Data MetaIndividual Patient Data Meta--Analysis of Time to Event OutcomesAnalysis of Time to Event Outcomes””, Stat , Stat in Med. 2005; 24:1307in Med. 2005; 24:1307--13191319

•• Software: Software: ““Comprehensive MetaComprehensive Meta--AnalysisAnalysis”” (CMA), Version 2.2.030(CMA), Version 2.2.030•• Software: Software: ““SASSAS””, Version 9.1, Version 9.1