Comparison of effect sizes associated with surrogate and final primary endpoints in
randomised clinical trials
1
Ciani O., Garside R., Pavey T., Stein K., Taylor R.S.
Background
Classic Definition for surrogatesDisease-centered characteristics Patient-centered characteristics
BiomarkersA characteristic that is objectively measured and evaluated as an indicator of normal, pathogenic or pharmacologic responses to a therapeutic intervention.
Final outcomeA characteristic
that reflects how patient feels, functions or
survives.
Surrogate outcomes
A biomarker that is intended to substitute and predict for a final
outcome.
e.g. LDL-cholesterolCardiovascular
Mortality
e.g. Intraocular pressure Loss of vision
Background
HTA-based Definition of surrogatesDisease-centered characteristics Patient-centered characteristics
BiomarkersA characteristic that is objectively measured and evaluated as an indicator of normal, pathogenic or pharmacologic responses to a therapeutic intervention.
Final outcomeA characteristic
that reflects how patient feels, functions or
survives.
Surrogate outcomes
A biomarker - or clinical or patient-relevant outcome - that is
intended to substitute and predict for a final
outcome, namely survival or HRQoL.
e.g. Rate of hip fracture Mortality/HRQoL
e.g. Event-free Survival Overall Survival
4
Objectives of the study
I. To study the association between primary endpoint (surrogate vs final) and treatment effect estimates in RCTs
II.To compare the risk of bias in trials reporting a surrogate endpoint vs trials reporting a final primary endpoint
5
Initial sample of abstracts (N = 639) Initial sample of abstracts (N = 639)
Excluded (N = 55)Not RCTs (N = 17)Economic evaluation studies (N = 11)Non interventional treatment (N = 25) Secondary analysis (N = 2)
Excluded (N = 55)Not RCTs (N = 17)Economic evaluation studies (N = 11)Non interventional treatment (N = 25) Secondary analysis (N = 2)
For outcomes classification (N = 584) For outcomes classification (N = 584)
Composite mixed outcomes (N = 73)
Composite mixed outcomes (N = 73)
Eligible for the study (N = 511) Eligible for the study (N = 511)
Surrogate outcomes based (N = 137)Surrogate outcomes based (N = 137) Final outcomes based (N = 137) Final outcomes based (N = 137)
Matching procedureMatching procedure
Final outcome trials (N = 101)Binary endpoint (N = 83)
Final outcome trials (N = 101)Binary endpoint (N = 83)
Excluded (N = 53)Equivalence/Non-inferiority study (N = 15) Unpooled Muliti-arm (N = 33)No analysable data (N = 5)
Excluded (N = 53)Equivalence/Non-inferiority study (N = 15) Unpooled Muliti-arm (N = 33)No analysable data (N = 5)
Surrogate outcome trials (N = 84)Binary endpoint (N = 51)
Surrogate outcome trials (N = 84)Binary endpoint (N = 51)
Excluded (N = 36)Composite mixed outcomes (N = 9)Early termination (N = 1)Equivalence/Non-inferiority study (N = 11)Unpooled Muliti-arm (N = 11)No analysable data (N = 4)
Excluded (N = 36)Composite mixed outcomes (N = 9)Early termination (N = 1)Equivalence/Non-inferiority study (N = 11)Unpooled Muliti-arm (N = 11)No analysable data (N = 4)
Methods
Study selection
6
Methods
Data extraction
Effect Size Binary endpoints: n/N data for each arm Continuous endpoints: SS, Mean, SD for each arm TEs(95%CI) as reported by authors
Study characteristics: sample size, follow-up, type of intervention, patient population, sponsor (i.e. FP, NFP and mixed), positive outcome in favour of the new treatment
Risk of bias: adoption of the intention to treat (ITT) principle, adequate randomized sequence generation and allocation concealment, double-blind/placebo-control
Surrogate outcomes: type of surrogate (i.e. imaging, histo/biochemical, instrumental, other), authors’ statement about validation and use of a substitute outcome
7
Methods
Data analyses
Primary Analysis Random-effects meta-analysis
Binary endpoints: TEs expressed as ORs Meta-regression models
Binary endpoints: Ratio of ORs (95%CI) ROR > 1 → greater TEs of the surrogate endpoints Adjustment for key trial characteristics
Sensitivity Analyses Pooled Relative Risk Ratios estimate (RRR) Combined continuous and binary endpoints ROR estimation Within-pair comparison of differences in ln(OR)
Secondary Analysis Logistic regression model
OR of reporting result in favour of the new treatment
Risk of bias assessment χ2 - test of methodological quality dimensions across groups
8
Results
Study characteristicsCharacteristics
Surrogate outcomes (N = 84)
Final outcomes (N = 101)
P-value
Intervention, N(%) 0.33
Pharmaceuticals 49 (58) 61 (60)
Medical Devices 7 (8) 7 (7)
Surgical procedures 4 (5) 8 (8)
Health promotion activities 7 (8) 2 (2)
Other therapeutic technologies 17 (20) 23 (23)
Sponsor, N(%) 0.86
Profit 24 (29) 28 (28)
Not-for-Profit 49 (58) 57 (56)
Mixed 11 (12) 16 (16)
Sample size, Median (IQR) 371 (162-787) 741 (300-4731) <0.001
Follow up, [days] Median (IQR) 255 (137-540) 180 (40-730) 0.73*Chi-square test, Fisher exact test, Mann-Whitney U test
9
Results
Comparison of TEs – primary analysis
Method of Analysis
(Nr of Surrogate trials vs. Nr of
Final Outcome trials)
Surrogate
outcome Trials
OR (95% CI)
Final outcome
Trials
OR (95% CI)
ROR (95% CI)Adjusted^
ROR (95% CI)
Primary analysis
Binary outcomes
(51 vs. 83)
0.51
(0.42 to 0.60)
0.76
(0.70 to 0.82)
1.47
(1.07 to 2.01)
1.46
(1.05 to 2.04)
ORs = Odds ratios pooled using DerSimonian & Laird random effects meta-analyses. ROR: Relative Odds Ratio; ^Adjusted for trial-level characteristics of clinical area of intervention, patient population, type of intervention, sponsor, journal, mean sample size and mean follow up time
10
Results
Comparison of TEs – sensitivity analyses Method of Analysis
(Nr Surrogate trials vs.
Nr Final Outcome trials)
Surrogate Trials
RR (95% CI)
Final Trials
RR (95% CI)
ROR or RRR
(95% CI)
Adjusted^
ROR or RRR
(95% CI)
Inclusion of risk ratios as
reported by authors
(57 vs. 86)
0.56
(0.48 to 0.65)
0.80
(0.75 to 0.86)
1.38
(1.12 to 1.71)
1.36
(1.08 to 1.70)
Inclusion of continuous
outcomes
(84 vs. 101)
0.46
(0.39 to 0.54)
0.68
(0.62 to 0.74)
1.44
(0.83 to 2.49)
1.48
(0.83 to 2.62)
Binary outcomes
matched-pairs
(43 vs. 43)
0.48
(0.39 to 0.59)
0.68
(0.61 to 0.77)
1.38
(1.01 to 1.88)-
RRR: Relative Risk Ratio; ^Adjusted for trial-level characteristics of clinical area of intervention, patient population, type of intervention, sponsor, journal, mean sample size and mean follow up time
11
Results
Risk of bias
Risk of Bias Assessment, N(%)Surrogate
outcomes (N=84)Final outcomes
(N=101)P-value
ITT adoption 62 (74) 83 (82) 0.17
Adequate Randomization sequence generation 54 (64) 65 (64) 0.99
Adequate Randomization allocation concealment 61 (73) 74 (73) 0.92
Double Blinding/Placebo control 42 (50) 43 (43) 0.31
*Chi-square test
12
Between-trial comparison of treatment effects
Possible role of smaller trial sample size in surrogate outcome trials
~40% ‘overestimation’ of TEs in surrogate outcomes trials
Consistent result across sensitivity analyses, confirmed by secondary analyses
Findings not explained by methodological quality or other key trial characteristics
Discussion and limitations
14
Main References1. Biomarkers Definitions Working Group. Biomarkers and surrogate endpoints:
preferred definitions and conceptual framework. Clinical Pharmacology and Therapeutics 2001; 69: 89–95.
2. Bucher, H. C. et al. Users' guides to the medical literature: XIX. Applying clinical trial results. A. How to use an article measuring the effect of an intervention on surrogate end points. Evidence-Based Medicine Working Group. JAMA, 1999: 282, 771-8.
3. Fleming TR, DeMets DL. Surrogate endpoints in clinical trials: Are we being misled? Annals of Internal Medicine 1996; 125: 605–13.
4. Lassere M. The Biomarker-Surrogacy Evaluation Schema: a review of the biomarker-surrogate literature and a proposal for a criterion-based, quantitative, multidimensional hierarchical levels of evidence schema for evaluating the status of biomarkers as surrogate endpoints .Statistical Methods in Medical Research 2007; 17: 303–340.
5. Taylor RS, Elston J. The use of surrogate outcomes in model-based cost-effectiveness analyses: a survey of UK Health Technology Assessment reports. Health Technol Assess 2009; 13(8).
6. Weir CJ, Walley RJ. Statistical evaluation of biomarkers as surrogate endpoints: a literature review. Stat Med 2006; 25: 183-203.