catalyst data monitoring prediction · 2010. 9. 13. · – remember most dsmb members are not...
TRANSCRIPT
![Page 1: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/1.jpg)
Data Monitoring of Clinical Trials Using Prediction
Scott EvansLingling Li, Hajime Uno, LJ Wei
East Campus, Reisman Lecture HallFeldberg/Reisman Complex
Beth Israel Deaconess Medical Center
April 14, 2010
![Page 2: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/2.jpg)
Special Thanks• Professor Jim Ware
• Kim Jackson
• Our hosts at BIDMC
• Colleagues– Sachiko Miyahara– Tzu-min Yeh– Huichao Chen– Jeanne Jiang– Lijuan Deng– Laura Smeaton
![Page 3: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/3.jpg)
Outline
• Setting
• Motivation
• Predicted Intervals
• Predicted Interval Plots (PIPs)
• Extensions
![Page 4: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/4.jpg)
Setting• Interim data from clinical trials are
monitored
• Treatment effects are estimated
• DSMBs (or DMCs) meet to review interim results and make recommendations regarding future trial conduct
![Page 5: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/5.jpg)
Practical Questions
• Should the trial be stopped?– For efficacy?– For futility?
• Are there non-efficacious arms that should be dropped?
• Should sample size be re-calculated?– Due to a lack of precision in estimating a parameter during trial
design (e.g., variability, control group response)
• Should the duration of follow-up be modified due to unexpected event rates?– E.g., with an event-time endpoint?
![Page 6: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/6.jpg)
Motivation• Answering these questions has:
– Ethical attractiveness• Fewer participants generally exposed to inefficacious and
potentially harmful therapies
– Economical advantages• Smaller expected sample sizes and shorter expected duration
than designs without interim analyses– Saving time, money, and other resources
– Public health advantages• Answers may get to the medical community more quickly
![Page 7: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/7.jpg)
Interim Analyses Methods
• Group sequential methods – Control α spending– E.g., Slud and Wei, O’Brien-Fleming, Lan-
DeMets, Pocock– Generally boundary driven with test statistics
• Conditional power/futility index
![Page 8: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/8.jpg)
Limitations of Many Traditional Methods
• Do not
– Provide estimates of effect or associated precision (only test statistics, p-values, and decision rules)
– Evaluate “clinical relevance”• Statistical significance is not the only consideration
– Information regarding the reasons for:• High p-values (or test statistics):
– Negligible effect vs. insufficient data vs. too much variation• Low p-values (or test statistics):
– Clinical significance?
![Page 9: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/9.jpg)
Limitations of Many Traditional Methods
• Inflexible with binding decision rules based usually on a single (primary) endpoint
– Desire to base decisions upon simultaneous assessment of many factors, such as:
• Safety data • Secondary endpoints• Quality of life• Benefit:risk assessment• Results of other trials• Scientific relevance• Availability of new alternative therapies• Cost:benefit considerations
![Page 10: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/10.jpg)
Repeated Confidence Intervals (RCIs)
• Sequential CIs– Simultaneous coverage control
• Uses principles of group sequential methods
• Provides estimates of effects sizes
• Allows for flexibility in decision making
• Jennison & Turnbull, Controlled Clinical Trials, 1984.• Mehta et.al., Statistics in Medicine, 2000.
![Page 11: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/11.jpg)
Repeated Confidence Intervals
In fa
vor o
f Mon
oIn
favo
r of C
omb
99.95%CI
98.6%CI95.5%CI
![Page 12: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/12.jpg)
Limitations of Repeated CIs
• At the interim, we wish to weigh the options of stopping vs. continuing
• Repeated CIs do not: – Provide formal evaluation of the ramifications of
continuing• What effect size estimates and associated precision will be
observed at the end of the trial? At the next interim?
• Thus how do we weigh the options?
![Page 13: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/13.jpg)
Need for Methods that:• Control error rates
• Are flexible to allow for expert DSMB judgment – Allow incorporation of other information into decision
• Provide effect size estimates and associated precision– Assess clinical relevance and statistical significance
• Provide information about decision alternatives
![Page 14: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/14.jpg)
Predicted Intervals
• Predict CI at future timepoint (e.g., end of trial or next interim analysis time) conditional upon:
1. Observed data2. Assumptions regarding future data (e.g., observed
trend continues, HA is true, H0 is true, best/worst case scenarios, etc.)
• Evans SR, Li L, Wei LJ, Drug Information Journal, 41:733-742, 2007.
![Page 15: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/15.jpg)
NARC 009• Randomized, double-blind, placebo-controlled,
multicenter, dose-ranging study of prosaptide (PRO) for the treatment of HIV-associated neuropathic pain
• Participants were randomized to 2, 4, 8, 16 mg/d PRO or placebo administered via subcutaneous injection
• Primary endpoint:– 6 week change from baseline in weekly average of random daily
Gracely pain scale prompts using an electronic diary
![Page 16: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/16.jpg)
NARC 009
• Designed for 390 participants equally allocated between groups
• Interim analysis conducted after 167 participants completed the 6-week double-blind treatment period
• Computed PIs
![Page 17: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/17.jpg)
Treatment N 95% CI forMean Change
95% CI for Diff1
95% PI forDiff2
95% PI forDiff3
Required Diff4
Placebo 31 (-0.35, -0.11)
2 mg 34 (-0.21, -0.04) (-0.04, 0.25) (-0.01, 0.21) (-0.16, 0.06) -0.54
4 mg 34 (-0.38, -0.12) (-0.19, 0.16) (-0.14, 0.10) (-0.23, 0.01) -0.45
8 mg 32 (-0.18, -0.02) (-0.01, 0.28) (0.03, 0.23) (-0.15, 0.05) -0.56
16 mg 36 (-0.34, -0.09) (-0.16, 0.19) (-0.11, 0.14) (-0.21, 0.04) -0.54
1: 95% CI for the difference in mean changes vs. placebo2: 95% PI for the difference in mean changes vs. placebo assuming full enrollment, assuming current trend3: 95% PI for the difference in mean changes vs. placebo assuming full enrollment, assuming per protocol, μplacebo = -
0.17 and μdrug = -0.344: Difference in mean changes needed in the remaining participants for the CI for the difference in mean changes to
exclude zero (in favor of active treatment) at the end of the trial
Interim Analysis Results: NARC 009
![Page 18: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/18.jpg)
NARC 009
• Sensitivity analyses shows that the futility assessment is robust
• Trial was discontinued by NARC DSMB for futility– Evans et. al., PLoS ONE, 2007.
![Page 19: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/19.jpg)
Predicted Intervals
• Intuitive
• Practical– Provides information on effect size and associated
precision (clinical relevance)– Invariant to study design (superiority vs. noninferiority)– Use with RCI theory to control error rates and allow
flexibility in decision process– Can use for:
• Binary, continuous, or time-to-event endpoints• 1 and 2 sample problems
![Page 20: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/20.jpg)
Issues
• Sensitivity analyses necessary to assess the robustness of results to varying assumptions– Strategic assessment of assumptions to be employed
• Need to assess impact of sampling variability
• Need concise and intuitive summaries for DSMBs– Too much information is not digestible
• Graphics are helpful– Too little information is not informative– Remember most DSMB members are not statisticians
![Page 21: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/21.jpg)
Predicted Interval Plots (PIPs)
• Evaluates the sampling variability associated with the assumed model using simulation
• Plots the simulated PIs under the model assumption
• Conditional power is readily available
• Li L, Evans SR, Uno H, Wei LJ (Statistics in Biopharmaceutical Research)
![Page 22: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/22.jpg)
PIP Construction
• Impose parametric assumptions for the unobserved data– Estimate or specify the values of the unknown parameters under
reasonable and strategic assumptions
• Simulate future data
• Combine the observed data with the simulated data
• Construct PIs using standard methods
• Repeat to obtain many simulated PIs
![Page 23: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/23.jpg)
PIPs: Construction Schema
Simulate future outcome
A simulated completedataset at the end of the trial or later interim
Calculate the “final” result
Observed dataset at interim point
+ Assumption about future data
Adjusted PIDSMB
![Page 24: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/24.jpg)
PIPs: Construction Schema
Simulate future outcome
A simulated completedataset at the end of the trial or later interim
Calculate the “final” result
Observed dataset at interim point
+ Assumption about future data
Adjusted PIDSMB
Repeat
![Page 25: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/25.jpg)
PIPs: Construction Schema
Simulate future outcome
A simulated completedataset at the end of the trial or later interim
Calculate the “final” result
Observed dataset at interim point
Adjusted PI
+ Assumption about future data
DSMB
Simulate many timesand get many adjusted PIs
![Page 26: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/26.jpg)
Minocycline for Cognitive Impairment in HIV
• Design– Randomized, blinded, placebo-controlled single
site study– Primary endpoint: 24 week change in
composite of standardized neuropsychological testing battery
– N=100
• DSMB reviewed results after ~40% information
![Page 27: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/27.jpg)
24-Week Change in NP Summary
TotalTotalTreatment ArmTreatment Arm
pp--valuevalueAA BB
U NP Sum ChangeU NP Sum Change 0.5930.593NN 4141 2020 2121# missing# missing 2424 1313 1111Mean (SD)Mean (SD) 0.41 (0.80)0.41 (0.80) 0.34 (0.84)0.34 (0.84) 0.49 (0.78)0.49 (0.78)Min, MaxMin, Max --1.43, 2.061.43, 2.06 --1.43, 2.061.43, 2.06 --1.00, 1.661.00, 1.66MedianMedian 0.580.58 0.600.60 0.520.52Q1, Q3Q1, Q3 --0.17, 0.840.17, 0.84 --0.30, 0.810.30, 0.81 0.17, 0.930.17, 0.93
![Page 28: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/28.jpg)
![Page 29: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/29.jpg)
![Page 30: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/30.jpg)
![Page 31: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/31.jpg)
Predicted Intervals Summary
AssumptionAssumption CurrentCurrent95% CI95% CI
Width ofWidth ofCurrent Current 95% CI95% CI
Median (Q1, Q3)Median (Q1, Q3)Width of Width of
PredictedPredicted95% CI95% CI
ProbabilityProbabilityto reject H0to reject H0
in favorin favorof Aof A
ProbabilityProbabilityto reject H0to reject H0
in favorin favorof Bof B
ObservedObservedTrendTrend --0.67, 0.380.67, 0.38 1.051.05 0.63 (0.61, 0.66)0.63 (0.61, 0.66) 0%0% 8%8%
NullNullHypothesisHypothesis --0.67, 0.380.67, 0.38 1.051.05 0.68 (0.65, 0.71)0.68 (0.65, 0.71) 0%0% 2%2%
AlternativeAlternativeHypothesisHypothesis --0.67, 0.380.67, 0.38 1.051.05 0.67 (0.64, 0.70)0.67 (0.64, 0.70) 19%19% 0%0%
![Page 32: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/32.jpg)
0-0.5 0.5
1234
567
89
10
Clinically Inferior
Clinically Superior Clinically Equivalent
Treatment B is preferred Treatment A is preferred
![Page 33: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/33.jpg)
0-0.5 0.5
1234
56
89
10
Clinically Inferior
Clinically Superior Clinically Equivalent
Treatment B is preferred Treatment A is preferred
7
![Page 34: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/34.jpg)
0-0.5 0.5
1234
567
89
10
Clinically Inferior
Clinically Superior Clinically Equivalent
Treatment B is preferred Treatment A is preferred
![Page 35: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/35.jpg)
Scenario Under Observed
Under H0
Under Ha
1 0% 0% 0%
2 31.6% 23% 0.2%
3 7.8% 2% 0%
4 0% 0% 0%
5 0.6% 4.6% 56.2%
6 60% 70.4% 24.8%
7 0% 0% 0%
8 0% 0% 18.8%
9 0% 0% 0%
10 0% 0% 0%
12
3
4
567
89
10
![Page 36: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/36.jpg)
Summary
• Futility analyses suggest low probability of a positive trial/superiority result with respect to primary endpoint
• DSMB evaluated if relevant effects could reasonably be ruled out
• DSMB evaluated if other reasons to continue (e.g., secondary endpoints)
• DSMB recommended early termination of the study
![Page 37: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/37.jpg)
![Page 38: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/38.jpg)
![Page 39: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/39.jpg)
Primary Comparisons and Power Considerations
Superiority
A total of 1700 primary events in the two treatment arms attains 85.9% power for detecting HR=0.85.
Non-inferiority
1850 primary events in these two treatment arms will provide 88.1% power if valsartan is actually 2.5% better than captopril
The total events = ½ (1700 + 1850 +1850) = 2700
Captopril 50 mg tidValsartan 160 mg bid
Captopril 50 mg tid + Valsartan 80 mg bid Captopril 50 mg tid
Captopril 50 mg tidValsartan 160 mg bid
![Page 40: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/40.jpg)
Example: Large CV Trial
• Design– Superiority trial comparing combination
therapy to monotherapy– Primary endpoint is time-to-CV-event– Sample size based on observing 2700 events– Interim planned after 900 and 1800 observed
events
![Page 41: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/41.jpg)
Final Result: VALIANT Study
![Page 42: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/42.jpg)
Final Result: VALIANT Study
Comparison Combo vs Captopril
Valsartan vs Captopril
Hazard Ratio 0.98 1.0
97.5%CI 0.89 to 1.09 0.90 to 1.11
Superiority Test P=0.73 P=0.98
Non-inferiority Test
NA P=0.004
![Page 43: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/43.jpg)
Example
Could these results been obtained more efficiently?We focus on the comparison of
Combo group vs. Mono-therapy groups
Captopril 50 mg tid
Valsartan 160 mg bidCaptopril 50 mg tid + Valsartan 80 mg bid
![Page 44: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/44.jpg)
Final Analysis Result (Comb vs Mono)
Time after Randomization
Sur
viva
l
CombMono
6M 12M 18M 24M
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
6M Survival [95%CI]
Comb : 90.6 [ 89.8 , 91.4 ]
Mono : 90.8 [ 90.2 , 91.4 ]
( Comb - Mono ): -0.2 [ -1.3 , 0.9 ]
12M Survival [95%CI]
Comb : 87.7 [ 86.7 , 88.6 ]
Mono : 87.1 [ 86.4 , 87.8 ]
( Comb - Mono ): 0.5 [ -0.8 , 1.8 ]
18M Survival [95%CI]
Comb : 84.4 [ 83.4 , 85.4 ]
Mono : 84.2 [ 83.5 , 84.9 ]
( Comb - Mono ): 0.2 [ -1.3 , 1.7 ]
Observed Events 6M 12M 18M 24MCombMono
1311 1708 2141 2508908 1396 1694 1910
No. at risk 6M 12M 18M 24MCombMono
4415 4265 4000 26488895 8513 8036 5283
The final HR [95%CI]0.975 [0.902, 1.054]
![Page 45: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/45.jpg)
Group Sequential Testing: O’Brien-Fleming
Observed test statistics
![Page 46: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/46.jpg)
Conditional Power
With 8% of chance, the significant result will be observed at the final.
Observed HR
?P<0.05
![Page 47: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/47.jpg)
Limitation of Conditional Power• Why is there a low probability (8%) of significance?
– No effect?– Not enough events?– Both?
• Conditional Power does not distinguish these
• Need more information
![Page 48: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/48.jpg)
Each Patient’s Entry and Follow-upwith calendar time
900 events11/6/2000
1800 events7/19/2001
Total events: 2,878Total Survivors: 11,825Total: 14,703
![Page 49: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/49.jpg)
Hazard Ratio
Perc
entil
e of
Poi
nt E
stim
ate
Dis
tribu
tion
0.9 1.0 1.1
525
5075
95Predicted Interval Plot
18M after the 2nd interim analysis (4Y from the start) Assumed HR = 0.975.
Current Interval[0.88, 1.08]
Length:0.20
In favor of Mono therapyIn favor of Comb
![Page 50: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/50.jpg)
Hazard Ratio
Perc
entil
e of
Poi
nt E
stim
ate
Dis
tribu
tion
0.9 1.0 1.1
525
5075
95Predicted Interval Plot
18M after the 2nd interim analysis (4Y from the start) Assumed HR = 0.975.
The final interval0.98 [0.90, 1.05]
Length: 0.15
Current Interval[0.88, 1.08]
Length:0.20
In favor of Mono therapyIn favor of Comb
![Page 51: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/51.jpg)
Sensitivity Analyses
• Perform sensitivity analyses to evaluate the effect of the uncertainty of the model assumption by creating PIPs for various assumed models– E.g., observed trend is true, HA is true, H0 is
true, optimistic/pessimistic case scenarios, etc.– Strategically chosen
![Page 52: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/52.jpg)
Hazard Ratio
Perc
entil
e of
Poi
nt E
stim
ate
Dis
tribu
tion
0.9 1.0
525
7595
Predicted Interval PlotHR = 0.85 (original alternative hypothesized value)
In favor of Mono therapyIn favor of Comb
![Page 53: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/53.jpg)
6M or 12M after the DSMB(Assumed HR = 0.975)
From DSM B From Start 95% CI Length
As of DSM B 2.5Y O bserved [0.88, 1.08] 0.20
6M later 3.0Y PI (m edian) [0.89, 1.07] 0.18
12M later 3.5Y PI (m edian) [0.90, 1.06] 0.16
18M later 4.0Y PI (m edian) [0.91, 1.06] 0.15
18M later 4.0Y (END) O bserved [0.90, 1.05] 0.15
![Page 54: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/54.jpg)
ACTG A5175• Phase IV, randomized, open-label, three-arm trial designed
to evaluate three ARV regimens for treatment-naïve HIV+ participants– One regimen contained two nucleoside reverse transcriptase
inhibitors (NRTIs) + an HIV-1 protease inhibitor (PI) and two regimens each containing two NRTIs + a non-nucleoside reverse transcriptase inhibitor (NNRTI)
– Primary endpoint = time to first of (virologic failure, new AIDS-defining OI, death)
• PIPs used to show that under reasonable assumptions, precision would not be appreciably increased with reasonable additional follow-up– Additional follow-up would not alter qualitative interpretation of
the trial
![Page 55: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/55.jpg)
Extensions• Bayesian analog (pending)
– Posterior probability of hypotheses– Side-by-side analyses to aid DSMBs in decision-
making– R2Winbugs
• Development Program Decisions– Using Phase II data to predict Phase III and make
go/no-go decisions
• Use of PIPs in Design
![Page 56: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/56.jpg)
Example: ACTG 5263
• A Randomized Comparison of Three Chemotherapy Regimens as an Adjunct to Antiretroviral Therapy for Treatment of Advanced AIDS-KS in Resource-Limited Settings
• Multinational
![Page 57: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/57.jpg)
ACTG 5263
• Liposomal doxorubicin = active control– expensive
• 2 primary comparisons vs. cheaper therapies– Etoposide + ART vs. liposomal doxorubicin + ART– BV + ART vs. liposomal doxorubicin + ART
• Primary endpoint– KS progression or death by 48 weeks (binary)
![Page 58: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/58.jpg)
ACTG 5263• Desire to make noninferiority claims
• Principles of defining the NI margin– Statistical
• Retain some effect of active control over placebo– Conceptual
• Maximum treatment difference that is clinically irrelevant• Largest treatment difference that is acceptable in order to gain other advantages of the
experimental intervention
• Difficult to define a single NI margin for settings with diverse resources
• Study sizes based on estimation and precision of the estimate (i.e., width of a CI) rather than hypothesis testing
– But desire to claim superiority or noninferiority will likely remain
![Page 59: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/59.jpg)
Type I error 0.048
Median of CI width 0.199
Non-inferiority Margin Power
0.1 0.524
0.12 0.681
0.15 0.866
Power 0.508
Median of CI width 0.197
Non-inferiority Margin Prop. of claiming non-inferiority
0.1 0.019
0.12 0.055
0.15 0.166
![Page 60: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/60.jpg)
Software
• Frontier Science (Beta version 0.01)– R interface– Menu driven with user setting parameters and
selecting options
• CBAR
• Grants
![Page 61: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/61.jpg)
![Page 62: Catalyst Data Monitoring Prediction · 2010. 9. 13. · – Remember most DSMB members are not statisticians. Predicted Interval Plots (PIPs) • Evaluates the sampling variability](https://reader034.vdocuments.us/reader034/viewer/2022052010/60204bdd567dfd683d2ecda7/html5/thumbnails/62.jpg)
Predicted Interval Plot
Kernel-smoothing is used to obtain the estimated density. The mode is identified as the value with the highest estimated density.