1 changing trial designs on the fly janet wittes statistics collaborative asa/fda/industry workshop...
TRANSCRIPT
1
Changing Trial Designs on the Fly
Janet WittesStatistics Collaborative
ASA/FDA/Industry Workshop September 2003
2
Context
Trial that is hard to redo• Serious aspect of serious disease• Orphan
3
Statistical rules limiting changes
To preserve the Type I error rate
To protect study from technical problems arising from operational meddling
4
Challenge
senserigor
5
6
Challenge
senselessrigor mortis
7
Scale of rigor
Over rigid Rigorous Prespecified methods for change – preserves Unprespecified but reasonable change Invalid analysis
• responders analysis• outcome-outcome analysis• completers
8
Consequences
No change during the study
OR
Potential for the perception that change caused by effect
9
Prespecified changes
Sequential analysis Stochastic curtailing Futility analysis Internal pilot studies Adaptive designs Two-stage designs
10
Problems
Technical Solved
Operational Risks accepted
Efficiency Understood
11
Add a DMC
What if it acts inconsistently with guidelines?
Something really unexpected happens?• DMC initiates change• Steering Committee initiates change
12
Reasons for unanticipated changes
Unexpected high-risk group Changed standard of care Statistical method defective Too few endpoints Assumptions of trial incorrect Other
13
Examples
1. Too much censoring; DMC extends trial
2. Boundary not crossed but DMC stops3. Unexpected adverse event4. Statistical method defective5. Event rate too low; DMC changes
design
14
#1 Endpoint-driven trial
Trial designed to stop after 200 deaths Observations different from expected
• Recruitment• Mortality rate
At 200 deaths, fu of many people<2 mo DMC: change fu to minimum 6 mo P-value: 0.20 planned; 0.017 at end
15
#2. Boundary not crossed
Endpoint• Primary: 7 day MI• Secondary: one-year mortality
Very stringent boundary
16
What DMC sees
Very strong result at 7 days No problem at 1 year Clear excess of serious adverse events
17
Haybittle-Peto bound (10%)
0
1
2
3
4
5
6
7
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
BoundsObserved
18
Haybittle-Peto bound (30%)
0
1
2
3
4
5
6
7
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
BoundsObserved
19
Haybittle-Peto bound (50%)
0
1
2
3
4
5
6
7
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
BoundsObserved
20
Haybittle-Peto bound (70%)
0
1
2
3
4
5
6
7
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
BoundsObserved
21
Haybittle-Peto bound (70%)
0
1
2
3
4
5
6
7
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
BoundsObservedOB
22
#3. Unexpected adverse event: PERT study of the WHI
Prespecified boundaries forBenefit HarmHeart attack StrokeFracture PEColon cancer Breast cancer
23
Observations
Benefit Harm----- Stroke
Fracture PEColon cancer Breast cancer
Heart attack
24
Actions
Informed the women about increased risk of stroke, heart attack, and PE
Informed them again Stopped the study
25
#4. Statistical method defective
Neurological disease 20 question instrument Anticipated about 20% would not come Planned multiple imputation- results:
• Scale: 0 to 80• Value for ID 001: 30 38 ? 42 28 ?• MI values: -22, 176
26
#5. Too few endpoints
Example: approved drug Off-label use associated with AE
• Literature: SOC event rate: 20 percent
Non-inferiority design - = 5 Sample size: 800/group
27
Observation
400 people randomized 0 events What does the DMC do?
28
Choices
Continue to recruit 1600 Stop and declare no excess Choose some sample size Tell the Steering Committee to
choose a sample size What if n=1? 2? 5? 10?
29
Conclusions
Ensure that DMC understands role Separate decision-making role of
DMC and Steering Committee Distinguish between reasonable
changes on the fly and “cheating” Expect fuzzy borders
30
Technical
Changing plans can increase Type I error rate• We need to adjust for multiple looks• How do we adjust for changes?
31
Operational
Unblind assessments
Subtle change in procedures
In clinical trials, the FDA and SEC