1 changing trial designs on the fly janet wittes statistics collaborative asa/fda/industry workshop...
Post on 27-Mar-2015
213 Views
Preview:
TRANSCRIPT
1
Changing Trial Designs on the Fly
Janet WittesStatistics Collaborative
ASA/FDA/Industry Workshop September 2003
2
Context
Trial that is hard to redo• Serious aspect of serious disease• Orphan
3
Statistical rules limiting changes
To preserve the Type I error rate
To protect study from technical problems arising from operational meddling
4
Challenge
senserigor
5
6
Challenge
senselessrigor mortis
7
Scale of rigor
Over rigid Rigorous Prespecified methods for change – preserves Unprespecified but reasonable change Invalid analysis
• responders analysis• outcome-outcome analysis• completers
8
Consequences
No change during the study
OR
Potential for the perception that change caused by effect
9
Prespecified changes
Sequential analysis Stochastic curtailing Futility analysis Internal pilot studies Adaptive designs Two-stage designs
10
Problems
Technical Solved
Operational Risks accepted
Efficiency Understood
11
Add a DMC
What if it acts inconsistently with guidelines?
Something really unexpected happens?• DMC initiates change• Steering Committee initiates change
12
Reasons for unanticipated changes
Unexpected high-risk group Changed standard of care Statistical method defective Too few endpoints Assumptions of trial incorrect Other
13
Examples
1. Too much censoring; DMC extends trial
2. Boundary not crossed but DMC stops3. Unexpected adverse event4. Statistical method defective5. Event rate too low; DMC changes
design
14
#1 Endpoint-driven trial
Trial designed to stop after 200 deaths Observations different from expected
• Recruitment• Mortality rate
At 200 deaths, fu of many people<2 mo DMC: change fu to minimum 6 mo P-value: 0.20 planned; 0.017 at end
15
#2. Boundary not crossed
Endpoint• Primary: 7 day MI• Secondary: one-year mortality
Very stringent boundary
16
What DMC sees
Very strong result at 7 days No problem at 1 year Clear excess of serious adverse events
17
Haybittle-Peto bound (10%)
0
1
2
3
4
5
6
7
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
BoundsObserved
18
Haybittle-Peto bound (30%)
0
1
2
3
4
5
6
7
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
BoundsObserved
19
Haybittle-Peto bound (50%)
0
1
2
3
4
5
6
7
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
BoundsObserved
20
Haybittle-Peto bound (70%)
0
1
2
3
4
5
6
7
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
BoundsObserved
21
Haybittle-Peto bound (70%)
0
1
2
3
4
5
6
7
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
BoundsObservedOB
22
#3. Unexpected adverse event: PERT study of the WHI
Prespecified boundaries forBenefit HarmHeart attack StrokeFracture PEColon cancer Breast cancer
23
Observations
Benefit Harm----- Stroke
Fracture PEColon cancer Breast cancer
Heart attack
24
Actions
Informed the women about increased risk of stroke, heart attack, and PE
Informed them again Stopped the study
25
#4. Statistical method defective
Neurological disease 20 question instrument Anticipated about 20% would not come Planned multiple imputation- results:
• Scale: 0 to 80• Value for ID 001: 30 38 ? 42 28 ?• MI values: -22, 176
26
#5. Too few endpoints
Example: approved drug Off-label use associated with AE
• Literature: SOC event rate: 20 percent
Non-inferiority design - = 5 Sample size: 800/group
27
Observation
400 people randomized 0 events What does the DMC do?
28
Choices
Continue to recruit 1600 Stop and declare no excess Choose some sample size Tell the Steering Committee to
choose a sample size What if n=1? 2? 5? 10?
29
Conclusions
Ensure that DMC understands role Separate decision-making role of
DMC and Steering Committee Distinguish between reasonable
changes on the fly and “cheating” Expect fuzzy borders
30
Technical
Changing plans can increase Type I error rate• We need to adjust for multiple looks• How do we adjust for changes?
31
Operational
Unblind assessments
Subtle change in procedures
In clinical trials, the FDA and SEC
top related