department of biostatistics & informatics colorado school...

4.4 GeneralCharacteristics(a) Boundary structure

(b) Design characteristics

(c) Case study:ROCKET-AF

*Background

*Fixed sample design

*Group sequential designevaluation

Bios 6649- pg 1

Bios 6649:Clinical Trials - Statistical Design and Monitoring

Spring Semester 2015

John M. KittelsonDepartment of Biostatistics & Informatics

Colorado School of Public HealthUniversity of Colorado Denver

c©2015 John M. Kittelson, PhD




*Background



Bios 6649- pg 2

Case Study : Design of ROCKET-AF trial

Background

I Atrial fibrillation is type of cardiac arrhythmia that causesblood to pool in the atrium.

I Pooled blood can clot; the clot can produce a stroke.I People with atrial fibrillation are given anticoagulation

treatment for prevention of clots

I All anticoagulation therapy can cause bleeding(sometimes major bleeding that results in death).




*Background



Bios 6649- pg 3


Background (cont.)

I Standard AF therapy uses Warfarin (Coumadin) foranticoabulation

I It can be difficult to get the correct Warfarin dose:I Doses that are too low increases risk of clots.I Doses that are too high increases risk of major bleeding.I All warfarin patients have their anticoagulation level (their

“INR") measured regularly to titrate the dose.

I New non-vitamin K antagonist oral anticoagulants(including rivaroxaban) have been developed.

I It is hypothesized that rivaroxaban will be as effective atpreventing stroke and will be easier to manage :

I without increased bleeding risk.I without measuring the anticoagulation level.




*Background



Bios 6649- pg 4


Definition of Treatment

I 20mg or 15mg (with poor kidney function) once daily

I Patients randomized to rivaroxaban or dose-adjustedWarfarin to target INR level of 2.0-3.0

I Treatment was double-blinded using a blinded INR valuewith centrally-adjustment of warfarin (or matching placebo)dose

I Primary conclusions based upon per-protocol instead ofintention-to-treat analyses




*Background



Bios 6649- pg 5


Primary endpoint(s)

I Primary efficacy endpoint (mostly ischemic/clottingevents):

* Stroke* Systemic ischemic event.* Secondary efficacy:

- Death (any cause)- Myocardial infarction

I Primary safety endpoint (major bleeding):* Intracranial hemorrhage* Drop in hemoglobin or transfusion* Gastrointestinal bleed* Any major bleed




*Background



Bios 6649- pg 6


Refinement of the primary endpoint

Primary endpoint: Comparison of hazards for event (censoredcontinuous data)

I Duration of followupI Accrual planned for about 30 months.I Minimum of 12-months follow-up planned for a total trial

duration of 3.5 years.

I Measures of treatment effect (comparison across groups)I Hazard ratio (Cox estimate; implicitly weighted over time):θ1 (rivaroxaban) and θ0 (Warfarin) = hazard of primaryoutcome (systemic ischemic event):

θ =θ1

θ0

I No adjustment for covariatesI Statistical information dictated by number of events (under

proportional hazards, statistical information is approximatelyD/4)




*Background



Bios 6649- pg 7


Definition of statistical hypotheses (non-inferiority design)

Design hypotheses

I H0 : θ ≥ 1.46 (Clinically important inferiority)I H+ : θ ≤ 1.0 (Benefit)

I A-priori estimated risk of primary outcome:I 2.3% per 100 person-years with WarfarinI 3.36% per 100 person-years with rivaroxaban if θ = 1.46.

I Needed to estimate number of patients to enroll

I FDA guidance on non-inferiority trials to be discussed:I in section 6 of course.I By project group 5.




*Background



Bios 6649- pg 8


Criteria for statistical evidence

I Type I error: Probability of falsely rejecting the nullhypothesisStandard:

I One-sided hypothesis test: 0.025

I Power: Probability of correctly rejecting the null hypothesis(1-type II error)

I 80% power (popular choice)I 95% power (used for ROCKET-AF trial)

I Sample size: (D = number of events)

D =

(1.96 + 1.645

log(1.46) − log(1)

)2

× 4 = 363




*Background



Bios 6649- pg 9


Determination of sample size

I Using D = 363 events

I Critical value = elog(1.46)−1.96√

4/363 = 1.1885

I 95% CI at critical value:(elog(1.46)−3.92

√4/363, 1.46

)= (0.967, 1.46).

I Sponsor decides to increase sample size to D = 405in order to “ensure a robust statistical result;" thus:

I Critical value = elog(1.46)−1.96√

4/405 = 1.202

I 95% CI at critical value:(elog(1.46)−3.92

√4/405, 1.46

)= (0.989, 1.46).

I Note: this may help to account for anticipated14% annual dropout rate.




*Background



Bios 6649- pg 10


Specification of fixed sample design using RCTdesign

I Definition of original design (with 363 events)

> survFixed <- seqDesign( prob.model = "hazard", arms = 2,+ null.hypothesis = 1.46, alt.hypothesis ="calculate",+ ratio = c(1., 1.), nbr.analyses = 1, test.type = "less",+ power = 0.95, alpha = 0.025,sample.size=363)> survFixedCall:seqDesign(prob.model = "hazard", arms = 2, null.hypothesis = 1.46,

alt.hypothesis = "calculate", ratio = c(1, 1), nbr.analyses = 1,sample.size = 363, test.type = "less", power = 0.95, alpha = 0.025)

PROBABILITY MODEL and HYPOTHESES:Theta is hazard ratio (Treatment : Comparison)One-sided hypothesis test of a lesser alternative:

Null hypothesis : Theta >= 1.46 (size = 0.025)Alternative hypothesis : Theta <= 1.00 (power = 0.950)(Fixed sample test)

STOPPING BOUNDARIES: Sample Mean scaleEfficacy Futility

Time 1 (NEv= 363) 1.1885 1.1885




*Background



Bios 6649- pg 11


Determination of sample size (cont.)

I Interpretation (inference at the boundary):

> seqInference(survFixed)Ordering *** a Boundary *** *** d Boundary ***

Time 1 Boundary 1.19 1.19MLE 1.19 1.19BAM 1.19 1.19RBadj 1.19 1.19

Mean MUE 1.189 1.189Mean P-value 0.025 0.025Mean 95\% Conf Int (0.967, 1.46) (0.967, 1.46)Time MUE 1.189 1.189Time P-value 0.025 0.025Time 95\% Conf Int (0.967, 1.46) (0.967, 1.46)

LR MUE 1.189 1.189LR P-value 0.025 0.025LR 95\% Conf Int (0.967, 1.46) (0.967, 1.46)




*Background



Bios 6649- pg 12



I Interpretation:

I How many subjects would se need to accrue to expect tohave 363 events?

I Expected number of events determined by assuming

I Exponential survival in placebo group with rate of 0.023events per 100 person-years

I Uniform accrual of patients over 2.5 yearsI Negligible dropout (we will revisit this assumption)




*Background



Bios 6649- pg 13



I In general, it necessary to know the expected number ofpatients required to obtain the desired operatingcharacteristics

I This is given by:

N =D

π0 Pr0[Event] + π1 Pr1[Event]

where D is the total number of required events and πi isthe proportion of patients allocated to group i




*Background



Bios 6649- pg 14



I Under proportional hazards, Pr[Event] for each groupdepends upon

1. The total followup (TL) and accrual (TA) time

2. The underlying survival distribution

3. The accrual distribution

4. Drop-out




*Background



Bios 6649- pg 15



I From the above, if we assume a uniform accrual patternwe have:

Pr[Event] =

∫ TA

0Pr[Event & Entry at t ]dt

=

∫ TA

0Pr[Event | Entry at t ] Pr[Entry at t ]dt

= 1−∫ TA

0Pr[No Event | Entry at t ] Pr[Entry at t ]dt

= 1− 1TA

∫ TA

0Pr[No Event | Entry at t ]dt (unif acc)

= 1− 1TA

∫ TA

0S(TL − t)dt




*Background



Bios 6649- pg 16


Specification of fixed sample design using RCTdesign

I In RCTdesign this is automated assuming exponentialsurvival using the function seqPHSubjects()

I For the ROCKET-AF trial we assumed

I Exponential survival in placebo group with rate of 0.023events per 100 person-years

I Uniform accrual of patients over 2.5 years

> seqPHSubjects( survFixed, lambda0=0.023, accrualTime=2.5, followupTime=1 )

accrualTime followupTime rate hazardRatio lambda0 nSubjects1 2.5 1 2364.8 1.46 0.023 5912.02 2.5 1 2886.5 1.00 0.023 7216.2




*Background



Bios 6649- pg 17



I Interpretation:

I In order to desire the required number of patients we wouldneed to accrue:

I N=2365 patients per year for 2.5 years if the null hypothesiswere true (Total of 5912 patients)

I N=2886 patients per year for 2.5 years if the alternativehypothesis were true (Total of 7216 patients)




*Background



Bios 6649- pg 18



I Notice the following aspects of this design:

I Hypotheses:I Non-inferiority bound represents nearly a doubling of the

hazard of the primary event.

I Sample size:I With 363 events and 95% power, the inference at the

boundary includes some unimportant benefit.95% CI: (0.967, 1.46)

I Approximately 5912 to 7216 patients will give 363 events.I Sponsor decides to recruit 14000 patients as explained in

protocol section 12.1.




*Background



Bios 6649- pg 19

Case Study : Design or ROAKCT-AF trial

Considerations in choosing a group sequential design

Principles in guiding initial choice of stopping rule

I Consider early stopping for 2 reasons:I Stop for inferiority (cannot rule out important harm:θ > 1.46)

I Stop for non-inferiority (rule out important harm)

I Early conservatismI Long-term benefit of high importanceI Early stopping precludes observation of long-term safety

dataI Potential safety concerns:

* Ability to stop for ‘ futility’ ≡ ability to stop for inferiority* Be careful not to continue with significant harm

I Number and timing of interim analysesI Trade-off between power and sample sizeI Determined by information accrual (events) but ultimately

scheduled on calendar time




*Background



Bios 6649- pg 20

ROAKCT-AF trial Interim analysis plan

From the trial protocol (synopsis):

Interim Analysis“An interim review of efficacy and safety data will beperformed when 50% of the primary efficacy events asreported by the investigators have occurred to assess theoption of stopping early for futility. The study may bestopped early due to lack of efficacy if it is unlikely toestablish non-inferiority on the primary efficacy endpoint ifthe study were to run to completion. There is no intentionto stop the study early to declare non-inferiority orsuperiority. However, in the presence of overwhelmingsuperiority of rivaroxaban versus warfarin, a conservativeHaybittle-Peto boundary will be used as a stoppingguidance, requiring no adjustments to the final analysis.."




*Background



Bios 6649- pg 21


Potential stopping rules

I Equally spaced analyses (Π1 = 0.50,Π2 = 1):OBF.eq <- update( survFixed, nbr.analyses=2,

P=c(1,1),alt.hypo=1,power="calculate")POC.eq <- update( survFixed, nbr.analyses=2,

P=c(0.5,0.5),alt.hypo=1,power="calculate")Asym.eq <- update( survFixed, nbr.analyses=2,

P=c(1.5,1),alt.hypo=1,power="calculate")

I Early interim analysis ( Π1 = 0.33,Π2 = 1):OBF.early <- update( survFixed, sample.size=c(0.33,1)*363,

P=c(1,1),alt.hypo=1,power="calculate")POC.early <- update( survFixed, sample.size=c(0.33,1)*363,

P=c(0.5,0.5),alt.hypo=1,power="calculate")Asym.early <- update( survFixed, sample.size=c(0.33,1)*363,


I Late interim analysis ( Π1 = 0.67,Π2 = 1):OBF.late <- update( survFixed, sample.size=c(0.67,1)*363,

P=c(1,1),alt.hypo=1,power="calculate")POC.late <- update( survFixed, sample.size=c(0.67,1)*363,

P=c(0.5,0.5),alt.hypo=1,power="calculate")Asym.late <- update( survFixed, sample.size=c(0.67,1)*363,





*Background



Bios 6649- pg 22

ROCKET-AF trial Interim analysis planComparison of designsseqPlotBoundary(OBF.eq,OBF.early,OBF.late,survFixed,fixed=F)

0 100 200 300

0.8

1.0

1.2

1.4

1.6

1.8

Number of Events

Haz

ard

Rat

io

● OBF.eq● OBF.early

● OBF.late● survFixed

●

●

●

●

●

●

●

●

●

●

●

●●●




*Background



Bios 6649- pg 23

ROCKET-AF trial Interim analysis planComparison of designsseqPlotPower(OBF.eq,OBF.early,OBF.late,fixed=F,reference=survFixed)

1.0 1.1 1.2 1.3 1.4 1.5

−0.

012

−0.

008

−0.

004

0.00

0

Hazard Ratio

Rel

ativ

e P

ower

(Lo

wer

)

survFixedOBF.eqOBF.early

OBF.latesurvFixed




*Background



Bios 6649- pg 24

ROCKET-AF trial Interim analysis planComparison of designsseqPlotASN(OBF.eq,OBF.early,OBF.late,survFixed,fixed=F)

1.0 1.2 1.4

240

260

280

300

320

340

360

380

Hazard Ratio

Sam

ple

Siz

e

Average Number of Events

OBF.eqOBF.earlyOBF.latesurvFixed

1.0 1.2 1.4

240

260

280

300

320

340

360

380

Hazard Ratio

Sam

ple

Siz

e

75th percentile

OBF.eqOBF.earlyOBF.latesurvFixed




*Background



Bios 6649- pg 25

ROCKET-AF trial Interim analysis plan

Examination of OBF.eq

I Stopping boundaries for OBF.eq on scale of observedstatistic

> OBF.eqCall:seqDesign(prob.model = "hazard", arms = 2, null.hypothesis = 1.46,

alt.hypothesis = 1, ratio = c(1, 1), nbr.analyses = 2, sample.size = 363,test.type = "less", power = "calculate", alpha = 0.025, P = c(1,

1))


Null hypothesis : Theta >= 1.46 (size = 0.0250)Alternative hypothesis : Theta <= 1.00 (power = 0.9477)(Emerson & Fleming (1989) symmetric test)


Time 1 (NEv= 181.5) 0.9649 1.4600Time 2 (NEv= 363.0) 1.1869 1.1869




*Background



Bios 6649- pg 26


Boundaries on various design scales

I Observe statistic (sample mean scale)

> seqBoundary(OBF.eq, scale="X" )STOPPING BOUNDARIES: Sample Mean scale

Efficacy FutilityTime 1 (N= 181.5) 0.9649 1.4600Time 2 (N= 363.0) 1.1869 1.1869




*Background



Bios 6649- pg 27



I Normalized Z statistic: Zj = zj = (θ̂j − θ∅)/se(θ̂j )

(Note: work with log HR)

> seqBoundary( OBF.eq, scale="Z" )STOPPING BOUNDARIES: Normalized Z-value scale

Efficacy FutilityTime 1 (N= 181.5) -2.7897 0.0000Time 2 (N= 363.0) -1.9726 -1.9726




*Background



Bios 6649- pg 28



I Fixed-sample p-value scale: Pj = Φ(zj )

> 1-seqBoundary( OBF.eq, scale="P" )STOPPING BOUNDARIES: Fixed Sample P-value scale





*Background



Bios 6649- pg 29



I Error Spending Function scale(to be defined later: section 4.6)

> seqBoundary(OBF.eq, scale="E" )STOPPING BOUNDARIES: Error Spending Function scale


> seqBoundary( OBF.eq, scale="E" )*.025STOPPING BOUNDARIES: Error Spending Function scale





*Background



Bios 6649- pg 30


Stopping rules (attachment to protocol):

I Stopping boundaries for futility.“...the study may be stopped earlier for futility of rivaroxabanif the estimated risk ratio of rivaroxaban versus warfarin onthe primary efficacy endpoint is more than 1.64."

I Stopping boundaries for superiority.‘There is also no intention to stop the study early to declaresuperiority of rivaroxaban over warfarin on the primaryefficacy endpoint. However, in the presence ofoverwhelming superiority of rivaroxaban over warfarin, aconservative Haybittle-Peto boundary (one-sided p-value <0.001) will be used as a stopping guidance, requiring noadjustments to the final analysis (because the alpha spenton the interim analysis of superiority is negligible). Thep-value for testing Superiority on the Primary EfficacyEndpoint of rivaroxaban over warfarin will be based onon-treatment data from the safety population using thePrimary Cox Model."




*Background



Bios 6649- pg 31


Protocol-specified stopping rules:

I Stopping boundaries for futility :I θ̂1 > d1 = 1.64I θ̂2 > d2 = 1.1885 (same as fixed-sample design)

I Stopping boundaries for efficacy:I P < 0.001 implies:

log(θ̂j )− log(θ∅)

se[log(θ̂j )]< Φ−1(0.001)

log(θ̂j ) < −3.0902

√4

181.5+ log(1.46)

log(θ̂j ) < −0.080321

θ̂j < 0.92282




*Background



Bios 6649- pg 32



I Stopping boundaries for futility :I θ̂1 ≥ d1 = 1.64I θ̂2 ≥ d2 = 1.1885 (same as fixed-sample design)

I Stopping boundaries for efficacy:I θ̂1 < a1 = 0.9228I θ̂2 < a2 = 1.1885 (same as fixed-sample design)

I Two ways to construct this design in seqDesignI Search for values of Pa and Pd that produce the above

stopping criteria(Only possible if there are 2 interim analyses)

I Use constrained (custom) boundaries




*Background



Bios 6649- pg 33



I Search (not automated) shows Pa = 1.154 andPd = 1.361 is a close approximation:

> Asym.eq <- update(survFixed, nbr.analyses=2,+ P=c(1.154,1.361),alt.hypo=1,power="calculate")> Asym.eqCall:seqDesign(prob.model = "hazard", arms = 2, null.hypothesis = 1.46,

alt.hypothesis = 1, ratio = c(1, 1), nbr.analyses = 2, sample.size = 363,test.type = "less", power = "calculate", alpha = 0.025, P = c(1.154,

1.361))


Null hypothesis : Theta >= 1.46 (size = 0.0250)Alternative hypothesis : Theta <= 1.00 (power = 0.9497)


Time 1 (NEv= 181.5) 0.9227 1.6401Time 2 (NEv= 363.0) 1.1879 1.1879




*Background



Bios 6649- pg 34



I Exact design using constrained boundary:

HPbnd <- c(log(1.46) - qnorm(1-0.001)*sqrt(4/181.5),0,0,log(1.64))HPbnd <- rbind(exp(HPbnd),seqBoundary(survFixed))HPbnd <- as.seqBoundary(HPbnd,scale="X")

> HPbndSTOPPING BOUNDARIES: Sample Mean scale

a b c dTime 1 0.9228 1.0000 1.0000 1.6400Time 2 1.1885 1.1885 1.1885 1.1885




*Background



Bios 6649- pg 35



I Exact design using constrained boundary:

RocketIA.2 <- update(survFixed, nbr.analyses=2,,alt.hypo=1,exact.constraint=HPbnd,power="calculate")

> RocketIA.2Call:seqDesign(prob.model = "hazard", arms = 2, null.hypothesis = 1.46,

alt.hypothesis = 1, ratio = c(1, 1), nbr.analyses = 2, sample.size = 363,test.type = "less", power = "calculate", alpha = 0.025,exact.constraint = HPbnd)


Null hypothesis : Theta >= 1.46 (size = 0.0250)Alternative hypothesis : Theta <= 1.00 (power = 0.9502)

Warning: exact.constraint specified, actual power and sizemay differ from the nominal values


Time 1 (NEv= 181.5) 0.9228 1.6400Time 2 (NEv= 363.0) 1.1885 1.1885




*Background



Bios 6649- pg 36

ROAKCT-AF trial Interim analysis planComparing protocol stopping rules with OBF.eq:seqPlotBoundary(OBF.eq,RocketIA.2,survFixed,fixed=F)

0 100 200 300

1.0

1.2

1.4

1.6

Number of Events

Haz

ard

Rat

io

● OBF.eq● RocketIA.2

● survFixed

●

●

●

●

●

●

●

●●●




*Background



Bios 6649- pg 37

ROAKCT-AF trial Interim analysis planComparing protocol stopping rules with OBF.eq:seqPlotPower(OBF.eq,RocketIA.2,fixed=F,reference=survFixed)

1.0 1.1 1.2 1.3 1.4 1.5

−0.

005

−0.

003

−0.

001

0.00

1

Hazard Ratio

Rel

ativ

e P

ower

(Lo

wer

)

survFixedOBF.eq

RocketIA.2




*Background



Bios 6649- pg 38

ROAKCT-AF trial Interim analysis planComparing protocol stopping rules with OBF.eq:seqPlotASN(OBF.eq,RocketIA.2,survFixed,fixed=F)

1.0 1.2 1.4

260

280

300

320

340

360

380

Hazard Ratio

Sam

ple

Siz

e

Average Number of Events

OBF.eqRocketIA.2survFixed

1.0 1.2 1.4

260

280

300

320

340

360

380

Hazard Ratio

Sam

ple

Siz

e

75th percentile

OBF.eqRocketIA.2survFixed




*Background



Bios 6649- pg 39

ROAKCT-AF trial Interim analysis planComparing protocol stopping rules with OBF.eq:seqPlotStopProb(OBF.eq)

1.0 1.1 1.2 1.3 1.4 1.5

0.0

0.2

0.4

0.6

0.8

1.0

OBF.eq

Hazard Ratio

Sto

ppin

g P

roba

bilit

y

1

1

1

11 1

1

1

1

1

1

2 2 2 2 2 2 2 2 2 2 2

Lower Upper




*Background



Bios 6649- pg 40

ROAKCT-AF trial Interim analysis planComparing protocol stopping rules with OBF.eq:seqPlotStopProb(RocketIA.2)

1.0 1.1 1.2 1.3 1.4 1.5

0.0

0.2

0.4

0.6

0.8

1.0

RocketIA.2

Hazard Ratio

Sto

ppin

g P

roba

bilit

y

1

1

1

11 1 1

1

11

1

2 2 2 2 2 2 2 2 2 2 2

Lower Upper




*Background



Bios 6649- pg 41

ROAKCT-AF trial Interim analysis planComparing protocol stopping rules with OBF.eq:seqPlotInference(OBF.eq)

●

●

0 100 200 300

1.0

1.4

Inference corresponding to futility boundary

Sample Size

1.46

1.19X

X

●

●

0 100 200 300

0.8

1.2

Inference corresponding to efficacy boundary

Sample Size

0.965

1.187X

X

o Observed X Adjusted




*Background



Bios 6649- pg 42

ROAKCT-AF trial Interim analysis planComparing protocol stopping rules with OBF.eq:seqPlotInference(RocketIA.2)

●

●

0 100 200 300

1.0

1.6

Inference corresponding to futility boundary

Sample Size

1.64

1.19

X

X

●

●

0 100 200 300

0.8

1.2

Inference corresponding to efficacy boundary

Sample Size

0.923

1.189

X

X

o Observed X Adjusted

department of biostatistics & informatics colorado school...

Documents