ba dgajewski tesellaberry

Building Efficient Comparative Effectiveness

Trials through Adaptive Designs, Utility

Functions, and Accrual Rate Optimization:

Finding the Sweet Spot

Byron J. Gajewski, PhD

Department of Biostatistics

University of Kansas Medical Center

Kansas City, Kansas, USA

July 23, 2013

1

http://en.wikipedia.org/wiki/Bad_(album)

2

Bayesian Adaptive Designs

(BAD) • No longer “a dream for statisticians only”

• Published not only in biostatistical journals but also clinical epidemiology and medical journals

• Save time and money and lean towards more ethical studies

• Scientific contribution to the design, implementation, and analysis of comparative effectiveness clinical trials

• Patient Centered Outcomes Research Institute (PCORI) advocates for their use

3

Comparative Effectiveness

• NASCAR (National Association for Stock Car Auto Racing)

• cryptogenic sensory polyneuropathy (CSPN) – What treatment for pain is the best? Off label and

approved drugs used in practice

• We built a BAD with efficiency for finding the best treatment in mind and found three key trial aspects – the Bayesian adaptive design parameters

– the utility function for weighing endpoints

– the patient accrual rate

• These three developmental parameters are vital for building adaptive, cost-effective comparative effectiveness designs.

4


• NASCAR

• Cryptogenic sensory polyneuropathy (CSPN) – What treatment for pain is the best? Off label and






5


• NASCAR







Non-diabetic

6


• NASCAR






• These three developmental parameters are vital for building adaptive, cost-effective comparative effectiveness designs

Non-diabetic

7


• NASCAR







Non-diabetic

8


• NASCAR







Non-diabetic

Performance Adaptive Investigation of

Neuropathic

Pain-Comparison of Treatments in

Real-Life Situations

(PAIN-CONTRoLS )

9

What has been done on BAD?

• Phase I-III clinical trials

– dose finding studies

– assessment of safety and efficacy in the

presence of historical prior information.

• In many cases these studies have a

functional form that is unique to

• Classical pharmaceutical clinical trials

(e.g. control group or dose)

10

What has not been done on

BAD? • A different challenge in comparative

effectiveness trials

– there is typically no control group

– investigating the relative effectiveness

– No dose structure to our problem

• We discuss the unique framework of BAD under

this setting

11

What we address here

• Combine endpoints with a utility function

• Optimize accrual

• Ex: PAIN-CONTRoLS

– Endpoints

– Models

– Simulation

• We find the “sweet spot” balancing

– Average number of patients needed

– Average length of time to finish the study

12

PAIN-CONTRoLS

• Five different drugs (e.g. Lyrica; Cymbalta;

Tramadol; Nortriptyline; Gabapentin)

• Multi-site trial (20 sites); accrual about 4-8

patients/week

• Nmax=600 (1.5-3.0 years)

• Endpoints:

– Efficacy: ½ or better drop in VAS score (baseline

to 12 weeks)

– Quit/Dropout: Drop treatment after 12 weeks

13

Combining Endpoints

• Combining two endpoints (Berry et al., 2010)

– We detail the building of a utility function here

• Scenario: Drug B > Drug A but higher quit rate

– What would that “quit rate” have to be in order for

Drug B to be clinically the same as Drug A?

14

Quit

15

Combining Endpoints

• Utility for Efficiency: 1 for 100% efficacy

and utility of 0 for 0% efficacy

• Utility for quit/ discontinue endpoint we

used utility of 0.75 at 0% quit/discontinue

with a drop to 0 at 100% quit/discontinue.

• Utility combination

U(E,Q)=E+.75-.75Q

16

Statistical Details

17

Basic analytic examples

• Example 1: one arm

• Example 2: two arms

18

Example 1: one arm

• Consider a tolerability endpoint for the

PAIN-CONTRoLS study and suppose the

endpoint is measured immediately after

randomization (Qi=1) or not (Qi=0)

– n=85 patients (fixed)

– SQ=Σqi

– θ quit rate (unobserved but random)

– Δ max. tolerated quit rate (fixed & known)

• Stopping rule: P(θ <Δ |SQ)> γ 19

Example 1: one arm

Period 1

(T1)

n1

Period 2

(T2)

n2

Time

20

Example 1: one arm

Period 1

(T1)

n1

Period 2

(T2)

n2

Time

Stop if P(θ <Δ |SQ)> γ

(Uniform-Binomial)

21

Example 1: one arm

Period 1

(T1)

n1

Period 2

(T2)

n2

Time

Stop if P(θ <Δ |SQ)> γ

(Uniform-Binomial)

Otherwise move on

22

Example 1: operating

characteristics 1. Sampling distribution is Qi|θ0~Bern(θ0), where θ0 is the true quit/discontinue rate.

2. Probability of stopping the trial early at period 1:

1

1 111

1 0 0

0 0 1

1 1Q QQ Q

Q

nn S n SS S

QS Q

nnP I d

SS n

,

where I(x>y) is 1 if x>y and 0 otherwise.

a. Expected time (T) of the trial is E(T)=P1T1+(1-P1)T2

b. Expected sample size (N) of the trial is E(N)=P1n1+(1-P1)85.

23




1

1

11 1

1 0 0

0 10

1 1Q QQ Q

Q

n SS n S

Q

nS

QS

nP

n

SI d

n S

,




24




1

1

11 1

1 0 0

0 0 1

1 1Q QQ Q

Q

n SS n S

Q

nS

QS

nP

n

SI d

n S

,




25




1

1

11 1

1 0 0

0 0 1

1 1Q QQ Q

Q

n SS n S

Q

nS

QS

nP

S

nI d

S n

,




26


characteristics

1. Sampling distribution is Qi|θ0~Bern(θ0), where θ0 is the true quit/discontinue rate.


1

1 111

0 0

0 0 1

1 1 1Q QQ Q

Q

nn S n SS S

QS Q

nnP I d

SS n

,


a. Expected time (T) of the trial is E(T)=P1T1+(1-P1)(T1+T2)


27

One arm: size and cost

(n1+n2=85)

30

40

50

60

70

80

90

25 35 45 55 65 75 85

E(n

) o

r E

(T)

in d

ays

n1

θ0=.2

Δ =.3, and γ =.8,

n1=30, 35, 40,…,80

T1=T2=28 days.

The probability of stopping early varies

from 0.4275 for n1=30 and jumps up to

0.7621 for n2=80.

E(N)

E(T)

28

“virtual response”

Example 2: two arms

• Similar notation, but stop if

P({θ1 < θ2 | SQ1,SQ2})> γ or if P ({θ1 > θ2 | SQ1,SQ2})>γ

• Operations

– Complicated closed form (Kawasaki &

Miyaoka, 2012)

– Then using a double sum across SQ1 and SQ2

would allow similar calculations for E(T) and

E(N) as done in Example 1

29

Gets more complicated fast

• Five arms and two endpoints

• Accrual patterns tend to be random and

staggered; not fixed

• Quickly complicate things for closed-form

analytic solutions

• Therefore, as advocated by Berry et al.

(2011), we utilize simulations

30

PAIN-CONTRoLS

• Virtual subject response for five arms

• Accrual patterns

• Design

• Adaptive randomization: allocation

• Simulation Algorithm

31

Virtual subject response for five

arms

• Null case

• Alternative case

0 .3, .3, .3, .3, .3e θ and 0 .2, .2, .2, .2, .2q θ

0 .3, .3, .3, .4, .5e θ and 0 .3, .3, .3, .25, .15q θ

32

Accrual patterns 1. mean number of accrued patients per week: ΛT.

2. 1{ }| ~T T T TN N Poisson , where T=1,2,3,…, and 0N =0. The patterns of ΛT

depend on two factors:

a. the number of sites actively enrolling patients into the study and

b. how fast the sites can enroll, which we assume is a constant λ0/2 for each:

0

0

0

0

, 0 <2

2 , 2 <4

3 , 4 <6

10 , 20

T

T

T

T

T

.

33

Design 1. Likelihood: SEjT|njT~Bino(njT ,θ

ej) and SQjT|njT~Bino(njT ,θ

qj).

2. Priors, 2logit ~ 0,100e

j N and 2logit ~ 0,100q

j N .

3. Posterior distributions, MCMC.

4. Our stopping criteria:

a. minimum of 200 subjects allocated.

b. Stop the trial if the probability the arm with the maximum utility > 0.90.

c. Utility | 0.75 0.75 |e q

jT j EjT j QjTU S S with maximum utility

max, 1 2 3 4 5max , , , ,T T T T T TU U U U U U .

d. The evaluation criteria: probability the arm with the maximum utility > 0.90.

34

Adaptive randomization:

allocation

max,*

Pr

1

jT T jT

j

jT

U U Var UV

n

35

Sweet Spot Algorithm (SSA) • Step 0: Set b=0

• Step 1: Set b=b+1.

• Step 2: Simulate the initial observed data.

• Step 3: estimate posterior parameters via simulation

and calculate the stopping rule and the possible

next allocation.

• Step 4: repeat steps 2 and 3 after collecting four

more weeks of data.

• Step 5: evaluate all of the data after collecting all of

the endpoints.

• Step 6: go to step 1 unless b=100, then stop.

36

Results

37

Pmax, N, and T predictive distributions

“Alternative Case” (Λ20=8)

0.2 0.4 0.6 0.8 10

50

100

Pmax

Count

200 300 400 500 6000

20

40

60

n

Count

40 60 80 100 1200

10

20

30

T

Count

38

Alternative Case • Success:

– 95% of the trials had early success

– 1% late success (trial goes to the maximum

sample size of 600)

– 4% of incomplete solutions

• Sample size:

– E(N)=302.2 subjects

– 80% of the trials being 362 or smaller.

• Length:

– E(T)=61.4 weeks

– longest trial taking 100 weeks

39

Expected size, time, and cost for

five arms (effect scenario)

E(N)= 7.4466Λ20 + 241.53

E(T)= 254.82(Λ20)-0.694

0

50

100

150

200

250

300

350

0 2 4 6 8 10 12

E(n

) o

r E

(T)

in w

eek

s

Λ20

40



E(N)= 7.4466Λ20 + 241.53

E(T)= 254.82(Λ20)-0.694

0

50

100

150

200

250

300

350

0 2 4 6 8 10 12

E(n

) o

r E

(T)

in w

eek

s

Λ20

E(Cost)= 7.4466 20Λ +241.53+ 1.25(254.82 20Λ -0.694)

Taking derivative w.r.t. 20Λ and solving we get

1/1.694ˆ 1.25*254.82*.694 /7.4466

=7.4

20Λ

41



E(N)= 7.4466Λ20 + 241.53

E(T)= 254.82(Λ20)-0.694

0

50

100

150

200

250

300

350

0 2 4 6 8 10 12

E(n

) o

r E

(T)

in w

eek

s

Λ20

350

400

450

500

550

600

0 2 4 6 8 10 12

E(C

)

Λ20

E(Cost)= 7.4466 20Λ +241.53+ 1.25(254.82 20Λ -0.694)

Taking derivative w.r.t. 20Λ and solving we get

1/1.694ˆ 1.25*254.82*.694 /7.4466

=7.4

20Λ

42

If we accrue faster will we get

less efficacy per unit? • No!

• For the accrual patterns the proportion

times we are successful (i.e. proportion ) is

between 0.96 and 1.00

• The margin of error if the true success rate

is about

0.98 (+/-1.96*sqrt(.98*.02/100)=+/-.0274)

43


five arms (null scenario)

E(N) = 0.3863Λ20 + 586.5

E(T) = 584.27(Λ20)-0.879

0

100

200

300

400

500

600

700

0 2 4 6 8 10 12

E(n

) o

r E

(T)

in w

eek

s

Λ20

44

Discussion: relative to fixed trial • Classical framework: fixed sample size but get

various endpoint efficacy knowledge

• We “flip” the approach to clinical trials design

– The effect we learn is fixed

– While sample size varies depending on the data

– BAD approach is a proxy for the scientific

knowledge

45

Discussion: various extensions

of SSA • Vary the number arms (say 2, 6, or more),

• One endpoint instead of two

• Change the maximum sample size from 600 to

higher

• Change to a minimal efficacy or a futility

stopping rule

• The accrual pattern could change

46

Discussion: Accrual • Starts off small and grows (Anisimov, 2011)

• Adaptive accrual => accrual prediction models

(e.g. Anisimov & Federov 2007; Gajewski,

Simon, and Carlson, 2008; Zhang and Long,

2010; & Anisimov, 2011) => update accrual

patterns are in real time

• For example,overpromise and under deliver

(e.g. Breau, 2006)

47

Discussion: Generalize • SSA algorithm extensions

– Time to event endpoints

– Ordinal or continuous or a mix of the two

endpoints

– Dynamic linear models for dose finding

studies

– Various types of hierarchical models

48

Discussion

• Sweet spot same for all drugs?

– Subjects cost differently by drug

• Generalizability to other Bayesian adaptive

clinical trials:

– adaptation rule

– utility function

– accrual should all be parameters considered

for optimizing the design of comparative

effectiveness research

49

Benefits of BAD: “B.A. Baracus”

trial design

http://www.a-team-inside.com/ba/bosco-b-a-baracus

• Hard work up-front

but worth it later

• Fit

• Efficient

• Very good at

getting answers

• Bad A**

50

Acknowledgements

• Co-authors:

– Scott M. Berry, PhD; Mamatha Pasnoor, MD;

Mazen Dimachkie, MD; Laura Herbelin, BS;

Richard Barohn, MD

• Frontiers: The Heartland Institute for

Clinical and Translational Research CTSA

UL1TR000001 (Barohn & Aaronson)

• Department of Biostatistics (e.g. matching

Frontiers effort) (Mayo) 51

QUESTIONS?

52

ba dgajewski tesellaberry

Health & Medicine

best treatment

bayesian adaptive designs

developmental parameters

patient accrual

key trial aspects

approved drugs

accrual rate optimization

utility functions