tessella webinar 6 18-2013 v3

A Novel Phase 3 Design Incorporating

Historical Information for the Development of

Antibacterial Agents

Jeff Wetherington, GSK

Kert Viele, Berry Consultants

Tessella Series Webinar

June 18th, 2013

Complicated Urinary Tract

Infections (1)

• Occurs in men and women

– structural or functional abnormalities of the urinary

tract

– hospitalized patients with significant medical or

surgical co-morbidities

• Major cause of hospital admission, extended hospitalizations morbidity, mortality, and excess healthcare costs

• Prescribing physicians have several options for empiric and pathogen-specific treatment

Bayesian Augmented Control for Antibacterial Agents 2

Complicated Urinary Tract

Infections (2)

Yet unmet treatment needs for patients with cUTI

continue to exist given the emergence and

prevalence of multi-drug resistance in uropathogens

Bayesian Augmented Control for Antibacterial Agents 3

Traditional Phase 3

• Randomized, parallel group

• Active controlled, non inferiority

• Narrow non inferiority margins

• Infeasible in the face of increasing unmet need

– >1500 patients enrolled into 2 independent trials

– >5 year clinical development programs

• Urgent need for novel clinical designs for

antibiotic drug development

Bayesian Augmented Control for Antibacterial Agents 4

Historical Studies

Bayesian Augmented Control for Antibacterial Agents 5

0.0

0.2

0.4

0.6

0.8

1.0

Naber 2009 Redman 2010

Doripenem Microbiological Eradication Rate

Fixed Design

• “Standard” 1:1 trial requires 750 subjects total

– 90% power and one sided α=0.025

– power for p=0.83 with noninferiority δ=0.10

– 20% dropout assumed

– note 375 patients on treatment

• Can we do better using the historical

information?

Bayesian Augmented Control for Antibacterial Agents 6

Goals

• Reduce sample size!

• Reduce sample size!

• Maintain

– controlled type I error (most complex…won’t be

able to get “complete” control)

– comparable power around (and slightly below per

expectation) p=0.83

– similar numbers of subjects on treatment (for

secondary analyses)

Bayesian Augmented Control for Antibacterial Agents 7

Preview

• Proposed design incorporates historical borrowing on the control arm using a hierarchical model.

• 20% reduction in sample size

• Similar power near or slight below p=0.83

• Slightly MORE subjects on treatment

• Type I error control

– in a region near 0.83

– based on E[type I error] for a range of perceived likely amounts of “drift” in the true control rate

Bayesian Augmented Control for Antibacterial Agents 8

Experimental parameters

• Dichotomous endpoint, p=Pr(ME)

• Control = Doripenum versus Treatment

• 20% dropout rate

• Goal is 90% for p=0.83 (NI δ=0.10) with

one sided α=0.025

• If possible, would like to leverage two

historical studies from control arm.

– Naber, 230 successes in 280 subjects (82.1%)

– Peninsula, 209 successes in 250 subjects (83.6%)

Bayesian Augmented Control for Antibacterial Agents 9

Notation and Model

• γ0 = logit(true current control rate)

• γ1 = logit(true Naber rate)

• γ2 = logit(true Peninsula rate)

• γ0,γ1,γ2 ~ N(μ,τ)

• π(μ)=N(1,1)

• π(τ2)=IGamma(α=0.001,β=0.001)

• Treatment effect θ ~ N(μθ=0,σθ=100)

• γ0+θ = logit(true treatment rate)

Bayesian Augmented Control for Antibacterial Agents 10

Intuition

• The three Doripenem arms are connected

through the “across studies” distribution

N(µ,τ).

– similar to random effects model on studies

– common model in meta-analysis

• τ is the most important parameter (across

study variance)

– τ≈0 corresponds to γ0≈γ1≈γ2

– τ large corresponds to no borrowing

Bayesian Augmented Control for Antibacterial Agents 11

Intuition • τ has a prior, and is estimated as part of the

model fitting.

• Datasets with high across study variation – produce higher estimates of τ

– the N(µ,τ) across distribution exerts less influence on each group (acts as less informative prior)

– less borrowing

• Datasets with low across study variation – produce lower estimates of τ

– the N(µ,τ) across study distribution can act as quite informative prior

– more borrowing

Bayesian Augmented Control for Antibacterial Agents 12

Intuition

• Only 3 Doripenem arms available

– so only three γ used to estimate τ

• Enough that borrowing is dynamic, but prior

will not fully wash out.

• Important to consider operating

characteristics (as always)

• Big goal

– borrow robustly when current data near p=0.83

– borrow less as current data diverges from p=0.83

Bayesian Augmented Control for Antibacterial Agents 13

Proposed Design A

• N=600 with 2:1 randomization and

borrowing

– 20% savings on N compared to fixed design

– 400 subjects on treatment, more than fixed

design

• Trial declared success if

– Pr(trt rate > ctrl rate – 10%)>0.975

Bayesian Augmented Control for Antibacterial Agents 14

Bayesian Augmented Control for Antibacterial Agents 15

Data

Bayesian Augmented Control for Antibacterial Agents 16

horizontal dashed

line is historical rate

Bayesian Augmented Control for Antibacterial Agents 17

control arm CIs

under none,full

and hierarchical

borrowing

Bayesian Augmented Control for Antibacterial Agents 18

Treatment CI

(same for all borrowing

as prior is noninformative)

Bayesian Augmented Control for Antibacterial Agents 19

Effective

borrowing

N=mean(p)*(1-mean(p)) / var(p)

numborrowed=N-233

Bayesian Augmented Control for Antibacterial Agents 20

probability of trial

success under each

kind of borrowing

Dynamic borrowing

• The effective amount of borrowing for

193/233 on control (almost identical to

history) is 225 of the 530 historical subjects

• Hierarchical modeling produces dynamic

borrowing.

– as the current control varies away from 82.9%

(history), the amount of borrowing decreases

Bayesian Augmented Control for Antibacterial Agents 21

Bayesian Augmented Control for Antibacterial Agents 22

Bayesian Augmented Control for Antibacterial Agents 23

Bayesian Augmented Control for Antibacterial Agents 24

Observed control proportion (x) versus

effective number of borrowed observations (y)

Summary

• The hierarchical model produces dynamic borrowing (through estimation of τ)

– Many historical subjects are borrowed when the current data is consistent with history

– Few historical subjects are borrowing when the current data is inconsistent

• Most effective (gains the most information) if the historical data is “on point”.

• If you happen to get inconsistent data, less borrowing occurs mitigating the costs.

Bayesian Augmented Control for Antibacterial Agents 25

Operating Characteristics

• For all designs, evaluated Pr(trial success) for

– control rates 0.780, 0.805, 0.830, 0.855, 0.880

– treatment rates 10% lower, 5% lower, equal, or

5% greater than control

• Treatment rates 10% lower were used to

compute type I error

• Equal treatment rates used to compute

power.

Bayesian Augmented Control for Antibacterial Agents 26

Operating Characteristics

Trt -10% Trt -5% Trt equal Trt +5%

Control 0.780 0.024 / 0.006 0.309 / 0.119 0.842 / 0.702 0.996 / 0.990

Control 0.805 0.026 / 0.007 0.321 / 0.229 0.874 / 0.861 0.996 / 0.997

Control 0.830 0.026 / 0.017 0.342 / 0.376 0.910 / 0.942 0.998 / 0.999

Control 0.855 0.028 / 0.045 0.372 / 0.564 0.929 / 0.966 1.000 / 1.000

Control 0.880 0.030 / 0.100 0.421 / 0.642 0.954 / 0.976 1.000 / 1.000

Bayesian Augmented Control for Antibacterial Agents 27

Table shows Pr(trial success)

Two entries per cell in the form FIXED / DESIGN A

Design aims to produce greater power AND lower type

I error for consistent control data, with 20% savings on N.

Here type I error reduced from 0.026 to 0.017 AND

power increased from 91% to 94.2%.

Type I error Power

Operating Characteristics

Trt -10% Trt -5% Trt equal Trt +5%

Control 0.780 0.024 / 0.006 0.309 / 0.119 0.842 / 0.702 0.996 / 0.990

Control 0.805 0.026 / 0.007 0.321 / 0.229 0.874 / 0.861 0.996 / 0.997

Control 0.830 0.026 / 0.017 0.342 / 0.376 0.910 / 0.942 0.998 / 0.999

Control 0.855 0.028 / 0.045 0.372 / 0.564 0.929 / 0.966 1.000 / 1.000

Control 0.880 0.030 / 0.100 0.421 / 0.642 0.954 / 0.976 1.000 / 1.000

Bayesian Augmented Control for Antibacterial Agents 28

Table shows Pr(trial success)

Two entries per cell in the form FIXED / DESIGN A

With slight reduction in true control rate, design still

obtains nearly equivalent power.

Type I error Power

Operating Characteristics

Trt -10% Trt -5% Trt equal Trt +5%

Control 0.780 0.024 / 0.006 0.309 / 0.119 0.842 / 0.702 0.996 / 0.990

Control 0.805 0.026 / 0.007 0.321 / 0.229 0.874 / 0.861 0.996 / 0.997

Control 0.830 0.026 / 0.017 0.342 / 0.376 0.910 / 0.942 0.998 / 0.999

Control 0.855 0.028 / 0.045 0.372 / 0.564 0.929 / 0.966 1.000 / 1.000

Control 0.880 0.030 / 0.100 0.421 / 0.642 0.954 / 0.976 1.000 / 1.000

Bayesian Augmented Control for Antibacterial Agents 29

Table shows Pr(trial success)

Two entries per cell in the form FIXED / DESIGN A

If true current control rate is higher than observed

historical rate, power is increased, but we also observe

inflated type I error.

Type I error Power

Operating Characteristics

Trt -10% Trt -5% Trt equal Trt +5%

Control 0.780 0.024 / 0.006 0.309 / 0.119 0.842 / 0.702 0.996 / 0.990

Control 0.805 0.026 / 0.007 0.321 / 0.229 0.874 / 0.861 0.996 / 0.997

Control 0.830 0.026 / 0.017 0.342 / 0.376 0.910 / 0.942 0.998 / 0.999

Control 0.855 0.028 / 0.045 0.372 / 0.564 0.929 / 0.966 1.000 / 1.000

Control 0.880 0.030 / 0.100 0.421 / 0.642 0.954 / 0.976 1.000 / 1.000

Bayesian Augmented Control for Antibacterial Agents 30

Table shows Pr(trial success)

Two entries per cell in the form FIXED / DESIGN A

If true current control rate is much lower than the observed

historical rate, then power is reduced (type I error rate

is negligible).

Type I error Power

Summary for Design A • Pros

– 20% reduction in sample size

– more patients on treatment

– increased or equivalent power for true control rates near observed historical data.

– essentially, there is a “sweet spot” where Design A dominates the fixed trial.

• Cons – inflated type I error for true control rates much

above observed historical control rates.

– decreased power for true control rates much below observed historical data

Bayesian Augmented Control for Antibacterial Agents 31

Design B

• As with design A, hierarchical borrowing and

2:1 randomization, with maximum N=600.

• Incorporates futility stopping

– interim analyses N=300, 400, 500, 600.

– trial stopped for futility if

Pr(non-inferiority)<0.15

• Evaluated operating characteristics as before,

including expected sample size.

Bayesian Augmented Control for Antibacterial Agents 32

Operating Characteristics

Trt -10% Trt -5% Trt equal Trt +5%

Control 0.780 0.006 / 0.005

456.0

0.119 / 0.133

572.5

0.702 / 0.692

598.5

0.990 / 0.994

599.7

Control 0.805 0.007 / 0.010

499.7

0.229 / 0.237

589.9

0.861 / 0.851

599.7

0.997 / 0.998

599.7

Control 0.830 0.017 / 0.019

537.9

0.376 / 0.376

595.7

0.942 / 0.938

599.7

0.999 / 0.999

599.7

Control 0.855 0.045 / 0.048

569.3

0.564 / 0.567

598.2

0.966 / 0.968

599.7

1.000 / 0.999

599.7

Control 0.880 0.100 / 0.096

580.2

0.642 / 0.638

598.3

0.976 / 0.970

599.7

1.000 / 0.999

599.7

Bayesian Augmented Control for Antibacterial Agents 33

Three entries per cell. Top two are Pr(trial success)

in the form DESIGN A / DESIGN B. Lower number

is expected N (max 600)

Can save another

60-140 subjects

under null hypothesis

depending on true

control rate.

Operating Characteristics

Trt -10% Trt -5% Trt equal Trt +5%

Control 0.780 0.006 / 0.005

456.0

0.119 / 0.133

572.5

0.702 / 0.692

598.5

0.990 / 0.994

599.7

Control 0.805 0.007 / 0.010

499.7

0.229 / 0.237

589.9

0.861 / 0.851

599.7

0.997 / 0.998

599.7

Control 0.830 0.017 / 0.019

537.9

0.376 / 0.376

595.7

0.942 / 0.938

599.7

0.999 / 0.999

599.7

Control 0.855 0.045 / 0.048

569.3

0.564 / 0.567

598.2

0.966 / 0.968

599.7

1.000 / 0.999

599.7

Control 0.880 0.100 / 0.096

580.2

0.642 / 0.638

598.3

0.976 / 0.970

599.7

1.000 / 0.999

599.7

Bayesian Augmented Control for Antibacterial Agents 34

Three entries per cell. Top two are Pr(trial success)

in the form DESIGN A / DESIGN B. Lower number

is expected N (max 600)

Very slight

changes to

power

Summary for Design B

• Similar Pros/Cons relative to fixed design

• Pros relative to design A

– can save 60-140 subjects on average when null hypothesis is true

– when drug is noninferior, design very rarely stops for futility.

– successful trials retain 400 subjects on treatment.

• Cons relative to design A

– very slight power loss (simulations all have 1% or less power loss)

Bayesian Augmented Control for Antibacterial Agents 35

Statistical Summary

• Hierarchical Modeling allows for competitive

alternatives to fixed designs with significant

sample size savings

• Possible risks are associated with drift in the

true control rate from observed historical

rate.

• Futility stopping can result in additional

sample size savings under null hypothesis

without significant cost to power.

Bayesian Augmented Control for Antibacterial Agents 36

Conclusions

• Hierarchical Modeling allows for

competitive alternatives to fixed designs with

significant reduction in cycle time

• Possible risks are associated with drift in the

true control rate from observed historical rate

• Futility stopping can result in minimizing

exposure to ineffective therapy under null

hypothesis without significant cost to power

Bayesian Augmented Control for Antibacterial Agents 37