An Experimental Paradigm for Developing Dynamic Treatment

Regimes

S.A. Murphy

Univ. of Michigan

March, 2004

Dynamic Treatment Regimes

Setting: Management of chronic, relapsing disorders such as alcohol, cocaine addiction and mental illness

Characteristics:

•May need a sequence of treatments prior to improvement

•Improvement marred by relapse

•Intervals during which more intense treatment is required alternate with intervals in which less treatment is sufficient

Dynamic Treatment Regimes are individually tailored treatments, with treatment type and dosage changing with ongoing subject need. Mimic Clinical Practice.

•Brooner et al. (2002) Treatment of Opioid Addiction

•Breslin et al. (1999) Treatment of Alcohol Addiction

•Prokaska et al. (2001) Treatment of Tobacco Addiction

•Rush et al. (2003) Treatment of Depression

EXAMPLE: Treatment of alcohol dependency. Primary outcome is a summary of heavy drinking scores over time

Treatment of Alcohol Dependency

Initial Txt Intermediate Outcome Secondary Txt

Monitor +Responder counseling

Monitor

Med B

Med ANonresponder

EM + Med B+ Psychosocial

Intensive OutpatientProgram

Responder Monitor +counseling

Monitor

Med A + Psychosocial Med B

Nonresponder

EM +Med B+Psychosocial

GOAL: Provide experimental methods for developing treatment assignment, i.e. decision, rules.

k Decisions

Observations made prior to jth decision

Action at jth decision

Primary Outcome:

for a known function u

A dynamic treatment regime is a vector of decision rules, one per decision

If the strategy is implemented then

for j=1,…., k.

Treatment of Alcohol Dependency

Initial Txt Intermediate Outcome Secondary Txt

Monitor +Responder counseling

Monitor

Med B

Med ANonresponder

EM + Med B+ Psychosocial

Intensive OutpatientProgram

Responder Monitor +counseling

Monitor

Med A + Psychosocial Med B

Nonresponder

EM +Med B+Psychosocial

Challenges

Two Challenges

•Delayed Effects

---sequential within-person randomization

•Dynamic Treatment Regimes are High Dimensional Multi-component Treatments

---series of randomized developmental trials prior to confirmatory trial.

Delayed Effects

Why choosing the best initial treatment on the basisof a randomized trial of initial treatments and choosing the best secondary treatment on the basis of a randomized trial of secondary treatments will not provide the best dynamic treatment regime.

Why you need data in which all treatment sequences are possible.

Treatment of Alcohol Dependency

Initial Txt Intermediate Outcome Secondary Txt

Monitor +Responder counseling

Monitor

Med B

Med ANonresponder

EM + Med B+ Psychosocial

Intensive OutpatientProgram

Responder Monitor +counseling

Monitor

Med A + Psychosocial Med B

Nonresponder

EM +Med B+Psychosocial

Treatment of Alcohol Dependency

Initial Txt Intermediate Outcome Secondary Txt

Monitor +Responder counseling

Monitor

Med B

Med ANonresponder

EM + Med B+ Psychosocial

Intensive OutpatientProgram

Responder Monitor +counseling

Monitor

Med A + Psychosocial Med B

Nonresponder

EM +Med B+Psychosocial

Summary:

Evaluating the initial treatments requires that we first calculate the mean primary outcome of patients for each

combination of secondary treatment, intermediate outcome and initial treatment.

The main point:

If the mean primary outcome to secondary treatment vary by initial treatment then we should use sequential

within-person randomization.

When might the mean primary outcome to each combination of secondary treatment,

intermediate outcome vary by initial treatment?

Causal Effects of Initial Txt

Unmeasured Common Causes

Initial Txt intermediate Secondary Primaryoutcome Txt Outcome

Noncausal Correlations

Unmeasured Common Causes

Initial Txt intermediate Secondary Primaryoutcome Txt Outcome

Delayed Effects: The Bottom Line•Are there unobserved but potentially important common causes of the primary and intermediate outcomes?

•Are there unobserved but potentially important causal pathways from initial treatment to primary outcome?

If yes to either of the above then use sequentially within-person randomized trials to develop good dynamic treatment regimes.

An Aside!

Medical Decision Making

Disease Yes/No?

Diagnostic Test Results Txt OutcomeTest

Sequential Experimental Designs

Parameters

Txt Outcome Txt Outcome

Individual 1 Individual 2

Adaptive Treatment Strategies are High Dimensional Multi-Component

Treatments

•when to start treatment?•which treatment to start?•when to step-up treatment?•which step-up treatment?•when to step down treatment to maintenance/monitoring?•which maintenance/monitoring treatment?•what information to use to make each of the above decisions?

Meeting the Challenges

Delayed Effects: Sequential within-person randomization: Randomize at each decision point.

High Dimensionality: Series of developmental randomized trials prior to a confirmatory trial (Box, Hunter and Hunter,1978, pg. 303).

Examples of sequentially within-person randomized trials:

•CATIE (2001) Treatment of Psychosis in Alzheimer’s Patients

•CATIE (2001) Treatment of Psychosis in Schizophrenia

•STAR*D (2001) Treatment of Depression

•Thall et al. (2000) Treatment of Prostate Cancer

Test Statistic and Sample Size Formula for Primary Analysis

Proposal

• Primary analysis in developmental trial is to discriminate between regimes with different initial treatments.

• In primary analysis consider dynamic treatment regimes with decision rules depending only on summaries of Xj’s (say Sj’s)

• In secondary analyses consider dynamic treatment regimes with decision rules depending on the Xj’s.

• Choose randomization probabilities to equalize the sample size across possible strategies.

Proposal

Estimate the mean of Y when the decision rules,

are followed:

The variance of the estimator is used to construct a sample size formula.

Randomization probability of treatment

Aj=aj given past:

Estimating Function:

Solve,

We obtain:

Primary Analysis: Test statistic to compare two

strategies with different initial treatments:

See Murphy, van der Laan and Robins (2001) for technical details.

Calculating Sample Sizes:

Treatment of Alcohol Dependency

Initial Txt Intermediate Outcome Secondary Txt

Monitor +Responder counseling

Monitor

Med B

Med ANonresponder

EM + Med B+ Psychosocial

Intensive OutpatientProgram

Responder Monitor +counseling

Monitor

Med A + Psychosocial Med B

Nonresponder

EM +Med B+Psychosocial

In this example:

Balanced design hence choosing the randomization probabilities to equalize the sample size across all possible regimes yields uniform randomization probabilities.

An estimator of the mean of Y under the decision rules is the average response of individuals whose treatment pattern is consistent with the rules:

The average response for individuals whose treatment pattern is consistent with the rules:

is the number of treatment alternatives at decision j

is response variance under treatment regime

Sample Size Formula:

where is the Type I error and

is the power of the test to detect a difference in

the mean response between regimes.

In our simple example:

Secondary Hypotheses

•Compare dynamic treatment regimes that begin with the same treatment; in this example, compare primary outcome to secondary treatments by levels of the summary intermediate outcome.

•Use an analysis that tests if other intermediate outcomes differentiate for whom each secondary treatment is best and if any pretreatment information differentiates for whom each initial treatment is best. (Murphy, 2003; Robins, 2003)

Discussion• Simulations indicate that sample size formula is

accurate for balanced designs.

• Secondary analyses can only explore dynamic treatment regimes that comply with the restrictions imposed by the experimental design.

• Trial design and analyses targeted at scientific goal.

Open Problems

• This setting requires development/generalization of Box's experimentation approach of several developmental trials, all based on randomization prior to a confirmatory trial.

• How could one use working assumptions on the form of delayed effects to speed up the developmental process? Use working assumptions to pool information. What kinds of working assumptions make sense? How do you detect potential violations of the working assumptions?

Open Problems

• Choosing randomization probabilities in unbalanced designs.

• Dealing with high dimensional X-- feature extraction--in secondary analyses.

This seminar can be found at

http://www.stat.lsa.umich.edu/~samurphy/seminars/seminar0304.ppt

The paper can be found at

http://www.stat.lsa.umich.edu/~samurphy/papers/ExperimentalEvidence.pdf

Principles in Designing a Sequentially Within-Person Randomized Trial

Principles in Designing a Sequentially Within-Person Randomized Trial

•Secondary treatment alternatives should vary by only a simple low dimension summary (responder status) instead of all intermediate outcomes (adherence, burden, craving, etc.).

•Collect intermediate outcomes that might be useful in ascertaining for whom each treatment works best; information that might enter into the decision rules.

Principles in Designing a Sequentially Within-Person Randomized Trial

•Choose a primary hypothesis that is both scientifically interesting and aids in developing the adaptive treatment strategy.

•Choose secondary hypotheses that further develop the adaptive treatment strategy and use the randomization to reduce confounding.