a fireside chat about precision medicine 3mm and a ... · strati cation free adaptive enrichment...
TRANSCRIPT
![Page 1: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/1.jpg)
A Fireside Chat about Precision Medicine
and a [somewhat] more informed discussion of
Stratification Free Adaptive Enrichment
Noah Simon
July, 5 2017
1 / 53
![Page 2: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/2.jpg)
An Dialogue about Precision Medicine
Let’s begin with a discussion about Precision Medicine.
2 / 53
![Page 3: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/3.jpg)
The following ideas may have partly been borrowed from...
He’s the one in the center... giving everyone else a hard time atseminar!
3 / 53
![Page 4: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/4.jpg)
What is Precision Medicine?
I will stick mostly to oncology...
Not because I know much there...
But I definitely know less about everything else!
4 / 53
![Page 5: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/5.jpg)
What is Precision Medicine?
The practice of medicine has always been about
I characterizing dysfunction
I treating based on specific characterizations
5 / 53
![Page 6: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/6.jpg)
What is Precision Medicine?
In the beginning this was based on simple observation alone:
you’ve been vomiting and missed your period −→ Pregnant
Now we have more sophisticated methods:
hCG in urine −→ Pregnant
In oncology, tumors are characterized using histology
6 / 53
![Page 7: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/7.jpg)
What is Precision Medicine?
My understanding is:
Medicine attempts to differentiate diseases...
to develop treatments that target specific disease characteristics
Precision medicine attempts to differentiate diseases...
more precisely?
to develop treatments that target specific disease characteristics
That reads like a high schooler “not-plagiarizing” an essay...
7 / 53
![Page 8: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/8.jpg)
What is Precision Medicine?
My understanding is:
Medicine attempts to differentiate diseases...
to develop treatments that target specific disease characteristics
Precision medicine attempts to differentiate diseases...
more precisely?
to develop treatments that target specific disease characteristics
That reads like a high schooler “not-plagiarizing” an essay...
7 / 53
![Page 9: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/9.jpg)
What is Precision Medicine?
My understanding is:
Medicine attempts to differentiate diseases...
to develop treatments that target specific disease characteristics
[Biomolecular] Precision medicine attempts to differentiate diseasesusing biomolecular profiling
to develop treatments that target specific biomolecular diseasecharacteristics
8 / 53
![Page 10: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/10.jpg)
What am I leaving out
Screening diagnostics
eg. cfDNA
Actionable prognostic biomarkers
eg. oncotypeDX
Often forgotten that the goal is to find actionable biomarkers
9 / 53
![Page 11: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/11.jpg)
Back to “Predictive Biomarkers”
Two common scenarios:
Developing a targeted treatment + diagnostic
Developing a new diagnostic, for an existing, non-targetedtreatment
10 / 53
![Page 12: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/12.jpg)
Targeted Treatments
30+ targeted cancer drugs1 with many different targets
The primary FDA-specified “biomolecular” indications were
I HER2/HR status
I KRAS/EGFR mutation
I BRAF mutation
Many with no “biomolecular indication”...
only approved in very specific cancer-types though!
(histology-based personalization!)
1from “Overview of FDA-approved Anti-Cancer Drugs Used for TargetedTherapy” WCRJ 2015; 2(3) e553
11 / 53
![Page 13: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/13.jpg)
The Road to Failure in Precision Medicine
Where have I seen little success?
Characterizing the [in]effectiveness of non-targeted treatments
Why do poor treatments tend not to work?
???
Why do I tend to miss free throws?
Because I keep forgetting to wear my lucky shirt...?
Or maybe because I’m generally bad at basketball...
12 / 53
![Page 14: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/14.jpg)
The Road to Failure in Precision Medicine
Where have I seen little success?
Characterizing the [in]effectiveness of non-targeted treatments
Why do poor treatments tend not to work?
???
Why do I tend to miss free throws?
Because I keep forgetting to wear my lucky shirt...?
Or maybe because I’m generally bad at basketball...
12 / 53
![Page 15: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/15.jpg)
The Road to Failure in Precision Medicine
Where have I seen little success?
Characterizing the [in]effectiveness of non-targeted treatments
Why do poor treatments tend not to work?
???
Why do I tend to miss free throws?
Because I keep forgetting to wear my lucky shirt...?
Or maybe because I’m generally bad at basketball...
12 / 53
![Page 16: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/16.jpg)
The Road to Failure in Precision Medicine
Where have I seen little success?
Characterizing the [in]effectiveness of non-targeted treatments
Why do poor treatments tend not to work?
Because they tend not to work...
Why do I tend to miss free throws?
Because I keep forgetting to wear my lucky shirt...?
Or maybe because I’m generally bad at basketball...
13 / 53
![Page 17: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/17.jpg)
The Road to Success in Precision Medicine?
What is the best place for statisticians on that road?
Is it building fancier methods?
(in some avenues things work pretty well with simple methods)
Or domain expertise?
Or some other option?
14 / 53
![Page 18: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/18.jpg)
Solve Easy Problems!
EE/CS does this well!
Very approximately solve useful + “easy” domain problems
Statistics seems to have more deep, but slow prodding phenotype.
sometimes the problems are messy...
15 / 53
![Page 19: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/19.jpg)
Now to the real talk!
Stratification-free Adaptive Enrichment Designs
16 / 53
![Page 20: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/20.jpg)
In Collaboration with...
17 / 53
![Page 21: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/21.jpg)
Biomarkers
Prognostic
Gives information on outcome independent of treatment
Generally not informative for treatment decisions
(Main effect term)
Predictive
Gives information on relative effectiveness of treatments
Informative for treatment decisions
(Interaction term)
18 / 53
![Page 22: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/22.jpg)
Developing a Drug and a Companion Biomarker
What is done now?
Have a targeted drug and potential clinical/genomic featuresof relevence.
Run a large clinical trial for general efficacy
Reserve a bit of α for sub-group exploration
Posthoc sub-group analysis
19 / 53
![Page 23: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/23.jpg)
Developing a Drug and a Companion Biomarker
Adaptive Enrichment Trial
Have a targeted drug and potential clinical/genomic features ofrelevence.
Concurrently
I Develop a biomarker to target treatment
I Enrich enrollment population based on biomarker
20 / 53
![Page 24: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/24.jpg)
An Adaptive Approach
Stratification
1. Break patients into arms by predefined biomarker
2. As the trial runs, discontinue arms which do not show benefit
3. At termination, cleverly test arms and determine beneficiarysubset
Strong Approach When
Biomarker is a simple score (1 dimensional)
Little adaptation is needed
21 / 53
![Page 25: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/25.jpg)
Stratification-free Enrichment
Strata simplify analysis but Strata are rigid
I “Beneficiary group” may not align well with strata
I Difficult to fine-tune
I Number of strata is exponential in number of features
Instead we use models...
22 / 53
![Page 26: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/26.jpg)
Stratification-free Enrichment
Strata simplify analysis but Strata are rigid
I “Beneficiary group” may not align well with strata
I Difficult to fine-tune
I Number of strata is exponential in number of features
Instead we use models...
22 / 53
![Page 27: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/27.jpg)
Example Biomarker Model
Something like
logit [pT (x)] = βT (0) +∑j
βT (j)xj
logit [pC (x)] = βC(0) +∑j
βC(j)xj
Make decisions based on
pT (x)− pC (x)
23 / 53
![Page 28: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/28.jpg)
The Plan
Enroll patients in blocks (k = 1, . . . ,K )
Use information from past blocks to develop/updatebiomarker
Use current biomarker to restrict enrollment in current block
Randomize new patients to arms (fixed weights)
At the end of the trial combine all the information to test null
24 / 53
![Page 29: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/29.jpg)
Frasian Statistics
1. Building biomarker/determining enrollment criteria Bayesian
2. Testing the null in a principled, model-free way Frequentist
25 / 53
![Page 30: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/30.jpg)
Testing the Null
Let’s begin at the end:
Assume
I we have a method for building/updating a biomarker
I and we can leverage this to restrict enrollment
How do we test the null?
pT (x) ≤ pC (x) for all x
26 / 53
![Page 31: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/31.jpg)
Testing the Null
Two designs which “obviously” work...
In reality only one controls type 1 error:
1. Permutation Test (within block)
2. Agglomerated z-test
27 / 53
![Page 32: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/32.jpg)
Permutation Test
Use the statistic
T = (# Treatment successes)− (# Control successes)
For null distribution of T , permute labels within strata.
28 / 53
![Page 33: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/33.jpg)
Agglomerated z-test
Use the statistic
T =∑
Tk
where
Tk =
√nkn
(p(T ,k) − p(C ,k)
2√
ppool ,k (1− ppool ,k) /nk
)(these are just weighted within block z-stats).
For null distribution of T , compare to N(0, 1)
29 / 53
![Page 34: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/34.jpg)
Testing the Null
Two designs which “obviously” work...
Which one controls type 1 error?
1. Permutation Test (within block) 7
2. Agglomerated z-test 3
30 / 53
![Page 35: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/35.jpg)
Testing the Null
Two designs which “obviously” work...
Which one controls type 1 error?
1. Permutation Test (within block) 7
2. Agglomerated z-test 3
30 / 53
![Page 36: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/36.jpg)
Failure of the Permutation Test
What goes wrong if we permute?
The prognosis of future patients contains information about theoutcome of previous blocks.
Thus, conditioning on the outcomes does not induce thepermutation distribution under the null.
31 / 53
![Page 37: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/37.jpg)
Failure of the Permutation Test
Consider a simple 2 block design:
Check if there is significance after the first block
I Yes? → enroll extremely poor (or good) prognosis group (lowvariability).
I No? → enroll only patients in middle (high variability)
Can easily see type 1 error in excess of 0.2
32 / 53
![Page 38: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/38.jpg)
Now for Part 1
How do we
I Construct/Update a Biomarker?
I Make Decisions about Enrollment?
I Decide treatment indication?
I Estimate treatment effect?
33 / 53
![Page 39: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/39.jpg)
Bayesian Modeling!
As a frequentist, those problems are hard.
Uncertainty in biomarker estimates needs to take sampling intoaccount
A Bayesian model (with a prior we believe) makes much of thisdisappear
We need to do this well enough that scientific inefficienciesdominate statistical inefficiencies.
34 / 53
![Page 40: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/40.jpg)
Decisions, Decisions, Decisions
How do we
Construct/Update a Biomarker?
35 / 53
![Page 41: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/41.jpg)
Constructing/Updating Biomarkers
Simple Version:
Assume
logit [pT (x)] = βT (0) +∑j
βT (j)xj
logit [pC (x)] = βC(0) +∑j
βC(j)xj
with a prior distribution Π on the βs
Update by considering Π (pT (x), pC (x)|past data)
Π (pT (x), pC (x)|past data) can ignore decision rule
36 / 53
![Page 42: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/42.jpg)
Decisions, Decisions, Decisions
How do we
Construct/Update a Biomarker?Condition on previous data
Make Decisions about Enrollment?
37 / 53
![Page 43: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/43.jpg)
Decisions, Decisions, Decisions
How do we
Construct/Update a Biomarker?Condition on previous data
Make Decisions about Enrollment?
37 / 53
![Page 44: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/44.jpg)
Making Enrollment Decisions
Need 2 things
I A utility function U(·)
I and a decision rule D which maximizes ED [U]
38 / 53
![Page 45: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/45.jpg)
Making Enrollment Decisions
What’s a good utility function?
Must combine power, and sensitivity:
One choice we like: expected future patient outcome
U (trial) =
∫X max E [pT (x)|trial] ,E [pC (x)|trial] dG (x) : reject∫X E [pC (x)|trial] dG (x) : reject
39 / 53
![Page 46: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/46.jpg)
Maximizing in Practice
Choosing the optimal rule is hard...
Instead we restrict to a simpler class: rules based on
Π (pT (x) > pC (x)|past data)
Still requires a lot of computational power; who do we turn to?
Very easy and cheap! (using MIT tool starcluster)
costs ∼ 1$/hour for 120 cores (if you are careful)40 / 53
![Page 47: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/47.jpg)
Decisions, Decisions, Decisions
How do we
Construct/Update a Biomarker?Condition on previous data
Make Decisions about Enrollment?Maximize a Utility Function
Decide treatment indication?
41 / 53
![Page 48: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/48.jpg)
Decisions, Decisions, Decisions
How do we
Construct/Update a Biomarker?Condition on previous data
Make Decisions about Enrollment?Maximize a Utility Function
Decide treatment indication?
41 / 53
![Page 49: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/49.jpg)
Deciding Treatment Indication
Who do we expect to benefit from treatment?
Treat those patients...
Ω = x st E [pT (x)|trial] > E [pC (x)|trial]
Can also take toxicity into account.
Could also use optimal rule from simpler class (eg. best linear rule)
42 / 53
![Page 50: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/50.jpg)
Decisions, Decisions, Decisions
How do we
Construct/Update a Biomarker?Condition on previous data
Make Decisions about Enrollment?Maximize a Utility Function
Decide treatment indication?Subset with expected posterior benefit
Estimate treatment effect?
43 / 53
![Page 51: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/51.jpg)
Decisions, Decisions, Decisions
How do we
Construct/Update a Biomarker?Condition on previous data
Make Decisions about Enrollment?Maximize a Utility Function
Decide treatment indication?Subset with expected posterior benefit
Estimate treatment effect?
43 / 53
![Page 52: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/52.jpg)
Estimating Treatment Effect
Given Ω; use expected posterior benefit∫Ω
(E [pT (x)|trial]− E [pC (x)|trial]) dG (x)
No selection bias (Assuming prior we trust)
Can substitute different (more conservative) prior here
44 / 53
![Page 53: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/53.jpg)
Decisions, Decisions, Decisions
How do we
Construct/Update a Biomarker?Condition on previous data
Make Decisions about Enrollment?Maximize a Utility Function
Decide treatment indication?Subset with expected posterior benefit
Estimate treatment effect?Expected posterior benefit
45 / 53
![Page 54: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/54.jpg)
Putting it all Together!
1. Choose a utility; and priors for pT , pC
2. Find an enrollment strategy to “optimize” this utility
3. Enroll patients in blocks according to:
I enrollment strategy using...
features, treatment labels and outcomes of previous blocks
4. At trial termination
I Test using blocked t-test
I Decide treatment indication based on posterior
I Estimate effect-size based on posterior
46 / 53
![Page 55: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/55.jpg)
Bayesian Vs Frequentist?
Biomarker Construction Bayesian
Enrichment Bayesian
Early Termination Bayesian
Testing the Global Null Frequentist
Estimating the Effect-size Bayesian
47 / 53
![Page 56: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/56.jpg)
Redesign of Bokemeyer et al.
Study of chemo vs chemo + targeted treatment (cetuximab)
Cetuximab targets EGFR
I In original trial only EGFR expression was considered
I The original trial did not find significance at 0.05 level.
I Later, tissue samples were re-assayed for KRAS mutation
I In KRAS wildtype-only samples, treatment effect was highlysignificant.
48 / 53
![Page 57: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/57.jpg)
Redesign of Bokemeyer et al.
I 337 patients equally randomized
I EGFR expression was a 3-level factor.
I Simulated trials: EGFR/KRAS and ORR based on empiricalrates from study.
I For adaptive enrichment, used 2 and 3 block designs withseparate working models
PT (ORR = 1|EGFR,KRAS) = βT0 + βT1EGFR + βT2KRAS
+ βT3(KRAS ∗ EGFR)
49 / 53
![Page 58: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/58.jpg)
Redesign of Bokemeyer et al.
Results averaged over 1000 simulated trials:
Non-Adapt 2 block 3 block
power 0.23 0.68 0.78true effect-size 0.06 0.21 0.21estimated effect-size 0.13 0.25 0.24
Effect-sizes were only calculated/estimated for successful trials.
Many more tables with discussion in the paper
50 / 53
![Page 59: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/59.jpg)
Take-aways
We can conduct a clinical trial
which controls type 1 error under the global null of no benefitfor anyone
in which we concurrently build a biomarker to determine thoselikely to benefit
and apply the biomarker to enrich our patient population (andincrease our power)
In addition, this biomarker is naturally constructed using aBayesian framework.
51 / 53
![Page 60: A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment Noah Simon July, 5 2017 1/53. ... Posthoc sub-group analysis 19/53. Developing a Drug](https://reader033.vdocuments.us/reader033/viewer/2022042202/5ea24d27d166c440bb4fab9c/html5/thumbnails/60.jpg)
The Papers
N. Simon, R. Simon, Adaptive Enrichment Designs for ClinicalTrials, Biostatistics 2013.
N. Simon, R. Simon, Using Bayesian Modeling in FrequentistAdaptive Enrichment Designs, Biostatistics 2017.
Cetuximab Study:
Bokemeyer and others, Fluorouracil, leucovorin, and oxaliplatinwith and without cetuximab in the first-line treatment of metasticcolorectal cancer, Journal of Clinical Oncology, 2009.
52 / 53