Emerging Diagnostic Technologies:
Proving The Clinical Application
Through Gold Standard Data
Lyssa Friedman
1Lyssa Friedman3/25/14
OutlineAsking the right questions: a stair-step approach to product development and selecting the right technical platform
Focus on novel diagnostics
Key components for success
Case studies
The role of clinical operations
3/25/14 Lyssa Friedman 2
Next Generation Sequencing (NGS)
2013
4Lyssa Friedman3/25/14
Nho, K. et al. Whole-exome sequencing and imaging
genetics identify functional variants for rate of change
in hippocampal volume in mild cognitive impairment
Braggio E, et al. Lessons from next-
generation sequencing analysis in
hematological malignancies
Mille FA, et al. Testing personalized
medicine: patient and physician
expectations of next-generation genomic
sequencing in late-stage cancer care
NGS in the news
3/25/14
Lyssa Friedman
5
Mar 17 2014
Mar 18 2014
Jan 14 2014 (Regeneron)
Mar 8 2012 (Complete Genomics)
Jan 26 2014
Oct 24 2013
NGS in the regulatory landscape
3/25/14 Lyssa Friedman 6
First high-throughput DNA sequencing analyzer
to receive U.S. Food and Drug Administration
(FDA) premarket clearance
MiSeqDxNov 2013
http://www.illumina.com
NGS in the clinic
3/25/14 Lyssa Friedman 7
• Indication: thyroid cancer
• ThyroSeq – 12 genes
• Ion AmpliSeq™
• Cystic fibrosis (CF), two products:
• CF 139-Variant Assay – 139 variants
• CF Clinical Sequencing Assay – additional variants
• Prenatal testing: verifitest – 7 fetal chromosomal abnormalities
• MiSeqDx
• Two products/indications:
• NeoTYPE MDS/CMML Profile – 16 genes, myelodysplastic
syndrome/chronic myelomonocytic leukemia (MDS/CMML)
• Solid tumor profile – 48 driver genes, multiple tumor types
• 7 common KRAS mutations, multiple cancer types
+ Laboratory Developed Tests (LDTs)
+ New platform for existing clinical indication
± Lower cost of goods sold (COGS)
Back to basics: What is the right tool for the job?
Honda Civic Jaguar XJ
Does the job for the price ✚ ✚
Safe and compliant ✚ ✚
Sexy and stylish ✚
The next best thing ✚
Because I can afford it ✚
3/25/14 Lyssa Friedman 8
Othertechnologies
NGS
✖
Choose the technical platform that suits the
business, clinical and performance needs
The tail wagging the dog
Broad clinical question
Target biomarkers
Selected platform
Business case
Clinical need
Reimbursement strategy
Commercial strategy
3/25/14 Lyssa Friedman 9
Avoid retrofitting the business, clinical, reimbursement and
commercial strategies to an early platform or assay decision
Instead, follow a stair-step approach
3/25/14 Lyssa Friedman 10
Business case
Clinical assessment
Biomarker
discovery
Risk assessment
•Technical
•Regulatory
Technical platform
and assay selection
Product
development
Product
launch
Setting the stage for successful product
development and launch
3/25/14 Lyssa Friedman 11
Define
• Market
• Unmet clinical need
• Gold standard Conduct early discussions
• Regulatory strategy
• Commercial strategy
• Reimbursement strategy Stay focused
• Clinical indication
• Discovery or in-licensing plan Remain agnostic regarding technical platform and assay requirements –
choose based on
• Clinical and market needs
• Test performance requirements
• Regulatory risks
• Technical risks
• COGS Allow the chosen technical platform and assay to serve the clinical,
business, commercial and reimbursement case
Consider these key components early
Gold standard
Regulatory strategy
Study design
Publication strategy
Reimbursement strategy
3/25/14 Lyssa Friedman 12
LDT vs FDA
3/25/14 Lyssa Friedman 13
LDT FDA
Companion diagnostics✔
Kit for distribution✔
Sole-source CLIA lab✔ ✔
Define the gold standard
• How do key stakeholders define the gold standard?– Physician community
– Literature/guidelines
– FDA
– Payers
• Is local diagnosis sufficient or is “expert” diagnosis needed? For expert diagnosis:– Individual vs. panel
– Blinded vs. conference
– Expert name recognition
• Apply gold standard label early in the discovery process – avoid waiting for development phase – Risk that labels will change changes in performance
3/25/14 Lyssa Friedman 14
Publications drive reimbursement
Well-designed studies with pre-defined
• Samples
• Clinical annotation
• Gold standard label
• Performance requirements
High-impact peer-reviewed publications
Positive coverage decisions
3/25/14 Lyssa Friedman 15
Studies Publications ReimbursementThe ACCE Evaluation Process for Genetic Testing
3/25/14 Lyssa Friedman 16
http://www.cdc.gov/genomics/gtesting/EGAPP/index.htm
http://www.cdc.gov/genomics/gtesting/ACCE/index.htm
• ACCE Model Project:
• Analytic validity
• Clinical validity
• Clinical utility
• Ethical/legal/social
• Publications in these areas
help build the reimbursement
case
Centers for Disease Control (CDC) Office of Public Health Genomics
Evaluation of Genomic Applications in Practice and Prevention (EGAPP™) Initiative
Three success stories
3/25/14 Lyssa Friedman 18
Publications
LDTMulti-
gene
Novel/
licensedNGS?
Clinical
validation
Analytic
verification
Clinical
utility
Cost
effectiveness
Genomic
Health
OncotypeDx✔ ✔ Novel No ✔ ✔ ✔ ✔
Veracyte
Afirma Thyroid
FNA Analysis✔ ✔ Novel No ✔ ✔ ✔ ✔
Illumina
verifi prenatal
test✔ ✔ Licensed Yes ✔ ✔ ✔ ✔
Cancer assays
Risk score (continuum)
3/25/14 Lyssa Friedman 19
•Samples: formalin-fixed paraffin-embedded tissue
•Platform: Reverse transcription polymerase chain reaction (RT-PCR),
mRNA
•Gold standard: centralized pathology
•Regulatory: CLIA LDT
Breast cancer
Risk of
recurrence
21 genes
Colon cancer
Risk of
recurrence
12 genes
Prostate cancer
Disease risk
assessment
17 genes
Bast RC & Hortobagyi. N Engl J Med 2004
Genomic Health Company Timeline
3/25/14 Lyssa Friedman 20
FFPE: formalin-fixed paraffin-embedded; RT-PCR: reverse transcription polymerase chain reaction; NSABP: National Surgical Adjuvant
Breast and Bowel Project; NCI: National Cancer Institute; ASCO: American Society of Clinical Oncology; SABCS: San Antonio Breast Cancer
Symposium; NEJM: New England Journal of Medicine; NCCN: National Comprehensive Cancer Network; DCIS: ductal carcinoma in situ
• Series A
• RNA extraction
protocol
• TaqMan (RT-PCR)
• Proof of concept
• Series B&C
• Commercial
strategy
• Series D
• ASCO,
SABCS
• Clinical
validation
• CLIA lab
• Breast assay
launch
• Clinical
validation
publication
(NEJM)
• Economic/cost
effectiveness
publication
• IPO
• Medicare
• Aetna
• Colon assay
development
• United
Healthcare
• NCCN
guidelines
• Colon assay
clinical
validation
• Breast assay
clinical utility
publications
• Colon assay
launch
• Social media
campaign
• Prostate
program
• Smartphone
app
• Breast NGS
data
• DCIS test
launch
• Prostate
clinical
validation
• NGS in
development
2000 ‘01 ‘02 ‘03 ‘04 ‘05 ‘06 ‘07 ‘08 ‘09 ‘10 ‘11 ‘12 ‘13
• Prostate
launch
http://www.genomichealth.com, http://www.oncotypedx.com
Suspicious thyroid neoplasm
Benign vs continued suspicious
result
3/25/14 Lyssa Friedman 21
Thyroid neoplasm with
indeterminate cytopathology
142 genes
•Samples: RNA-preserved fine needle aspirate biopsy
•Platform: Custom microarray (Affymetrix), mRNA
•Gold standard: centralized pathology
•Regulatory: CLIA LDT on FDA-cleared microarray platform
Alexander EK, et al. N Engl J Med 2012
Veracyte Company Timeline
3/25/14 Lyssa Friedman 22
• Series A
• RNA extraction
protocol
• Clinical validation
data
• Clinical validation
publication
• Series B
• Thyroid benign classifier
launch
• Cost effectiveness
publication
• Clinical validation
publication (NEJM)
• Analytic verification
publication
• Clinical utility
publication
• Medicare
• NCCN guidelines
• 2nd clinical utility
publication
• Gene discovery IPF
• United Healthcare
• Aetna
• IPO
2008 2009 2010 2011 2012 2013 2014
• Thyroid
malignancy
classifier launch
NCCN: National Comprehensive Cancer Network; IPF: idiopathic pulmonary fibrosis
http://veracyte.com, http://afirma.com
Fetal chromosomal abnormalities
3/25/14 Lyssa Friedman 23
(Verinata)
Blood screen
7 fetal chromosomal abnormalities
•Samples: maternal blood
•Platform: massively parallel sequencing (MPS) of maternal plasma
cell-free DNA (cfDNA)
•Gold standard: previously existing and accepted prenatal aneuploidy
screening
•Regulatory: CLIA LDT on FDA-cleared DNA sequencing analyzer
Bianchi DW, et al. N Engl J Med 2014
Verinata Company Timeline
3/25/14 Lyssa Friedman 24
• Stanford license
• Clinical validation
publication
• Clinical utility publication
• Name change (fr. Artemis
Health)
• Series C
• Launch
• Product expansion
• Cost effectiveness
publication
• Fundraising
• Series D
• Acquisition (Illimuna)
2009 2011 2012 2013 2014
• Clinical validation
publication (NEJM)
http://www.illumina.com, http://www.verifitest.com
Poised for success:
Stay laser-focused on goals
Take a stair-step approach
Novel diagnostics
3/25/14 Lyssa Friedman 25
GoalsStaying on target is a continuous process
3/25/14 Lyssa Friedman 26
Clearly define
•Success
•Milestones
•Timelines
•Risks Stay
•Focused
•Flexible
•Nimble Communicate with transparency
•Speedbumps
•Roadblocks
•Brick walls Mitigate risk
Question assumptions (avoid drinking the Kool-Aid)
Learn quickly from failures
Successful companies choose the technical platform
that suits business, clinical and test performance needs
3/25/14 Lyssa Friedman 27
Business case
Clinical assessment
Biomarker
discovery
Risk assessment
•Technical
•Regulatory
Technical platform
and assay selection
Product
development
Product
launch
While managing many of these processes
non-sequentially and simultaneously
3/25/14 Lyssa Friedman 28
Business case
• Market
– Who are your key customers?
– What is the clinical indication?
– Where is your geographical target over the next 3 years? 5 years?
10 years?
– What is the payer landscape?
• Early consideration of risks
– Who are your investors?
– What return do your investors expect on their investment in 3 years?
5 years? 10 years?
– Who are your competitors?
– What is the regulatory landscape? Potential minefields?
– What is the technical landscape? Potential minefields?
• Performance requirements and gold standard– What does the market need in order to trust the test
• Equipment and reagents– How competitive is the space? FDA-cleared vs.
research use only (RUO)? How will this affect• Supply agreements: cost, room to negotiate
• Reagents: cost, availability, stability, need for verification
• Software verification and validation (V&V)– What capital purchases and internal resources are
needed?
– How will V&V affect time and cost?
3/25/14 Lyssa Friedman 29
Business case, cont’d
• Unmet clinical need
– What is the unmet clinical need?
– What are the pain points? How can you ease your
physician customer’s pain?
– What is the current clinical workflow? How can you
minimize disruption of that workflow?
– What will it take to engender physician trust in your
test? Performance? Gold standard? Publications?
Guidelines? Influential KOLs? FDA-clearance?
3/25/14 Lyssa Friedman 30
Clinical assessment
• Gold standard
– Is there label consistency from discovery through
development?
– Is there stakeholder buy-in for the gold standard?
• Physician community
• Literature/guidelines
• FDA
• Payers
3/25/14 Lyssa Friedman 31
Clinical assessment, cont’d
• Target sample: lessons learned from prospective
sample collection
– How are they collected and handled?
– Where are they collected? Clinic, radiology, pathology?
– What is the impact on sample collection on the patient?
Special procedures, extra visits, discomfort, anesthesia?
– How disruptive will collection and handling be to current
workflow?
– Will special training be needed?
– Will special storage facilities be needed?
– How many specialties will need to be involved? Clinic
physician, radiology, pathology?
3/25/14 Lyssa Friedman 32
Clinical assessment, cont’d
• KOLs – how are they selected and engaged?– Protocol development
– Development of key clinical questions
– Publications
– Guideline committees
– Influence: US/ex-US
– Study participation
• Non-KOL investigators– Peer influence
– Early adopters
– Patient population
– Clinic access - workflow
3/25/14 Lyssa Friedman 33
Clinical assessment, cont’d
• Clinical utility: so what?
– Patient outcomes
– Societal cost-effectiveness
– Savings to payers
• Performance requirements
• Publication strategy
• Reimbursement strategy
3/25/14 Lyssa Friedman 34
Clinical assessment, cont’d
Disease
Test Present Absent Total
PositiveTrue positive
A
False positive
BA + B
NegativeFalse negative
C
True negative
DC + D
Total A + C B + D A + B + C + D
SensitivityA
A+CProbability test positive when disease present
SpecificityD
B+DProbability test negative when disease not present
Positive predictive value
PPV
A
A+BProbability disease present when test positive
Negative predictive value
NPV
D
C+DProbability disease absent when test negative
PrevalenceA+C
A+B+C+D
Performance requirements: what type of test?
3/25/14 Lyssa Friedman 35
Performance requirements: things to
think about
• Will the test performance lead to actionable patient results?
– What difference does the test make?
• What performance will the community accept?
– What are the barriers to uptake? How much education will the community need in order to understand test performance?
• How well does the gold standard perform?
– The test cannot outperform the gold standard
3/25/14 Lyssa Friedman 36
Publication and reimbursement strategy
3/25/14 Lyssa Friedman 37
Publications
Protocol
Clinical validation study
Recognized gold standard
Independent test set
Clinical
validity
Analytic
validity
Clinical
utility
Clinical
guidelines
Positive
coverage
decisions
Samples
Discovery
Samples
Study planning
Samples
Analytic testing
Samples
Validation
3/25/14 Lyssa Friedman 38
•Simultaneous and interdependent processes
•Continuously re-assess risks and re-focus priorities
throughout the continuum
Processes are not sequential
• Use same gold standard from discovery through development
– Changes can affect test performance
• Continue other activities in parallel
– Stability and kit testing
– Planning: validation, product development, reimbursement, publication and commercial strategies
• Ongoing and continuous risk assessment with focused and aggressive timelines
– Building the bicycle while riding it (downhill at 60 mph!)
3/25/14 Lyssa Friedman 39
Biomarker discovery
• Will platform change from discovery to
product development? If so, anticipate
changes, with associated delays, to
– Gene set
– Equipment and reagents
– Algorithm development
– Test performance
3/25/14 Lyssa Friedman 40
Technical risk assessment
• LDT vs FDA clearance– LDT
• CLIA lab: buy or build?
• Equipment
• Documentation
• Staffing
– FDA• Documentation – design control
• Early meetings with agency
• Kit and partnership strategies
• Equipment and reagents– FDA-cleared vs RUO
3/25/14 Lyssa Friedman 41
Regulatory risk assessment
• Clearer platform decision at this stage with
a stair-step approach
– FDA-cleared vs. RUO
– Reagents
– Cost of goods
– Regulatory pathway
3/25/14 Lyssa Friedman 42
Technical platform and assay selection
What assay and technical
platform?
3/25/14 Lyssa Friedman 44
RT-PCR: reverse transcriptase polymerase chain reaction; FISH: fluorescent in
situ hybridization; SKI; spectral karyotype imaging; cDNA: complementary DNA
Genomic information Indications
PCRMultiple copies of targeted DNA
sequenceLarge DNA targets
RT-PCR Amplified RNARNA viruses
mRNA or large DNA translocations
qPCR Real-time PCRReal-time quantification of multiple
copies
FISHOligonucleotide probes attached to
cDNA targetOncogene amplification, translocations
Microarrays Thousands of transcripts and probes
Gene expression “fingerprints”
Single nucleotide polymorphisms
(SNPs)
Sequencing Precise order of nucleotides within DNAMolecular biology, evolutionary biology,
metagenomiccosts
Netto GJ, et al. Proc (Bayl Univ Med Cent) 2003
NGS: things to think about
• Equipment is costly – can it be leveraged across projects in lab?
• Computational challenges
– Sequencing produces raw data requiring assembly into longer sequences
– Sequences may not be assigned to specific chromosomes
• When transferring existing assay to new platform full validation is needed – technology transfer not sufficient
3/25/14 Lyssa Friedman 45
• Transfer assay from R&D – conduct studies on platform to be used commercially
• LDT: conduct studies in CLIA lab by approved personnel– Stability – collection, handling transport
– Analytic sensitivity – input, dilution
– Analytic specificity – interfering substances
– Assay robustness
– Reproducibility
– Limits of detection
– Quality control
3/25/14 Lyssa Friedman 46
Product development
• Clinical sites = early adopters
• Partner with commercial group for
knowledge transfer
– Customer pain points and workflow
– Nuances of patient population and physician
specialty
– Key contact people at customer sites
3/25/14 Lyssa Friedman 47
Product launch
Clinical operations: not just for accruing
samples…
Sample accrual
Pristine annotation
Gold standard labels
Thorough data management and data auditing
Prospective protocols conducted according to Good Clinical Practices
3/25/14 Lyssa Friedman 49
…We are the first ear to the voice of the
customer and thus a key to success
50Lyssa Friedman3/25/14
• Articulate the clinical question– Unmet need
– Pain points
– Workflow challenges
• Define the gold standard
• Grow influential KOL relationship, which are critical to– Publications
– Physician uptake
– Guideline acceptance
• Cultivate PI/site relationships: early commercial adopters
• Develop critical protocols
• Assess and mitigate risk
• Define clinical utility
• Publish abstracts and manuscripts that drive guideline and
physician uptake and reimbursement