Submission of Microarray Data:Dealing Effectively with Data Quality Issues and Information Content Necessary to Develop an Expression Database

Advisory Committee for Pharmaceutical Science

June 10, 2003

Challenge: Over 50% of drug failures due to efficacy/tox.

Problem: 20 million US patients exposed to drugs withdrawn between Sept. ’97 and Sept. ’98

Problem: Only 1 in 10 IND’s become approved drugs -- Our current methods of candidate characterization are only 10% accurate

Solution: Bridge genomics and chemistry to broadly understand a compound’s effects in genomic terms

Vision and Challenge

The VisionBetter compounds submitted

Safer compounds submitted and approvedLower approval time

Assumptions -- Sponsor side

Sponsor providing data to support IND or NDA

Data is part of a larger package and not the sole and only evidence provided

Sponsor has on going array efforts with trained staff or will contract with CRO that meets these requirements

Assumptions – Agency side

Agency is willing to develop and train staff so that the data is meaningfully interpreted and a balanced view is taken

– Sponsor is concerned about overly reactive view – Sponsor is concerned about the future liability from public disclosure

Agency is able to accept data in a community defined standard format and has capacity to assess quality

- Agency is prepared for all current technologies - Agency will keep up with future technology improvements

Agency desires to deposit submitted data into an internal database for use by staff during future evaluations

Agency understands that context is essential for a balanced view

Array measurements similarities to traditional measurements, but…

ALT elevation is associated with liver damage- Treated group value is 55 ± 7 (SE) [range: 30-75], n=5- Control group value is 50 ± 4 (SE) [range: 15-110] n=15- No treated animals outside range of controls

– Conclusion: ALT not significantly changed by treatment, consistent with good liver safety.

Gene expression data on 5 RNAs associated with liver damage

- No treated animal outside range of controls– Conclusion: 5 RNAs associated with liver toxicity not significantly

changed, consistent with good liver safety.

RNA A RNA B RNA C RNA D RNA E

Ratio to control 0.9 1.0 1.2 0.8 1.4

SEN (Treated, Untreated)

0.1(5,15)

0.2(5,15)

0.1(5,15)

0.4(5,15)

0.5(5,15)

Range(significant at p<0.05)

1.1-0.8n.s.

1.2-0.7n.s.

1.4-0.9n.s.

0.6-1.1n.s.

1.7-0.9n.s.

… but, microarray data has differences from conventional measurements

Both agency and community has lower familiarity with technology

– Will improve with experience

Concern that the survey might uncover “confounding” factors- Sponsor concerned about a “overly reactive view”

- FDA concerned about sponsor missing important findings

Less “scientific agreement” about how to interpret and meaning

– Pattern matching is less familiar to biological community

– Requires a different mindset than single gene focus of the past

Perception that microarray data is lower quality and noisier

– Technology has improved

– Carefully conducted experiments are accurate and predictive

Summary: What should be provided by sponsor to FDA MIAME/MAGE-ML compliant descriptions of experiment(s) and

electronic submission of all data

Minimum experimental design metrics similar to that required for any other biological experiment

“Youth” of technology requires that additional quality data be submitted to demonstrate competency of experimenters

Sponsors interpretation of data in “scientific paper style” format

Comparison to community accepted RNA biomarkers and compared to Bench Mark Drugs and Toxicants

– Interpretation vs. context of current drugs failed drugs and toxicants

Sponsor provided data to assure QUALITYThemes Measurement vs. lab historical values

Measurement vs. external standards» RNA external standard

Measurement vs. internal standards» A few spike-in standards

•Different than other agency submissions due to “youth” of technology•Need to assure competency of experimenter•Need to assure internal/external consistency •Need to assure consistency with historical values

Experiment is complex but has several points where critical evaluations needed

Laboratory Information Management System (LIMS)

In VivoBiology

In VitroBiology

RNAIsolation

TargetPreparation Hybridization Data

Loading

Laboratory Information Management System (LIMS)

RNAIsolation

TargetPreparation Hybridization

286 steps needed to prepare microarray

Several quality control check points

Independent of platform

Experimental design minimums

Minimum of biological triplicate

Minimum of three untreated or mock or vehicle treated controls, processed contemporaneously

Minimum of three external standard RNA’s, processed contemporaneously

Minimum three spike-in RNA’s each sample

Treated

Untreated -Control

RNA standard External Control

3 Spike RNA’s

RNA used in experiments Ratio of 28S to 18S

• Mean• Standard Deviation• Range• Traces for all RNA samples

Data provided for each the following:

Samples in dataset

Historical for similar tissue/cell prepared in lab

RNA external standard, contemporaneous with dataset

Hybridization and array Q.C.

Data for experimental samples Array average signal to background ratio Array average background Array average raw signal Log dynamic range Average Raw signal intensity for each of three spike-in RNA’s

Comparative data (Mean, SD and range) Historical for similar samples (match for tissue or cell type) Historical average for RNA standard Historical average for each of spike-in RNA’s Average for contemporaneous RNA standard Average for contemporaneous controls

Experiment internal and external consistency

Experimental samples, correlation coefficient (CC) with

1. each other

2. contemporaneous controls

3. contemporaneous external standard RNA

4. historical external RNA standards

5. historical similar tissue or cell line samples

Mean, SD and range in all cases

Sponsor provided “Scientific literature style interpretation”

Abstract Significance relative to application Brief methods Summary of quality evidence described earlier Results Discussion Conclusion relative the application under consideration Conclusions in context with other drugs and toxicants

Report helps the agency help you

Summary: What should be provided by sponsor to FDA MIAME/MAGE-ML compliant descriptions of experiment(s) and

electronic submission of all data

Minimum experimental design metrics similar to that required for any other biological experiment

“Youth” of technology requires that additional quality data be submitted to demonstrate competency of experimenters

Sponsors interpretation of data in “scientific paper style” format

Comparison to community accepted RNA biomarkers and compared to Bench Mark Drugs and Toxicants

– Interpretation vs. context of current & failed drugs and toxicants

Internal FDA contextual databaseMicoarray technology requires stepping into the coming age of

electronic data submissions

Paper submission of microarray data is not useful

Used by agency to develop a better understanding of technology

Used by the agency to allow a look at the data in the context of other submissions

Contextual database is highly useful to provide meaning and balance to interpretation

– Same biology is present regardless of data preparation technology

Once the data is available to the agency how should they evaluate?

Objectives Promote balanced view of the data React to truly significant events Ground the analysis in the context of real world

effects of drugs, failed drugs, toxicants and standards

Need a reference database for these purposes

What is necessary to provide contextual grounding? Database containing …

Wide diversity of successful drugs, failed relatives, toxicants and standards

– Needed to understand pharmacologic mechanism– Needed to understand toxicity mechanism

Multiple tissues with dose and time context

Linkage of expression data to orthogonal data including:- Pharmacology including on and off target sites of action- Histology- Clinical Chemistry- Hematology- Chemical Structure

In vivo and in vitro data for successful bridging domains

Benefits of using a reference database to the agency Provides context to interpret events seen with

candidates – An example drawn from DrugMatrix

EGF-Receptor (p<0.01)

cKit-Oncogene(p<0.01)

BCL2 Oncogene(p<0.001)

Meloxicam

Torsemide

Nislodipine

Rosigolitazone

Norethindrone

Grenisteron

Torsemide

Clotrimazole

Valproic Acid

Simvastatin

Beta-estradiol

Carboplatin

Busulfan

Clofibric Acid

Melatonin

Sparfloxacin

Withdrawn Diethylstilbesterol

Oncogenes elevated by approved drugs in liver

Context promotes balanced view of data

Summary: What should be provided by sponsor to FDA MIAME/MAGE-ML compliant descriptions of experiment(s) and

electronic submission of all data

Minimum experimental design metrics similar to that required for any other biological experiment

“Youth” of technology requires that additional quality data be submitted to demonstrate competency of experimenters

– Additional quality data beyond that required of established technologies

Sponsors interpretation of data in “scientific paper style” format

Comparison to community accepted RNA biomarkers and compared to Bench Mark Drugs and Toxicants

– Interpretation vs. context of current drugs and toxicants

Conclusions Looking forward Micro Array technology is ready to contribute today

– Simple assurances of quality are needed

– Contextual databases to allow meaningful interpretation are available

Developing understanding and consensus around meaningful RNA biomarkers

Requirements beyond normal verification of data quality will diminish as community sophistication using technology improves

Analysis of data collected off several platforms is quite possible

- Same biology is found regardless of the platform

Clinical applications to accessible human tissues will become common

Improved predictive power of animal studies making truly predictive animals models a reality

Addresses the Problem: Patients exposure to drugs which are subsequently withdrawn

Addresses the Problem: Only 1 in 10 IND’s become approved drugs -- Our current methods of candidate characterization are only 10% accurate

Result

Helps realize the VisionBetter compounds submitted

Safer compounds submitted and approvedLower approval time