1

FDA Industry WorkshopFDA Industry WorkshopStatistics in the FDA & IndustryStatistics in the FDA & Industry

The FutureThe Future

David L DeMets, PhD

Department of Biostatistics & Medical Informatics

University of Wisconsin School of Medicine & Public Health

2

TopicsTopics

• Training/Certification Needs

• Academic/Industry Collaborations

• Attack on Clinical Trials & Statistics

• CT Costs & Data Management

• Statistical Methodology Issues

3

Globalization of Clinical TrialsGlobalization of Clinical Trials

• Rate of discovery increasing• Translational into practice is not fully

realized– Screening– Prevention– Treatment

• Declining Recruitment in US • More trials becoming multinational

4

Common CoreKnowledge

ClinicalTrialist

Clinician

Statistician

BehavioralScientist

ClinicalPharm

NIH Roadmap: NIH Roadmap: Discipline of Clinical ResearchDiscipline of Clinical Research

5

Clinical Research Training:Clinical Research Training: a multidisciplinary workforce a multidisciplinary workforce

• In USA, number of clinical researchers is not increasing

• Previous training “on the job”, sort of “trial and error” approach

• Rigorous training programs in USA are just starting – NIH Roadmap Initiative

• Many disciplines now involved in clinical research without formal training in this science

• Threat of the “silver tsunami”– 40% of Clinical Researchers in USA over age 50

• World wide training challenges

6

Training Pyramid in Training Pyramid in Patient-Oriented ResearchPatient-Oriented Research

PhD

MS Degree

Certificate Degree

Workshops

7

Biostatistician CrisesBiostatistician Crises

• Increasing demand for statistician/biostatisticians in academia, industry & government

• Supply of MS and especially PhD trained biostatisticians relatively constant over past two decades

• Domestic students in biostatistics in very short supply

• Crises not fully appreciated

8

Academic – Industry CT Academic – Industry CT PartnershipsPartnerships

• Industry CT funding levels similar to NIH • Need to continue developing relationships• Can be a win-win for all Phases I, II & III• Four key elements

– Independent Steering Committee– Independent Statistical Center– Independent Data Monitoring Committee– Freedom to publish

• Journals beginning to require investigator independence

9

Central Units (Labs, …)

Clinical Centers

Patients

Data Management Center (DMC)

Sponsor

Institutional Review Board

Independent Data Monitoring

Committee (IDMC)

Steering Committee

Statistical Analysis Center (SAC)

Regulatory Agencies

A Clinical Trial ModelA Clinical Trial Model

10

Challenge: Attack on Clinical Challenge: Attack on Clinical Trials & StatisticsTrials & Statistics

• Pending Congressional Legislation

• Wall Street & WSJ

• Some Patient Advocacy Groups

11

Senate Bill 1956Senate Bill 1956

• A proposed amendment to Federal Food, Drug & Cosmetic Act

• Known as the ACCESS Ammendment

• A three tiered approval system

• More responsive to “the needs of seriously ill patients”

12

Proposed Three Tier ApprovalProposed Three Tier Approval

• Tier I– Based on Phase I information– Based on clinical, not statistical analysis– May require post approval studies

• Tier II– Based on surrogates or biomarkers

• Tier III– Traditional requirements

13

Some Issues in Some Issues in Proposed LegislationProposed Legislation

• Challenge of placebo controlled studies

• De-emphasize statistical analysis-no disapprovals solely on the basis of statistical analysis or 95% CIs

• Evidence may be based on uncontrolled studies such as case histories, observational studies, mechanism of actions, computer models…

• Outcome data may be a surrogate or biological marker

14

CT Statistical Methodology IssuesCT Statistical Methodology Issues

• Surrogate Outcomes

• Composite Outcomes

• Non-inferiority Designs

• Adaptive Designs

• Gene Transfer Designs

• Safety Monitoring

15

Surrogate Response Variables Surrogate Response Variables • Used as a substitute for Clinical Endpoint

• May lead to smaller or shorter studies

• Requirements (Prentice, 1989)T = True clinical endpoint

S = Surrogate Z = Treatment

• Sufficient Conditions1. S is informative about T (predictive)

2. S fully captures effect of Z on T

• Concern:– Correlation is not Causation

– Pathways often more complex

– Other side effects not seen

16

Failures of Potential SurrogatesFailures of Potential Surrogates

• Nocturnal Oxygen Therapy Trial (NOTT)– 24 vs 12 hour oxygen in COPD patients– Pulmonary Function tests (NS)– Survival (p<0.001)

• CAST– Patients with cardiac arrhythmias– Arrhythmias suppressed– Terminated with increased mortality

• Ref (Fleming & DeMets, Annals Intern Med, 1996)

17

Failures of Potential SurrogatesFailures of Potential Surrogates

• Inotropic Drugs in Heart Failure– Improved heart function but increased

mortality– PROMISE, PROFILE, VEST,….

• Lipid lowering but no survival benefit– Women’s Health Initiative & HRT– Increased risk of clotting (PE, DVTs)

• Ref (Fleming & DeMets, Annals Intern Med, 1996)

18

Composite Endpoint Composite Endpoint RationaleRationale

• Defined as having occurred if any one of several components is observed– e.g. death, MI, stroke, change in severity,…..

• May reduce Sample Size by increasing event rates– Assumes each component sensitive to

intervention– Otherwise, power can be lost

• May avoid competing risk problem– Death is a competing risk to all other morbid

events, probably not independent

19

Problems with Problems with Composite OutcomesComposite Outcomes

• Interpretability if individual components go in different directions– e.g. WHI global index–

• Death: similar• Fractures: positive• DVTs, PEs: negative

• Relevance of a mixed set of components– Trials are adding softer outcomes

• Could have a loss of power if some components not responsive

• Failure to ascertain components

20

Non-Inferiority DesignsNon-Inferiority Designs

• Design to compare a new intervention with an accepted/proven standard– “As good as” with respect to a primary– Has some other advantage (cost, less toxic, less

invasive,…..)

• Must define a degree of non-inferiority or indifference, δ– Choice is somewhat arbitrary– Absolute or relative scale

21

Difference in EventsTest – Standard Drug

(Antman et al)

22

Non-Inferiority MethodologyNon-Inferiority Methodology

a) Comparison: New Treatment vs. Standard:RRa

Upper CI must be less than δ

b) Estimate of standard vs. placebo: RRb Based on literature

c) Imputed effect of New Trt vs. placebo (RRc)

RRc = RRa x RRb

23

Challenges for Non-Inferiority Challenges for Non-Inferiority DesignsDesigns

• Current paradigm makes all non-inferiority trials vulnerable

• Relevance of standard vs placebo historical estimate

• Fraction of standard benefit to be retained

• Choice of δ for current trial

24

Adaptive DesignsAdaptive Designs• Many Adaptive Designs in Use

– Baseline Driven (based on risk profile)– Total Event Driven Designs– Group Sequential Designs

• Benefit or Harm• Futility

– Drop the Losing Arm

• Statistical & Logistical issues worked out for these

• Not a Frequentist vs Bayesian Issue

25

Adaptive DesignsAdaptive Designs

• Adjusting design during trial– Sample size– Primary outcome

• Current interest very high• A need exists to be adaptive or flexible• Some statistical methods developed• Still many statistical debates• Many remaining issues related to logistics

& potential for introducing bias

26

Monitoring of Clinical TrialsMonitoring of Clinical Trials

• Shalala– Death of gene transfer patient– NEJM (2000)– Press Release (2000)

• IRBs often not provided sufficient information to evaluate clinical trials fully

• NIH will require monitoring plans for Phase I, II and III trials - guidelines

• FDA issued guidelines for Data & Safety Monitoring Boards and IRBs (2001, 2005)

• Post Cox II issues– Rapid access vs long term safety

27

IRB Safety IRB Safety Monitoring ProblemMonitoring Problem

• IRBs review trial design and ethics

• IRBs responsible for patient safety

• Drowning in SAE reports, not useful

• Inadequate infrastructure to be able to provide adequate safety monitoring

• For some multicenter trials, an alternative process exists (i.e. DMC)

• For single center trials, patient “safety” monitoring provided is now inadequate

28

Safety & Observational DataSafety & Observational Data

• Long term RCT follow-up for low rate SAEs not common

• Have turned to observational data as a supplement

• Serious limitations to argue causality due to confounding and bias

• Statistical analysis can take us only so far• Need to understand better what can be

learned

29

Reducing Trial Costs Reducing Trial Costs

• DCRI Workshop: Hypothetical Trial Example– 60-70% of cost site related, half due to site

monitoring– Could reduce costs 40% by reducing CRFs

& monitoring site visits• DCRI CT example: Ongoing site monitoring

improved regulatory compliance but little on trial data results & conclusions

• Breast Cancer Fraud Case – Academic network; Intense audit did not alter the results (<1% error), NEJM 1995

30

Need for Change in Need for Change in Site MonitoringSite Monitoring

• Current system is “out of control”• Educate/train clinical sites & investigators• Focus data collected & limit the extraneous• Set priorities on monitoring key variables:

– eligibility– primary and secondary outcomes, – serious adverse events (SAE)

• Sample audit the rest• Use more statistical QC methods• Standardize CRFs and data management

31

Challenge: Gene Transfer TrialsChallenge: Gene Transfer Trials

• NIH Re-Combinant Advisory Committee (RAC)

• RAC reviews new gene transfer trials• Mostly very early phase studies• Designs often not appropriate

– No objectives clearly stated– Borrowed from other settings that are not

relevant• Design guidelines need further

development

32

SummarySummary

• With current discovery rate, future appears very promising

• Significant challenges exist• Most are solvable but will require

collaboration from academia, regulators & sponsors

• Failure is not an option – we need evidence based medicine

• Every challenge is an opportunity