trends and innovations in quantitative safety evaluation–bias and multiplicity –statistical...

Quantitative Safety Evaluation

Trends and Innovations in Quantitative Safety Evaluation

Aloka Chakravarty, Ph.D. Division of Biometrics VII (DBVII) Office of Biostatistics, CDER, FDA [email protected]

The views expressed by the speaker of this talk are her own and do not necessarily

represent those of FDA

TICTS - Quantitative Safety Evaluation Aloka Chakravarty

Outline

• Introduction - FDA, Office of Biostatistics and DBVII

• Advances in Meta-Analysis for Safety assessment

• International collaboration in Sentinel Initiative

• Trends in Observational Studies

• Benefit-Risk Framework

• Concluding remarks

2

10 10 70 170 40 10

300+ Statisticians at FDA and Growing!

7


DBVII - Focus Areas

• Focus on the quantitative assessment of safety • Development phase of product development • Pre-market risk assessment • Post-market risk assessment

• Expertise in areas unique to the safety assessment • Meta-analysis • Randomized trials primarily to evaluate safety • Design and analysis of observational studies • Graphical and computational methods • Analyses of registry and health care databases

5


Outline







6


PDUFA V Goal: Advancing the science of meta-analysis methodologies

• Develop a dedicated resource for meta-analysis in the FDA regulatory context

• Hold a public meeting to discuss and to obtain input on the use of meta-analyses in the FDA regulatory context

• Publish a draft guidance on FDA’s intended approach to the use of meta-analyses in the FDA’s regulatory review process (by end of FY 2015)

• Publish a final or revised draft guidance within 1.5 years of the close of the public comment period

7


Public meeting: meta-analyses in the FDA regulatory context

• Took place on November 25, 2013 at FDA Campus

• White paper available prior to meeting

• Invited panel and opportunities for public to participate

• Four focused sessions – Bias and multiplicity

– Statistical Methods

– Individual Study Quality

– Overall Meta-Analysis Quality

• Material available at: http://www.fda.gov/Drugs/NewsEvents/ucm370686.htm

Levenson CIOMS WG X February 2014 Meeting

8


Personal observations from workshop (1 of 3)

• Pre-specification of protocol and analysis plan is key

• Protocol and analysis plan should be placed in public repository prior to conduct of MA

• Traditional statistical significance and multiplicity considerations may not be feasible in safety MA

• Consideration of effect sizes, sensitivity analyses, and subgroups are important in evaluating MA

• Outcome and exposure definitions should be specific and have good ascertainment


9


Personal observations from workshop (2 of 3) • Patient-level data is important

• Might be advantageous to focus on well-defined set of high-quality trials

• Other trial inclusion issues: index trial, safety trials, small trials issues

• For safety, there is a preferences for fixed-effect models


10


Personal observations from workshop ( 3 of 3) • Traditional weighting methods such as inverse

variance and DerSimonian and Laird may not be appropriate for safety data

• Bayesian methods may offer some advantages

• Different situations call for different efforts in meta-analysis


11


NSAIDs and CV risk AC meeting

• February 10 and 11, 2014

• Discussed class labeling of NSAIDs and CV risk

• New information

– Meta-analysis of RCTs, direct and indirect comparisons

– Observational studies

• Discussed subtleties of using meta-analysis of heterogeneous trials

– Duration, dose, comparator, inclusion criteria

• Meta-analysis? Dedicated safety trial? Observational studies?


12


Outline







13


Evaluation of Post-Market Safety Issues

Signal detection

Hypothesis testing in a

formal pharmaco-epi study

Signal validation

Sentinel

Signal validation

Signal refinement

14


Potential Capabilities of Sentinel

• Improving FDA’s capability to identify and evaluate safety issues in near real time

• Enhancing FDA’s ability to evaluate safety issues not easily evaluated with the passive surveillance systems currently in place – Expanding FDA’s access to subgroups and special

populations (e.g., the elderly) – Expanding FDA’s access to longer term data – Expanding FDA’s access to adverse events occurring

commonly in the general population (e.g., myocardial infarction, fracture) that tend not to get reported to FDA through its passive reporting systems

15


Components of Sentinel Initiative

• Mini-Sentinel

• Observational Medical Outcomes Partnership (OMOP) – now called IMEDS

• Federal Activities

– Federal Partners Collaboration

– Federal Partners Working Group

• International activities

17


International Discussions (1)

• Europe – European Network of Centers for Pharmaco-epidemiology and

Pharmaco-vigilance (ENCePP)

• Create a “network of excellence” consisting of research and medical-care centers, healthcare databases, electronic registries and existing networks to strengthen post-marketing monitoring to facilitate the conduct of safety related post-approval studies

– IMI/PROTECT • To develop and validate tools and methods that will enhance AE data

collection, active signal detection, create standards for pharmaco-epidemiology studies, and means to integrate all data know about a product for evaluation of risk : benefit

– EU-ADR • Design, develop and validate a computerized system that exploits data

from electronic healthcare records and biomedical databases for the early detection of adverse drug reactions; complementary to existing systems, have more power and detect signals earlier

18


International Discussions (2)

• Canada – Drug Safety and Effectiveness Network (DSEN)

• Enable research by linking researchers through a new virtual network, creating a national agenda of research based on priorities identified by decision-makers, provide funding for research to assess the risks and benefits of drug products that are on the market.

• Japan – Utilization of Electronic Medical Records (EMR) and

Claims Data in Pharmacovigilance • Secure access to EMR database including claim data to

assess drug safety through ADR incidence survey and using a pharmacopeia approach

19


Outline




• Trends in Observational Studies • Benefit-Risk Framework


20


Postmarketing Requirements (PMRs)

• Under FDAAA, post-marketing studies and clinical trials can be required to assess risk related to the use of a drug

• May be required at the time of approval or when new safety information becomes available

• Studies include observational, laboratory, or animal

• Before requiring a post-marketing clinical trial, FDA must find that a post-marketing study will not be sufficient


Guidance: Pharmacoepidemiology with electronic health care data

• Detailed protocol with clear objectives and hypothesis – Data sources

– Study design

– Comparator selection

– Exposure and outcome ascertainment

– Confounding adjustment

– Analysis methods

– Sample size and power

– Potential biases and impact on findings

– QA/QC

• “In certain circumstances, FDA might request access to the original analytic data”


Observational Studies at CDER: Influence/Future

• What role does/should observational studies have in CDER regulatory decisions?

• Don’t believe any relative risk less than 2?

• Carefully design studies to remove and evaluate bias?

• Use empirical measures of level of evidence?


Study Limitations and Strengths

Limitations

• Not randomized

• Potential misclassification

• Potential unmeasured confounding

• Endpoints not independently

validated

• Part D data not used previously for

research

• Limits of observational study in

terms of estimate sizes?

Strengths

• Large size

• Entire eligible population

• Close similarity in baseline

characteristics

• Previously validated end points

• Complete death ascertainment

• Consistency across sub-

analyses

25


Conclusion on Observations Studies for Safety at CDER • More and more common

• Perhaps more complex than RCTs in design and review

• Guidance on design and review

• Mixed views of their usefulness

• Future: Need to establish their merit, in particular validity


Outline





• Benefit-Risk Framework • Concluding remarks

27


CDER Benefit-Risk Framework

• A need for a more structured benefit-risk assessment in the review process – communicate the reasoning behind CDER’s decisions

– Which benefits/risks or other factors were considered?

– How was evidence interpreted?

– How were risks and benefits weighed?

– Keep the “big picture” in mind during a complex, detailed review

• Initiated in 2009 with the recognition that effective decision support begins with an understanding of how regulators think about the decision

28 28


Risk-Benefit - A Balancing Act

• CDER examined formal quantitative methods, but had some concerns

Reducing complex considerations into a single scale cannot capture the nuanced assessments in decisions

Quantitative analysis risks obscuring subjective expert judgment

• Structured qualitative approach currently the CDER norm

Approach best reflects the reality that B-R assessment is a qualitative exercise grounded in quantification of various data

Flexible to accommodate more complex supporting quantitative analyses that can aid, rather than replace, expert judgment

Rigorously communicates the basis for decisions in words

OB working actively to develop a quantitative approach

29 29


CDER Framework Development

• Used case studies of prior regulatory decisions to develop a conceptual framework

– Conducted interviews of key review disciplines on select challenging, less obvious decisions to identify the range of benefits and risks

– Developed question-based prompts to guide Framework completion

• Pilot-tested the framework in on-going pre-market reviews

– Evaluated and further refined the Framework and the question-based prompts

– Focused on implementation of the Framework in the review process

Decision Factor

Evidence and Uncertainties

Conclusions and Reasons

Analysis of Condition

Summary of evidence:

Conclusions (implications for

decision):

Current Treatment

Options



decision):

Benefit Summary of

evidence:


decision):

Risk Summary of

evidence:


decision):

Risk Management



decision):

Benefit-Risk Summary and Assessment

30


The Rows: Therapeutic Area Information

• Analysis of Condition and Current Treatment Options establish the context for the regulatory decision

• FDA’s understanding of disease severity and the current treatment armamentarium can shape how reviewers think about a drug’s benefit-risk assessment

• Patients who live with a disease are in a unique position to contribute to this understanding

• Patient-Focused Drug Development in PDUFA V

– This initiative is a systematic and more expansive approach to

obtaining the patient perspective on disease severity and the

adequacy of existing treatment options for 20 diseases

31 31


The Rows: Product-Specific Information

• Benefit analyzes the submitted data to assess the strengths and weaknesses of the clinical trials in demonstrating the benefit of the drug as well as safety signals identified in the data

• Risk characterizes any safety signals identified in the submitted data

• Risk Management describes any actions necessary to manage safety concerns in the post market setting or requirements for additional evidence to characterize the risk

32 32


The Columns: Evidence and Conclusions

Evidence and Uncertainties • What you know (facts)

• What you don’t know (uncertainties and underlying assumptions)

• How good are the data?

Conclusions and Reasons • What do you make of the data and uncertainties?

• Analysis of the information and its clinical relevance

• Drawing conclusions within each key consideration

33

Benefit Risk Summary & Assessment – A balanced written analysis

of the factors and their tradeoffs that summarizes the resulting regulatory recommendation or action

33


Benefit—Risk Summary Assessment

• What is the problem? (Analysis of Condition)

• What other potential interventions exist? (Current Treatment Options)

• What is the benefit of the proposed intervention? (Benefit)

• What am I worried about? (Risk)

• What can I do to mitigate/monitor those concerns? (Risk Management)

• The Benefit-Risk Summary Assessment is a succinct written analysis that integrates the conclusions from each decision factor, showing how the key evidence, uncertainties and conclusions were weighed in reaching a regulatory decision.

34


Outline







35


Concluding Remarks

• Quantitative safety evaluation (QSE) is a multi-disciplinary review approach

• Complex process, needs careful consideration throughout the lifecycle

• Regulators should be involved at all stages • Study requirements should be firmly

grounded in science • Innovation should be encouraged at each step

36


Questions

Acknowledgements: • Dr. Lisa LaVange

• Dr. Mark Levenson

37

trends and innovations in quantitative safety evaluation–bias and multiplicity –statistical...

Documents