![Page 1: Evidence Evaluation & Methods Workgroup: Developing a Decision Analysis Model](https://reader035.vdocuments.us/reader035/viewer/2022062410/568164a1550346895dd6930a/html5/thumbnails/1.jpg)
Evidence Evaluation & Methods Workgroup:
Developing a Decision Analysis Model
Lisa A. Prosser, PhD, MSSeptember 23, 2011
![Page 2: Evidence Evaluation & Methods Workgroup: Developing a Decision Analysis Model](https://reader035.vdocuments.us/reader035/viewer/2022062410/568164a1550346895dd6930a/html5/thumbnails/2.jpg)
Overview
• Limitations of evidence review• Brief introduction to decision analysis• Case study: Newborn screening for MCADD• Application to hyperbilirubinemia
![Page 3: Evidence Evaluation & Methods Workgroup: Developing a Decision Analysis Model](https://reader035.vdocuments.us/reader035/viewer/2022062410/568164a1550346895dd6930a/html5/thumbnails/3.jpg)
ACHDNC: Evidence Evaluation Methods Working Group
• Convened in April, 2011• Charged with evaluating evidence review
methods• Considered modeling to assist in evidence
synthesis and generation• Hyperbilirubinemia case study
![Page 4: Evidence Evaluation & Methods Workgroup: Developing a Decision Analysis Model](https://reader035.vdocuments.us/reader035/viewer/2022062410/568164a1550346895dd6930a/html5/thumbnails/4.jpg)
Decision Analysis (DA)
• Systematic approach to decision making under conditions of uncertainty
• Requires explicit consideration of each aspect of the decision problem:– Defining full set of alternatives– Choices regarding timing of implementation– Uncertainties involved– Assigning relative values to full set of possible outcomes
• Identifies alternative estimated to result in maximum benefit and uncertainty associated with that projection
![Page 5: Evidence Evaluation & Methods Workgroup: Developing a Decision Analysis Model](https://reader035.vdocuments.us/reader035/viewer/2022062410/568164a1550346895dd6930a/html5/thumbnails/5.jpg)
Advantages of Modeling
• Can evaluate existing & un-tested alternatives
• Can simulate head-to-head comparisons
• Requires explicit definition of assumptions• Can identify sources of uncertainty/prioritize future
research
• Allows for extension of time horizons
![Page 6: Evidence Evaluation & Methods Workgroup: Developing a Decision Analysis Model](https://reader035.vdocuments.us/reader035/viewer/2022062410/568164a1550346895dd6930a/html5/thumbnails/6.jpg)
Decision Analysis & Child Health Policy
• Decision analytic modeling approach can provide insights into comparative effectiveness and cost-effectiveness
• Especially salient for child health by providing approach for projecting long-term outcomes
• Cost-effectiveness results increasingly considered
![Page 7: Evidence Evaluation & Methods Workgroup: Developing a Decision Analysis Model](https://reader035.vdocuments.us/reader035/viewer/2022062410/568164a1550346895dd6930a/html5/thumbnails/7.jpg)
Decision Analysis & Newborn Screening
• Incorporation of modeling into the evidence review process:– Simple models– Health outcomes– No cost-effectiveness analysis (yet)
• Initial goal is to project health benefits and potential harms
![Page 8: Evidence Evaluation & Methods Workgroup: Developing a Decision Analysis Model](https://reader035.vdocuments.us/reader035/viewer/2022062410/568164a1550346895dd6930a/html5/thumbnails/8.jpg)
Case Study: Expanded Newborn Screening for MCADD
• Expansion of newborn screening programs using MS-MS
• Incremental costs very low• Total costs of screening and follow-up not
characterized• Higher incidence with newborn screening
Source: Prosser et al., Pediatrics, 2010
![Page 9: Evidence Evaluation & Methods Workgroup: Developing a Decision Analysis Model](https://reader035.vdocuments.us/reader035/viewer/2022062410/568164a1550346895dd6930a/html5/thumbnails/9.jpg)
MCADD Example: Methods
• Develop a decision analytic model to project health benefits, health risks, and costs of expanded newborn screening
• Model inputs using primary and published data supplemented by expert opinion:– Event probabilities (e.g., outcomes of metabolic disease, test
characteristics)– Costs of screening and clinical outcomes– Effects on health-related quality of life (HRQOL)
• Limited evidence especially for long-term outcomes• Analysis: Projected long-term clinical outcomes and cost-
effectiveness
![Page 10: Evidence Evaluation & Methods Workgroup: Developing a Decision Analysis Model](https://reader035.vdocuments.us/reader035/viewer/2022062410/568164a1550346895dd6930a/html5/thumbnails/10.jpg)
MCADD Example: Decision Analytic Model
Costs
Probabilities
Health State
Values
Clinical Outcomes• Cases• Hospitalizations•Deaths
Economic Outcomes• Costs• QALYs
Screening Outcomes• False positives• Follow-up required
Newborn Screening
Simulation Model
![Page 11: Evidence Evaluation & Methods Workgroup: Developing a Decision Analysis Model](https://reader035.vdocuments.us/reader035/viewer/2022062410/568164a1550346895dd6930a/html5/thumbnails/11.jpg)
MCADD Example: Decision Analytic Model
BLACK BOX
Costs
Probabilities
Health State
Values
Clinical Outcomes• Cases• Hospitalizations•Deaths
Economic Outcomes• Costs• QALYs
Screening Outcomes• False positives• Follow-up required
![Page 12: Evidence Evaluation & Methods Workgroup: Developing a Decision Analysis Model](https://reader035.vdocuments.us/reader035/viewer/2022062410/568164a1550346895dd6930a/html5/thumbnails/12.jpg)
MCADD Newborn Screening Simulation Model
Hypothetical Cohort of Newborns
Lifetime MCADD
Submodel with
Screening
Newborn Screening Program
Submodel
NoNewborn
Screening Program
Lifetime MCADD
Submodel with Clinical Identification
![Page 13: Evidence Evaluation & Methods Workgroup: Developing a Decision Analysis Model](https://reader035.vdocuments.us/reader035/viewer/2022062410/568164a1550346895dd6930a/html5/thumbnails/13.jpg)
MCADD Newborn Screening Submodel
Lifetime MCADD
Submodel with Screening
Newborn Screening
Test
Normal Result/No Further Follow-up
Inadequate Sample/
Repeat Test
Out-of-Range Value/Repeat
Referral to Pediatrician/
Metabolic Center
False Positive
Presumed Diagnosis
True Positive
![Page 14: Evidence Evaluation & Methods Workgroup: Developing a Decision Analysis Model](https://reader035.vdocuments.us/reader035/viewer/2022062410/568164a1550346895dd6930a/html5/thumbnails/14.jpg)
Lifetime MCADD Submodel
From Newborn Screening Program
Submodel
Normal
Intellectual Disability
Dead
Hospitalization
![Page 15: Evidence Evaluation & Methods Workgroup: Developing a Decision Analysis Model](https://reader035.vdocuments.us/reader035/viewer/2022062410/568164a1550346895dd6930a/html5/thumbnails/15.jpg)
MCADD Example: Projected Outcomes
Clinical Ident. Screening ∆
Population 100,000 100,000 -
Children w/MCADD 5.88 (0.01) 8.4 (0.01) 2.52
FP screen N/A 20 (0.02) 20
Costs (lifetime) $630,710 $1,629,482 $998,778
QALYs 2,976,780.08 2,976,827.03 46.95
C/E ratio $21,273 (395)
Source: Prosser et al., Pediatrics, 2010
![Page 16: Evidence Evaluation & Methods Workgroup: Developing a Decision Analysis Model](https://reader035.vdocuments.us/reader035/viewer/2022062410/568164a1550346895dd6930a/html5/thumbnails/16.jpg)
MCADD Example: Projected Long-term Outcomes
0 10 20 30 40 50 60 70 80 900
1
2
3
4
5
6
Dev. Delay (Screen)
Dev. Delay (Clinical)
Cum. Death (Screen)
Cum. Death (Clinical)
Age
Num
ber
of C
ases
Source: Prosser et al., Pediatrics, 2010
![Page 17: Evidence Evaluation & Methods Workgroup: Developing a Decision Analysis Model](https://reader035.vdocuments.us/reader035/viewer/2022062410/568164a1550346895dd6930a/html5/thumbnails/17.jpg)
MCADD Example: Policy Implications
• Model predicts short- and long-term outcomes– Screening test and follow-up results– Projected number of children with condition, cases of
developmental delay, hospitalizations, deaths– Costs– Quality-adjusted life years
• Results robust to changes in underlying assumptions• Results sensitive to test costs, but not false positive
rate
![Page 18: Evidence Evaluation & Methods Workgroup: Developing a Decision Analysis Model](https://reader035.vdocuments.us/reader035/viewer/2022062410/568164a1550346895dd6930a/html5/thumbnails/18.jpg)
Application to Hyperbilirubinemia
• Create decision analysis model
• Expert panel input
• Project short- and long-term health outcomes
![Page 19: Evidence Evaluation & Methods Workgroup: Developing a Decision Analysis Model](https://reader035.vdocuments.us/reader035/viewer/2022062410/568164a1550346895dd6930a/html5/thumbnails/19.jpg)
Projected Health OutcomesSelected Outcomes Clinical
AssessmentUniversal Screening
Screening outcomesShort-term:
HospitalizationsLong-term:
Acute bilirubin encephalopathyChronic bilirubin encephalopathyUnaffected
![Page 20: Evidence Evaluation & Methods Workgroup: Developing a Decision Analysis Model](https://reader035.vdocuments.us/reader035/viewer/2022062410/568164a1550346895dd6930a/html5/thumbnails/20.jpg)
Anticipated Findings
• Projected health outcomes and associated uncertainty
• Identification of key parameters • Process for specifying assumptions on health
benefits and potential risks
![Page 21: Evidence Evaluation & Methods Workgroup: Developing a Decision Analysis Model](https://reader035.vdocuments.us/reader035/viewer/2022062410/568164a1550346895dd6930a/html5/thumbnails/21.jpg)
Moving forward
• Adding decision analysis to the evidence review can provide:– Approach for evidence synthesis– Method for specifying assumptions
• Incorporation of cost/cost-effectiveness analysis
![Page 22: Evidence Evaluation & Methods Workgroup: Developing a Decision Analysis Model](https://reader035.vdocuments.us/reader035/viewer/2022062410/568164a1550346895dd6930a/html5/thumbnails/22.jpg)
Questions