value of information calculations to inform and prioritize clinical research investments david...
TRANSCRIPT
![Page 1: Value of Information Calculations to Inform and Prioritize Clinical Research Investments David Meltzer MD, PhD The University of Chicago](https://reader031.vdocuments.us/reader031/viewer/2022032705/56649dd45503460f94acb594/html5/thumbnails/1.jpg)
Value of Information Calculations to Inform and Prioritize
Clinical Research Investments
David Meltzer MD, PhDThe University of Chicago
![Page 2: Value of Information Calculations to Inform and Prioritize Clinical Research Investments David Meltzer MD, PhD The University of Chicago](https://reader031.vdocuments.us/reader031/viewer/2022032705/56649dd45503460f94acb594/html5/thumbnails/2.jpg)
Overview
• Cost-effectiveness analysis has long been used to assess the value of medical treatments and the information that comes from diagnostic tests– Cost-effectiveness measured in Cost/QALY– Net health benefits
= gain in QALYs – opportunity cost of spending in QALYs– Net monetary benefit
= $ value of improved health - costs• Newer value of information techniques have extended
these tools to assess the value of medical research
![Page 3: Value of Information Calculations to Inform and Prioritize Clinical Research Investments David Meltzer MD, PhD The University of Chicago](https://reader031.vdocuments.us/reader031/viewer/2022032705/56649dd45503460f94acb594/html5/thumbnails/3.jpg)
Research as Value of Information: Analogy to Diagnostic Testing
Test
Don’t Test
S
H
S
H
Max{pU(T|S)+(1-p)U(T|H), pU(N|S)+(1-p)U(N|H)}
U(T|S)
U(N|H)
pU(T|S)+(1-p)U(N|H)
![Page 4: Value of Information Calculations to Inform and Prioritize Clinical Research Investments David Meltzer MD, PhD The University of Chicago](https://reader031.vdocuments.us/reader031/viewer/2022032705/56649dd45503460f94acb594/html5/thumbnails/4.jpg)
Value of Information Approach to Value of Research
• Without information– Make best compromise choice not knowing true state of the
world (e.g. don’t know if intervention is good, bad)• With probability p: get V(Compromise|G)• With probability 1-p: get V(Compromise|B)
• With information– Make best decision knowing true state
• With probability p: get V(Best choice|G)• With probability 1-p: get V(Best choice|B)
• Value of information = E(outcome) with information - E(outcome) w/o information
= {p*V(Best choice|G) + (1-p)*V(Best choice|B)} - {p*V(Compromise|G) + (1-p)*V(Compromise|B)}
= Value of Research
![Page 5: Value of Information Calculations to Inform and Prioritize Clinical Research Investments David Meltzer MD, PhD The University of Chicago](https://reader031.vdocuments.us/reader031/viewer/2022032705/56649dd45503460f94acb594/html5/thumbnails/5.jpg)
Practical Applications of Value of Information
• VOI requires modeling population value of information
where
• VOI based on decision models– IVOI modeled with decision model– UK (NICE): Alzheimer’s Disease Tx, wisdom teeth removal
• Minimal modeling approaches to VOI– IVOI comes (nearly) directly from clinical trial– US (NIH): CATIE Trial of atypical antipsychotics
• Bound with more limited data (burden of illness)
( ) ( )tt
t
VOI D t I t N IVOI ( )
( )
in
t
t
is time preference discount factor
D t is depeciation of knowledge over time
I t is extent of implementation
N is number of eligible individuals each cohort
IVOI is individual VOI
![Page 6: Value of Information Calculations to Inform and Prioritize Clinical Research Investments David Meltzer MD, PhD The University of Chicago](https://reader031.vdocuments.us/reader031/viewer/2022032705/56649dd45503460f94acb594/html5/thumbnails/6.jpg)
“Bayesian Value of information analysis: An application to a policy model of Alzheimer's disease.”
![Page 7: Value of Information Calculations to Inform and Prioritize Clinical Research Investments David Meltzer MD, PhD The University of Chicago](https://reader031.vdocuments.us/reader031/viewer/2022032705/56649dd45503460f94acb594/html5/thumbnails/7.jpg)
Uncertainty in Incremental Net Benefits
![Page 8: Value of Information Calculations to Inform and Prioritize Clinical Research Investments David Meltzer MD, PhD The University of Chicago](https://reader031.vdocuments.us/reader031/viewer/2022032705/56649dd45503460f94acb594/html5/thumbnails/8.jpg)
Cost-Effectiveness Acceptability Curve
![Page 9: Value of Information Calculations to Inform and Prioritize Clinical Research Investments David Meltzer MD, PhD The University of Chicago](https://reader031.vdocuments.us/reader031/viewer/2022032705/56649dd45503460f94acb594/html5/thumbnails/9.jpg)
Value of Research by Value of Health
![Page 10: Value of Information Calculations to Inform and Prioritize Clinical Research Investments David Meltzer MD, PhD The University of Chicago](https://reader031.vdocuments.us/reader031/viewer/2022032705/56649dd45503460f94acb594/html5/thumbnails/10.jpg)
Value of Research by Time Horizon
![Page 11: Value of Information Calculations to Inform and Prioritize Clinical Research Investments David Meltzer MD, PhD The University of Chicago](https://reader031.vdocuments.us/reader031/viewer/2022032705/56649dd45503460f94acb594/html5/thumbnails/11.jpg)
Contributors to Value of Research
![Page 12: Value of Information Calculations to Inform and Prioritize Clinical Research Investments David Meltzer MD, PhD The University of Chicago](https://reader031.vdocuments.us/reader031/viewer/2022032705/56649dd45503460f94acb594/html5/thumbnails/12.jpg)
Practical Applications of Value of Information
• VOI requires modeling population value of information
where
• VOI based on decision models– IVOI modeled with decision model– UK (NICE): Alzheimer’s Disease Tx, wisdom teeth removal
• Minimal modeling approaches to VOI– IVOI comes (nearly) directly from clinical trial– US (NIH): CATIE Trial of atypical antipsychotics
• Bound with more limited data (burden of illness)
( ) ( )tt
t
VOI D t I t N IVOI ( )
( )
in
t
t
is time preference discount factor
D t is depeciation of knowledge over time
I t is extent of implementation
N is number of eligible individuals each cohort
IVOI is individual VOI
![Page 13: Value of Information Calculations to Inform and Prioritize Clinical Research Investments David Meltzer MD, PhD The University of Chicago](https://reader031.vdocuments.us/reader031/viewer/2022032705/56649dd45503460f94acb594/html5/thumbnails/13.jpg)
Expected Value of Researchon the Comparative Cost-Effectiveness
of Antipsychotics Drugs
David Meltzer MD PhD
Department of Medicine, Department of Economics, Harris School of Public Policy, &
University of Chicago CERTChicago IL
(Joint work with Anirban Basu PhD, University of Chicago & Dr. Herbert Y. Meltzer, Vanderbilt University)
![Page 14: Value of Information Calculations to Inform and Prioritize Clinical Research Investments David Meltzer MD, PhD The University of Chicago](https://reader031.vdocuments.us/reader031/viewer/2022032705/56649dd45503460f94acb594/html5/thumbnails/14.jpg)
The Clinical Antipsychotic Trials in Intervention Effectivness (CATIE)
• $42.6 million, NIMH-funded randomized trial of Atypical Antipsychotic Drugs (A-APDs) and a Neuroleptic (Perhphenazine) in patients with established schizophrenia
• Major findingso Discontinuation rates similar with A-APDs and Perphenazineo Perphenazine cost-effective first-line treatment
• Impacto Frequently discussed in coverage decisionso Some have argued results should be considered definitive
![Page 15: Value of Information Calculations to Inform and Prioritize Clinical Research Investments David Meltzer MD, PhD The University of Chicago](https://reader031.vdocuments.us/reader031/viewer/2022032705/56649dd45503460f94acb594/html5/thumbnails/15.jpg)
The Clinical Antipsychotic Trials in Intervention Effectivness (CATIE)
• Limitationso Continuation was major endpointo Limited precision in estimates of effectiveness, costso Small differences in effectiveness/costs across many
persons could be of great value
• Important to know value of potential future researcho Help prioritize individual research opportunitieso Facilitate rational investment decisions
![Page 16: Value of Information Calculations to Inform and Prioritize Clinical Research Investments David Meltzer MD, PhD The University of Chicago](https://reader031.vdocuments.us/reader031/viewer/2022032705/56649dd45503460f94acb594/html5/thumbnails/16.jpg)
CATIE Cost-Effectiveness Results
Monthly Costs Mean (sd) ($)
QALY Mean (sd)
ICER ($/QALY)
Perphenazine 817 ( 728) 0.722 (0.0064) -
Olanzapine 1619 (1442) 0.723 (0.0063) 9,624,000
Risperidone 1635 (1457) 0.706 (0.0066) Dominated
Quetiapine 1680 (1497) 0.721 (0.0065) Dominated
(Ref: Rosenheck et al , 2006; Private Communications with Dr. Rosenheck)
Only statistically significant difference:
QALYPerphenazine > QALYRisperidone (p-val < 0.001)
![Page 17: Value of Information Calculations to Inform and Prioritize Clinical Research Investments David Meltzer MD, PhD The University of Chicago](https://reader031.vdocuments.us/reader031/viewer/2022032705/56649dd45503460f94acb594/html5/thumbnails/17.jpg)
Aims
Primary Aim: To determine the expected value of more precise determination of effects of AAPDs and Perphenazine on costs and QALYs.
Secondary Aim: To determine the optimal sample size for a future trial of the effects of AAPDs and Perphenazine on costs and QALYs
![Page 18: Value of Information Calculations to Inform and Prioritize Clinical Research Investments David Meltzer MD, PhD The University of Chicago](https://reader031.vdocuments.us/reader031/viewer/2022032705/56649dd45503460f94acb594/html5/thumbnails/18.jpg)
Methods Used for Value of Research
Expected value calculated based on the welfare of the prevalent cohort over their lifetimes and the welfare of next 20 incident cohorts over their lifetimes
3% discounting was used
![Page 19: Value of Information Calculations to Inform and Prioritize Clinical Research Investments David Meltzer MD, PhD The University of Chicago](https://reader031.vdocuments.us/reader031/viewer/2022032705/56649dd45503460f94acb594/html5/thumbnails/19.jpg)
Simulated Distribution of Mean QALYS(Based on uncertainty around CATIE results)
Den
sity
.65 .7 .75 .8 .85E(QALY)/per patient per year
Olanzapine: 0.723 (0.0063)Quetiapine: 0.721 (0.0065) Risperidone: 0.706 (0.0066) Perphenazine: 0.722 (0.0064)
![Page 20: Value of Information Calculations to Inform and Prioritize Clinical Research Investments David Meltzer MD, PhD The University of Chicago](https://reader031.vdocuments.us/reader031/viewer/2022032705/56649dd45503460f94acb594/html5/thumbnails/20.jpg)
Simulated Distribution of Mean Costs(Based on uncertainty around CATIE results)
Den
sity
0 5000 10000 15000E(QALY)/per patient per year
Olanzapine: $1606 (1421)Quetiapine: $1685 (1485) Risperidone: $1621 (1439) Perphenazine: $ 810 ( 723)
![Page 21: Value of Information Calculations to Inform and Prioritize Clinical Research Investments David Meltzer MD, PhD The University of Chicago](https://reader031.vdocuments.us/reader031/viewer/2022032705/56649dd45503460f94acb594/html5/thumbnails/21.jpg)
Realizations of Value of Research Over Time
02
46
81
01
2V
alu
e (
in B
illio
n $
)
2007 2017 2027 2037 2047 2057 2067 2077 2087YEAR
Incident in 2012-2036Incident in 2011Incident in 2010Incident in 2009
Incident in 2008Incident in 2007Prevalent Cohort
Value of Future Research to Prevalent and Incident Cohortsat $50k/QALY
Total Value to Prevalent Cohort: $207 billionTotal Value to Each Incident Cohort: $6.6 billion
Total Value to Prevalent & Next 20 Incident Cohorts: $342 billion
![Page 22: Value of Information Calculations to Inform and Prioritize Clinical Research Investments David Meltzer MD, PhD The University of Chicago](https://reader031.vdocuments.us/reader031/viewer/2022032705/56649dd45503460f94acb594/html5/thumbnails/22.jpg)
Value of Research and Acceptability Profile
.5.5
2.5
4.5
6.5
8.6
Pr
300
320
340
360
380
400
Va
lue
(in b
ilio
n $)
10000 30000 50000 70000 90000 110000 130000 150000Marginal WTP for a QALY
Max VofRPr(CATIE decision is incorrect)
![Page 23: Value of Information Calculations to Inform and Prioritize Clinical Research Investments David Meltzer MD, PhD The University of Chicago](https://reader031.vdocuments.us/reader031/viewer/2022032705/56649dd45503460f94acb594/html5/thumbnails/23.jpg)
Optimal Sample Size for A Future Trial• Traditional (Deterministic) Power Calculations (at $50K/QALY)
• Largest effect size in NMB between an atypical & perphenazine based on CATIE results: $15,680 (sd=$315,000) vs. $26,296 (sd=$140,000)
• Sample size required for alpha = 0.05 & power = 0.80: 8,300 for each arm
• Power associated with n of CATIE = 400/arm & alpha = 0.05: 10%
![Page 24: Value of Information Calculations to Inform and Prioritize Clinical Research Investments David Meltzer MD, PhD The University of Chicago](https://reader031.vdocuments.us/reader031/viewer/2022032705/56649dd45503460f94acb594/html5/thumbnails/24.jpg)
Net Expected Value of Sample Information(at $50K, $100K and $150K/QALY)
|
|
|
|
|
||v3
003
253
50E
VS
I (in
Bill
ion
s)
5000 10000 15000 20000 25000 30000 35000 40000 45000 50000Sample size for each arm
at $50K/QALYat $100K/QALY
at $150K/QALY
Cost of Research: $3 mill + (sample size*4)*($5000/month)*18 months
Optimal sample size for each arm = 22,500
![Page 25: Value of Information Calculations to Inform and Prioritize Clinical Research Investments David Meltzer MD, PhD The University of Chicago](https://reader031.vdocuments.us/reader031/viewer/2022032705/56649dd45503460f94acb594/html5/thumbnails/25.jpg)
CATIE VOI Conclusions
• The value to more precisely establishing the cost-effectiveness of typical/atypical antipsychotics is enormous.
• The results of CATIE should not be viewed as definitive.
• Further studies of the comparative cost-effectiveness of typical/atypical antipsychotics with adequate sample size to answer such questions have high expected value.
• Optimal sample sizes may be exceptionally large, raising interesting questions as to how clinical trials on such a scale might be executed.
• Large scale social experiments may provide an interesting model for such studies.
![Page 26: Value of Information Calculations to Inform and Prioritize Clinical Research Investments David Meltzer MD, PhD The University of Chicago](https://reader031.vdocuments.us/reader031/viewer/2022032705/56649dd45503460f94acb594/html5/thumbnails/26.jpg)
Conclusions• Value of information (VOI) analysis can be used to develop
prospective estimates of the value of research• VOI can be used to prioritize areas of clinical research or choose
among study designs• Decision models or minimal modeling approaches
– Additional minimal model applications in progress– Erlotinib (Tarceva) (+gemcitabine) in advanced pancreatic CA– Azithromycin vs. Augmentin in acute sinusitis
• Incorporating VOI into research prioritization is a work in progress– Use by funding agencies?– Use by investigators?
![Page 27: Value of Information Calculations to Inform and Prioritize Clinical Research Investments David Meltzer MD, PhD The University of Chicago](https://reader031.vdocuments.us/reader031/viewer/2022032705/56649dd45503460f94acb594/html5/thumbnails/27.jpg)
Acknowledgments
• Collaborators: Anirban Basu Ph.D., Jeanette Chung Ph.D., Ties Hoomans Ph.D., Herbert Meltzer M.D.
• Funding: AHRQ BCBS Evidence-Based Practice Center, AHRQ Hospital Medicine and Economics Center for Education and Research in Therapeutics, Best Practice Inc, National Institute of Aging, National Institute of Mental Health