alec miners, london school of hygiene and tropical ... · lecture overview part 1 what is (health)...
TRANSCRIPT
Lecture Overview Part 1
What is (health) economics? Rationale for economic evaluation What is an economic evaluation? What type of questions can it address? The different types of economic evaluation
Decision rules Pros and cons of each approach
Part 2 How to undertake an economic evaluation
With special reference to decision modelling
How results are used? Some examples / experiences from the National Institute for Health and Clinical
Excellence Technology Appraisal Committee perspective
Summary
2
Economics is about …
Limited resources
Unlimited “wants”
Choosing between which ‘wants’ we can ‘afford’ given our resource ‘budget’
What is Health Economics and How Might it Improve Health (Care)? Health Economics is the application of economics to
health / health care
Broadly speaking can be divided into two work streams
Understanding behaviour
Informing choices
Economic Evaluation
4
Why Economic Evaluation? In perfectly competitive private markets, allocation
of goods and services is left to market forces Interaction of supply and demand
Leading to efficient allocation of resources
Market allocation of health care ‘fails’ Imperfect information, externalities etc
Provision of health care therefore not left entirely to the market
5
Why Economic Evaluation? Therefore some level of Government / non-market
intervention in health care
But basic problem of limited resources remains
Decisions still needed on what to buy
Information on ‘value for money’ is still needed
In a non-market situation, this information is missing
How to generate this information?
Economic Evaluation – a technique of measuring efficiency in areas where there is no market
6
What does an Economic Evaluation look like?
“The comparative analysis of alternative courses of action in terms of their costs and consequences” (Drummond et al 2005)
“Based on the common sense notion that a decision to do or not to do something should depend weighing up the advantages (benefits) and disadvantages (costs)” (Morris et al 2007)
7
Schematic of an Economic Evaluation Treatments A v B
Choice
Costs A
(Drugs etc) Health Outcomes A
Costs B
(Doctor time etc) Health Outcomes B
8
Time
The Types of Questions that can be Answered What combinations of HIV drugs should people receive,
NNRTIs, protease inhibitors?
Should patients be able to switch HIV therapies?
How should disease progression be monitored?
At what clinical stage should treatment be started / stopped
In what age groups should mammography screening be undertaken?
Which is the most cost-effective type of bed net to prevent malaria?
Etc………
9
Types of Economic Evaluation Cost-benefit analysis (CBA) Monetary valuation of outcomes
Cost-effectiveness analysis (CEA) Natural unit of outcome eg. change in mortality
Cost-utility analysis (CUA) QALYs or DALYs used to measure / value outcomes
Cost-minimisation analysis (CMA) Outcomes are equivalent
They address different levels of efficiency
10
Cost-Benefit Analysis Oldest form of evaluation (circa 1960)
All benefits are measured in monetary terms Benefits don’t have to be confined to ‘health’
Net Benefit (£) = (BenefitB - BenefitA) – (CostB - CostA)
Addresses issues of allocative efficiency Where marginal benefits > marginal costs
Do the benefits outweigh the costs / is the net benefit positive?
Decision rule: Yes they do, adopt
11
Discrete Choice Experiment to Value Health Benefits – HIV Testing
Question 1 A B
Location Home Doctors office
Collection method Urine Swab
Test results (time) 1 Week Immediate
Cost $10 $18
12
Tick the option you would prefer □ □
Repeat for a number of different questions with varying attribute levels
13
Discrete Choice - HIV Testing Example WTP (95% CI)
(Test)
Draw blood – swab
$19 to $36
(Test)
Draw blood – urine
$15 to $33
Philips et al, Jounral of Health Services Research 2002
Pros and Cons of CBAs Can establish allocative efficiency and promote
allocatively efficient technologies
Theory based, maximise welfare, which can include not just health benefits but also processes of care e.g. Improved survival, quality of life AND that you
prefer having (say) treatment at home rather than in a hospital – ie processes of care
Can compare across sectors eg. health vs defence
Major limitation is said to be the (acceptability of) methods used to value health Methodological challenges
14
Cost-Effectiveness Analysis Arose because of ‘difficulties’ of using CBA in
health
No attempt is made to ‘value’ health outcomes
Uses ‘natural / clinical’ outcome measures
Typically outcome chosen is one dimensional
Eg. events averted, infections prevented, heart attack prevented, life years gained etc.
15
Cost-Effectiveness Analysis Traditional statistic generated is an incremental cost-
effectiveness ratio (ICER)
ICER= mean cost B - mean cost A
mean effect B - mean effect A
16
E-learning Devices v Dietary Advice
Mean Life years Mean costs
DA 14.800 £5,115
E-Learning 14.915 £5,707
Incremental 0.015 £591
17
Mean cost per additional life year= approx. £39,433
Miners et al. unpublished
Pros and Cons of CEAs One dimensional outcomes (eg. life-years)
Restricted to comparisons of technologies that have similar types of effect
Liver transplant versus treatments for pain?
Allocative efficiency within a clinical area
Practical problems of interpreting results
£/ $/ Euro per prevented infection?
Determining decision willingness to pay threshold
If ICER < WTP threshold, then adopt(£10,000 < £12,000)
18
Cost-Minimisation Analysis Narrowest form of evaluation
Similar to CEA
Used where there is strong evidence that health outcomes are similar
Therefore compare costs between alternatives and determine which is the most economically efficient
Where more benefit can be produced at the same cost or the same benefit produced at lower cost (lecture and seminar 4)
19
CMA example: Wellwood et al 1998 (BMJ)
Laparoscopic versus open mesh repair for inguinal hernia
RCT of 403 patients
Health outcomes broadly similar
Quality of life, recurrence rates
Laparoscopic cost on average £335 more per procedure
Although less costly if more disposable items were used
20
Pros and Cons of CMAs Very hard to establish different health care
technologies produce equivalent outcomes
Seen by some health economists as a defunct form of evaluation (Briggs)
Certainly open to misuse, equivalent outcomes are often assumed rather than proved
21
Cost-Utility Analysis Arisen because of limitations with CEA
One dimensional outcomes
Not being able to compare outcomes across different clinical settings
CUAs mostly measure and value health outcomes in terms of Quality-Adjusted Life-Years (QALYs)
Disability-Adjusted Life-Years
QALYs combine length of life with quality of life
1 QALY = 1 year in perfect health
22
QALYs Utility value of 1 = perfect health
Utility value of 0 = death
Health states between dead and perfect health have a value somewhere in between 0 and 1 1 year in perfect health = 1 QALY
1 year in half perfect health followed by instant death = 0.5 QALY
3 years in half perfect health = 1.5 QALYs
Are negative utility values possible?
Time –trade off / standard gamble
23
24
Putting the ‘Q’ in QALYs: Time Trade-off
25
Time Trade-off The subject is offered two alternatives
State i for time t (life expectancy of an individual with the chronic condition) followed by death
Healthy for time x < t followed by death
Time x is varied until the respondent is indifferent between the two alternatives.
Utility for hi = x/t
If x=4 years and t=10 years, utility for health state i =0.4
Pros and Cons of CUAs Type of economic evaluation of choice for National
Institute for Health Clinical Excellence (NICE)
Can be used to assess allocative efficiency, but only with respect to health care
Eg. should access to HIV treatments be prioritised over bed net provision to prevent malaria?
Can not compare say health v defence as with CBA
Ignores process issues of health care which might be important eg. shorting waiting times
26
CUA Decision Rules Practical problem of knowing willingness to pay threshold
for an additional QALY ICER < threshold WTP, then adopt
NICE states ‘threshold’ is between £20,000 to £30,000 per additional QALY
Not based on empirical evidence, rather case history ‘Achilles heel’ of economic evaluation?
Studies suggest between £19,000 to £70,000 per QALY But all have methodological limitations
World Bank (1993) <$25 per DALY averted is ‘highly attractive’
27
Part 1 Summary Markets for health care ‘fail’ Economic evaluation generates information on efficiency
in non-market situations By providing comparative information on the costs and benefits of
treatment
Different forms of evaluation exist, they differ in terms of how they assess benefits and the types of efficiency they assess Significant issues of getting threshold willingness to pay levels
wrong
Lots of methodological issues about how economic evaluation is / should be undertaken
But what is the alternative??
28
How To Undertake And Use An Economic Evaluation
29
Economic Evaluation Output Traditional statistic generated is an incremental cost-
effectiveness ratio (ICER)
ICER= mean cost B - mean cost A
mean effect B - mean effect A
30
Question Then Becomes... .....how to estimate these ‘mean’ costs and effects
Broadly speaking there are essentially three types of economic evaluation ‘framework’
Either an evaluation
alongside a (randomised) clinical trial
using a decision model
or a mix of the two (in good practice this tends to be the case)
31
What is a Decision Model? A decision-model is an explicit quantitative
mathematical method of combining / synthesising information from more than once source
Simplifies the complexity of the real world
Decision aid
The aim of a decision model is to make explicit the best options / decision
32
Why Model? Think about what needs to be estimated?
And why a RCT type framework might be limited?
Mean costs and benefits of all relevant options
Over a relevant time horizon?
Outcomes expressed in relevant units such as QALYs / DALYs
Using all relevant evidence
Other RCTs might already exist
The long and short or it is that it is very rare (impossible?) to base an EE on a single RCT alone
33
*Buxton et al. 1997 Health Economics
6(3):217-227
Time Horizon The time horizon should be sufficient to reflect any
differences in costs and health outcome
RCTs have limited follow-up period
Statins for the prevention of myocardinal infarction (MI) / death
A number of RCTS eg. WOSCOPS and 4S, 3-5 years in duration
Statins associated with significant reduction in both events
Treatment not stopped after 4 years!
People might have a MI in year 6 or after...
Therefore, need to extrapolate results
34
Linking Outcome Measures HIV-related (drug) trials typically not powered to detect
differences in patient survival
To see survival changes would take too long
Primary outcome is usually some immunological / virological endpoint Cost per additional virological success?
Cost per % change in CD4 count?
Not very helpful from an economics perspective
Therefore models are often used to link changes in CD4 count / viral load to probabilities of death, opportunistic infections and costs (see previous Markov model)
35
Including All Relevant Comparators A comparison of options A vs B might be
misleading, if C, D and F also exist!
Likely that a trial of all options does not exist
Methods exist to incorporate evidence from networks of pair wise trials, to allow simultaneous comparisons
Sophisticated form of a meta-analysis
36
NICE Advanced Breast Cancer Guideline (2009, Willis et al) CUA of chemotherapy sequences for ABC
Lots of different drug and palliative care options
Lots of 2-armed RCTs
17 different possible sequences were identified
Solution was to perform an ‘indirect treatment comparison’
37
NICE Advanced Breast Cancer Guideline (2009, Willis et al) CUA of chemotherapy sequences for ABC
Lots of different drug and palliative care options
Lots of 2-armed RCTs
17 different possible sequences were identified
Solution was to perform an ‘indirect treatment comparison’
38
More Than One Relevant RCT NICE Guidance on treatments for ovarian cancer
ICON 3 is a large RCT of treatments for ovarian cancer
Results were largely in favour of the taxane treatment arm
But 3 other RCTs (eg. OV10) exist
2 of which suggested the control was more effective
A ‘good’ economic evaluation should ‘synthesise’ (meta-analyse) results from all 4 RCTs
39
Infectious Diseases Ongoing NICE Public Health Guidance on
detecting cases of hepatitis C in intravenous drug users
Assessing the cost-effectiveness of vaccines / interventions for infectious diseases Treatments affect networks of people, not just the
individual who receives the treatment
Difficult to assess outcomes in the context of RCT, often because its difficult to track interactions between people eg. passing on a cold
Idea behind ‘herd immunity’ and why immunisation programmes are associated with ‘positive’ externalities
ie actions of one have impacts on health of others
40
Time Taken to Perform an Evaluation The UKs National Institute for Health and Clinical
Excellence (NICE) require economic evaluations to be submitted in order for reimbursement decisions to be made Interventions considered cost-effective will be available
on the NHS
Scope document issue outlining issues such as relevant comparators
People submitting evidence typically have <1 year to construct evidence
Long enough for a clinical study?
41
Other Reasons RCTs might not exist in the first instance
Or did not measure all relevant information
Generalising the results from one setting to another
Eg. using US clinical trials relating to kidney transplants for European decision making when entire infrastructure is different
Eg. types of donor
Methods of storing / transporting the graft
Assessing the value of undertaking clinical research in the first instance
Ie pre trial modelling
Expected value of perfect information analysis
42
Economic Evaluations Alongside Clinical Trials Common source for effectiveness (efficacy) and
economic data Relatively easy data collection Comparability of groups (internal validity)
Strength of randomisation
Allows statistical analysis or relevant inputs (costs) and outcomes (health) .....but limitations are inherent.....the answering
being modelling Some do not believe that RCT based economic
evaluations have any role at all
43
What Does a Decision Model Look Like? There is no universally agreed definition
But all include
A relevant structure representing a disease or process
Probabilities of events occurring
Costs
Health outcomes
44
Decision Tree: Diagnosis Of Knee Injuries
45
Surgical Diagnosis
Patient attends withknee injury
Listed forarthroscopy
Surgery
No surgery
Resolution ofsymptoms
Residualsymptoms
Resolution ofsymptoms
Residualsymptoms
Pathprobability
Pathcost
QALY
Expected cost = £410Expected QALYs = 46.00
0.68 500 47.5
0.12 500 40
0.12 50 47.5
0.08 50 40
0.4
0.6
0.15
0.85
0.8
0.2
46 Pathway probability eg. 0.8 x 0.85 = 0.68
MRI Diagnosis
Patient attendsA&E with kneeinjury
No seriousabnormality
Conservativemanagement
Conservativemanagement
Abnormality
Offersurgery
Surgery
No surgery
Symptomspersist
Symptomspersist
Surgery
Surgery
No surgery
No surgery
Path probability Pathcost
QALY
MRI
0.05
0.95
0.80
0.20
0.95
0.05
0.800.20
0.400.60
0.400.60
0.900.10
0.500.50
0.900.10
0.0475
50
47.5
0.0020
550
47.5
40
47.5
47.5
47.5
47.5
47.5
47.5
0.0002
0.0001
0.0001
0.632
0.1847
0.0205
0.0091
0.01347
0.1444
0.00381
0.0038
0.0057
A
A
A
A
A
A
A
A
B
B
B
B
B
B
50
50
550
550
550
550
550
50
50
50
50
50
40
40
40
40
40
0.96
0.04
0.700.30
0.90
0.10
0.90
0.10
A = Resolution of symptomsB = Residual symptoms
Expected cost = £244Expected QALYs =46.92
47
Incremental Cost-Effectiveness Ratio Is MRI cost-effective compared to listing patients for
arthroscopy for patients attending A&E?
Yes, MRI is less costly and more effective compared to surgical listing
ICER = (£244-£410)/(46.92 QALYs-46 QALYs)
Leads to a negative ratio, which are never reported as they have more than one potential interpretation MRI is instead said to be ‘dominant’
Or arthroscopy is said to be ‘dominated’
48
Decision Trees: Limitations Useful for ‘simple’ problems
Decision trees do not ‘efficiently’ model events that occur repeatedly Trees become too ‘bushy’ and become too unwieldy to
manage
Decision trees implicitly assume that events occur instantaneously That is there is no formal consideration of time
Sometimes it is important that variables are related to time Eg. the probability of all cause mortality increases with age.
49
Solution: Markov Models • Main difference is an explicit consideration /
incorporation of time and facilitate modelling repeat events
• At any time, individuals are in one of a finite set of health states
• Health states are (typically informed) by disease process and reflect progression of the disease
• Individuals move from state to state according to a set of transition probabilities over stated ‘time cycles’. • Eg a month or a year
50
Basic Information Requirements For A Markov Model • Draw model structure
– Reflecting relevant health states
• Calculate and Assign
– Probabilities associated with ‘transitions’ between states in relation to cycle length
– Assign starting distribution (where to start hypothetical patients)
– Assign costs and health benefits to health states
– When to stop analysis (termination rule)
51
Example of a HIV-Related Markov model
52
Chancellor et al.(1997)
Pharmacoeconomics
A
B
C
D
Markov Models: Limitations Sometimes difficult to define a disease in terms of a set
of mutually exclusive health states
Markov models are often said to be ‘memory-less’
Probability of disease progression only depends on the health state at the beginning of the previous cycle
However there are ways around this
Add more health states
Fitting extra variables with conditional statements
Tunnel states
Individual based simulations and ‘tracker / label’ variables
53
Discrete Event Simulation (DES) Very powerful and flexible modelling approach
Based on microsimulation techniques
Markov modelling (MM) is discrete with respect to time Fixed cycle length eg. 1 year based on probabilities Pre specified set of mutually exclusive health states
DES is discrete with respect to events (eg. death, change in weight, development of diabetes) Typically time is not fixed, samples to form an order of events with
variable times between A ‘time to’ analysis, time ‘jumps’ from events to events
No fixed health states, more or less anything ‘competing’ can be incorporated Therefore much more flexible but programming is more complex and
layering on probabilistic sensitivity analysis can be difficult
54
Basic Features of a DES Construct lists of
Events (eg. non disease related death, type 2 diabetes, weight gain, CVD)
Activities (eg. death following CVD)
Typically assumed to be timeless when they occur
Data required on
Time to events of interest (eg. survival curves, predictive risk equations)
Costs
Effects / utilities
55
An Example To examine the clinical and cost-effectiveness of e-
learning devices for obese people Eg internet based ‘help’ programmes
Decision model based
Life time horizon
NHS cost perspective
Outcome expressed in terms of QALYs
Treatment effect driven by changes in body mass index using a series of interlinked equations to predict times
to events
56
Weight Loss Model (Discrete Event Simulation)
57
Miners et al
HTA, in press
E-learning Devices v Dietary Advice
QALYs Mean costs
DA 12.48 £5,115
E-Learning 12.49 £5,707
Incremental 0.01 £591
58
Mean cost per QALY = approx. £86,321
Miners et al. unpublished
What is a Good Decision Model? One that accurately reflects future events at time t+1?
But you won’t know this at time t!
One that helps to make better decisions at time t compared with not having a model?
Is this testable?
Is a road map a bad map if you are using it to navigate a plane?
Probably, but does this make it a bad map?
Depends on context
Difficult question to answer
$Weinstein et al. 2001 Value in Health 4(1):348-61
Problems With Models O’Brien – ‘Frankenstein’s Monster’
Combining heterogeneous pieces of information
Reality of systematically identifying all relevant evidence?
Illogic of working assumptions
Lack of transparency
Need for careful and detailed write up.
Ability to manipulate results
60
O’Brien et al. 1995 Medical Care. 34(12): DS5-DS10
61
How Results Are Used It can be very difficult to understand a decision model
Submissions can either come from manufacturer’s or from independent (university-based) assessment teams
Important that submissions are accompanied by a detailed text report and computer coding is submitted
Need to be confident that the two match!
Manufacturer submissions are critiqued by assessment teams
62
How Results Are Used Committee meetings (for Technology Appraisals
Program) tend to be split in two First – discussion of the underlying clinical evidence
Second – discussion on the cost-effectiveness evidence
If underlying clinical evidence is considered to be weak, then c0st-effectiveness discussions tend to be short Weak – poor quality or no RCT data
Is this sensible?
Something that is clinically poorer but much cheaper could be the most cost-effective option?
63
How Results Are Used All submitted economics evaluations are decision
model based
The cost-effectiveness discussions can be complex, although committee members are involved, the assessment team and manufacturers are presents to aid understanding
Often considerable amount of time is spent trying to understand why manufacturer models and assessment team models lead to different conclusions But this can be / is a useful process
Highlights area’s which need to be considered
64
Ideally Will Reflect ‘Reference Case’ Perspective on costs National Health Service
Perspective on outcomes All health effects on individuals
Measure of health effects QALYs
Source of data for
measurement of health
related quality of life
Reported directly by patients and/or
carers
Source of data for
measurement of health
related quality of life
Representative sample of the public
Discount rate 3.5% per year for costs and health
effects
Equity weighting An additional QALY has same weight
for all patients 65
In General..... If a technology is underpinned by good evidence of
clinical effect, and is associated with an incremental cost per QALY gained of less than £30,000, then it is recommended for use on the NHS
Above this, have to be other reasons for giving a positive recommendation
Eg. No other active treatment options, ‘end of life considerations’
66
No Fixed Threshold…
Probability
of rejection
Cost per QALY (£’000) 10 20 30 40 50
0
1
Why NICE doesn’t have a fixed cost effectiveness threshold?
NICE
DECISIONS Equality & Diversity
legislation Innovation
Social Value Judgements
Extent of
uncertainty
Additional health benefits
Cost per QALY
Breakdown of recommendations up to TA199
all technologies n %
recommended for routine use or under specific circumstances 321 83
‘no’ or ‘only in research’ 64 17
cancer drugs
recommended for routine use or under specific circumstances 70 72
‘no’ or ‘only in research’ 27 28
Experience so far
• 200 appraisals published to end Sept 2010
(56 STAs, 144 MTAs, 385 individual decisions)
Recommendations >£30,000 per QALY
Topic ICER (‘000)
Severity End of life Significant
innovation
Disadvantaged
population
Children Corporate
responsibility
Riluzole
38-42
Temozolomide
(glioma)
35
Trastusumab
(breast cancer)
37.5
Imatinib
(CML)
36-65
Bortezomib
(myeloma)
32.5
Pemetrexed
(mesothelioma)
34.5
Sunitinib (renal
cancer
55
Insulin pumps
Uncertain
After Decisions Have Been Made Important that when initial and final decisions are
explained in detail, and linked to evidence in order to be transparent
Technical team at NICE responsible for this, along with experienced medical writers
Appeal process exists, although relatively few appeal points are upheld
71
Future Challenges? Value based pricing is being proposed
Where the price of the drug the NHS is prepared to pay will directly reflect anticipated health (and other?) benefits
Problems and challenges of implementing this are likely to be signficant
Eg. At the moment the committee tries to establish whether an ICER is generally above or below £20-£30,000 per QALY
In future, might have to be much more precise
72
Summary Economic evaluation has been used by NICE and
other institutes to inform technology reimbursement decision for a number of years
It is imperfect, but it does seem to work
Certain structural arrangements have helped
National data on unit costs, life tables, utility data
Important that these data are collected if they do not exist
Processes are just as important as methods
73