Editorial

10.1586/14737167.8.4.329 © 2008 Expert Reviews Ltd ISSN 1473-7167 329www.expert-reviews.com

Using economic evidence as a support tool for policy decisions: Herculean or Sisyphean effort?Expert Rev. Pharmacoeconomics Outcomes Res. 8(4), 329–332 (2008)

Dominika Wranik, PhDSchool of Public Administration, Dalhousie University, Halifax, Nova Scotia B3H3J5, CanadaTel.: +1 902 494 3764Fax: +1 902 494 7023dwl@dal.ca

“At the practical level the use of economic evidence in policy making could be compared with the use of caution by

teenage snowboarders.”Most healthcare systems have a public com-ponent. Public sector involvement is largein some systems such as the UK, Canada orAustralia, and smaller in other systems suchas the USA. Regardless of size, the issuesfaced by government include the allocationof funding of healthcare therapies and pro-grams, and the funding of drugs on a for-mulary. At the conceptual level, there is abroad consensus between policy makersand economists that resources should beused efficiently and that therefore economicanalysis should be used. At the practicallevel, the use of economic evidence in pol-icy making could be compared with the useof caution by teenage snowboarders.

Basics of economic analysis in healthcareThe market fails to allocate resources tothe healthcare sector and within the sectorin a way that is socially optimal. The mar-ket for healthcare fails for several specu-lated and some evidenced reasons. Theseinclude large positive externalities createdby healthcare care, asymmetry of informa-tion between buyers and sellers and largeuncertainties; the latter two resulting in anonindependent demand curve.

Economic analysis of efficiency is usedfor resource allocation decisions when themarket fails to do so. Efficiency can be esti-mated using cost–benefit, cost–effective-ness (CE) or cost–utility (CU) analysis.Only the latter two are used in healthcare,since the monetary valuation of healthbenefits is not acceptable on ethical and onpractical grounds.

Both a CE and CU analysis follow asimilar logic. Both are comparative analy-ses and both compare the cost per unit ofoutcome of competing therapies. The unitof outcome is measured differently. CEanalysis uses a physical unit of outcome,which could be an interim marker of aclinical study (e.g., cholesterol level reduc-tion) or a broad measure of overall healthbenefit (e.g., life years gained). The CUanalysis adjusts a life year gained throughtherapy by a valuation of utility of this life.The reason is that a year of perfect healthis not equivalent to a year with fatigue orto a year spent in a wheelchair.

Suppose there are three competingtherapies, A, B and C. In the simplestcase, all three aim at the same goal, areindicated for the same condition andtheir effect can be measured in the sameunits of outcome (e.g., cholesterol levels).An estimation of cost per cholesterol levelreduction allows us to discriminatebetween these three therapies. The econo-mist’s decision would be to choose thetherapy with the lowest CE ratio. Alreadythe simplest case is fraught with practicaland methodological challenges, which aredescribed below.

The real-life scenario is generally farmore complicated. Clinical benefit canrarely be measured in identical physicalunits for each therapy. The decisionmight be between an acute therapy forasthma and a preventive therapy toreduce asthma and diabetes incidence.The decision might also be between anew therapy for breast cancer and smok-ing cessation therapy. In these situations,

330 Expert Rev. Pharmacoeconomics Outcomes Res. 8(4), (2008)

Editorial Wranik

economists estimate a CE ratio expressed in terms of life yearsgained. Already at this point methodological problems arise,since the estimation of life years gained from any therapyrequires assumptions and modeling techniques that are oftensubject to debate.

The next possibility is to discount the life years gained esti-mation for years lived in subperfect health. In principle the ideais widely accepted, since rarely do scholars disagree that a yearof perfect health is preferable to a year of life with an illness. Inpractice, the assignment of utility values to particular healthstates is a bone of contention. The myriad of argumentsrevolves around techniques used to elicit preferences from peo-ple, as well as the population from which to draw a sample ofrespondents. In addition, the discounting of life years usingutility values is said to discriminate against older patients andto disregard equity considerations [1–4].

Aside from methodological difficulties, which are not thefocus of discussion here, the process by which CE and CUstudies are used to support policy decisions is fraught with chal-lenges. Below is an elaboration on the challenges associatedwith the use of economic analysis as is intended in theory.Many of these have been described in the literature. Not dis-cussed in the literature are the difficulties caused by the chrono-logical order of the decision-making process and the timing ofdecision making.

A large number of barriers to the use of economic evidence inpolicy making have been described in the literature. These areclassified by Eddama et al. into institutional and political fac-tors, cultural reasons and methodological problems [5]. The clas-sification omits the challenges associated with knowledge trans-lation and process. Many barriers arise due to themisunderstandings between economists and policy makers.

Policy makers in world of economicsThe policy maker is reluctant to use and publicly announcethe use of economic evaluations as indented by the economist.The public and especially the media are quick to label eco-nomic evidence as heartless and not compassionate. This rep-utation is created by the misunderstanding of what economicevaluation is and what its purpose is. Literature refers to a cul-tural tendency to place a higher weight on evidence of effec-tiveness (and other criteria) than evidence of cost. This ten-dency renders the reliance on CE/CU studies as politically notfeasible [1,3–7]. This line of arguments demonstrates a lack ofunderstanding of what economic evaluation does. First, eco-nomic evaluation is based on the assumption of clinical effec-tiveness. No economist will claim that cost is more importantthan effectiveness. All economists presume that clinical effec-tiveness has already been demonstrated by a clinicalresearcher. Second, economic evaluation is used to allocatehealthcare dollars in a fair manner; what might appear non-compassionate towards one person in fact compassion forsociety as a whole.

The lack of understanding, knowledge and transparency can,among other factors, be the driver behind political decisionmaking. The policy maker is also overwhelmed by the amountof detailed technical knowledge necessary to understand eco-nomic evaluation. Economists are quite good at making theirwork inaccessible to the untrained person. Policy makers rarelyhave training in health economics and therefore have a poorknowledge of CE and CU analyses. This stands in the way ofusing existing economic studies and even more so in the way ofconducting them [4,6–9].

Even those with some degree of training discover a generallack of transparency in health economic evaluations, a lack ofguidelines to follow and poorly specified decision-making crite-ria [2,7,8]. The clinical evaluation of new drugs is heavily regu-lated and monitored. Economic evaluation, on the other hand,has little regulation and monitoring. Guidelines that exist arenot nearly as comprehensive and rigorous as those for clinicaltrials. Many are concerned with the bias that is introduced intoeconomic evaluation by virtue of their sponsorship; it is sug-gested that economic studies that are funded by the pharma-ceutical industry are bound to be fraught with bias [6,9]. Thisproblem is circumvented in clinical trials by way of externalregulatory bodies.

Economists in the world of policy making On the flipside of the coin is the economist who enters theworld of policy making and has to adapt her approaches to thisworld. The economist understands the methods and underly-ing assumptions behind economic analysis. She knows that aneconomic evaluation is not meant to replace a clinical evalua-tion, but rather is to be used only if the clinical trial demon-strated effectiveness. She is trained to operate within the mazeof methodological approaches and guidelines that are availablefor economic analysis. She has a complete and thoroughunderstanding of the textbook. What are the challenges theeconomist encounters upon entering the policy world?

The economist quickly discovers that the policy world is notarranged as described in her textbook. This description mightlead her to believe that she will face a choice between a setnumber of therapies, A, B and C. Each will be assigned a CEratio or, most often, an incremental CE ratio (ICER) to guidethe choice whenever new therapies are to replace an existingone. She also expects that additional consideration will be givento equity and fairness concerns, which are not fully accountedfor in the economic analysis alone.

The reality is that therapies do not enter the market simulta-neously, therapies already on the market were often not evalu-ated in terms of efficiency and the economist is exposed to onlya small portion of the market. To make life more complicated,access to information is limited. The economist is presentedwith a CE/CU study of one therapy that just entered the mar-ket and asked “Is this therapy cost effective?” Of course, thequestion cannot be answered without further qualification. The

www.expert-reviews.com 331

Using economic evidence as a support tool for policy decisions Editorial

posing of this question demonstrates a lack of understanding ofwhat CE analysis is. CE analysis is a comparative analysis andits results tell us little, unless related to the CE ratios of othertherapies or to the budget.

The first option implies that one of several therapies for aspecific condition is to be chosen. On other grounds, it wasalready decided that this condition requires a therapy. The pol-icy maker’s question becomes “Which one is most cost effec-tive?”. This the economist could answer, except she finds thatonly one therapy is proposed for the condition and it is com-pared with an existing standard of care in the analysis. There isan ICER with no criteria for judgment.

The specification of a benchmark (e.g., US$50,000 per qual-ity-adjusted life year) as the cutoff for funding new therapiescould circumvent the problem of not having a comparator.There is no such benchmark, however, because it is politicallynot attractive to specify a cutoff value. There are many ethicalreasons for this. Some diseases, such as cancer, are notoriouslymore expensive and would always come out as not cost effec-tive. The benchmark is also not attractive because it is very sim-ple to scrutinize in the media: a clear simple commitment isalways shunned by government.

The last option is to interpret a CE ratio within the contextof the healthcare budget. If a series of therapies were presented,the economist could choose therapies with the lowest ICERsuntil the budget was exhausted. In practice, once more, theapproach fails because it turns out that the size of the budget isnot clear. The exact amount assigned to cancer therapies, forinstance, is not known and the amount assigned rarely matchesthe amount actually spent. In addition, as mentioned, the ther-apy under consideration is on the table in isolation from othertherapies. Is it cost effective? There is no answer.

Supposing that the aforementioned obstacles are overcome,there remains an additional difficulty with generalization orresults and transportation to local contexts. An efficiency eval-uation is typically performed once (if at all) for any particulartherapy. Any estimation of CE or CU requires the specifica-tion of resource use and costs for the therapy and all compara-tors, as well as a specification of utility values. Resource useand costs vary between regions and countries. A CE or CUstudy conducted in the UK may produce different results thanone conducted in Canada. In addition, contextual factors suchas population characteristics, socioeconomic characteristics orcultural characteristics are not taken into account in a CE/CUstudy, but might have great influence over the effectivenessand efficiency of a therapy within the local context [1,4,5]. Fur-thermore, a CE or CU ratio is calculated as a per year or perquality-adjusted life-year cost.

The last task of the health economist, after all other hurdleshave been jumped, is to find funding for a new efficient therapy.The public purse is characterized by rigidities both between andwithin budgets. Funding will not be shifted from the educationbudget to the healthcare budget, even if all healthcare interven-tions are shown to be more cost effective than educational pro-gramming. (This is probably good news for the healthcarebudget, since most social interventions appear more cost effectivethan healthcare in improving population health, but that isbeside the point). Similarly, there is rigidity within the healthcarebudget and shifting of resources is difficult [3,4,9]. The rigidity issupported by incentive structures within the public sector.Bonuses and performance reviews of program managers are tiedto the size of their program, therefore no manager is interested inhaving their program reduced, even if not cost effective. Thisrigidity might appear paradoxical to the lack of a clear specifica-tion of budget size, however, there is logic to the madness. Theinability to shift resources is an internal public sector challengethat is difficult to track and therefore not likely to come underthe scrutiny of the media. A fixed budget size lends itself tomedia attack, as do all precise monetary specifications. Together,these two characteristics lead to ever increasing budgets – we addfunding to the total pot, cannot shift within the pot and rarelyare able to cut funding, since the latter is politically not attractive.

To sum up, the road to using health economics in health pol-icy as was intended by the economist is long and filled withobstacles. Some are likely to remain: policy decisions will alwaysinclude an element of the political, for instance. Others can beovercome, however, making the impossible task possible, thoughchallenging: training can be offered to policy makers, resourcescan be devoted to economic analysis and interpretation and thereputation of economic studies can be changed.

AcknowledgementsI would like to thank my student Amelia DeMarco for assistance with theliterature review for this editorial.

Financial & competing interests disclosureDominika Wranik is an Assistant Professor at the School of PublicAdministration at Dalhousie University, where she teaches HealthEconomics and Program Evaluation. For the past 2 years, DominikaWranik has been sitting on an advisory committee to the Nova Scotiagovernment. The committee evaluates clinical, economic and otherevidence that is available on drugs that are being considered for publicfunding. She has been in charge of reviewing economic evaluations. Theviews expressed are her own and do not represent those of the advisorycommittee or Dalhousie University.

The author has no other relevant affiliations or financial involvementwith any organization or entity with a financial interest in or financialconflict with the subject matter or materials discussed in the manuscriptapart from those disclosed.

No writing assistance was utilized in the production of this manuscript.

“…the road to using health economics in health policy as was intended by the economist is long

and filled with obstacles.”

332 Expert Rev. Pharmacoeconomics Outcomes Res. 8(4), (2008)

Editorial Wranik

References1 Robinson R. Limits to rationality: economics,

economists and priority setting. Health Policy 49, 13–26 (1999).

2 Drummond M, Brown R, Fendrick AM et al. Use of pharmacoeconomics information – report of the ISPOR task force on use of pharmacoeconomic/health economic information in health-care decision making. Value Health 6(4), 407–416 (2003).

3 Zwart-van Rijkom JEF, Leufkens HGM, Busschbach JJV et al. Differences in attitudes, knowledge and use of economic evaluations in decision making in the Netherlands, Pharmacoeconomics 18(2), 149–160 (2000).

4 van Velden ME, Severens JL, Novak A. Economic evaluations of healthcare programmes and decision making: the influence of economic evaluations on different healthcare decision-making levels. Pharmacoeconomics 23(11), 1075–1082 (2005).

5 Eddama O, Coast J. A systematic review of the use of economic evaluation in local decision making. Health Policy 86, 129–141 (2008).

6 Hoffman C, Stoykova BA, Nixon BA et al. Do health care decision makers find economic evaluations useful? The findings of focus group research in UK health authorities. Value Health 5(2), 71–79 (2002).

7 Williams I, Bryan S. Understanding the limited impact of economic evaluation in health care resource allocation: a conceptual framework. Health Policy 80, 135–134 (2007).

8 Willimans I, Bryan S. Cost–effectiveness analysis and formulary decision making in England: findings from research. Soc. Sci. Med. 65(10), 2116–2129 (2007)

9 van Gool K, Gallego G, Haas M, Viney R, Hall J, Ward R. Economic evidence at the local level: options for making it more useful. Pharmacoeconomics 25(12), 1055–1062 (2007).