using respondent uncertainty to mitigate hypothetical bias

Using Respondent Uncertainty to MitigateHypothetical Bias in a Stated Choice Experiment

Richard C. Ready, Patricia A. Champ, and Jennifer L. Lawton

ABSTRACT. In a choice experiment study, will-ingness to pay for a public good estimated fromhypothetical choices was three times as large aswillingness to pay estimated from choices requiringactual payment. This hypothetical bias was relatedto the stated level of certainty of respondents. Wedevelop protocols to measure respondent certainty inthe context of a choice experiment, and to calibratehypothetical choices using these certainty measures.While both the measurement of respondent certaintyand the use of certainty measures to calibrateresponses are complicated by the multiple-choicenature of choice experiments, calibration successful-ly mitigated hypothetical bias in this application.(JEL Q51)

I. INTRODUCTION

Stated preference (SP) approaches tononmarket valuation involve asking studyparticipants to make hypothetical trade-off(s) between their wealth and the non-market good(s) of interest. Many studieshave compared SP responses to actualbehaviors involving choices with real mon-ey at stake. List and Gallet (2001) conduct-ed a meta-analysis of results from 29 suchstudies and found that values estimatedfrom hypothetical responses were threetimes as large (on average) as thoseestimated from observed behavior in choic-es involving real money commitments.Little and Berrens (2004) expanded Listand Gallet’s meta-analysis by adding morestudies and including more variables thatmeasured methodological differencesamong studies. They also found hypothet-ical willingness to pay (WTP) to be approx-imately three times that of actual WTP.Murphy et al. (2005) conducted a similar

meta-analysis using only WTP studies.They found that hypothetical WTP was,on average, 2.6 times as large as actualWTP. The difference between hypotheticalvalues and actual payment values is referredto as hypothetical bias.

There has been interest in finding ways tomodify how SP questions are asked and/orcalibrate the values obtained to eliminate oradjust for hypothetical bias. One approach,used primarily with the dichotomous choice(DC) contingent valuation (CV) format,involves identifying SP respondents who areunsure of their preferences. There is evi-dence that respondents who are unsurewhether they would pay a specified amountin return for an increase in a public goodtend to say yes to a DC CV valuationquestion (Ready, Navrud, and Dubourg2001; Berrens et al. 2002). These unsurerespondents can be identified with a follow-up question of the form, ‘‘How sure are youthat you would choose the option youindicated?’’ (Li and Mattson 1995). Acommon approach to mitigating hypothet-ical bias is to recode DC CV ‘‘yes’’responses where the respondent reports alow level of certainty to ‘‘no’’ responses,and accepting ‘‘yes’’ responses only if theyare given with high confidence (Champ etal. 1997; Johannesson, Liljas, and Johans-son 1998; Champ and Bishop 2001). Littleand Berrens (2004), in their meta-analysis ofvalidity studies, found that studies thatcalibrated responses based on the respon-

The authors are, respectively, associate professor,Pennsylvania State University; economist, U.S. ForestService, Rocky Mountain Research Center; statisticalconsultant, Pitney Bowes MapInfo, Ann Arbor. Thisresearch was partially funded through agreement 00-JV-11221617-198 with the U.S. Forest Service, RockyMountain Research Station.

Land Economics N May 2010 N 86 (2): 363–381ISSN 0023-7639; E-ISSN 1543-8325

E 2010 by the Board of Regents of theUniversity of Wisconsin System

dent’s stated level of certainty had lesshypothetical bias.

While this ‘‘certainty threshold’’ ap-proach to calibration has been used pri-marily for the DC CV format, the choiceexperiment (CE) format1 has increasinglybeen used to value multidimensional non-market goods. An advantage of the CEformat is that it can be used to value bothdiscrete changes in quantities of publicgoods and marginal changes in the attri-butes of those goods. Recent split samplecriterion validity studies have found thathypothetical bias can exist for the CEformat as well (e.g., Lusk and Schroeder2004; List, Sinha, and Taylor 2006). Couldhypothetical bias in CEs also be related torespondent uncertainty? When more thantwo options are presented to a respondent,measurement of respondent uncertaintybecomes more complicated, as does appli-cation of certainty threshold calibration. Tosee this, consider a respondent who choosesone option out of three available, but stateslow confidence in that choice. Should thatrespondent be recoded to one of the othertwo options? Which one? Should calibra-tion depend on whether the chosen optionhas a higher cost than the other options?

In this paper we explore these issues. Thespecific objectives of this study are to (1)construct a choice experiment with hypo-thetical and real payment treatments anddetermine whether hypothetical bias exists inthe experiment, (2) develop a method formeasuring respondent uncertainty in CEsurveys, (3) determine whether hypotheticalbias is related to respondent uncertainty, and(4) determine whether respondent statementsof uncertainty can be used to mitigatehypothetical bias in this application.

II. PREVIOUS RESEARCH ON SPRESPONDENT UNCERTAINTY

Why might SP respondents be uncertainover their behavior or their preferences?

First, there may be details of the valuationscenario that are not completely described,including the exact characteristics of thegoods and how they would be provided(Hanemann and Kristrom 1995). Second,the respondent may have insufficient timeto evaluate his own preferences, insufficientexperience with the good, or insufficientmotivation to invest the time and effortneeded to fully consider the choice task andoptimize over his preferences (Alberini,Boyle, and Welsh 2003; Loomis and Ek-strand 1998). It has been argued that sometrade-offs are inherently impossible to makewith precision, such as trade-offs thatinvolve moral issues, but that even in thosecases respondents can place upper andlower bounds on their WTP (Opaluch andSegerson 1989; Kooten, Krcmar, and Bulte2001).

Respondent Uncertainty in DC Valuation

Traditional SP questions do not accom-modate uncertainty. For example, in theDC CV format, the only allowable respons-es are ‘‘yes’’ and ‘‘no.’’ A respondent who isunsure what choice she would make if thedecision context were real is forced into oneof those two responses. Several studies(Welsh and Poe 1998; Loomis and Ekstrand1998; Ready, Navrud, and Dubourg 2001;Berrens et al. 2002) have found that DCrespondents who self-identify as beingunsure over their preferences tend to say‘‘yes’’ to such a forcing question. Theauthors speculate that this form of yea-saying could be responsible for much of thehypothetical bias seen in DC values.

If some DC CV ‘‘yes’’ responses are madeby respondents who are actually unsureover their preferences, it may be possible toidentify those individuals and adjust (cali-brate) their responses to account for thatuncertainty. Several studies have attemptedto do so using information from certaintyfollow-up questions. One of the first wasthat of Li and Mattson (1995), who askedall DC respondents how certain they wereof their DC response, on a scale from 0% to100%. They used these responses as state-

1 This format is also referred to, variously, as theattribute-based method, the stated choice format, andsometimes conjoint analysis, though that term is also usedfor techniques that rate options.

364 Land Economics May 2010

ments of probability, so that a respondentwho said ‘‘yes’’ with a confidence level of 80was interpreted as having an 80% probabil-ity of actually choosing ‘‘yes’’ and a 20%likelihood of actually choosing ‘‘no.’’ Theyadjusted the usual likelihood function toinclude both possible outcomes and theirprobabilities for each respondent. Theyfound that accounting for respondent un-certainty decreased estimated WTP valuesby 50% relative to the standard approach toanalyzing DC responses.

Several subsequent studies have exploredhow to best measure respondent uncertain-ty and use those measurements to adjustestimated WTP values (Loomis and Ek-strand 1998; Berrens et al. 2002). Othershave investigated the role that respondentuncertainty plays in motivating differencesamong elicitation formats in estimatedWTP (Welsh and Poe 1998; Ready, Nav-rud, and Dubourg 2001). While the findingsof these studies generally support thehypothesis that respondent uncertaintycan lead to hypothetical bias, they lackexternal validity criteria against which theirWTP estimates can be compared.

Champ et al.’s (1997) was the first study toinclude certainty follow-up questions in asplit sample study with hypothetical and realmoney treatments. In the context of dona-tions for a public good, they asked DChypothetical payment respondents how cer-tain they were of their response, with a 1- to10-point scale from ‘‘very uncertain’’ to‘‘very certain.’’ They found the proportionof respondents who said that they woulddonate in the hypothetical treatment waslarger than the proportion who actuallydonated. However, if they applied a certaintythreshold of 10, that is, counted as positiveresponses only the ‘‘yes’’ responses with astated certainty level of 10, the hypotheticalWTP estimate was not statistically differentfrom the actual donation WTP estimate.This type of adjustment, where only thehighest response level is considered a positiveresponse, is called a ‘‘top box’’ analysis in themarketing literature.

Champ and Bishop (2001) applied asimilar approach and found that a certainty

threshold of 8 results in equivalence be-tween the hypothetical and real treatments.They also found that the characteristics ofrespondents who met this certainty thresh-old closely matched the characteristics ofthose who donated in the actual treatment.They take this as evidence that the certaintystatements can be used to identify whichindividual respondents in the hypotheticaltreatment would actually make a donationif placed into a real choice situation.

Other validity studies have used thecertainty threshold approach and havefound that it can mitigate hypothetical bias,though there is some variability in thethreshold level that provides the closestmatch between hypothetical and actualWTP values. While many studies havefound that the certainty threshold that bestmitigates hypothetical bias is at the very thetop of the certainty scale (Blomquist,Blumenschein, and Johannesson 2008;Champ et al. 1997), others have found thata somewhat lower threshold works best(Champ and Bishop 2001; Ethier et al. 2000;Vossler et al. 2003; Poe et al. 2002). Allstudies found that hypothetical bias existed,and that calibration using a fairly strictcertainty threshold (7 or higher on a scale of10) was needed to mitigate that bias. Studiesthat used verbal descriptions of certaintylevels instead of ordinal scales find similarresults (Blumenschein et al. 1998, 2008;Johannesson, Liljas, and Johansson 1998).

An alternative to the certainty thresholdapproach to calibration is to estimate astatistical bias function (Johannesson et al.1999; Blomquist, Blumenschein, and Jo-hannesson 2008). This can be done onlywhen hypothetical and real choices areobserved for the same respondents. Forrespondents who said ‘‘yes’’ to the hypo-thetical DC question, a probit regression isestimated that predicts the probability therespondent will actually choose the yesoption when faced with a real choice. Theresponse to a certainty follow-up to thehypothetical DC question is used as anexplanatory variable in the probit regres-sion. Johannesson et al. (1999) found thatthe certainty follow-up response was posi-

86(2) Ready, Champ, and Lawton: Hypothetical Bias in Choice Experiments 365

tively related to the probability of an actual‘‘yes’’ choice. They also found that theprobability of an inconsistent response washigher for higher prices. They used theprobit regression to predict actual behaviorbased on the hypothetical responses and thecertainty statements and found that such acalibration successfully mitigated the hypo-thetical bias. They also found that acommon statistical bias function could beestimated for two different private goods.Blomquist, Blumenschein, and Johannes-son (2008) updated Johannesson et al.’s biasfunction by including additional data for athird good, a health treatment protocol.

Note that calibration using a certaintythreshold or statistical bias function is notmade for hypothetical ‘‘no’’ responses.Studies that have compared hypotheticaland actual behavior for the same respon-dent find that hypothetical ‘‘no’’ respon-dents rarely choose the ‘‘yes’’ option whenprovided the opportunity in a real paymentsituation (e.g., Johannesson et al. 1999).There appears to be an asymmetry in howrespondent uncertainty biases hypotheticalresponses.

Hypothetical Bias in CE Valuation

The few CE studies that have evaluatedhypothetical bias have largely reportedresults consistent with the DC hypotheticalbias studies. These studies can be dividedinto studies that include an opt-out ‘‘nopurchase’’ option and those that do not.Studies valuing attributes of private goodsusually include an opt-out option, in whichcase it is possible to estimate the probabilityof purchase of the good as well as themarginal WTP for changes in attributes ofthe goods. When an opt-out option is notprovided, it is only possible to estimatemarginal WTP for attributes of the good.

Lusk and Schroeder (2004) comparehypothetical and real choices over purchaseof steaks with different qualities. Theyfound that the probability of buying a steakwas significantly greater in the hypotheticaltreatment than in the real payment treat-ment, but that marginal WTP for changes

in steak attributes were similar between thetwo treatments. Similarly, List, Sinha, andTaylor (2006) found that the probability ofbuying a sports trading card was overstatedin hypothetical choices, but that marginalWTP for improvements in card quality weresimilar in the hypothetical and real treat-ments.

For a public good, List, Sinha, andTaylor found that hypothetical CE re-spondents overstated their likelihood ofdonating for the good by 60%. Likewise,respondents in the hypothetical paymenttreatment who chose to donate also over-stated the probability of choosing the moreexpensive donation (versus the less expen-sive donation) by 100%.

Taylor, Morrison, and Boyle (2007)found, for both a private good and a publicgood, that hypothetical bias existed in theproportion of stated choice respondentswho opted in (i.e., chose an option withpositive cost), and that the magnitude of thebias was greater when the hypotheticaltreatment used a provision rule that wasnot incentive compatible. When the provi-sion rule was not incentive compatible,marginal WTP estimates from hypotheticalchoices were also biased upward by 30% to35% relative to actual payment choices.They found mixed results on whethermarginal WTP estimates from hypotheticalchoices were biased when an incentivecompatible provision rule was used.

Cameron et al. (2002) found that theproportion of CE respondents who chose toparticipate in a green energy program washigher in the hypothetical payment treat-ment than in the actual payment treatment,and that WTP estimates from hypotheticalpayment responses exceeded those estimat-ed from actual payments, but the differenc-es in WTP were not statistically significant.Because they were limited to only one actualpayment scenario, Cameron et al. could notexplore whether marginal WTP for attri-butes differed between the real and hypo-thetical payment treatments.

Turning to studies that did not include anopt-out option, Carlsson and Martinsson(2001) asked respondents about hypotheti-


cal choices over different donations towildlife programs, followed by identicalreal choices. In this within-sample compar-ison, they found no significant differences inmarginal WTP for attributes of the dona-tion options. Johansson-Stenman andSvedsater (2007) essentially repeated Carls-son and Martinsson’s experiment (withdifferent money amounts and programs)but used a split-sample design with a real-only treatment and a hypothetical then realtreatment. In contrast to Carlsson andMartinsson, Johansson-Stenman and Sved-sater found that marginal WTP for attri-butes estimated from hypothetical choiceswere significantly larger than those from thereal-only treatment. Alfnes and Rickertsen(2002) compared hypothetical and realchoices over steaks with different charac-teristics. They found that marginal WTP forsteak attributes estimated from hypotheti-cal payment choices were 5 to 11 timeshigher than those estimated from realsecond-price auctions. Blamey and Bennett(2001) compared hypothetical choices overtoilet paper purchases to real purchasesobtained from scanner data. They foundthat parameters of models estimated fromhypothetical and real choices differed, butthat the hypothetical models did a fairlygood job predicting the aggregate marketshare of products with green attributes. Dueto limited variation in the set of realproducts available, they were unable tocompare marginal WTP for product attri-butes between the hypothetical and realpurchase treatments.

To summarize, in almost all studies,hypothetical CE responses generated larg-er WTP estimates than did choices involv-ing actual payments. There is someevidence that, when the study includes anopt-out option, hypothetical bias may bemore of an issue in determining theproportion of respondents who choosean option that costs money and inestimating aggregate WTP for a unit ofthe good than it is in choices among thecostly options and estimates of marginalWTP for attributes of the good, thoughthat evidence is mixed.

Respondent Uncertainty in CE Valuation

Only two studies have been conductedthat attempted to calibrate CE responsesbased on certainty follow-up questions.Norwood (2005) compared hypotheticalpayment CE responses to actual paymentDC choices in a public good experimentwith undergraduate students, where thecontributions to and payoff from the publicgood were points toward their grade. Eachhypothetical choice question had two do-nation options and an opt-out option.Norwood found that a random utilitymodel estimated from hypothetical CEresponses overstated the proportion ofstudents who would contribute to the publicgood, as compared to the actual paymentDC treatment.

For respondents who chose one of thetwo donation options in a CE question, afollow-up question asked how certain therespondent was that she would actuallydonate the amount indicated to the publicgood, if given the opportunity. Unfortu-nately, this form of certainty follow-up issomewhat ambiguous. Respondents couldinterpret the question as asking, ‘‘If givenan actual choice among these three options,how sure are you that you would actuallychoose the option you indicated?’’ Alterna-tively, it could be read as, ‘‘If given theopportunity to actually make the donationyou indicated, how sure are you that youwould actually choose to make the dona-tion?’’ In the first case, the choice is amongall three options, while in the second it isbetween two. A respondent might be highlycertain that he would make a donation, butunsure which of the donation choices heprefers. Such a respondent would answerthe first form of the question with a lowlevel of confidence, but answer the secondwith a high level. It is not clear howrespondents interpreted the question.

Given a low level of confidence, howshould a CE response be recalibrated? In athree-option choice, the respondent couldbe reassigned to either of the other twooptions. Norwood chose to recode all low-certainty CE responses to the opt-out


option. Using this calibration procedure,Norwood found that, with a certaintythreshold of 6, the estimated random utilitymodel predicted a donation rate thatmatched the actual donation rate from thereal treatment. Note that this calibrationprotocol assumes away the possibility thatunsure respondents who indicate in thehypothetical choice question that theywould choose one costly option wouldactually chose the other costly option whenfaced with a real payments choice.

Olsson (2005), in a survey valuing fishstocks and water quality in Sweden, used asimilar protocol for measuring respondentuncertainty in a CE. He asked only onecertainty follow-up question, after all CEquestions were answered. He found thatWTP estimates based on the CE questionswere much larger than those from open-ended or DC valuation questions. Due tothe nature of the good, he could not have areal payment treatment.

Olsson found that applying a certaintythreshold protocol similar to that used byNorwood (where uncertain respondents arerecoded to the opt-out option), decreasedestimated WTP for the program, butincreased marginal WTP for attributes ofthe program. This result is counterintuitive.None of the previous CE studies with actualpayment treatments found that marginalWTP values from hypothetical choices weresmaller than those estimated from realchoices. A protocol designed to eliminatehypothetical bias should therefore notincrease marginal WTP estimates. Theseresults suggests that recoding all uncertainrespondents to the opt-out option gives upimportant information about the marginaleffects of attributes.

In this paper we develop new ways tomeasure respondent uncertainty in CEs thatdistinguish between uncertainty overwhether the respondent would opt out oropt in from uncertainty over which opt-inoption she would choose. We also developprotocols for using that information tocalibrate CE responses in ways that allowswitching from one costly option to anothercostly option.

III. METHODS

Measuring Respondent Uncertainty in CEs

Holmes and Adamowicz (2003) provide agood overview of the theory and practice ofchoice experiments. In each CE question,respondents are presented with sets ofchoice options that vary in the levels ofseveral attributes, and are asked whichoption they prefer from each set. The utilitythat respondent i receives from option j istypically assumed to be a linear function ofthe option’s attribute levels, Aj. Extendingthe usual model, we consider the possibilitythat the utility from each choice includestwo random components, so that the utilityto respondent i from choice j is given by

Uij~X1

k~1

bkAjkzbppjzeijzgij , [1]

where ßk is the marginal utility of attributek, Ajk is the level of nonprice attribute k inalternative j, and ßp is the marginal utility ofthe price attribute, pj. Since an increase inprice decreases income, 2ßp is the marginalutility of income. The first randomcomponent, eij, captures individual-specificfactors that are known to the respondentbut unknown to the researcher. Thesecond random component, gij, reflects therespondent’s own uncertainty over herpreferences.2

Prior to making a binding choice, therespondent may not be sure which optionshe will actually choose. However, she canevaluate her probability of choosing eachoption. If gij is independent and identi-cally distributed according to a Gumbeldistribution, then her subjective probabilityof choosing option j from choice set C isgiven by

2 The reader may be concerned that, with twoindependent error components, the multinomial logitmodel will no longer be appropriate. The distribution of asum of two Gumbel-distributed random terms can bevery closely approximated by another Gumbel distribu-tion. If the error terms are normally distributed, then theirsum will be normally distributed, and a multinomialprobit will still be appropriate.


P( j)~

exp (X

k

bkxjkzbppjzeij)X

m[C

exp (bkxmkzbppmzeim): [2]

Note that the respondent knows the valuesof e when evaluating these probabilities.Ideally, when asked a hypothetical choicequestion, the respondent would choose theoption with the highest subjective probabil-ity. However, experimental evidence onrespondent behavior in DC settings sug-gests that respondents who are forced tomake a hypothetical choice without know-ing their values of g tend to choose morecostly options than they choose in actualpayment situations.

If we wish to calibrate hypotheticalchoices using information about respondentuncertainty, we need to be able to measurethat uncertainty. One approach would be tohave the respondent report his subjectiveprobability for each of the options present-ed. This would likely be a difficult task forthe respondent, however. To make therespondent’s task easier, we stay withquestions of the form, ‘‘How sure are youthat . . .’’ Experience with DC formats isthat respondents believe that they are ableto answer that type of question reliably. Aquestion of the form, ‘‘How sure are you ofyour previous answer?’’ provides informa-tion about P(j) only for the chosen option.In order to fully recover certainty over alloptions, though, it is necessary to ask morethan one of these questions.

The choice sets used in this study includethree options: two costly options, which wecall the opt-in options, and a no cost (opt-out) option. These will be respectivelyreferred to as options A, B, and O, thoughthey were not labeled as such in the surveyinstrument. There are several possible se-quences of follow-up certainty questions thatshould generate similar information aboutrespondent uncertainty. In this study, weemployed two different question sequences.

Our first question sequence (sequence S1)is as follows. For respondents who choseoption A or B in a hypothetical choice, the

respondent was first asked a follow-upquestion of the form3

Q1-S1: How sure are you that you would choose theoption you indicated, instead of one of the other twooptions?

This provides information on P(A) orP(B), depending on which costly option waschosen. We call this the respondent’s‘‘unconditional choice certainty.’’ Thisquestion is similar to but more specific thanthose used by Norwood and Olsson. If therespondent indicated a level of certainty inQ1-S1 less than 100%, she was then asked aquestion of the form

Q2-S1: How sure are you that you would choose eitheroption A or option B, instead of option O?

This provides information on P(A)+P(B).We call this the respondent’s ‘‘opt-incertainty.’’

The second sequence of certainty ques-tions (sequence S2) is as follows. Forrespondents who chose option A or B, thefirst follow-up question was of the form

Q1-S2: You indicated that you would choose one ofthe two options that would cost you money. How sureare you that you would choose option A (B), instead ofoption B (A)?

This provides information on P(A|A<B)or P(B|A<B). We call this the respondent’s‘‘conditional choice certainty.’’ It is condi-tional on choosing one of the two costlychoices. Regardless of the answer to thisquestion, the respondent was then asked thesame opt-in certainty question as in se-quence 1, that is, Q2-S2 took the same formas Q2-S1.

In both sequence S1 and sequence S2,respondents who initially chose the opt-outoption, O, were asked a follow-up questionof the form

Q3: How sure are you that you would choose optionO, instead of one of the other two options?

We call this the respondent’s ‘‘opt-outcertainty.’’ It provides information on P(O).

3 Actual wording of the questions and layout of theresponse screens are available from the authors.


The certainty follow-up questions are sum-marized in Table 1.

Other question sequences are possible,but pretests showed that respondents foundthe questions used in these two sequenceseasy to understand and intuitive.

Study Design

The study was conducted using under-graduate students at Pennsylvania StateUniversity. The public good valued waswildlife rehabilitation. A state-licensedwildlife rehabilitator, Centre Wildlife Care(CWC), collaborated on the study. CWCaccepts and cares for injured or abandonedwild animals, with the goal of releasingthem back into the wild. If this is notpossible, the animals are cared for incaptivity for life. The operations of CWCare funded entirely through donations oftime and money. Not all animals can beaccepted for rehabilitation due to time andmoney constraints.

In the CEs, respondents were offered theopportunity to donate money sufficient tocare for one animal. In each choice ques-tion, the respondent was offered twodonation options and an opt-out option.Respondents were told that CWC must turnaway animals because of limited resources.Respondents were told that, if they chose adonation option, the money would be usedto rehabilitate one animal of the type they

chose. If they chose not to donate, thatanimal would be turned away and wouldpresumably die. CWC agreed to use thedonations for the types of animals indicat-ed, so the respondents choices had realconsequences.

The option attributes were type of animal(mammal, bird, or turtle), common versusless common, whether the animal caneventually be returned to the wild, and thedonation required of the participant, whichtook values of 0 for the opt-out option and$5 or $10 for the opt-in options.4 So that theamount donated equaled the actual cost ofrehabilitation (cost estimates were obtainedfrom CWC), the study provided a donationmatch of $5 for every donation made by arespondent. An example of a hypotheticalchoice is shown in Figure 1.

The good used in this study—rehabilita-tion of an injured or abandoned wildanimal—is a quasi-public good. It is non-rival in that everyone can benefit from itonce it is provided, but it is excludable inthat the respondent will benefit from thatunit of the good only if she pays thedonation. Further, in contrast to quasi-public goods used in previous studies (e.g.,Taylor, Morrison, and Boyle 2007; Cum-

TABLE 1

CERTAINTY TYPES FOR A CHOICE QUESTION WITH TWO COSTLY OPTIONS, A AND B, AND A COSTLESS OPT-OUT

OPTION, O

OptionChosen Question No. Certainty Type Example Question

ProbabilityMeasure

No. of Cases

Version 1 Version 2

A or B Q1-S1 Unconditionalchoicecertainty

How sure are you that youwould choose A insteadof B or O?

P(A) or P(B) 160 —

A or B Q1-S2 Conditionalchoicecertainty

How sure are you that youwould choose A insteadof B?

P(A|A<B) orP(B|A<B)

— 139

A or B Q2-S1 andQ2-S2

Opt-in certainty How sure are you that youwould choose A or B insteadof O?

P(A) + P(B) 153 139

O Q3 Opt-out certainty How sure are you that youwould choose O insteadof A or B?

P(O) 168 193

4 Based on pretests, no snakes or amphibians wereincluded (due to revulsion toward those animals), and noendangered species were included (due to overwhelmingpreference for those animals).


mings and Taylor 1999; Champ et al. 1997),the good used in this study is time sensitive.The provision rule was clearly explained torespondents. If a donation is not made, ananimal will be turned away that wouldotherwise be rehabilitated. The respondentcannot free-ride on the future donations byothers for that specific animal. For arespondent who values rehabilitation forthat specific animal more than the postedcost, the optimal strategy is to make thedonation, and vice versa. Therefore, eventhough the payment vehicle was describedto respondents as a donation, and it wouldlegally be considered a donation for taxpurposes, it functioned as a posted price foran excludable good.

After each hypothetical choice, the cer-tainty follow-up questions were asked. Foreach certainty question, respondents indi-cated their level of certainty with a sliderbar (Figure 2). The choice set was shownwith coloring and an arrow or arrows, andthe labels next to the slider bar werecustomized to clearly indicate the optionor options to which the certainty questionapplied.

Respondents were paid $20 to partici-pate. Surveys were conducted individuallyin a laboratory on a computer. In eachchoice task, respondent was presented withfour choice sets. The same set of choices wasused for hypothetical and real tasks. For

real choice tasks, respondents were told thatone of the four choices would be binding,and would be randomly selected at the endof the survey. At the end of the survey, theadministrator determined which choice wasbinding and subtracted any donation fromthe $20 incentive.

The study include three survey treat-ments: hypothetical choices with certaintyfollow-up sequence S1 followed by realchoices (treatment H1), hypothetical choic-es with certainty follow-up sequence S2followed by real choices (treatment H2),and real choices only (treatment R).

IV. RESULTS

A total of 249 surveys were completed, 82in the H1 treatment, 83 in the H2 treatment,and 84 in the R treatment. The computer-based survey would not allow participantsto skip questions, and all who started thesurvey completed it.

Table 2 shows the proportion of choiceswhere the respondent chose an opt-inoption for hypothetical and real choices ineach treatment. A Pearson chi-square testwas conducted to test the null hypothesisthat the frequency of commitments forAnimal A, Animal B, or No Donationfollowed a common distribution across thethree real commitment datasets and acrossthe two hypothetical commitment datasets.

FIGURE 1EXAMPLE OF A HYPOTHETICAL CHOICE SET


For all four choices, for both real and forhypothetical datasets, the test failed toreject the null hypothesis of common choicebehavior. Similarly, tests conducted on theopt-in rates (i.e., without distinguishingbetween Animal A and Animal B) showedno statistical differences across treatments.There is no evidence, therefore, of treatmenteffects within either the hypothetical or thereal treatments.

Does Hypothetical Bias Exist?

However, there were large differences inthe opt-in rates between the hypotheticaltreatments and the real treatments (Ta-ble 2). The opt-in rate for hypotheticalchoices was about three times as large, onaverage, as for choices involving realpayments. This difference was statisticallysignificant at the 1% level, regardless ofwhether the test was conducted using opt-inrates for individual questions or the averageopt-in rate for all four questions combined.This is clear evidence of hypothetical bias

and demonstrates bias both within sampleand between samples.

Behavior in the real choices in treatmentsH1 and H2 was similar to that in treatmentR, so that there does not appear to be anysequencing effect. It is therefore useful toexplore within-respondent patterns of be-havior. Table 3 shows how many timesrespondents in the H1 and H2 treatmentsfollowed each of five response patterns. Ofrespondents who chose to opt in in thehypothetical choice, only 26.8% chose todonate for that same animal when present-ed with a real choice. Most who opted in inthe hypothetical choice opted out in the realchoice (65.6%), but there were some re-spondents who chose one animal in thehypothetical choice but then chose the otheranimal in the real choice (7%). The calibra-tion method used by Norwood and Olssonwould have incorrectly reclassified theserespondents to the opt-out option.

In contrast, almost all respondents whoopted out in the hypothetical choices optedout when faced with real choices (99.5%).This result is similar to that seen in the DCCV studies and implies any calibrationprotocol should not change hypotheticalopt-out responses, regardless of the level ofcertainty associated with those responses.

Multinomial logit (MNL) models wereestimated for the hypothetical and realpayment choices, pooled across treatments.When all attributes in the design wereincluded in the model, none were found to

FIGURE 2CERTAINTY FOLLOW-UP QUESTION Q1-S1

TABLE 2

HYPOTHETICAL AND REAL OPT-IN RATES

Treatment

Opt-in Rate:Hypothetical

ChoicesOpt-in Rate:Real Choices

R (n 5 336) — 14.3%H1 (n 5 328) 48.8% 15.2%H2 (n 5 332) 41.9% 16.0%


be statistically significant other than dona-tion amount. This may be because ofheterogeneity in preferences. For example,some respondents may prefer a bird, whileothers prefer a mammal. A simple modelwas estimated that included only thedonation amount (COST; $5 or $10), acommon alternative specific constant forthe two options that result in rehabilitatingan animal (ANIMAL; 51 for the two opt-in options, 0 for the opt-out option), andthe ANIMAL alternative specific constantinteracted with a measure of how high apriority the respondent placed on rehabili-tating wildlife (PRIORITY; ranging from 05 low priority to 10 5 high priority).

Table 4 presents the MNL results. Theestimated model based on real paymentchoices is presented in the first resultscolumn of Table 4, while the estimatedmodel based on hypothetical paymentchoices is presented in the second column.In both cases, all parameter estimates arestatistically significant at the 1% level. Themost important difference between the realand hypothetical models is that the margin-al disutility of COST estimated from realchoices is over twice as large as thatestimated from hypothetical choices. Alikelihood ratio test showed that the twomodels differed at the 1% level of signifi-cance. This result also held true if only theresponses from treatment R (the real-onlytreatment) were used to estimate the modelfor real choices.

For both the real payment and hypothet-ical payment models, utility from rehabili-tating an animal increases with the PRI-ORITY score. Evaluating PRIORITY at itsmean (6.43), it is possible to calculate themean WTP to rehabilitate one animal. If we

assume that WTP is bounded from below at0, then mean WTP estimated from thehypothetical choices was $5.29, versus $1.68from real choices. A Monte Carlo approachwas used to simulate sampling variability inthe mean WTP estimates and generate 95%confidence intervals. Using Poe, Severance-Lossin, and Welsh’s (1994) methods ofconvolutions, the difference between realand hypothetical WTP estimates was foundto be statistically significant at the 1% level.The same results were obtained when thereal choices only from treatment R wereused.

While the estimated model does not allowus to calculate marginal WTP for changesin the attributes of the good, it does allow usto calculate marginal changes in WTPassociated with a change in the respondent’scharacteristics. Here, a one-unit increase inthe PRIORITY score increased WTP esti-mated from real payment choices by $0.55,but increased WTP estimated from hypo-thetical payment choices by $1.33, a statis-tically significant difference at the 1% level.We therefore see hypothetical bias in bothtotal WTP and marginal WTP. This resultis inconsistent with the findings of Lusk andSchroeder (2004) and of List, Sinha, andTaylor (2006), who found hypothetical biasin total WTP but not in marginal WTP.

Is Hypothetical Bias Related toRespondent Uncertainty?

Having established that hypothetical biasis present, the next research objective is toexamine the relationship, if any, betweenrespondent uncertainty and hypotheticalbias. Analysis of the certainty responsesfrom the H1 treatment showed internal

TABLE 3

PATTERNS OF RESPONSES IN HYPOTHETICAL TREATMENTS

Hypothetical Choice Real Choice Category Proportion

Opts in (n 5 299)Opts in, and chooses same animal Yes-Yes-Same (YYS) 26.8%Opts in, but chooses other animal Yes-Yes-Different (YYD) 7.7%Opts out Yes-No (YN) 65.6%

Opts out (n 5 361)Opts in No-Yes (NY) 0.5%Opts out No-No (NN) 99.5%


inconsistencies. As constructed, a respon-dent’s opt-in certainty should be higherthan his unconditional choice certainty.That is, you should be more certain thatyou will choose some animal than that youwill choose a specific animal. However,many H1 respondents expressed higherlevels of certainty about which animal theywould rehabilitate compared to whetherthey would donate at all. We conclude thatthe sequence S1 of certainty follow-upquestions used in the H1 treatment doesnot work well. We believe that respondentswere confused about the distinction be-tween opt-in certainty and unconditionalchoice certainty. We do not use the resultsfrom the unconditional choice certaintyquestion in subsequent analysis. Becausethe opt-in certainty question in treatmentH1 was identical to that used in treatmentH2, we continue to consider the results fromthat question in our analysis.

Table 5 shows the average certaintylevels for each response pattern for bothhypothetical treatments. Respondents whoopted in in the hypothetical choice, andthen opted in in the real choice (YYS orYYD) had higher opt-in certainty thanthose who opted out in the real choice(YN). Further, respondents who chose thesame donation option in the real treatment(YYS) had higher conditional choice cer-tainty than those who chose the otherdonation option (YYD). Information fromthese follow-up questions is likely, there-fore, to be useful in predicting which

respondents will follow which responsepattern. Meanwhile, NN respondents hadhigher opt-out certainty than NY respon-dents, but there were very few of the latter.

A probit regression analogous to Johan-nesson et al.’s (1999) statistical bias functionwas estimated for choices where the respon-dent opted in in the hypothetical choice. Itpredicts the probability of a YYS or YYDpattern versus a YN pattern. The results areshown in Table 6. All estimated parametersare statistically significant at the 1% level.For both hypothetical treatments, theprobability of opting in in the real choicewas higher as the opt-in certainty increased.A likelihood ratio test showed no differencein the model parameters between the twotreatments. This provides strong evidencethat stated opt-in certainty can be used topredict which hypothetical respondents willactually opt in when faced with a realchoice.

TABLE 5

MEAN CERTAINTY LEVELS BY TREATMENT AND

RESPONSE PATTERN

ResponsePattern

TreatmentH1: Opt-inCertainty

TreatmentH2: Opt-inCertainty

Treatment H2:Conditional

ChoiceCertainty

YYS 6.7 8.1 7.5YYD 7.4 6.1 5.5YN 4.7 4.6 5.2NN 6.8 7.3 —NY 4.9 1.0 —

Note: YYS, yes-yes-same; YYD, yes-yes-different; YN, yes-no;NN, no-no; NY, no-yes.

TABLE 4

MULTINOMIAL LOGIT ESTIMATION RESULTS, DEPENDENT VARIABLE 5 OPTION CHOSEN

Variable Description All Treatments: Real ChoicesTreatments H1 and H2:

Hypothetical Choices

ANIMAL 1 5 opt-in choice0 5 opt out

22.713*** (0.462) 22.023*** (0.355)

PRIORITY * ANIMAL 0 5 low priority10 5 high priority

0.388*** (0.051) 0.332*** (0.043)

COST $0, $5, or $10 20.351*** (0.039) 20.142*** (0.025)Sample size 996 660Log likelihood 2447.059 2612.510Mean WTP (95% CI) $1.68 ($1.35, $2.07) $5.29 ($4.65, $6.40)

Note: Standard error in parentheses. WTP, willingness to pay.*** Significant at p # 0.01.


The second issue to address is whether wecan identify which respondents will followthe YYD pattern versus the YYS pattern.Here we use the conditional choice certaintymeasures from the H2 treatment. Initialanalysis of the pattern of responses showedthat the YYD pattern was more likely insituations where the option chosen in thehypothetical choice was more expensivethan the other opt-in option. To capturethis effect, a relative cost dummy variablewas constructed equal to 1 if the optionchosen in the hypothetical choice was moreexpensive than the other opt-in option, and0 if it was less expensive or the same cost.

A probit regression predicting the prob-ability of a YYS pattern versus a YYDpattern was estimated for the H2 treatment.Results are shown in Table 7. The proba-bility that the respondent will stick with thesame animal in the real choice, versusswitching to the other animal, was higherif the respondent stated higher conditionalchoice certainty, and was lower if the initialanimal chosen was more expensive. Botheffects were statistically significant at the1% level.

Based on these results, there is strongevidence that respondent uncertainty isrelated to inconsistent choices, in this caseYN and YYD. Respondents who opted inin the hypothetical choice but had lowerstated opt-in certainty were more likely toswitch to the opt-out option in the realchoice. Respondents who opted in in thehypothetical choice but had lower condi-tional choice certainty and/or originallychose an animal with higher cost were morelikely to donate for the other animal whenpresented with the real choice.

These results, which are consistent withthe findings of Johannesson et al. (1999)and Blomquist, Blumenschein, and Johan-nesson (2008), are useful in that they resolvea criticism of using respondent certaintymeasures to calibrate hypothetical respons-es. This criticism can be best described inthe context of DC CV. If the proportion ofrespondents who say ‘‘yes’’ in a hypothet-ical treatment is larger than the proportionwho actually purchase the good in atreatment with real payments, then for anycontinuous measurable attribute of therespondent, there is some threshold value

Treatment H2: Hypothetical Choices

Treatment H2: Calibrated Hypothetical Choices

Without Option Switching With Option Switching

22.249*** (0.488) 24.306*** (0.733) 23.412*** (0.714)

0.344*** (0.059) 0.380*** (0.077) 0.384*** (0.078)

20.132*** (0.036) 20.085 (0.056) 20.210*** (0.057)332 332 332

2296.154 2172.273 2166.402$5.12 ($4.24, $7.61) $1.71 (undefined) $1.56 ($1.16, $2.23)

TABLE 4

(Extended)

TABLE 6

PROBIT REGRESSION MODEL: DEPENDENT VARIABLE 5 1 IF YYS OR YYD RESPONSE PATTERN, 5 0 IF YNRESPONSE PATTERN

Variable Description Treatment H1 Treatment H2 Pooled

Intercept 51 21.812*** (0.299) 21.910*** (0.318) 21.874*** (0.217)Opt-in certainty 0 5 completely unsure

10 5 100% sure0.23*** (0.05) 0.26*** (.046) 0.25*** (0.03)

Sample size 160 139 299Log likelihood 285.156 272.966 2158.481

Note: Standard error in parentheses. YYS, yes-yes-same; YYD, yes-yes-different; YN, yes-no.*** Significant at p # 0.01.


that, if used to calibrate the hypotheticalresponses, would give the ‘‘correct’’ pro-portion of yes responses. That this can bedone using respondent certainty as thecalibrating metric does not prove thathypothetical bias is related to respondentuncertainty. However, the within-sampleresults in Tables 6 and 7 show that respon-dent uncertainty is indeed related to hypo-thetical bias, affirming the use of respon-dent certainty as a calibration metric.

Can Hypothetical Choices Be Calibrated UsingRespondent Uncertainty Measures?

Respondents who give hypothetical an-swers that are inconsistent with their realbehavior tend to state lower levels ofcertainty in their choices. The certaintyfollow-up measures might therefore beuseful in calibrating hypothetical choices.We consider two different calibration pro-tocols. The first is similar to that used byNorwood and by Olsson. In this protocol,hypothetical choices where the respondentopted in but had low opt-in certainty arerecoded to the opt-out option, but choicesare never switched from one opt-in optionto the other opt-in option. A certaintythreshold of 7 or higher was used, because itbest matched the actual donation rate.5

The second calibration rule allows for thepossibility of switching from one opt-inoption in the hypothetical choice to the

other opt-in option in the real choice. Thecalibration rule used here is shown dia-grammatically in Figure 3. For hypotheti-cal opt-in choices, there is first a decisionwhether the respondent would opt in in thereal choice situation. This rule is the same asin the first protocol; respondents with lowopt-in certainty are recoded to the opt-outoption. For hypothetical opt-in choiceswith high opt-in certainty, however, asecond determination must be made wheth-er the respondent would switch to the otheropt-in option (i.e., follow a YYD pattern).Information on the conditional choicecertainty is used to identify these respon-dents, but the switching rule varies depend-ing on the relative cost of the two opt-inoptions.

Three cases exist. The first case is wherethe opt-in option chosen in the hypotheticalchoice is less expensive than the other opt-inoption. In treatments H1 and H2, suchrespondents almost never followed a YYDpattern (this occurred in only 1 response outof 158). Accordingly, the calibration rule inthis case does not switch any respondents tothe more costly opt-in option. The secondcase is where the two opt-in options havethe same cost. In this case, most respon-dents who opted-in in both the hypotheticaland the actual choice stayed with the sameoption (i.e., followed the YYS pattern),though some (38%) did switch. The calibra-tion rule in this case switches a respondentfrom one opt-in option to the other only ifher conditional choice certainty is very low(less than 5). The third case is where the opt-in option chosen in the hypothetical choice

5 Continuous measures of certainty from the sliderbars were rounded to the nearest 0.1 before the thresholdwas applied, so the actual threshold used was 6.95.

TABLE 7

PROBIT REGRESSION MODEL: DEPENDENT VARIABLE 5 1 IF YYS RESPONSE PATTERN, 5 0 IF YYDRESPONSE PATTERN

Variable Description Parameter Estimate

Intercept 51 20.068 (0.580)Conditional choice certainty 0 5 completely unsure; 10 5 100% sure 0.17** (0.08)Relative cost dummy 51 if option chosen is more expensive

than other option20.923** (0.419)

Sample size 52Log likelihood 223.761

Note: Standard error in parentheses. YYS, yes-yes-same; YYD, yes-yes-different.** Significant at p # 0.05.


is more expensive than the other opt-inoption. In this case, more respondentsfollowed the YYD pattern (i.e., switchedto the cheaper opt-in option) than followedthe YYS pattern (57% vs. 43%). Accord-ingly, the calibration rule uses a higherconditional choice certainty threshold thanin Case 2. Respondents are assumed toswitch to the cheaper opt-in option if theirconditional choice certainty is less than 7,the same threshold level used to identify YNrespondents. Figure 3 shows the number ofcases where each type of recalibration wasapplied for the responses from treatmentH2. The resulting proportions of respon-dents following the YYS, YYD, and YNpatterns closely match the actual propor-tions measured in treatments H1 and H2.

For both calibration protocols, hypo-thetical opt-out choices were left un-changed, regardless of the stated opt-outcertainty.

The hypothetical choices from treatmentH2 were calibrated using both protocols,and MNL models were estimated. Resultsfor calibrated and uncalibrated data areshown in the third through fifth resultscolumns of Table 4, along with calculatedmean WTP and 95% confidence interval onmean WTP for a respondent with averagePRIORITY score. Both calibration proto-cols resulted in estimates of mean WTP thatwere similar to, and not significantlydifferent from, mean WTP estimated fromreal choices. Similarly, calibration removedthe impact of hypothetical bias on marginalWTP estimates. The marginal change inWTP associated with a one-unit increase inPRIORITY was $0.70 when estimated fromthe data calibrated without option switch-ing, and $0.59 when estimated from datacalibrated with option switching. Neithercalibrated marginal WTP differed signifi-cantly from the marginal WTP estimated

FIGURE 3CALIBRATION RULE ALLOWING SWITCHING BETWEEN OPT-IN OPTIONS


from real payment choices, which was$0.55.

While both calibration rules appear toremove hypothetical bias from both totaland marginal WTP estimates, there areimportant differences between them. Inparticular, the estimated parameter for theCOST attribute is negative but not signif-icantly different from zero for the calibra-tion rule without switching. As a conse-quence, the simulated 95% confidenceinterval for the estimated mean WTP isundefined. In contrast, the estimated pa-rameter for the COST attribute is signifi-cantly different from zero for the calibra-tion rule with switching, and for the realchoices. Likelihood ratio tests show that theMNL model estimated from the hypothet-ical choices calibrated using the protocolwithout option switching is significantlydifferent from the model estimated fromreal choices, but that the model for the datacalibrated using the protocol with optionswitching is not significantly different fromthe model estimated from real choices.Allowing for option switching appears tobe important in preserving price sensitivityin the choices. If calibration without allow-ing for option switching results in lowerestimates of the marginal disutility ofexpenditures, this could help explain whyOlsson found that calibration resulted inincreases in the marginal WTP estimates.

V. DISCUSSION

The main conclusions of this research are(1) hypothetical bias does exist in CE data,(2) hypothetical bias in CE data is related torespondent uncertainty, and (3) certaintyfollow-up questions can be used to calibratehypothetical CE data. The pattern of resultssuggests that hypothetical bias exists notonly in the respondents’ propensity tochoose a costly option, versus the opt-outoption, but also when respondents chooseamong costly options. Calibration proto-cols should therefore include the possibilityof the respondent switching from one costlyoption to another costly option, instead of

always assuming that unsure respondentswould switch to the opt-out option.

When the number of options presented tothe respondent is larger than two, measure-ment of respondent uncertainty becomesmore complicated. There exists more thanone sequence of possible certainty questionsthat could be used. We found that not allsequences perform equally well. Moreresearch is needed on how respondentsinterpret certainty follow-up questions in amultiple-choice context, and how the ques-tions can be asked in ways that enhancetheir reliability and usefulness. Berrens et al.(2002), in the context of DC CV, proposeusing direct questions about the probabilityof each choice. We believe that this wouldbe difficult for respondents in a CE setting,but it is worth exploring.

The results show that calibration musttake into account the possibility of switch-ing between opt-in options. More workremains to be done in developing suchcalibration rules. In particular, the certaintythresholds used here for option switchingshould be researched further. We chose acertainty threshold for the opt-in decisionthat matched the calibrated hypotheticalopt-in rate to the actual opt-in rate.However, we had too few observations tobe able to do such frequency matchingwhen setting the thresholds for optionswitching. Studies with larger sampleswould be needed to find option switchingthresholds that are more firmly rooted inempirical measurement of behavior. Workis also needed to determine whether thecertainty thresholds that mitigate hypothet-ical bias are consistent across studies andacross goods. Studies that calibrate CV datahave consistently concluded that the cer-tainty threshold that performs best liesbetween 7 and 10. Several CE studies willbe needed valuing different goods before wecan determine whether CE calibrationprotocols are transferable across goods.

Although studies, including this one,have consistently found an empirical rela-tionship between respondent uncertaintyand hypothetical bias, there has notemerged a consensus on the behavioral


mechanism underlying that relationship.Norwood, Lusk, and Boyer (2008) offerup two possible utility-theoretic explana-tions why respondent uncertainty mightlead to hypothetical bias: risk aversion andcommitment cost. According to the riskaversion hypothesis, individuals who areuncertain about the utility they will derivefrom a good may be more risk averse whenthey have to actually pay for a good thanwhen they are in a hypothetical paymentsetting, so that actual payment values arelower than contingent values. The commit-ment cost hypothesis argues that if individ-uals are uncertain about the value of agood, expect to learn more about that valuein the future, but are forced to make apurchase decision today, they will statelower WTP than they would if they had nouncertainty about the value of the good.Commitment cost is defined as the differ-ence between WTP with certainty and WTPwith uncertainty. If commitment cost isonly considered in actual payment situa-tions, the difference between actual andhypothetical payments (hypothetical bias)will be more pronounced as uncertaintyincreases.

Our results cannot directly confirm orrefute either hypothesis. However, respon-dents in our experiment were explicitly toldthat they would not be receiving any newinformation prior to making their decision.Still they expressed uncertainty over theirbehavior, which was related to hypotheticalbias. This suggests that commitment costswere not an important mechanism in thiscase. More work is needed to develop andtest alternative conceptual models of therelationship between respondent certaintyand hypothetical bias.

References

Alberini, Anna, Kevin Boyle, and Michael Welsh.2003. ‘‘Analysis of Contingent Valuation Datawith Multiple Bids and Response OptionsAllowing Respondents to Express Uncertain-ty.’’ Journal of Environmental Economics andManagement 45 (1): 40–62.

Alfnes, Frode, and Kyrre Rickertsen. 2002. ‘‘SC-X:Calibrating Stated Choice Surveys with Exper-

imental Auction Markets.’’ Discussion Paper#D-5/2002, Department of Economics andSocial Sciences, Norwegian University of LifeSciences, Aas, Norway.

Berrens, Robert P., Hank Jenkins-Smith, Alok K.Bohara, and Carol L. Silva. 2002. ‘‘FurtherInvestigation of Voluntary Contribution Con-tingent Valuation: Fair Share, Time of Contri-bution, and Respondent Uncertainty.’’ Journalof Environmental Economics and Management44 (1): 144–68.

Blamey, Russell, and Jeff Bennett. 2001. ‘‘Yea-saying and Validation of a Choice Model ofGreen Product Choice.’’ In The Choice Model-ling Approach to Environmental Valuation, ed.Jeff Bennett and Russell Blamey. Cheltenham,UK: Edward Elgar.

Blomquist, Glenn C., Karen Blumenschein, andMagnus Johannesson. 2008. ‘‘Eliciting Willing-ness to Pay without Bias using Follow-upCertainty Statements: Comparisons betweenProbably/Definitely and a 10-Point CertaintyScale.’’ Working paper, Department of Eco-nomics, University of Kentucky, Lexington.

Blumenschein, Karen, Magnus Johannesson,Glenn C. Blomquist, Bengt Liljas, and RichardM. O’Conor. 1998. ‘‘Experimental Results onExpressed Certainty and Hypothetical Bias inContingent Valuation.’’ Southern EconomicJournal 65 (1): 169–77.

Blumenschein, Karen, Glenn C. Blomquist, Mag-nus Johannesson, Nancy Horn, and PatriciaFreeman. 2008. ‘‘Eliciting Willingness to Paywithout Bias: Evidence from a Field Experi-ment.’’ Economic Journal 118 (525): 114–37.

Cameron, Trudy A., Gregory L. Poe, Robert G.Ethier, and William D. Shulze. 2002. ‘‘Alterna-tive Non-market Value-Elicitation Methods:Are the Underlying Preferences the Same?’’Journal of Environmental Economics and Man-agement 44 (3): 391–425.

Carlsson, Fredrik, and Peter Martinsson. 2001.‘‘Do Hypothetical and Actual Marginal Will-ingness to Pay Differ in Choice Experiments?’’Journal of Environmental Economics and Man-agement 41 (2): 179–92.

Champ, Patricia A., and Richard C. Bishop. 2001.‘‘Donation Payment Mechanisms and Contin-gent Valuation: An Empirical Study of Hypo-thetical Bias.’’ Environmental and ResourceEconomics 19 (4): 383–402.

Champ, Patricia A., Richard C. Bishop, ThomasC. Brown, and Daniel W. McCollum. 1997.‘‘Using Donation Mechanisms to Value Non-use Benefits from Public Goods.’’ Journal ofEnvironmental Economics and Management 33(2): 151–62.


Cummings, Ronald G., and Laura O. Taylor. 1999.‘‘Unbiased Value Estimates for EnvironmentalGoods: A Cheap Talk Design for the Contin-gent Valuation Method.’’ American EconomicReview 89 (3): 649–65.

Ethier, Robert G., Gregory L. Poe, William D.Schulze, and Jeremy Clark. 2000. ‘‘A Compar-ison of Hypothetical Phone and Mail Contin-gent Valuation Responses for Green-PricingElectricity Programs.’’ Land Economics 76 (1):54–67.

Hanemann, W. Michael, and Bengt Kristrom.1995. ‘‘Preference Uncertainty, Optimal Design,and Spikes.’’ In Current Issues in EnvironmentalEconomics, ed. Per Olov Johansson, BengtKristrom, and Karl-Goran Maler. Manchester,UK: Manchester University Press.

Holmes, Thomas P., and Wiktor L. Adamowicz.2003. ‘‘Attribute Based Methods.’’ In A Primeron Nonmarket Valuation, ed. Patricia A.Champ, Thomas Brown, and Kevin Boyle,171–219. Dordrecht: Kluwer Academic Pub-lishers.

Johannesson, Magnus, Glenn C. Blomquist, KarenBlumenschein, Per-Olov Johansson, Bengt Lil-jas, and Richard M. O’Conor. 1999. ‘‘Calibrat-ing Hypothetical Willingness to Pay Respons-es.’’ Journal of Risk and Uncertainty 18 (1):21–32.

Johannesson, Magnus, Bengt Liljas, and Per-OlovJohansson. 1998. ‘‘An Experimental Compari-son of Dichotomous Choice Contingent Valu-ation Questions and Real Purchase Decisions.’’Applied Economics 30 (5): 643–47.

Johansson-Stenman, Olof, and Henrik Svedsater.2007. ‘‘Hypothetical Bias in Choice Experi-ments: Within versus Between Subject Tests.’’Working Papers in Economics 252. Goteborg:Department of Economics, Goteborg University.

Kooten, G. Cornelis van, Emina Krcmar, andErwin H. Bulte. 2001. ‘‘Preference Uncertaintyin Non-market Valuation: A Fuzzy Approach.’’American Journal of Agricultural Economics 83(3): 487–500.

Li, Chuan-Zhong, and Leif Mattsson. 1995.‘‘Discrete Choice under Preference Uncertainty:An Improved Structural Model for ContingentValuation.’’ Journal of Environmental Econom-ics and Management 28 (2): 256–69.

List, John A., and Craig A. Gallet. 2001. ‘‘WhatExperimental Protocol Influence DisparitiesBetween Actual and Hypothetical Stated Val-ues? Evidence From a Meta-Analysis.’’ Envi-ronmental and Resource Economics 20 (3):241–54.

List, John A., Paramita Sinha, and Michael H.Taylor. 2006. ‘‘Using Choice Experiments to

Value Non-market Goods and Services: Evi-dence from Field Experiments.’’ Advances inEconomic Analysis and Policy 6 (2): Article 2.

Little, Joseph, and Robert Berrens. 2004. ‘‘Ex-plaining Disparities between Actual and Hypo-thetical Stated Values: Further InvestigationUsing Meta-Analysis.’’ Economics Bulletin 3(6): 1–13.

Loomis, John, and Earl Ekstrand. 1998. ‘‘Alterna-tive Approaches for Incorporating RespondentUncertainty When Estimating Willingness toPay: The Case of the Mexican Spotted Owl.’’Ecological Economics 27 (1): 29–41.

Lusk, Jayson L., and T. C. Schroeder. 2004. ‘‘AreChoice Experiments Incentive Compatible? ATest with Quality Differentiated Beef Steaks.’’American Journal of Agricultural Economics 86(2): 467–82.

Murphy, James J., P. Geoffrey Allen, Thomas H.Stevens, and Darryl Weatherhead. 2005. ‘‘AMeta-analysis of Hypothetical Bias in StatedPreference Valuation.’’ Environmental and Re-source Economics 30 (3): 313–25.

Norwood, F. Bailey. 2005. ‘‘Can CalibrationReconcile Stated and Observed Preferences?’’Journal of Agricultural and Applied Economics37 (1): 237–48.

Norwood, F. Bailey, Jayson L. Lusk, and TracyBoyer. 2008. ‘‘Forecasting Hypothetical Bias: ATale of Two Calibrations.’’ In EnvironmentalEconomics, Experimental Methods, ed. Todd L.Cherry, Stephan Kroll, and Jason F. Shogren.New York: Routledge.

Olsson, Bjorn. 2005. ‘‘Accounting for ResponseUncertainty in Stated Preference Methods.’’Presented at the EAERE Congress, Bremen,Germany, June 23–26.

Opaluch, James J., and Kathleen Segerson. 1989.‘‘Rational Roots of ‘Irrational’ Behavior: NewTheories of Economic Decision-Making.’’Northeastern Journal of Agriculture and Re-source Economics 18 (2): 81–95.

Poe, Gregory L., Jeremy E. Clark, Daniel Ron-deau, and William D. Schulze. 2002. ‘‘ProvisionPoint Mechanisms and Field Validity Tests ofContingent Valuation.’’ Environmental and Re-source Economics 23 (1): 105–31.

Poe, Gregory L., Eric K. Severance-Lossin, andMichael P. Welsh. 1994. ‘‘Measuring theDifference (X – Y) of Simulated Distributions:A Convolutions Approach.’’ American Journalof Agricultural Economics 76 (4): 904–15.

Ready, Richard C., Stale Navrud, and W. RichardDubourg. 2001. ‘‘How Do Respondents withUncertain Willingness to Pay Answer Contin-gent Valuation Questions?’’ Land Economics 77(3): 315–26.


Taylor, Laura O., Mark D. Morrison, and Kevin J.Boyle. 2007. ‘‘Exchange Rules and the IncentiveCompatibility of Choice Experiments.’’ Work-ing paper, Center for Environmental andResource Economic Policy, North CarolinaState University.

Vossler, Christian A., Robert G. Ethier, Gregory L.Poe, and Michael P. Welsh. 2003. ‘‘Payment

Certainty inDiscreteChoiceContingentValuationResponses: Results from a Field Validity Test.’’Southern Economic Journal 69 (4): 886–902.

Welsh, Michael P., and Gregory L. Poe. 1998.‘‘Elicitation Effects in Contingent Valuation:Comparisons to a Multiple Bounded DiscreteChoice Approach.’’ Journal of EnvironmentalEconomics Management 36 (2): 170–85.


using respondent uncertainty to mitigate hypothetical bias

Documents