Studies in Neuroscience, Psychology andBehavioral Economics

Martin ReuterChristian Montag Editors

Neuroeconomics

Studies in Neuroscience, Psychologyand Behavioral Economics

Series editors

Martin Reuter, Bonn, GermanyChristian Montag, Ulm, Germany

More information about this series at http://www.springer.com/series/11218

Martin Reuter • Christian MontagEditors

Neuroeconomics

123

EditorsMartin ReuterDifferentielle and Biologische PsychologieRheinische Friedrich-Wilhelms-UniversitätBonn

BonnGermany

Christian MontagDepartment of Molecular Psychology,Institute for Psychology and Education

Ulm UniversityUlmGermany

and

Key Laboratory for NeuroInformation,School of Life Science and Technology,Center for Information in Medicine

University of Electronic Scienceand Technology of China

ChengduPeople’s Republic of China

ISSN 2196-6605 ISSN 2196-6613 (electronic)Studies in Neuroscience, Psychology and Behavioral EconomicsISBN 978-3-642-35922-4 ISBN 978-3-642-35923-1 (eBook)DOI 10.1007/978-3-642-35923-1

Library of Congress Control Number: 2016948772

© Springer-Verlag Berlin Heidelberg 2016This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or partof the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmissionor information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilarmethodology now known or hereafter developed.The use of general descriptive names, registered names, trademarks, service marks, etc. in thispublication does not imply, even in the absence of a specific statement, that such names are exempt fromthe relevant protective laws and regulations and therefore free for general use.The publisher, the authors and the editors are safe to assume that the advice and information in thisbook are believed to be true and accurate at the date of publication. Neither the publisher nor theauthors or the editors give a warranty, express or implied, with respect to the material contained herein orfor any errors or omissions that may have been made.

Printed on acid-free paper

This Springer imprint is published by Springer NatureThe registered company is Springer-Verlag GmbH GermanyThe registered company address is: Heidelberger Platz 3, 14197 Berlin, Germany

Preface

Dear scientists, students, and readers,The present book, Neuroeconomics, was originally intended as the cornerstone

of the Springer book series Studies in Neuroscience, Psychology and BehavioralEconomics. It was not our idea to write (or edit) a book before or close to retire-ment. So it took a while before Springer, who had contacted MR to write a book onNeuroeconomics, persuaded us to do it. One prerequisite for MR was that CMagreed to do this jointly with MR, and CM did agree!! We have worked togetherside by side for many years and become very close friends. Springer ultimatelyconvinced us, not only to publish this book, but to edit a whole book series.Meanwhile a first book, Internet Addiction—Neuroscientific Approaches andTherapeutical Interventions, which we edited, was published early in 2015, formingthe first publication of this series and more books will appear in the future.

Science has become more and more interdisciplinary and so new scientificdisciplines emerge—like neuroeconomics, which is a joint venture between neu-roscientists, psychologists, and behavioral economists. The focus of interest inneuroeconomics lies on human decision-making under an economic perspective.“Economics” refers not only to monetary transactions, but also to all kinds of costsand benefits associated with decisions. Before a decision is reached and an actualaction is exerted, cognitive and affective processes are active and these processesoriginate in the brain. Therefore, if one is interested in the question why peoplebehave impulsively or rather rationally and in a manner guided by self-interest, therole of the brain has to be taken into account. Differences in the hard-wiringof the brain or functional differences in brain activity help to explain variation inhuman decision-making. Research topics like this are at the core of the youngdiscipline of neuroeconomics.

Neuroeconomics has adopted and expanded games and paradigms frombehavioral economics and psychology, and uses concepts from diverse disciplineslike addiction research (e.g., reward or temporal discounting) and applies nearly allkinds of neuroscience techniques to the study of human decision-making.

In 2009, the Center for Economics & Neuroscience (CENs) was founded at theUniversity of Bonn. There are three arms to the CENs; comprising a behavior

v

economics lab (Armin Falk), an imaging center (Bernd Weber), and MR’s geneticlaboratories. The collaboration with the colleagues of the CENs has alerted us to theproblems related to interdisciplinary work in the field of neuroeconomics. Even thestatistical methods preferred to analyze a given data set differ between psycholo-gists and economists. Finding a common language is sometimes cumbersome, butat the same time offers researchers the chance to learn from our colleagues. This isfurther outlined by an example: behavioral economists are particularly fascinated bythe opportunities offered by the neurosciences. However, they typically have notcome across these techniques during their undergraduate and postgraduate training.We have often been asked for a scholarly introduction to molecular genetics, thefield in which we are specialized. A comparable demand exists for other neuro-scientific techniques. Thus, we decided not only to publish a textbook on neuroe-conomics, but to enrich the book by a broad methods section in which the mostcommon neuroscientific techniques, ranging from molecular techniques includinggenetics and hormone analysis, to brain imaging are introduced in a scholarlymanner by experts in the field. This methods section is so far unique for a neu-roeconomics book. We are convinced that many scientists and students will findthey have an interest in this methods section, even if they are not primarily inter-ested in neuroeconomics. We hope that this potential readership becomes aware ofthis special feature of the book.

The book comprises eight sections, starting with an introduction into neuroe-conomics (1) followed by an overview on frequently applied experimental para-digms (games) in neuroeconomic research (2). In the next section, the molecularbasis of human decision-making is addressed (3). Here, the focus is on the role ofhormones, neurotransmitters, and (their underlying) genes that have been reportedto be of relevance for the field. Section four focusses on environmental and situ-ational factors (4) and section five on social contexts influencing humandecision-making (5). From the synopses of sections 3, 4, and 5, it becomes apparentthat the successful prediction of human behavior must include nature and nurture,as well as situational factors related to the decision (e.g., framing effects).Section six presents translational and developmental approaches to neuroeconomicsincluding, among other valuable contributions, chapters on decision-making inchildren and among patients suffering from mental illness (6). An article on neu-romarketing demonstrates how knowledge from neuroeconomics research can beapplied in real life. For this reason, this chapter has been labeled AppliedNeuroeconomics (7). Hopefully this section can be extended in the future; we arevery confident that the applicability of basic neuroeconomic research willincreasingly be acknowledged. The culmination of the present book constitutes theabove-mentioned methods section, in which eight different neuroscience techniquesare introduced (8).

The completion of this book took longer than planned, but now that it is finished,we are very satisfied with the product. We are happy to have received contributionsfrom so many highly regarded experts in this field. Thank you all for your strongcontributions and for your patience. Big thanks also go to Éilish Duke, for hercritical redaction of our chapters.

vi Preface

We thank all the readers interested in this work. We hope that we meet yourexpectations and are thankful for your criticisms and comments.

We also want say thank you to our beloved wives [Anette (MR) and Susanne(CM)] and to our families, friends, and colleagues for their never ending supportand love over all the years. Their support came long before this book project began.

Bonn, Germany Martin ReuterUlm, Germany Christian MontagJune 2016

Preface vii

Contents

1 Neuroeconomics—An Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 1Martin Reuter and Christian Montag

Part I Games in Experimental Economics

2 Game Theory in Neuroeconomics . . . . . . . . . . . . . . . . . . . . . . . . . . . 13Claudia Civai and Daniel R. Hawes

Part II Molecular Basis of Human Decision Making

3 Hormones and Economic Decisions . . . . . . . . . . . . . . . . . . . . . . . . . . 41Amos Nadler and Paul J. Zak

4 Genes and Human Decision-Making . . . . . . . . . . . . . . . . . . . . . . . . . 67Martin Reuter and Christian Montag

5 Monoamines and Decision-Making Under Risks . . . . . . . . . . . . . . . . 85Hidehiko Takahashi

Part III Environmental/Situational Factors InfluencingHuman Decision Making

6 Decision-Making Under Uncertainty . . . . . . . . . . . . . . . . . . . . . . . . . 99Dominik R. Bach

7 Emotion Regulation and Economic Decision-Making . . . . . . . . . . . . 113Renata M. Heilman, Andrei C. Miu and Daniel Houser

8 How the Experience of Time Shapes Decision-Making . . . . . . . . . . . 133Marc Wittmann and Martin P. Paulus

9 Framing Effects: Behavioral Dynamics and Neural Basis . . . . . . . . 145Xiao-Tian Wang, Lilin Rao and Hongming Zheng

ix

10 The Influence of Costs, Benefits and Their Interactionon the Economic Behaviour of Consumers . . . . . . . . . . . . . . . . . . . . 167Luca Panzone and Deborah Talmi

Part IV Decision Making in Social Contexts

11 Individual Differences in Decision-Making: A Neural TraitApproach to Study Sources of Behavioral Heterogeneity . . . . . . . . . 191Kyle Nash and Daria Knoch

12 Altruistic Punishment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211Alexander Strobel

Part V Translational and Developmental Approaches toNeuroeconomics

13 Brain SEEKING Circuitry in Neuroeconomics: A UnifyingHypothesis for the Role of Dopamine-Energized Arousalof the Medial Forebrain Bundle in Enthusiasm-GuidingDecision-Making . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231Jaak Panksepp and Cristina G. Wilson

14 The Psychology and Psychobiology of Simple Decisions:Speeded Choice and Its Neural Correlates . . . . . . . . . . . . . . . . . . . . 253David K. Sewell and Philip L. Smith

15 A Neurocognitive Perspective on the Developmentof Social Decision-Making. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293Geert-Jan Will and Berna Güroğlu

16 Neuroeconomic Approaches in Mental Disorders . . . . . . . . . . . . . . . 311S. Lis and P. Kirsch

Part VI Applied Neuroeconomics

17 Consumer Neuroscience and Neuromarketing . . . . . . . . . . . . . . . . . 333Bernd Weber

Part VII Neuroscience Methods in Neuroeconomics

18 Skin Conductance Measures in Neuroeconomic Research . . . . . . . . 345Dominik R. Bach

19 Electroencephalography: Current Trendsand Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359Stefan Debener, Cornelia Kranczioch and Maarten De Vos

20 Functional Magnetic Resonance Imaging (fMRI) . . . . . . . . . . . . . . . 375Sebastian Markett

x Contents

21 Structural MRI: Morphometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399Christian Gaser

22 Diffusion Tensor Imaging (DTI) and Tractography . . . . . . . . . . . . . 411Theodor Rüber, Christian Erich Elger and Bernd Weber

23 Molecular Genetics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443Martin Reuter, Andrea Felten and Christian Montag

24 Hormones. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463Robert Miller and Clemens Kirschbaum

25 Eye Movements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481Ulrich Ettinger and Christoph Klein

Appendix A: Neuroanatomy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503

Contents xi

Chapter 1Neuroeconomics—An Introduction

Martin Reuter and Christian Montag

Abstract The present chapter provides an introduction into the young discipline ofneuroeconomics and into the present Neuroeconomics book. Historical aspects,core concepts and future research avenues are presented.

1.1 Historical Aspects

Neuroeconomics is a very young scientific discipline that constitutes an interdis-ciplinary symbiosis of economics, psychology and the neurosciences. The generalaim of neuroeconomics is to study human decision-making with a focus on theneural mechanisms thereof. The official establishment of the discipline was markedby the foundation of The Society for Neuroeconomics in 2005.

Research in this field is prolific and of high quality, however, scepticismremains, especially among those scientists who retain a purist vision of theirrespective disciplines. History has taught us that great achievements are madepossible only by the combined expertise of scientists from different fields. Forexample, only through such successful interdisciplinary research could man have

M. Reuter (&)Department of Psychology, University of Bonn, Bonn, Germanye-mail: martin.reuter@uni-bonn-diff.de

M. ReuterCenter for Economics and Neuroscience (CENs), University of Bonn, Bonn, Germany

C. MontagDepartment of Molecular Psychology, Institute of Psychology and Education,Ulm University, Ulm, Germanye-mail: christian.montag@uni-ulm.de

C. MontagKey Laboratory for NeuroInformation, Center for Information in Medicine,School of Life Science and Technology, University of Electronic Scienceand Technology of China, Chengdu, People’s Republic of China

© Springer-Verlag Berlin Heidelberg 2016M. Reuter and C. Montag (eds.), Neuroeconomics, Studies in Neuroscience,Psychology and Behavioral Economics, DOI 10.1007/978-3-642-35923-1_1

1

flown to the moon; astronomers working in isolation could never have achieved thisdream, but with input from other disciplines (e.g. informatics, mathematics, phy-sics, etc.) mankind’s dream of walking on the moon became a reality.

Cognitive neuroscience, which emerged during the 1970s, is the youngestmember of the neuroeconomic trio, while the disciplines of psychology and eco-nomics have been around for over one hundred years. For decades the two disci-plines seemed to live in an uneasy parallel, arguably ignoring each other. This issurprising, given that the understanding of human behaviour is intrinsic to bothdisciplines. Of note, the scientific worldviews and the methodological approachesutilized by each discipline differ dramatically. Whereas economists try to establish aformal theory explaining human behaviour in an axiomatic way, psychologistsbuild and refine theories through an empirical approach. Roughly speaking,economists have traditionally favoured a theoretical—and psychologists anempirical—approach. Since the launch of the journal of Experimental Economics in1998 (and in view of the chairs for behavioural economics newly created atUniversities throughout the world), it is clear that this strict differentiation betweenthe theoretical economics and empirical psychology no longer holds. Nevertheless,such historical traditions are of importance; even today the two disciplines showmarked differences that are far-reaching, which manifest in different theoreticalfoundations and methodological and statistical approaches, all of which serve toundermine successful interdisciplinary research efforts.

Whereas economics had not made direct acquaintance with the neurosciencesprior to the establishment of the new discipline of neuroeconomics, the idea ofinvestigating the role of the brain in human behaviour is an old one in psychology.For decades psychologists have used electroencephalography (EEG; see the methodChap. 19 by Debener et al. in this book) to investigate cognitive and emotionalprocesses. Therefore, the invention and scientific application of magnetic resonancetomography (MRI; for an introduction see the method chapters on MRI by Markett(fMRI; Chap. 20), Gaser (MRI; Chap. 21) and Rüber (DTI; Chap. 22) in this book)in the 1990s proved a logical step for psychologists interested in subcortical pro-cesses that are not explicitly measurable through EEG. The subdiscipline ofBiological Psychology makes use of all kinds of techniques that characterize theneurosciences, incorporating, in addition to EEG and MRI, genetics, psychophys-iology, endocrinology, etc. In order to help bridge the gap between the “subdis-ciplines” of neuroeconomics, the present book deliberately features a broadmethods section, which gives a scholarly introduction into neuroscience techniquesrelevant to this field (see PART VII of this book).

1.1.1 Economic Models and Their Parallels in Psychology

As mentioned above, economic models of human behaviour are axiomatic and tryto establish algorithms valid for all participants across different situations. This ideais mirrored in classical experimental psychology, with the difference that

2 M. Reuter and C. Montag

experimental psychology uses experimental conditions to analyse behaviour. InPersonality Psychology, however, the central tenet recognizes that large variabilityexists across participants, even in strictly controlled experimental settings or naturalenvironments; a phenomenon referred to as individual differences.

A prominent economic model in neoclassical economics is utility maximization.According to utility maximization, people make their best choices according to theirdesires, knowledge and resources. The term utility does not refer to a good’squantity or monetary value per se in determining the decisions of an agent, but tothe utility they obtain from the item. According to Marshall (1920, p. 78) “utility iscorrelative to desire and want”, but desire and want can only be inferred indirectlyby “the price which a person is willing to pay for the fulfilment or satisfaction of hisdesire”. Although utility maximization makes correct predictions in a wide range ofsettings and situations including politics, markets and social life, its validity hasbeen questioned, e.g. by Prospect Theory (Kahneman and Tversky 1979). Whereasthe concept of expected utility, which originates from Utility Maximization Theory,postulates that alternative choices are valued by weighting the hedonic utility ofpossible outcomes against the chances of those outcomes actually occurring (e.g. ingambles), Prospect Theory claims that people do not always show a numericalevaluation of probabilities, but that outcomes are valued according to two aspects: areference point (reference-dependent value) and an absolute utility. Thereference-dependent value is thought to represent the valuation of past experiencesand future aspirations and is therefore related to learning (past) and motivation(future). Most prominently, Prospect Theory explains why people grant moreweight to losses than to gains, a phenomenon called Loss Aversion. There isempirical evidence across different cultures and ethnicities that, on average, lossesare valued about twice as large as equal-sized gains. Of note, Prospect Theory hasgained empirical support from the neurosciences. Using an fMRI study, Tom andcolleagues have demonstrated that different brain activity patterns are correlated tothe amount of gains and losses (Tom et al. 2007). Interestingly, they did not identifydifferent brain circuits coding for gains and losses, but instead identified a uniquesystem—the ventral striatum—that has become famous in the neurosciences as thebrain’s reward centre. Gains were expressed by an increase—and losses by adecrease—of the BOLD (blood oxygen level dependent) response in the ventralstriatum.

Utility Maximization Theory focuses on economic decisions taken by a singleagent in isolation. In contrast, Game Theory has extended the idea of utilitymaximization to social situations, e.g. it makes predictions of how the choices ofother peoples influence the choice of an individual. Behavioural economics (partlyinfluenced by psychology) has developed a battery of different games (e.g. TrustGame, Public Goods Game, Prisoner’s Dilemma, etc.; for an introduction to eco-nomic games see Chap. 2 by Civai and Hawes in this book), which test theassumptions made by Game Theory. However, the empirical data do not alwaysyield support for the theory. Naturally, people take into account the choices—orputative choices—of others when making their decisions, but their behaviour isoften incongruent with the traditional economic view of man as a homo

1 Neuroeconomics—An Introduction 3

economicus. It is stated that the homo economicus makes decisions guided byself-interest (i.e. the maximization of personal benefit), that his decisions arecompletely rational and that all information necessary for making a choice isavailable. Results from the dictator game where player 1 (the dictator) has to splitan endowment with an anonymous person (player 2) show that people do notbehave in a manner congruent with that expected according to the view of man as ahomo economicus (i.e. to take all the money and to award no money at all to player2) (Camerer 2003). Instead, cross-cultural studies have shown that the “dictator” isfar more cooperative, with mean allocations to the receiver (player 2) of about 28 %(Engel 2011). Based on the fact that this game, in its original version, is played as a‘one shot’ game, i.e. the dictator has no reason to fear punishment from player 2 in asubsequent interaction; the dictator game is thought to be a measure of purealtruism.

In addition to the influence of others on people’s choices (Game Theory), thereare other crucial factors that influence economic behaviour. One of the mostprominent factors studied in neuroeconomics is the relationship of the time lagbetween the decision and its consequences, referred to as temporal discounting.Interestingly, psychologists have investigated this topic for decades as delay ofgratification (Mischel et al. 1989). In his seminal ‘Marshmallow Study’ at StanfordUniversity in 1972, Mischel devised an experiment in which children were affordedthe opportunity to ‘earn’ marshmallows. If the children could resist eating the firstmarshmallow they were offered, they were promised a second one, i.e. they wouldreceive two marshmallows instead of one. The duration each child resisted thetemptation to eat the initial marshmallow was analysed, and it was subsequentlyinvestigated whether or not delaying gratification correlated with future success.While the majority of the approximately 600 child participants attempted to resistthe urge to eat the first marshmallow, only one-third delayed gratification longenough to get the second marshmallow. Analyses suggested that the age of thechildren was a crucial factor in influencing the child’s success on this task. Withincreasing age, the ability to defer gratification increases. These findings have sincebeen extended to adult samples, using various kinds of reinforcement. Intelligence(positive association) and gender (females were superior in resisting an immediatesmall reward in favour of a delayed bigger reward; evolutionary factors are dis-cussed to account for this gender effect) turned out to be further prominent pre-dictors of the ability to defer gratification. Under the label temporal discounting thisphenomenon was investigated by means of functional magnetic resonance imaging(fMRI). McClure et al. (2004a) could identify distinct neural systems responding toimmediate and delayed rewards. Whereas the limbic system is activated byimmediate rewards (t = 0) the prefrontal cortex responds to both immediate anddelayed rewards (t > 0), but more so when the delayed option is chosen. Thesefindings hold true for monetary reinforcement as well as for primary rewards, e.g.sex (McClure et al. 2004b). The dissociation between cortical and subcortical brainregions with respect to immediate rewards supports the role of the limbic system(comprising the ventral striatum that is also named “the reward centre”) for drivesand instincts and the role of the prefrontal cortex for impulse control and cognitive

4 M. Reuter and C. Montag

processes. The latter are essential for evaluating offers and for deferring rewardsuntil a future time point. There is plenty of evidence that the more a person dis-counts a delayed reward, the more likely that person is to exhibit a range ofbehavioural problems, including clinical disorders (e.g. drug addiction, impulsecontrol disorders). The ventromedial prefrontal cortex (vmPFC) has shown to beinvolved in impulse control and in individual propensity to engage in risky beha-viours (Bechara et al. 2000, 2002).

1.2 What We Have Learned from Animal Models

The crucial question when referring to findings from animal research is whetherresults can be extrapolated to humans. Preclinical trials—typically conducted inrodents—for the development of new drugs targeted at the treatment of humandiseases, clearly answer this question with “yes”. Excellent animal models for arange of psychological phenomena, e.g. anxiety, are available and do allow forpredictions of the anxiolytic effects of a certain substance in humans. Even for thosemore complex behaviours relevant to the field of neuroeconomics, animal modelexist. For example, Chen et al. (2006) were able to demonstrate that Capuchinmonkeys are able to use tokens to purchase food from experimenters and that theyprefer to trade with those experimenters who offer the best deals for their “money”.In other words, even New World monkeys understand the principles of the market.Nonetheless, it is evident that the transfer from animal model to human is notalways successful or feasible. Ethical concerns are a crucial consideration in thisrespect.

The invention of imaging techniques [e.g. MRI, positron emission tomography(PET)] has made it possible to study the human brain during task performance.Although PET imaging requires the administration of a radioactive ligand into thecentral blood system, it is a safe technique that can be used for research purposeswith humans. More invasive techniques, such as microdialysis (a sampling tech-nique for the continuous measurement of free, unbound concentrations of neuro-transmitters or hormones in the extracellular fluid of brain tissue) or single-cellrecordings (for assessment of the firing rate of neurons) in the living brain are, ofcourse, not possible in healthy humans for ethical reasons. However, the neuro-sciences have provided many groundbreaking animal studies with broad relevanceto neuroeconomics. As mentioned above, reinforcement and reward are crucial fordecision-making, although other context variables also have a tremendous influenceon our choices. The biological system most prominently related to reward is thedopamine (DA) system (Schultz et al. 1997). Its relevance was first identified in thecontext of studies on drug addiction. It was suggested that the dopaminergic systemis the final common pathway of reward since almost all substances with thepotential of causing addiction act via the DA system, either directly or indirectly(Spanagel and Weiss 1999). These findings could be extended to naturally occur-ring rewards (primary reinforcers like food or sex). The crucial question of how the

1 Neuroeconomics—An Introduction 5

DA system could encode signals of reward is best studied in animal studies (for areview see Schultz 2013).

In a seminal study by Tobler et al. (2005), the activity of midbrain dopamineneurons in Macaque monkeys was recorded while cues signalled the probability ofreceiving a primary reinforcer (juice) of varying magnitude. This experiment triedto explain how the brain disentangles the probability and magnitude of reward.Keeping the probability of reward constant, the firing rate of the DA neuronsincreased monotonically with the expected liquid volume. The DA neurons werealso able to encode the expected reward value, i.e. the combination of magnitudeand probability. In a further step the authors conducted an experiment in which thereward outcomes were explicit rather than probabilistic. They used conditionedstimuli that explicitly predicted various amounts of liquid (p = 1). For example, aconditioned stimulus normally indicates the deliverance of 0.15 ml juice. Theysubsequently followed the conditioned stimulus with an unpredicted stimulus;either a smaller (0.05 ml) or larger (0.50 ml) volume of liquid; in response to whichthe firing rate of the dopaminergic neurons decreased or increased respectively. In afinal experiment, Tobler et al. used one stimulus that predicted the delivery of eithera small or a medium volume of juice with equal probability and a second stimulusthat predicted a medium or a large volume with equal probability. Results indicatedthat for both conditioned stimuli, the deliverance of the, respectively, larger stim-ulus resulted in an increase—and that the deliverance of the, respectively, smallerstimulus resulted in a decrease—of the neuronal firing rate. Surprisingly, theidentical medium volume delivery had opposite effects on neuronal activity,depending on the prediction. The prediction is in turn influenced by a framingeffect. A medium amount of juice is attractive when compared to a small volume ofjuice, but unattractive in comparison to a large volume. The authors argue that,given the infinite number of reward values that are possible, this is an adaptiveprocess. Thus, the firing rates of the dopaminergic neurons adapt to the coding ofreward value in order to have a greater capacity for coding the likelihood of rewardoutcomes. Results showed that dopaminergic neurons encode a combination ofmagnitude and probability; the so-called expected reward values and that theresponse of the dopaminergic neurons depends on framing effects (for a concisereview on the behavioural dynamics and neural basis of the framing effect pleasesee Chap. 9 by X.T. Wang et al. in this book).

The effects of expected reward have a discrete neural signature in humandecision-making, as demonstrated in a seminal study by Preuschoff et al. (2006).Using a simple gambling task in an fMRI setting, the authors varied expectedreward and risk in an uncorrelated manner. Risk is a consideration because manydecisions in daily life have to be made under conditions of uncertainty. Expectedreward and risk were both represented in dopaminergic innervated brain regions,however, there was a temporal dissociation in their processing. The brain firstprocesses information related to reward expectancy and later risk information.Besides the aforementioned study by Preuschoff et al., there are numerous examplesin the literature of findings from animal studies being mirrored in neuroeconomicstudies on humans. For example, Roiser et al. (2009) investigated the influence of

6 M. Reuter and C. Montag

framing effects on human decision-making and its neural activation patterns. Theyfound that amygdala activation was stronger in those trials where participants madechoices in congruence with—compared with those made counter to—the frame, butthat this effect was only apparent in subjects carrying the short allele (s-allele) of theserotonin transporter polymorphism (5-HTTLPR; for more information on geneticssee Chaps. 4 and 23 by Reuter and Montagin this book), a genetic variant related toneuroticism, depression and anxiety (Roiser et al. 2009).

1.2.1 Validation of Theoretical Models on Human DecisionMaking in Animals

As described above economists have developed theories (e.g. utility maximization;game theory, etc.) to predict human decision-making. Researchers from cognitivepsychology and mathematics have established such theoretical models, albeit with adifferent focus. The best studied of these models try to explain choices via thesimplest form of decision an individual can make—the choice between two alter-natives. The focus here lies on the interdependency of choice probability andresponse time (RT). The most familiar expression of this relationship is the speed–accuracy trade-off, which characterizes the decision-maker’s dilemma of beingforced to negotiate between the competing demands of response speed and responseaccuracy (Bogacz et al. 2010). Many decisions are based on information thataccumulates over time. Although the probability of making a correct or favourabledecision increases with the amount of information we have gathered, sometimes weare forced to make quick and ill-informed decisions (e.g. to prevent harm). Thedevelopment of Sequential Sampling Models has increased the theoretical under-standing of such decision processes, however, it was the empirical validation inanimal models (i.e. single-cell recordings in monkeys) that initially helped to testand refine these models. David Sewell and Philip Smith (see Chap. 14) provide athrilling and comprehensive introduction to a research area in which theoreticalmathematical frameworks and computational neuroscience meets empirical neu-rophysiological animal research. Through recent advances in imaging techniques,these models have now also been successfully tested in humans (Forstmann et al.2010).

1.3 Ecological Validity

One of the most severe criticisms of neuroeconomic research is the frequent lack ofecological validity in studies. What can we learn from human decisions that areregistered in fMRI scanners; a loud environment where movement is extremelyrestricted and where social interaction partners are presented—if at all—via video

1 Neuroeconomics—An Introduction 7

glasses (Mäki 2010). Imaging techniques like MRI, and PET are fantastic tools forallowing us to register brain activity, even in subcortical brain regions, while stimuliare processed. However, these techniques are not made for field studies, in whichparticipants are observed in their natural environment. However, history hasdemonstrated that experimental approaches applied in the laboratory can indeedprovide valuable insights into human behaviour and have thereby helped to legit-imize the discipline of experimental economics. The same success is demonstrablefor neuroeconomic studies using imaging techniques. Neuroeconomics permanentlystrives to establish ecological validity in any way possible. Implementing monetaryreward in the behavioural games is one of these provisions. Decisions must berelated to real consequences for the decision-maker, in order to be ecologicallyvalid. It can be assumed that engagement in an economic game, which is played formonetary stakes, allows even the (fMRI) scanner environment to fade into thebackground.

Imaging studies are still common in neuroeconomics and have greatly boostedthe success of the discipline. However, alternative neuroscientific techniques thatare not limited to scanner facilities or laboratories are becoming increasinglyprevalent. Molecular genetics is a key example in this instance. Behaviour can bestudied in participants’ natural environment and the participant subsequently pro-vides a cell sample (e.g. by means of a noninvasive buccal swap) for geneticanalyses. This approach ensures that participants are not influenced by the exper-imenter while exhibiting their natural behaviour. Most economists embarking uponneuroeconomic study are initially unaware that molecular genetics can provideinformation on the brain. Genes code for neurotransmitters, hormones, receptorsand enzymes relevant for brain metabolism. Static genetic variants, called poly-morphisms, exert a permanent influence on these gene products, by modulating theexpression or the structure of gene products. In recent years a new field has grownfrom molecular genetics: epigenetics. Epigenetics dispels the ancient myth thatgenes are like an unstoppable computer programme, started at the moment when thesemen and egg of our parents have fused. Prior to the introduction of epigenetics,genetic research often occasioned strong resentment among the general population,as it was considered synonymous with fatalism—a thing you cannot change.Epigenetic research has served to change this view of genes as destiny.Epigeneticists have demonstrated that the environment can and does influence ourgenes; not the static genetic polymorphisms, but rather the expression of our genes,by changing the methylation patterns of the genes. Thus, the relationship betweengenes and behaviour/environment is bidirectional (for a more detailed introduction,please see the genetics Chap. 23 in the methods section of this book and Chap. 4“Genes and Human Decision Making”).

Genetic approaches are not limited to field studies, but are also suitable forlaboratory experiments. Neuroeconomics studies have used this method success-fully and it will certainly become more and more important in the field. In a seminalstudy, Israel et al. (2009) have reported an association of a single nucleotidepolymorphism (SNP; rs1042778) on the oxytocin receptor gene (OXTR) andprosocial fund allocations in the dictator game. This finding was replicated in an

8 M. Reuter and C. Montag

independent sample and serves to corroborate animal and human studies indemonstrating the pivotal role of the hormone oxytocin for prosocial behaviour (fora review see Ebstein et al. 2009).

1.4 Future Perspectives in Neuroeconomics

No matter how strongly neuroeconomists strive to improve prediction models onhuman decision-making through use of neuroscientific methods, criticism willalways be present. It is impossible to convince every sceptic that biological vari-ables can help us to better understand human behaviour and that neuroscientificapproaches are helpful in verifying and refining theoretical economic models. Onthe whole, however, most criticism pertains to serious concerns, which must betaken seriously. The exciting possibilities offered by neuroscientific methods carrywith it the risk of overselling the findings (Rubinstein 2008). The mass mediacontributes to this by exaggerating its reports of solid scientific work. We take thisopportunity to discuss two such examples. We recently published a neuroeco-nomics study entitled “Investigating the genetic basis of altruism: The role of theCOMT Val158Met polymorphism” (see a detailed description of this study inChap. 4 in this book). The newspapers wrote articles on this study with headlineslike this: “Altruism gene makes people generous”. It is obvious that altruism is not amonogenetic phenotype, but is subject to influence both from many genes, and fromenvironmental effects. Therefore, there cannot be “an (a single)” altruism gene. Thesecond example demonstrates that researchers sometimes tend to oversell theirscientific findings. Kuhnen and Chiao (2009) published an article based on a sampleof 65 participants entitled “Genetic determinants of financial risk-taking”. TheScientific American reported this study with the headline “My genes made meinvest: DNA implicated in financial risk-taking”. One can debate the connotationsof the word “determinants”, but it is obviously related to “determinism”, implyingthat there are no other sources of variance relevant for risk-taking, besides the5-HTTLPR polymorphism investigated in this study. For the sake of ScientificAmerican, it must be noted that the word “implicated” reflects the scientific value ofthis study very well, much better, in our opinion, than the phrase “genetic deter-minants”. Thus, a modest interpretation of scientific results in the field of neuroe-conomics is essential to increase the respectability of the discipline.

It is obvious that the methodological spectrum of neuroscientific techniques hasdramatically increased over the last years. Neuroeconomics is no longer limited tofMRI studies. We see EEG-, genetic-, endocrinological-, and TMS—studies, toname but a few methods, and the use of such methods will dramatically increase infuture research. The paradigms and games used in neuroeconomic research will alsobecome more and more elaborate in the endeavour to disentangle the subcompo-nents involved in economic decision making. Finally, the introduction of fieldstudies will further enrich the spectrum by allowing researchers to test laboratoryhypotheses in “real life”.

1 Neuroeconomics—An Introduction 9

References

Bechara A, Tranel D, Damasio H (2000) Characterization of the decision-making deficit ofpatients with ventromedial prefrontal cortex lesions. Brain 123(Pt 11):2189–2202 (Nov 2000).Erratum in: Brain 132(Pt 7):1993 (Jul 2009)

Bechara A, Dolan S, Hindes A (2002) Decision-making and addiction (part II): myopia for thefuture or hypersensitivity to reward? Neuropsychologia 40(10):1690–1705

Bogacz R, Wagenmakers EJ, Forstmann BU, Nieuwenhuis S (2010) The neural basis of thespeed-accuracy tradeoff. Trends Neurosci 33(1):10–16

Camerer CF (2003) Behavioral game theory: experiments in strategic interaction. PrincetonUniversity Press, Princeton, NJ

Chen MK, Lakshminaryanan V, Santos LR (2006) The evolution of our preferences: evidencefrom capuchin monkey trading behavior. J Polit Econ 114(3):517–537

Ebstein RP, Israel S, Lerer E, Uzefovsky F, Shalev I, Gritsenko I, Riebold M, Salomon S,Yirmiya N (2009) Arginine vasopressin and oxytocin modulate human social behavior.Ann NY Acad Sci 1167:87–102

Engel C (2011) Dictator games: a meta-study. Exp Econ 14:583–610Forstmann BU, Anwander A, Schäfer A, Neumann J, Brown S, Wagenmakers EJ, Bogacz R,

Turner R (2010) Cortico-striatal connections predict control over speed and accuracy inperceptual decision making. Proc Natl Acad Sci USA 107(36):15916–15920

Kahneman D, Tversky A (1979) Prospect theory: an analysis of decision under risk. Econometrica:J Econom Soc 47(2):263–291

Kuhnen CM, Chiao JY (2009) Genetic determinants of financial risk taking. PLoS One 4(2):e4362.doi:10.1371/journal.pone.0004362

Israel S, Lerer E, Shalev I, Uzefovsky F, Riebold M, Laiba E, Bachner-Melman R, Maril A,Bornstein G, Knafo A, Ebstein RP (2009) The oxytocin receptor (OXTR) contributes toprosocial fund allocations in the dictator game and the social value orientations task.PLoS ONE 4(5):e5535

Mäki U (2010) When economics meets neuroscience: hype and hope. J Econ Methodol 17:107–117

Marshall M (1920) Principles of economics: an introductory volume, 8th edn. Macmillan, LondonMcClure S, Laibson D, Lowenstein G, Cohen J (2004a) Separate neural systems value immediate

and delayed rewards. Science 306:503–507McClure SM, Ericson K, Laibson DI, Loewenstein G, Cohen JD (2004b) Time discounting for

primary rewards. J Neurosci 27(21):5796–5904Mischel W, Shoda Y, Rodriguez ML (1989) Delay of gratification in children. Science 244:933–

938Preuschoff K, Bossaerts P, Quartz SR (2006) Neural differentiation of expected reward and risk in

human subcortical structures. Neuron 51(3):381–390Roiser JP, de Martino B, Tan GC, Kumaran D, Seymour B, Wood NW, Dolan RJ (2009) A

genetically mediated bias in decision making driven by failure of amygdala control. J Neurosci29(18):5985–5991

Rubinstein A (2008) Comments on neuroeconomics. Econ Philos 24:485–494Schultz W (2013) Updating dopamine reward signals. Curr Opin Neurobiol 23(2):229–238Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science

275(5306):1593–1599Spanagel R, Weiss F (1999) The dopamine hypothesis of reward: past and current status. Trends

Neurosci 22(11):521–527Tobler PN, Fiorillo CD, Schultz W (2005) Adaptive coding of reward value by dopamine neurons.

Science 307:1642–1645Tom SM, Fox CR, Trepel C, Poldrack RA (2007) The neural basis of loss aversion in

decision-making under risk. Sci 315(5811):515–518

10 M. Reuter and C. Montag

Part IGames in Experimental Economics

Chapter 2Game Theory in Neuroeconomics

Claudia Civai and Daniel R. Hawes

(Spade): If you kill me, how are you going to get the bird? If Iknow you can’t afford to kill me till you have it, how are yougoing to scare me into giving it to you?(Gutman): “Well, sir, there are other means of persuasionbesides killing and threatening to kill.”(Spade): “Sure, but they’re not much good unless the threat ofdeath is behind them to hold the victim down. See what I mean?If you try something I don’t like I won’t stand for it. I’ll make ita matter of your having to call it off or kill me, knowing youcan’t afford to kill me.”(Gutman): “I see what you mean. That is an attitude, sir, thatcalls for the most delicate judgment on both sides, because, asyou know, sir, men are likely to forget in the heat of action wheretheir best interests lie and let their emotions carry them away.”

—The Maltese Falcon

Abstract Game theory and contemporary decision theory provide the mathemat-ical foundation of economics. Neuroeconomics, which principally concerns itselfwith the integrative study of brain, mind and behavior, builds on this mathematicalfoundation while also drawing heavily from the repository of experimental para-digms that have grown out of economic game theory and behavioral economics.Game theory is central to neuroeconomics primarily because it constitutes a formalmathematical framework with which to bridge insights occurring at different levelsof neuroeconomic analysis. In particular, game theoretic principles can be used toexpress neuroscientific ideas about the brain, psychological concepts regarding thehuman mind, and economic predictions of human behavior, thereby making thesedifferent ideas more rigorously relatable to each other. In this chapter we provide anontechnical introduction to game theory and its relation to neuroeconomics. It hasbeen written as an overview of the basic concepts most likely to be encountered inneuroeconomic research. The first part of the chapter introduces the reader to the

C. Civai (&)Donders Institute for Brain, Cognition and Behaviour, Radboud University,Nijmegen, Netherlandse-mail: c.civai@donders.ru.nl

D.R. HawesInstitute of Food and Resource Economics, University of Bonn, Bonn,Deutschlande-mail: daniel.hawes@uni-bonn.de

© Springer-Verlag Berlin Heidelberg 2016M. Reuter and C. Montag (eds.), Neuroeconomics, Studies in Neuroscience,Psychology and Behavioral Economics, DOI 10.1007/978-3-642-35923-1_2

13

basic concepts and philosophical underpinnings of game theory in relation toneuroeconomics. The second part is an introduction and discussion of commongames, including the games featured in the other chapters of this book.

2.1 Introduction

In the well-known movie “The Maltese Falcon” (Wallis and Huston 1941)Humphrey Bogart’s character (Spade) attempts to ‘call bluff’ on his adversaries’threats. He does so by what amounts to game theoretic reasoning—namely byputting into relation each man’s objectives, their individual beliefs, and their sharedknowledge of the situation—and deduces that the adversary, Gutman, cannotpossibly afford to kill him for risk of never learning the Maletese Falcon’swhereabouts. Gutman of course concludes with the cautioning appeal to keep inmind the potential for internal conflict between reason and emotion, and howrational objectives may become lost in its wake.

Neuroeconomics relates to this scene as an extension of economic analysis,traditionally preoccupied with the prediction of behavior, into the realm of the brainand the mind (the internal processes that give rise to behavior). Neuroeconomics isinherently interdisciplinary, but draws most heavily from economic decision theory,psychology, and neuroscience. Game theory, as the mathematical foundation ofeconomic decision theory, is central to neuroeconomics, because of the relativelynew disciplines’ intellectual history (i.e., because neuroeconomics lends fromeconomics, and economics uses game theory), but also because it has the potentialto function as the mathematical foundation upon which neuroeconomic theorymight be developed further into a discipline of its own right (i.e., the properties ofgame theory make it an attractive candidate for trying to develop an integrativetheory of brain, mind, and behavior).

The purpose of this chapter is to introduce researchers coming to neuroeco-nomics from fields other than economics to the basic principles and ideas thatfeature in game theory. While outlining the basic tenets of game theory, we alsoattempt to draw philosophical and historical connections that link game theory tothe goals and of neuroeconomics.

Game theory, generally speaking, is the study of mathematical objects whosestates and properties interact. At the level of neuroscience, game theory could beused to describe the interactive or competitive firing of neuron populations. At thelevel of psychology, game theory could be used to describe the emergence ofmental states in relation to interacting cognitive processes, or the emergence ofbehavior from competing mental states. Traditionally, in economic analysis, gametheory has been used to predict the behavior of multiple, interacting, intelligentdecision-makers. Hence, game theory can be viewed as a general tool for the logicalconsideration of strategic and nonstrategic relationships at multiple levels ofanalysis, as long as the concepts being used have representation as numbers and therelationships are closed or bounded correspondences. Economic examples, which

14 C. Civai and D.R. Hawes

represent the mathematical objects of game theory as the behaviors of intelligentdecision-makers, are generally intuitive, instructive, and easy to develop, whereforethis consideration forms the common approach in most textbooks, and also in thischapter. However, we hope to draw the reader’s attention to the fact that gametheory itself, being a mathematical language and logic system, has far widerapplication and that fundamental issues regarding measurement of utility throughgames, and how this relates to theory development in physics, remains outside ofthe scope of this chapter. The classic book Theory of Games and EconomicBehavior (in particular chapter 1) remains invaluable on this account(von Neumann and morgenstern 1944).

As neuroeconomics becomes more consolidated as a stand-alone discipline, weexpect the importance of game theory for neuroeconomics to increase, and havereasoned hope for the potential of game theory to function as a bridge acrossdifferent levels of brain–mind behavior analysis. This latter part, concerning thefuture of game theory in neuroeconomics, remains personal conjecture and is onlyhinted at or mentioned in passing throughout this chapter; however, it informs muchof our thinking in choosing which elements of game theory to focus on for thisintroduction, and how to present the basic principles. It makes sense, therefore, toprovide a quick note on our definition of neuroeconomics.

Neuroeconomics as an interdisciplinary scientific approach aimed at discovering/creating a unified theory of human behavior and cognition, via integrative study ofmind, brain, and behavior relationships.

Practically, the neuroeconomic enterprise combines concepts, methods, andtechnological tools from neuroscience (see Chap. 8) with formal analysis ofdecision-making, typically drawing heavily from economic decision theory as wellas psychology.

As a procedure, neuroeconomics aims to experimentally link the neurophysio-logical and behavioral constituents of the decision process to each other, and thenconceptually relate these links to psychological concepts of mental activity viaformal models, typically in the tradition of economics and decision theory.

With this definition of neuroeconomics in place, the formal models developedwithin game theory enable mathematically precise descriptions of the decisionprocess, which in turn allow prediction, specification, and comparison of neuralactivity presumed to underlie decision-making. Additionally, experimental para-digms developed within experimental game theory (i.e., games) are prominentlyfeatured in neuroeconomics research, where the goal is often to differentiatebetween competing economic models of the decision process (Glimcher andRustichini 2004; Rustichini 2005; Camerer et al. 2005), or to investigate competingdescriptions of mental processes thought to underlie a particular kind of decision orbehavioral phenomenon.

Because it is a common source of confusion, the final note of this introductionpoints to game theories’ conceptual birth in expected utility theory and connectionto the economic idea of rational decision-making (Mongin 1997): As will bediscussed later, strict economic rationality is not necessary for game theoreticinquiry, and many game theoretic applications in neuroeconomics are explicitly

2 Game Theory in Neuroeconomics 15

geared toward understanding decision processes for which individuals systemati-cally violate the predictions of rational choice theory and expected utility maxi-mization in one way or another. Furthermore, the interested reader is encouraged toconsider differences in the economic meaning of rationality compared to thenoneconomic colloquial meaning of rationality, for example, by consultingAnthony Downs’ book “An economic theory of democracy.” (1957).

The remainder of this chapter elaborates on the basic elements described in theabove introduction. It concludes with a compendium of games most commonlyencountered in the neuroeconomics literature, including a digest of each of thegames featured in this book.

2.1.1 Basic Terms and Definitions

This chapter is a nontechnical introduction to game theory1; however, the mainstrengths of game theory derive precisely from its mathematical exactness andrigorous definitions, wherefore some minimal recourse to formal terminologyappears unavoidable. We therefore begin by introducing some basic terms anddefinitions that will appear throughout this chapter, first among which are those thatdescribe an economic Game.

A formal game consists of three basic objects:

1. Players are independent decision-makers, mathematically represented in termsof their utility functions; i.e., by a function that assigns ordinal preference rankto all possible outcomes of the game.

2. Actions are full descriptions of the actions each player may take during thegame. These action descriptions may differentiate between different points oftime, situational circumstances or—more generally—stages of the game.

3. Payoffs are full descriptions of the outcome (and consequently utility) experi-enced by each player for each possible combination of actions that may occurduring the game.

Additional objects used to describe games may include

(a) Information Sets are full descriptions of the information available to eachplayer in each stage of the game. Information pertains to the actions availableto the players, their utility functions, as well as the current history and possibletrajectory of stages in the game.

(b) Environments are nonstrategic mechanisms capable of influencing any of theabove elements of the game, typically in a probabilistic manner. For example,the environment may impose random restrictions on what type of actions are

1Comprehensive, technical treatments can be found for example in Myerson (2013) and Osborneand Rubinstein (1994).

16 C. Civai and D.R. Hawes

available at a given stage of the game, or may influence the payoff distributionfor game outcomes.

(c) Strategies are probability distributions over all actions for each stage of thegame. Strategies are full descriptions of what action should be chosen for eachpossible stage of the game. A strategy can describe a specific action for eachstage, in which case it is referred to as a pure strategy. Alternatively, a strategymay assign probabilities to multiple actions for any given stage, in which caseit is referred to as a mixed strategy. Importantly, a strategy specifies an actionor mixture of actions for every possible contingency of the game.

In nondegenerate2 games, the actions taken by all of the players collectivelyinteract to determine the payoffs and consequent utilities experienced by eachplayer individually. This gives rise to the strategic considerations, which lie at theheart of game theory.

The major objectives of game theoretic inquiry may therefore be described as theaim to:

1. Formulate applicable game descriptions for real-world decision problems;2. Develop and apply solution concepts to positively describe or normatively

prescribe the strategic reasoning processes that are used by players in gamesituations;

3. Develop, refine, and apply equilibrium concepts that describe stable patterns ofstrategic reasoning and behavior between the players.

4. Identify the existence and properties of such equilibriums.

The above elements of a game are described and presented in either a so-calledExtensive Form, or as a so-called Normal Form representation.

Extensive Form representations are depictions of games by the way of math-ematical graphs. In these graphs the nodes represent the different stages of thegame, and the edges (the lines connecting the nodes) represent the actions that leadto each stage. Extensive Form games specify exactly the possible sequence ofactions and their resulting outcomes for a game, and are therefore particularly usefulwhen the order in which players choose during the game is relevant to its outcome.An annotated example is given in Fig. 2.1a.

Normal Form representations of games are more commonly used for games inwhich the order of moves is irrelevant, as is the case in so-called one-shot,simultaneous move games, i.e., games in which the resulting outcome is determinedby considering all chosen actions simultaneously.3 Normal Form games are pre-sented as matrices, with each cell of the game matrix containing a vector of final

2That is, games that can not be reduced to sets of mere one-player decision problems in which theactions of other players are irrelevant to each players utility.3Think for example of a secret ballot election, in which each player enters a vote into a computer,and the computer then decides the election outcome by considering all such submitted votes.

2 Game Theory in Neuroeconomics 17

payoffs for each possible combination of actions (for all players) that may occur inthe game.4 An annotated example is given in Fig. 2.1b.

The well-known Prisoner’s Dilemma (PD) provides a useful example fordemonstrating the concepts introduced thus far, and both Fig. 2.1a, b are depictionsof this game. The PD is a one-shot, simultaneous move game between two players.Each player in the game has two actions available to him or her. We call these

Fig. 2.1 a Generic Prisoner’s Dilemma (normal form). Rows indicate actions available to Player 1({C, D}) and columns indicate actions of Player 2 ({c, d}). Cells list the utility of eachcombination of actions. Utility to player 1 is listed first in black font. Utility to player 2 is listedsecond and using red font. We have highlighted the Nash equilibrium outcome in the bottom rightcell, corresponding to choice of D, and d by the two players. b Generic Prisoner’s Dilemma(extensive form). For illustration we also show a strategic form version of the PD. Nodes indicateplayers’ “turns” in the game. Player 1 moves first at the initial node of the game (open circle).Edges indicate actions available at each node ({C, D} for player 1, {c, d} for player 2). Dottedlines connecting nodes indicate information sets. Two nodes belonging to the same information setcannot be distinguished by the player. Hence, player 2—at the time of her move—does not knowwhether player one has chosen C or D. Final nodes contain the utility to each player. We haveagain highlighted the Nash equilibrium for this game. Although the extensive form and strategicform of the prisoner’s dilemma are equivalent, not all extensive form games can be transformedinto an equivalent normal form game

4Note that all extensive form games can be transformed into normal form, but not all normal formgames may have an equivalent extensive form; hence there also exists a qualitative difference.

18 C. Civai and D.R. Hawes

actions {C} and {D} for player 1, and {c, d} for player 2. The intuition behind thePD was originally suggested by Flood and Dreshner in the 1950s, later formalizedby Albert Tucker (Poundstone 1992), and goes as follows: two criminals are beingquestioned by the police; both are facing the choice of whether to defect ({D} or{d}) on their partner in crime by providing incriminating evidence to the police, orto remain cooperative ({C}, or {c}) by not revealing any information duringinterrogation. The possible actions to each player are therefore to cooperate or todefect ({C, D}) for player one, as well as for player two ({c, d}). In the case thatboth prisoners remain cooperative ({C, c}) the police lacks evidence for a fullconviction and both criminals go to jail for only a short period of time; in the casethat both prisoners defect on each other ({D, d}), they both land in jail. Theinteresting scenario that creates the dilemma results from what happens when oneplayer defects while the other remains cooperative ({C, d} or {D, c}): in this casethe defector goes free entirely, while the defected upon goes to jail for an extendedperiod of time. The possible outcomes to the game are therefore ({C, c},{C, d},{D,c}, and {D, d}), and the payoffs are given by the utility experienced for the differingamounts of jail time (in the example we let these utilities be 0, 2, 4, and 6).Assuming that both players prefer less time in jail to more time in jail (i.e., theyprefer higher utility values among the outcomes in Fig. 2.1b), each player in thisgame fares best when he or she is the only one to defect, and worst if he or she is theonly one to cooperate. Additionally, each player also prefers the outcome of mutualcooperation to the outcome of mutual defection.

Standard game theoretic reasoning predicts an equilibrium outcome for thePrisoner’s Dilemma in which both players defect. This is because the outcome frommeeting cooperation by the other player with defection (i.e., the best outcome) ispreferred to the outcome from mutual cooperation; at the same time meetingdefection by the other player with defection also is preferred to being the onlycooperator in the game (i.e., the worst outcome). Hence defecting is a best responseto whatever the other player does, and thus mutual defection becomes an equilib-rium response in the game.

The terms best response and equilibrium deserve special note in this context, asthey relate to a very particular solution concept in game theory, known as NashEquilibrium, Nash Solution Concept, or Best Response Equilibrium. This conceptdefines equilibriums for finite noncooperative5 games as instances in which eachplayer is playing a best response given the strategies being played by all otherplayers in the game. A strategy, in turn, is defined as a best response if and only ifits expected utility is at least as large as that obtained from any other possiblestrategy the player may play.6 Hence, in Nash equilibrium no player has any

5Game theory divides games into cooperative and noncooperative types. All of the games in thischapter are noncooperative games, and we therefore do not spend a lot of time discussing thedifference between the two classes of games, which can be found elsewhere (e.g., Myerson 2013).6The concept is named after John Nash, among other contributions, for his work on such equi-librium points in n-person games (Nash 1950).

2 Game Theory in Neuroeconomics 19

incentive to unilaterally change his or her chosen response in the game, since it is abest response to what all other players are choosing at the time.

Returning to the PD, it is also worth noting that defection is a so-called dominantstrategy for both players. This means that defecting is optimal regardless of whatother players choose. Not all games have dominant strategies, however, and theyare not necessary for Nash equilibrium. We will see examples of such games later inthis chapter.

2.1.2 Rationality and Expected Utility Maximization

Standard game theoretic solutions to decision situations such as the Prisoner’sDilemma prescribe the optimal behavior of a rational decision-maker with unlim-ited analytical resources under the goal of utility maximization. However, humananalytical resources are necessarily limited, introspection may be noisy, preferencesmay be uncertain, and any host of context-dependent affective responses maycontribute to the decision process. Furthermore, decision-makers may rely onheuristics and intuitions (i.e., emotions or gut feelings) in order to reduce cognitivecosts (Simon 1972, 1990; Tversky and Kahneman 1974; Damasio et al. 1991;Gigerenzer 2004; Glöckner and Betsch 2008; Glöckner and Hochman 2011). Suchpsychological features and behavioral strategies may be behaviorally indistin-guishable from rational choice behavior in some contexts, but lead to systematicbiases and inconsistencies in others. Investigation of such behavior has a longhistory in economics (e.g., Allais 1979; Ellsberg 1961; Loomes and Sugden 1982),and has given rise to extensions (e.g. Friedman and Savage 1948; Aumann 1997),and modifications such as bounded rationality (Selten 1990) and Prospect Theory(Kahneman and Tversky 1979). Consequently, economic theory has produced awealth of models and theory refinements that individually address instances inwhich standard economic model possess poor predictive validity.7 To the extentthat none of these competing models and refinements approaches the economicideal of a unified theory of human behavior, neuroeconomics is seen as a disci-plined approach to answering unresolved questions in economics: for example byjuxtaposing competing models [e.g., by investigating whether a unified system, orseparate systems compute the value for present versus future rewards (McClureet al. 2004)]; by investigating the rules that govern for whom, when and underwhich circumstances one model, or model specification, is more applicable thananother [e.g., by generating insights into the neurobiology underlying differenttypes of decision-makers (Coricelli and Nagel 2009; Bhatt et al. 2010), and learnedstrategic behaviors (Hawes et al. 2012)]; or by helping to establish the precise role

7Note that Economic Theory is primarily concerned with generalizable predictive validity, and thatcontent validity regarding the process via which a decision is actually made, is at best secondary,but probably irrelevant, to as-if modeling in economics.

20 C. Civai and D.R. Hawes