![Page 1: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/1.jpg)
Computational Approaches to Economic Valuation & Strategy Choice
Colin CamererAntonio Rangel
Caltech
![Page 2: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/2.jpg)
Outline
• Brief history of the role of computation in Economics• Models of valuation and simple choice (Rangel)• Models of strategic choice and learning in games (Camerer)• Computational issues at different levels: individuals, firms,
markets (Camerer)• Future directions of research
![Page 3: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/3.jpg)
I
Brief history of the role of computation in Economics
![Page 4: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/4.jpg)
Computation is at the heart of economic problems
Consider some typical problems:•Individual: What snack should I pick out of the buffet table?•Individual: Optimal investment portfolio?•Firm: Price setting and production selection problem•Market system: price formation
Thus, one would expect computational based models of decision-making to be common in Economics
![Page 5: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/5.jpg)
This is not the case:Traditional deliberate ignorance of computational detail
• The triumph of “as if” modelling (economic behaviorism)-- Pareto (1987):“Pure political economy has therefore a great interest in relying as little as possible on the domain of psychology” -- Friedman (1953)Test predictions of theory rather than realism of assumptionsà can ignore computational detail
• Fictional stand-ins for computation- Walrasian auctioneer- equilibrium in games
• Underlying computational processes are modeled in REDUCED FORM
![Page 6: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/6.jpg)
Traditional view (cte.)
• Axioms are considered primitives (logic vs biology as constraint on choices)
• A developed preference for general mathematical proof over simulation – Study of “procedural rationality” algorithms
(Simon) did not gain traction– Distrust of complicated many-equation
macroeconomic models & simulations– Little post-1990 taste for SFI agent-based
modeling
![Page 7: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/7.jpg)
Neuroeconomics: neurobiologically based computational models of decision-making
Goals of Neuroeconomics:
1. What computations are carried out by the brain to make different types of economics decisions?
2. How are these computations implemented by the brain?
3. What are the implications of this knowledge for economics, finance, education, AI, marketing, … ?
![Page 8: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/8.jpg)
Computation is at the core of Neuroeconomics
COMPUTAT.MODELS
JUGDMENT& DM
NEUROSCIENCE PSYCHOLOGY
ECONOMICAPPLICATIONS
THERAPEUTICAPPLICATIONS
BUSINESSAPPLICATIONS
A.I.
![Page 9: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/9.jpg)
II
Neuroeconomic Models of Valuation and Simple Choice
![Page 10: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/10.jpg)
Example I: Reward Prediction Learning
Event: trial 1 trial 2 trial 3 …
Time: 1 2 3 4 5 6 7 8 9 10 11 12 ….
Cue
wait
Rew
ard
Cue
wait
Rew
ard
Cue
wait
Rew
ard
• Brain’s problem: learn to predict size & timing of rewards that follow each type of cue
• Temporal-difference learning algorithms have been designed in CS to solve this problem (Sutton & Barto (1998))
![Page 11: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/11.jpg)
How can the brain learn the reward function?
Notation:- True value of state s: mean r(s)- pt(s) = computed predicted value at beginning of triat t (= brain’s best guess about the state’s true value)- t(s) = r(s) - pt(s) = error signal in trial t This error term is extremely important: it serves as THE teaching signal!
![Page 12: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/12.jpg)
Learning Algorithm
Step 1. Arbitrarily initialize the decision values p1(s) for all s
Step 2. Every trial t:-- begin with pt(s)-- measure actual reward-- Compute error (t)-- Update the DV for a and c active in trial as follows:
pt+1(s) = pt(s) + (t) where -> (0,1) is a learning rate
• Under very general conditions, E(pt(a|c)) -> E(r(s)) for all s
![Page 13: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/13.jpg)
How well do TD algorithms describe brain’sreward learning?
Cue Reward
TD-ErrorsBefore Learning
TD-ErrorsDuring Learning
TD-Errors AfterLearning if UnexpectedOmission of reward
![Page 14: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/14.jpg)
Single unit recordings from VTA dopamine neurons revealed that these neurons produce responses consistent with TD - learning:
Can we find evidence of TD-error signals in monkeys’ brains?
Schultz [1998]
![Page 15: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/15.jpg)
R
+6 +3
+54
-6
-30
-3
p<<0.001
What brain areas show activation that What brain areas show activation that correlates w/ TD-error signals in humans?correlates w/ TD-error signals in humans?
From O’Doherty et.al. [2003]
CS+ trials
at
tim
e o
f C
S
![Page 16: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/16.jpg)
Example II: Role of Visual Attention in Simple Choice
?
![Page 17: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/17.jpg)
Model: Three Parallel Processes
Visualattention
DVscomputation
Comparator
g(e)= L,R
• e=time elapsed since beginning of choice trial
dL(e), dR(e)
Choose L,R,or wait
![Page 18: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/18.jpg)
Visualattention
DVComputation
Comparator
g(e)
g(e)
dL(e), dR(e)
choose g(e) orswitch
switch
![Page 19: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/19.jpg)
Visual attention process
First fixation: Stochastic bottom-up process
•P0 = Prob first fixation to L
•Exponential latency: Pr(First fixation begins at t)= 1- B.e- t
Subsequent fixations: top-down process
•Follow the commands of the comparator process
![Page 20: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/20.jpg)
Value construction process
t
0
v+
v-
![Page 21: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/21.jpg)
Comparator process:
• During each fixation, the comparator either chooses g(e) or sends a signal to the visual system to switch gaze
• Length of each fixation stochastic:-- d = duration of current fixation-- Pr(comparator evaluates at d)= 1- A.e- d
• Decision made as follows:-- rx(e) = d(tx(e)) - d(ty(e))
•-- Choose g(e) with probability
-- Wait (and switch fixation) with prob
• Always switch after first fixation
![Page 22: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/22.jpg)
Model predictions
• Behavioral: S-shaped choice probabilities
• Process: RTs and #saccades increase with choice difficulty
• Performance:- Importance of first fixation: P(choice=best|fist-fixation=best)> P(choice=best|fist-fixation=worse)- First look bias: for items with similar value P(choice=L|fist-fixation=L)> P(choice=L|fist-fixation=R)- …
![Page 23: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/23.jpg)
Test
Present until a choiceis made
Enforce2000 msfixation
+ ++
1000 ms
Collect eye-fixations @ 50 Hz
![Page 24: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/24.jpg)
Results
QuickTime™ and aTIFF (PackBits) decompressorare needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
![Page 25: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/25.jpg)
Summary
• Computation is at the core of the nascent field of Neuroeconomics
• Goal is to (1) describe the computation and processes that the brain uses to make decisions and (2) establish their neural basis
• Test the computational processes directly using modern neuroscience and psychology tools -- from fMRI to eye tracking
• Feasibility of the research agenda has already been proven
• Novel insights into DM are already being generated by this class of models.
![Page 26: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/26.jpg)
III
Models of Strategic Choice & Learning in Games
![Page 27: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/27.jpg)
Some theoretical interest in computational models
• Finite-state automata (Rubinstein, Neyman, et. al.)
• Computational complexity (Gilboa-Zemel on NP-hard games)
• Not linked to data or practical problems
![Page 28: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/28.jpg)
Cognitive hierarchy models of limited strategic thinking
• Selten (1998): – “The natural way of looking at game situations…is not based on
circular concepts, but rather on a step-by-step reasoning procedure”
• Cognitive hierarchy– “Level 0” use a heuristic (e.g. randomize)– “Level k” best-respond to choices of level 0-(k-1)– Axiom f(k)/f(k-1) 1/k (k-th step increasingly difficult)
f(k)=e/k! (Poisson)– Limit as often converges to equilibrium– Simpler than equilibrium in some ways
easier to compute predictions no problem of multiple equilibria
![Page 29: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/29.jpg)
Limited planning ahead in bargaining (Science, 03)
3-stage bargaining
1: $5 p1 offers
2: $2.50 p2 offers
3: $1.25 p1 offers
(0,0) if rejected
![Page 30: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/30.jpg)
E.g. “P-beauty contest” (Ho et al AER 98)pick x in [0,100], x closest to (2/3) of average
wins
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
5 15 25 35 45 55 65 75 85 95
relative frequency
equilibrium=0
data
CH prediction (_=1.5)
![Page 31: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/31.jpg)
“Choosing” computations are different than “belief formation” computations
Bhatt-Camerer GEB 2005
![Page 32: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/32.jpg)
Field application: “Cold opening” of movies (unavailable to critics for Friday review)
Studios do not let worst movies get reviewed…”cold” opening increases box office
![Page 33: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/33.jpg)
EWA learning in games: Generalized reinforcement
• Reinforcement, fictitious play linked (Econometrica 99)• Update attractions to strategy j from payoff
A ij (t) - A i
j (t-1) = [*π(s ij,s-i (t)) -A i
j (t-1)]/(ϕN(t-1)+1) = prediction error/increasing weight
is “imagination” of counterfactual payoffsϕ is recency weight
Typical values: N(0)=1, ϕ=.8, weights go from .56 .20
• Can replace , ϕ with “self-tuning” functions (JET ’07) • Can add “sophistication”– players know others are learning
(JET 02, GEB 06)
![Page 34: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/34.jpg)
Example: Price matching with loyalty rewards (Capra, Goeree, Gomez, Holt AER ‘99)
• Players 1, 2 pick prices [80,200] ¢
Price is P=min(P1,,P2)
Low price firm earns P+50
High price firm earns P-50• What happens?
– Theory: competition drives prices to 80
![Page 35: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/35.jpg)
1
35
79
80
81~9091~100101~110111~120121~130131~140141~150151~160161~170171~180181~190191~200
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Prob
Period
Strategy
Thinking fEWA1
35
79
80
81~9091~100101~110111~120121~130131~140141~150151~160161~170171~180181~190191~200
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Prob
Period
Strategy
Empirical Frequency
![Page 36: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/36.jpg)
IV
Computational Issues at different levels:individuals, firms, markets
![Page 37: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/37.jpg)
Levels of computational modelling in economics
• Individuals (what you’ve seen)• Firms
– Firms as hierarchies of imperfectly informed individuals (Radner-Van Zandt)Optimal hierarchies for aggregating formation
• Mechanism design– Computability as an individual rationality constraint (Ledyard)
• Markets– Markets as computational mechanisms
• Computing equilibria (Judd, Kearns et al)
– Smart markets: Hybrids of bids and optimal combination (e.g. combinatorial “package auctions” e.g. PCS spectrum)
– Information aggregation• Markets ‘compute’ probabilities of events (e.g. prediction markets)
![Page 38: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/38.jpg)
Prediction markets
• Began with basic research: 20 yrs to wide use • Plott and Sunder (1982 Econometrica): • Markets for “contingent claims”• Pay $1 if an event occurs. Prices reveal probabilities• Markets are $-weighted opinion polls of self-selected
respondents• Iowa Political Markets 1988 (http://www.biz.uiowa.edu/iem/)• Markets for political events predict surprisingly accurately• Tradesports 2002 (http://www.tradesports.com/) et al• Used by some companies, policy markets• See Wolfers & Zitzewitz J Econ Perspectives 04
![Page 39: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/39.jpg)
Six hours earlier (9pm EST Oct 26 ‘06): Guess about Karl Rove non-indictment appears in Intrade price drop…36 hrs before Oct 28 Libby indictment
BID
Qty Price
1 25.1
10 25.0
1 24.6
1 24.4
1 24.2
1 23.7
1 23.6
3 23.4
11 23.3
27 23.2
16 23.0
13 22.5
10 22.0
20 21.0
11 20.0
ROVE.INDICTED.31DEC
ASK
Price Qty
29.9 1
30.0 3
31.9 1
32.0 2
32.7 2
33.0 10
34.9 4
35.0 20
39.0 5
40.0 11
50.0 10
68.8 5
70.0 99
72.0 4
74.9 1
![Page 40: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/40.jpg)
Google news at 1:46am EST Oct 27: Will Karl Rove be indicted?
•Rove critics again turn up the volume New York Daily News - Oct 27 1:18 AMWith rampant rumors of a soon-to-drop indictment in Special Counsel Patrick Fitzgerald 's CIA leak investigation, the Karl Rove literary business is booming. •Rove's Last Campaign Washington Post - Oct 26 11:31 AMWill Karl Rove, architect of President Bush's improbable political career, snatch one last victory from the jaws of defeat? (Or at least avoid getting indicted?) Something appears to have provoked special prosecutor Patrick J. Fitzgerald into a last-minute flurry of activity centered............ •Leak Counsel Is Said to Press on Rove's Role New York Times - Oct 25 7:25 PMThree days before the grand jury is set to expire, Patrick Fitzgerald appeared to be trying to determine Karl Rove's role in the outing of a C.I.A.'s officer's identity. •Libby, Rove Await Indictment Decisions By Martin Sieff, UPI Senior News Analyst Washington DC (UPI) Oct 25, 2005 Space War - Oct 26 9:53 PMWashington seethed with rumors and speculation Tuesday night on the eve of the expected announcement of possible indictments in the Valerie Plame CIA leak probe.
![Page 41: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/41.jpg)
Current (3/15) “prices” of Scooter Libby pardon
Legal - I. Lewis Libby
Contract Bid Ask Last Vol Chge
I. Lewis (Scooter) Libby Pardon
LIBBY.DEC07.PARDON
I.Lewis Libby to be
pardoned by 31 Dec 2007
M 10.0 17.0 9.5 1012 0
LIBBY.EOT.PARDON
I.Lewis Libby to be
pardoned by the end of President
Bush's term in office
M 62.0 63.2 63.2 948 -0.2Mar 16 - 3:18AM GMT
![Page 42: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/42.jpg)
V
Future of computational models of decision in Economics
![Page 43: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/43.jpg)
@ the Individual level
• It will look like theoretical neuroscience• Focus on modeling the neural and psychological
processes involved in decision-making • Modeling constraints provided by neural, psychological,
and behavioral data• Models will be tested with techniques such:
- fMRI- electrophysiology- TMS- eyetracing- behavioral predictions
![Page 44: Computational Approaches to Economic Valuation & Strategy Choice](https://reader035.vdocuments.us/reader035/viewer/2022062721/56813868550346895da0186e/html5/thumbnails/44.jpg)
@ the firm and market levels
• Will build on the properties of the individual level models
• Model the interactions of many agents • Goal will be to improve our understanding of:
- auctions- price formation in markets- financial markets dynamics- macroeconomic performance and policy