cs 182 lecture 28: neuroeconomics j.g. makin april 27, 2006

CS 182Lecture 28: Neuroeconomics

J.G. Makin

April 27, 2006

Decisions, Uncertainty, and the BrainPaul Glimcher (2003); MIT Press

• Thesis: neuroscience has been dominated by the reflex paradigm

• Alternative: investigations rooted in economics, evolution, game theory, and probability

Reflex Theory

• Model: Input-Association-Output

– (think of trying to explain language this way)

– Even ANNs?

• Methodology: thoroughly constrain the environment

– “Isn’t this how science is done?”

– Obscures a system-level view

• Has this really led researchers astray?

• Why are there so many questions on this slide?

Glimcher 2003

Reflex Theory (con’t)

Challenges to “naïve” reflex theory

• T. Graham Brown and the Half-Center Oscillators [This is not the name of a band, as far as I know, though it should be]– Sherrington: stimulus for walking from

enteroceptive or interoceptive sources only

• Reafference and Efference Copy (Von Holst and Mittelstaedt)– [Glimcher actually has these confused]

Reflex Theory (con’t)

Challenges to “naïve” reflex theory

• T. Graham Brown and the Half-Center Oscillators [This is not the name of a band, as far as I know, though it should be]– Sherrington: stimulus for walking from

enteroceptive or interoceptive sources only

• Reafference and Efference Copy (Von Holst and Mittelstaedt)– [Glimcher actually has these confused]

An Alternative

Behavior is structured

– by goals (cf. shoulder reflex)

– by optimization strategies in the face of uncertainty

– Specification of the problem on the basis of function rather than implementation (Marr)

– In particular, the problem is an optimization problem

– Conclusion: Neuroscience needs probability theory, economics, evolutionary theory, and game theory

Reflex Theory (con’t)

What reflex theory doesn’t address

– the shoulder “reflex” (Paul Weiss)

– foraging

– mate selection

– exploratory behaviors

– Language & thought

An Alternative

Behavior is structured

– by goals (cf. shoulder reflex)

– by optimization strategies in the face of uncertainty

– Specification of the problem on the basis of function rather than implementation (Marr)

– In particular, the problem is an optimization problem

– Conclusion: Neuroscience needs probability theory, economics, evolutionary theory, and game theory

I: Optimization

• Q: Optimization with respect to what?

• A: Inclusive fitness but modularized. Evolution provides the goals, economics the optimization techniques

• Do we have a prayer at specifying the optimum?

– Phototransduction near the quantum limit

– Hair cells can detect individual fluid molecule collisions

– Convergent Evolution: Cichlid fish of Tanzania

II: Uncertainty: Epistemological

• Reflex theory dominated by deterministic responses to input (from a highly constrained set)

• Alternative: in general, we suffer from epistemological uncertainty, so we have to optimized in an indeterminate world

Uncertainty (con’t)

• An empirical test of foraging economics:

the prey model, Parus major

• View foraging as an optimization problem: choose the probability p_i of attacking the prey i that maximizes the rate at which energy is gained

• Solution:

– “zero-one” rule

– “independence from encounter inclusion rate” principle

Uncertainty (con’t)

• Frequencies of large and small mealworms were varied

• Small mealworms always had larger handling time

• Prediction (from optimal sol’n):

– Preference for large worms as their freq. increases, regardless of small worm freq. (by IEIR principle)

– If the bird couldn’t get all the worms, it should give up entirely on the small ones (by the zero-one rule)

• Result: yes and no (only 85% selective)

• Maybe this is an optimal strategy after all…

Epistemological Uncertainty & the Brain:A Series of Studies

• Input-association-output model: sensory-parietal-motor

• Lateral intraparietal area (LIP) and monkey saccades:

– Monkeys trained to perform task w/juice reward

– Invariant to input stimulus (light or button or whatever)

– Position-encoding

– Conclusion: command signal (Mountcastle)

Epistemological Uncertainty & the Brain (con’t)

• Lateral intraparietal area (LIP) and monkey saccades:

– Fixation and saccade tasks w/eccentric light

– Weak activation on fixation, but increasingly active over trials of saccade task

– Conclusion: attentional enhancement (Goldberg)

Epistemological Uncertainty & the Brain (con’t)

• Lateral intraparietal area (LIP) and monkey saccades:

– Memory saccade task: target is extinguished but LIP neuron still fires—until the motor command is executed

– Conclusion: motor intention (Gnadt & Anderson)

Platt & Glimcher: encoding the probability of pay-off

Epistemological Uncertainty & the Brain (con’t)

Epistemological Uncertainty & the Brain (con’t)

Probability experiment

Epistemological Uncertainty & the Brain (con’t)

Value experiment

III: “Irreducible” Uncertainty & Game Theory

• Static environment Dynamic competition with other agents

• Then the optimal approach is given by game- theoretic approaches

• In these cases, the optimum often involves (purposefully) random behavior

Uncertainty & Game Theory (con’t)

• Example 1: Chicken

Uncertainty & Game Theory (con’t)

• Conclusion: Smith is best served by behaving non-deterministically, but with probability 0.647 of being a chicken. (Ditto for Jones.)

• If Jones finds non-randomness in the distribution of Smith’s choices, he can predict above chance which option Smith will pick—and win.

• Random behavior is the optimal solution, so: we shouldn’t expect behavior to look deterministic (contrast w/reflex theory).

Intermezzo: How Random Are We?

• Paper, scissors, rocks

• Dice, viscera divination, etc.: technological breakthrough (Jaynes)

• Unconscious vs. conscious behaviors; natural selection vs. “rational actors”

• Pigeons, babies, and adults: the matching rule and cognitive load (and reward)

Game Theory and Ethology

• Duck foraging

– Two feeders at opposite ends

– 33 ducks

– Rate of food depends on feeder, but the more ducks in an area the worse it is

– Where to sit?

Game Theory & Ethology (con’t)

• Person 1: 2-gram bread ball every 5 sec

• Person 2: 2-gram bread ball every 10 sec

Game Theory & Ethology (con’t)

Game Theory & Human Behavior:Work or Shirk

Insp = -5

Game Theory & Human Behavior:Work or Shirk (con’t)

Insp = -50

• Experiment: subjects play against a computer program which looks for statistical regularities in its opponent’s plays and tries to exploit them

• Subjects are only told that they can make money by playing

• 150 trials, then the pay-off matrix switches (unannounced)

• Guess how human beings played….

Game Theory & Human Behavior:Work or Shirk (con’t)

• 150 trials, one pay-off matrix, vis-à-vis the Nash equilibrium?

Game Theory & Human Behavior:Work or Shirk (con’t)

Game Theory & Human Behavior:Work or Shirk (con’t)

• Work-shirk-work-shirk yields 50% behavior. Shannon entropy of choices?

Game Theory & Human Behavior:Work or Shirk (con’t)

Game Theory & Human Behavior:Work or Shirk (con’t)

• Switching between pay-off matrices?

Game Theory & Human Behavior:Work or Shirk (con’t)

Game Theory & Human Behavior:Work or Shirk (con’t)

Game Theory & the Brain

• Repeat the game, this time with monkeys instead of humans

• Simultaneously record from parietal area LIP

• Prediction: if these neurons encode expected utility, then they will fire at constant rates over various movements and various rewards (contrast Platt & Glimcher 1999)

• Now we have an experiment that yields non-deterministic behavior but about which predictions of lawful actions can nevertheless be made

Game Theory & the Brain (con’t)

• Across trials:

– Monkeys behave (near?) optimally: their behaviors track the Nash equilibrium

– LIP neurons do not track the Nash equilibrium suggesting that they are, in fact, encoding (relative) expected utility

• Play-by-play:

– The relative expected value on any given play does vary slightly, given the randomness of play

– Positive correlation b/n this fluctuating expected value and fluctuations in LIP neurons

Game Theory & the Brain (con’t)

Neuroeconomics & Language

• Skinner’s Verbal Behavior

• Programs that are more than input/output

• Bayes Nets for utility as well as beliefs

• Minimum description length: grammar

• Minimum description length: evolution

Neuroeconomics & Language

• “The paradox disappears only if we make a radical break with the idea that language always functions in one way, always serves the same purpose: to convey thoughts—which may be about houses, pains, good and evil, or anything else you please.” (Sec. 304)