the computing brain: focus on decision-making cogs 1 march 5, 2009 angela yu [email protected]
TRANSCRIPT
![Page 2: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/2.jpg)
Understanding the Brain/Mind?
Behavior Neurobiology
CognitiveNeuroscience
A powerful analogy: the computing brain
≈
![Page 3: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/3.jpg)
An Example: Decision-Making
![Page 4: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/4.jpg)
An Example: Decision-MakingWhat are the computations involved?
2. UncertaintyTHUR
74/61
3. Costs:
Health? Coolness?
1. Decision
![Page 5: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/5.jpg)
Monkey Decision-Making
Random dots coherent motion paradigm
![Page 6: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/6.jpg)
Random Dot Coherent Motion Paradigm
30% coherence 5% coherence
QuickTime™ and aVideo decompressor
are needed to see this picture.
QuickTime™ and aVideo decompressor
are needed to see this picture.
![Page 7: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/7.jpg)
How Do Monkeys Behave?
Accuracy vs. Coherence <RT> vs. Coherence
(Roitman & Shadlen, 2002)
![Page 8: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/8.jpg)
What Are the Neurons Doing?Saccade generation system
(Britten, Shadlen, Newsome, & Movshon, 1993)
Preferred Anti-preferred
Time Time
Response
25.6%
Time Time
Response
99.9%
MT tuning function
Direction ()
(Britten & Newsome, 1998)
MT: Sustained response
![Page 9: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/9.jpg)
What Are the Neurons Doing?Saccade generation system
(Roitman & Shalden, 2002)
LIP: Ramping response
(Shadlen & Newsome, 1996)
![Page 10: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/10.jpg)
A Computational Description
1. Decision: Left or right? Stop now or continue?
(t): input(1), …, input(t) {left, right, wait}
![Page 11: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/11.jpg)
Timed Decision-Making
L R L R L R
wait wait
![Page 12: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/12.jpg)
A Computational Description
2. Uncertainty: Sensory noise (coherence)
![Page 13: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/13.jpg)
Timed Decision-Making
L R L R L R
wait wait
QuickTime™ and aVideo decompressor
are needed to see this picture.
QuickTime™ and aVideo decompressor
are needed to see this picture.
QuickTime™ and aVideo decompressor
are needed to see this picture.
![Page 14: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/14.jpg)
A Computational Description
3. Costs: Accuracy, speed
cost() = Pr(error) + c*(mean RT)
Optimal policy * minimizes the cost
![Page 15: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/15.jpg)
+: more accurate
-: more time
Timed Decision-Making
L R L R L R
wait wait
QuickTime™ and aVideo decompressor
are needed to see this picture.
QuickTime™ and aVideo decompressor
are needed to see this picture.
QuickTime™ and aVideo decompressor
are needed to see this picture.
![Page 16: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/16.jpg)
Mathematicians Solved the Problem for Us
Wald & Wolfowitz (1948):
Optimal policy:
• accumulate evidence over time: Pr(left) versus Pr(right)
• stop if total evidence exceeds “left” or “right” boundary
“left” boundary
“right” boundary
Pr(left)
![Page 17: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/17.jpg)
Model NeurobiologySaccade generation system
Model LIP Neural Response
![Page 18: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/18.jpg)
Model BehaviorSaccade generation system
Model RT vs. Coherence
boundary
Coherence
![Page 19: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/19.jpg)
Putting it All TogetherSaccade generation system
MT neurons signal motion direction and strength
LIP neurons accumulate information over time
LIP response reflect behavioral decision (0% coherence)
Monkeys behave optimally (maximize accuracy & speed)
![Page 20: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/20.jpg)
Understanding the Brain/Mind?
Behavior Neurobiology
CognitiveNeuroscience
A powerful analogy: the computing brain
≈
![Page 21: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/21.jpg)
![Page 22: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/22.jpg)
Sequential Probability Ratio Test
Log posterior ratio
undergoes a random walk:
rt is monotonically related to qt, so we have (a’,b’), a’<0, b’>0.
b’
a’
0rt
In continuous-time, a drift-diffusion process w/ absorbing boundaries.
![Page 23: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/23.jpg)
Generalize Loss Function
We assumed loss is linear in error and delay:
Wald also proved a dual statement:
Amongst all decision policies satisfying the criterion
SPRT (with some thresholds) minimizes the expected sample size <>.
This implies that the SPRT is optimal for all loss functions that
increase with inaccuracy and (linearly in) delay (proof by contradiction).
(Bogacz et al, 2006)
What if it’s non-linear, e.g. maximize reward rate:
![Page 24: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/24.jpg)
Neural ImplementationSaccade generation system
LIP - neural SPRT integrator? (Roitman & Shalden, 2002; Gold & Shadlen, 2004)
Time
LIP Response & Behavioral RT LIP Response & Coherence
![Page 25: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/25.jpg)
Caveat: Model Fit Imperfect
Accuracy Mean RT RT Distributions
(Data from Roitman & Shadlen, 2002; analysis from Ditterich, 2007)
![Page 26: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/26.jpg)
Fix 1: Variable Drift Rate(Data from Roitman & Shadlen, 2002; analysis from Ditterich, 2007)
(idea from Ratcliff & Rouder, 1998)
Accuracy Mean RT RT Distributions
![Page 27: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/27.jpg)
Fix 2: Increasing Drift Rate(Data from Roitman & Shadlen, 2002; analysis from Ditterich, 2007)
Accuracy Mean RT RT Distributions
![Page 28: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/28.jpg)
A Principled ApproachTrials aborted w/o response or reward if monkey breaks fixation
(Data from Roitman & Shadlen, 2002)
(Analysis from Ditterich, 2007)
• More “urgency” over time as risk of aborting trial increases
• Increasing gain equivalent to decreasing threshold
![Page 29: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/29.jpg)
Imposing a Stochastic Deadline(Frazier & Yu, 2007)
Loss function depends on error, delay, and whether deadline exceed
Optimal Policy is a sequence of monotonically decaying thresholds
Time
Probability
![Page 30: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/30.jpg)
Timing Uncertainty(Frazier & Yu, 2007)
Timing uncertainty lower thresholds
Time
Probability
Theory applies to a large class of deadline distributions:
delta, gamma, exponential, normal
![Page 31: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/31.jpg)
Some Intuitions for the Proof(Frazier & Yu, 2007)
ContinuationRegion
Shrinking!
![Page 32: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/32.jpg)
• Review of Bayesian DT: optimality depends on loss function
• Decision time complicates things: infinite repeated decisions
• Binary hypothesis testing: SPRT with fixed thresholds is optimal
• Behavioral & neural data suggestive, but …
• … imperfect fit. Variable/increasing drift rate both ad hoc
• Deadline (+ timing uncertainty) decaying thresholds
• Current/future work: generalize theory, test with experiments
Summary
![Page 33: The Computing Brain: Focus on Decision-Making Cogs 1 March 5, 2009 Angela Yu ajyu@ucsd.edu](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfca1a28abf838ca940c/html5/thumbnails/33.jpg)