1 decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult,...
Post on 22-Dec-2015
220 views
TRANSCRIPT
![Page 1: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/1.jpg)
1
Decision making in basketball• 2-point shot: easier, fewer points
• 3-point shot: more difficult, more points
Kobe BryantLA Lakers
31.6 PPG (2006-7)
3P attempts: 398 (23%)
2P attempts: 1,359 (77%)
Chris BoshToronto Raptors
26.3 PPG (2006-7)
35 (3%)
1,059 (97%)
3P success: 34%
2P success : 50%
34%
50%
![Page 2: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/2.jpg)
2
NBA best 100 players (2006-2007)
2
3
3
N
N N
3
2 3
I
I I
N2,3 = # of 2,3 points shots
I2,3 = # 2,3 points earned0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
Bryant
Bosh
The matching law
![Page 3: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/3.jpg)
3
The reward schedule
, 1 , 2 ,R t r A t A t A t
![Page 4: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/4.jpg)
4
Herrnstein, JEAB, 1961
1
11
1 2
I
I I
1
1 2
N
N N
The matching law
![Page 5: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/5.jpg)
5
Sugrue, Corrado & Newsome, Science, 2004
The matching law
![Page 6: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/6.jpg)
6
The matching law
Gallistel et al., unpublished
![Page 7: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/7.jpg)
7
Nj = # of attempts at alternative j investment in j
1 1
1 2 1 2
N I
N N I I
equal returns
Ij = # of points earned from alternative j income from j
The matching law
1 2
1 2
I I
N N
E 1 E 2R A R A
![Page 8: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/8.jpg)
8
MATCHING MAXIMIZING
The matching law is very general. It is found in many animal types as well as humans, under very different experimental conditions.
![Page 9: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/9.jpg)
9
freq [drugs]1–freq [work]
Example: addiction model
0 0.2 0.4 0.6 0.8 1
E[R|A=drugs]
after Herrnstein and Prelec, J Econ Perspect, 1991
![Page 10: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/10.jpg)
10
Example: addiction model
0 0.2 0.4 0.6 0.8 1
freq [drugs]1–freq [work]
E[R|A=drugs]E[R|A=work]
matching
after Herrnstein and Prelec, J Econ Perspect, 1991
![Page 11: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/11.jpg)
11
Example: addiction model
freq [drugs]1–freq [work]
0 0.2 0.4 0.6 0.8 1
E[R|A=drugs]E[R|A=work]E[R]
maximizing matching
after Herrnstein and Prelec, J Econ Perspect, 1991
![Page 12: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/12.jpg)
12
Question:What is the neural basis of the matching law?
![Page 13: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/13.jpg)
13
It is generally believed that learning is due to changes in the efficacy of synapses
Kennedy, Science, 2000
0.4 μm
![Page 14: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/14.jpg)
14
Question:What is the neural basis of the matching law?
Question: What microscopic plasticity rules underlie adaptation to matching behavior?
![Page 15: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/15.jpg)
15
Question:What is the neural basis of the matching law?
Hypothesis: the matching law results from synaptic plasticity that is driven by the covariance of reward and neural activity
![Page 16: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/16.jpg)
16
Question:What is the neural basis of the matching law?
Hypothesis: the matching law results from synaptic plasticity that is driven by the covariance of reward and neural activity
![Page 17: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/17.jpg)
17
• two random variables X, Y
Cov
Var Var
[ , ]
[ ] [ ]
X Yr
X Y
E[ ] E[ ]X X X Y Y Y
Covariance is a measure of dependence
X Y X Y
X Y
X Y
Cov , E[ ]
E[ ]
E[ ]
• correlation coefficient:
• covariance:
![Page 18: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/18.jpg)
18
Covariance
Cov[X ,Y ] 0
Cov[X ,Y ]0
Cov[X ,Y ] 0
![Page 19: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/19.jpg)
19
Hypothesis: the matching law results from synaptic plasticity that is driven by the covariance of reward and neural activity
![Page 20: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/20.jpg)
20
Synaptic plasticity• Local signals affect synaptic
efficacies. Popular theory: Hebbain plasticity
pre postW DS S
• Global signals affect synaptic efficacies. Popular theory: dopamine gates Hebbian plasticity (Wickens)
pre postW S S
![Page 21: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/21.jpg)
21Schultz, Dayan & Montague, Science, 1997
![Page 22: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/22.jpg)
22
Synaptic plasticity• Local signals affect synaptic
efficacies. Popular theory: Hebbain plasticity
pre postW DS S
• Global signals affect synaptic efficacies. Popular theory: dopamine gates Hebbian plasticity (Wickens)
ED R R R
• Popular theory: dopamine codes the mismatch between reward and expected reward (Schultz)
pre postW S S
![Page 23: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/23.jpg)
23
Synaptic plasticity
pre postW DS S
D R
pre postW R S S
E [ ] Cov[ , ]pre post pre postW R S S R S S E
Average trajectory approximation
![Page 24: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/24.jpg)
24
Covariance-based plasticity rules
W R R N
W R N N
E
E
E Cov[ , ]W R N
Average trajectory approximation:
N=Spre , N=Spost , N=SpreSpost
![Page 25: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/25.jpg)
25
The matching law
Stationary state of covariance-based plasticity
Hypothesis:
covariance-based synaptic plasticity The matching law
outline:
Cov , 0 R N
![Page 26: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/26.jpg)
26
Assumptions
1. E[N|A=i] ≠ E[N|A≠i]
2. The dependence of the reward R on neural activity N is through the action A.
N1
N2
N3
N4N5
A R
hidden variables
neurons
action reward
![Page 27: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/27.jpg)
27
Suppose that Assumptions 1 and 2 are satisfied
Cov , 0 E | 1 E | 2jR N R A R A The matching law
Theorem
![Page 28: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/28.jpg)
28
Intuition
A Raction reward
Nneuron
Cov , 0 E | ER N R A i R
• In general R depends on A • If, as a result of the policy used by the subject,
R becomes independent of A then R also becomes independent of N
![Page 29: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/29.jpg)
29
W R S S E
![Page 30: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/30.jpg)
30
W R S S E
W RS
![Page 31: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/31.jpg)
31
W R S S E
W RS
ij i jpre postW R R S M E
![Page 32: 1 Decision making in basketball 2-point shot: easier, fewer points 3-point shot: more difficult, more points Kobe Bryant LA Lakers 31.6 PPG (2006-7) 3P](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d7e5503460f94a60f1a/html5/thumbnails/32.jpg)
32
The matching law
Hypothesis:Covariance based synaptic plasticity underlies the matching law
Cov , 0 R N
Summary
Loewenstein & Seung, PNAS, 2006Loewenstein, PLoS Comp Biol, 2008
Theorem:
Disclaimer:There are learning rules that converge to Cov[R,N]=0 that are not driven by covariance