week lecture 4
DESCRIPTION
LecutreTRANSCRIPT
-
Week lecture 4
Rescorla-Wagner Model; Neurobiology of Prediction Error
-
Surprisingness of the US
Robert Rescorla & Allan Wagner The model is a mathematical expression of surprise:
Learning will occur only when the subject is surprised - that is, when what happens is different from what the subject expected to happen
-
Blocking (Leon Kamin)
Group Phase 1 Phase 2 Test Result Blocking L-US L & T-US T no CR
Control L & T-US T CR
This experiment is important because it shows that:
1. Conditioning is not an automatic result of CS-US pairings
2. For conditioning to occur, the CS must be informative and the US surprising
-
V = associative strength between CS and US
Vmax = maximum associative strength
V = change in associative strength
on each conditioning trial
Measure of size of CR during CS-US conditioning trials
Reality
Theory
-
Quantification of surprisingness of the US
More surprise
Less surprise
V = Vmax - Vn Vn = strength of the association at the beginning of trial n
Vn = change is the strength of the association produced by trial n
-
Learning curves can differ in terms of: 1. Vmax
2. Rate of acquisition
Vn = (Vmax - Vn)
Vmax is determined by the magnitude of the US
and relate to the salience of the CS and the US, respectively. Their values are between 0 and 1.
Vn = (Vmax -Vn)
-
Rescorla-Wagner model: valuable predictions
This model precludes quantitative predictions but allows interesting qualitative predictions (increases, decreases, and more).
Vn = (Vmax -Vn)
ACQUISITION Assume = 0.3 and Vmax = 1
Trial Vn Vn = (Vmax - Vn) 1 0.00 V1 = 0.3 (1 - 0.00) = 0.30
2 0.00 + 0.30 V2 = 0.3 (1 - 0.30) = 0.21
3 0.00 + 0.30 + 0.21 V3 = 0.3 (1 - 0.51) = 0.15
4 0.00 + 0.30 + 0.21 + 0.15 V4 = 0.3 (1 - 0.66) = 0.10
-
Trial Vn Vn = (Vmax - Vn) 1 0.00 V1 = 0.3 (1 - 0.00) = 0.30
2 0.00 + 0.30 V2 = 0.3 (1 - 0.30) = 0.21
3 0.00 + 0.30 + 0.21 V3 = 0.3 (1 - 0.51) = 0.15
4 0.00 + 0.30 + 0.21 + 0.15 V4 = 0.3 (1 - 0.66) = 0.10
1 Ext 0.00 + 0.30 + 0.21 + 0.15 + 0.10 V5 = 0.3 (0 - 0.76) = - 0.22
Vmax = 0
CONDITIONED INHIBITORS have negative associative strength
EXTINCTION The weakening of a conditioned response when a CS is presented by itself
-
BLOCKING
When two CSs are used (a & b), the association or expectation at the beginning of a trial would be the sum of the strengths of each of the stimuli present
Vab = Va + Vb
Therefore, the amount of conditioning on a compound trial in which a and b occur together would be
Va = Vb = (Vmax - Vab)
In the blocking group, if the Vmax for the light (L) = 1.0, then:
VL = 1.0 at the end of Phase 1 (because of extensive L conditioning)
when the light and the tone (T) are presented in combination on trial 1 of Phase 2
VLT = VL + VT = 1.0 + 0 = 1.0
Therefore, the amount of conditioning to the T in the blocking group after 1 trial of conditioning with the LT compound is:
VT = (Vmax - VLT) = 0.3 (1.0 - 1.0) = 0
-
Tone ------>shock Light ------>shock
Tone + Light-------> shock
Tone ?
Assume that only few trials were given before the compound trial, and that Vmax = 1 and = 0.3
VT = 0.2 and VL = 0.2, thus VTL = 0.4
VT = VL = 0.3 (1.0 - 0.4) = 0.18
The model predicts an increase in associative strength for both T and L when presented during the compound trial
But, if there was extensive conditioning before the compound trial such that:
VT = 0.9 and VL = 0.9, thus VTL = 1.8
VT = VL = 0.3 (1.0 - 1.8) = - 0.24
Therefore, the model predicts a decrease associative strength for both T and L when presented during the compound trial
OVEREXPECTATION EFFECT
-
Rescorla (1970) - Extensively trained rats
Tone ---------->shock
Light ---------->shock
Experimental group (E) Tone + Light----------> shock
Control group (C) Nothing
-
No conditioning to the CS does not mean no conditioning at all
Contextual stimuli
Context
-
Trial 1 CS + Context ------> US = + associative strength to compound Trial 2 Context alone ------>US = + associative strength to context
Trial 20 CS + Context ------> US = - associative strength to compound
Trial 21 Context alone ------> US = + associative strength to context
When the US is not contingent to the CS, conditioning will be strong to contextual (background) stimuli but not to the CS
= context
-
Some problems with the Rescorla-Wagner model
1. Exclusive focus on the surprisingness of the US Nicholas Mackintosh John Pearce & Geoffrey Hall
The Mackintosh Model The Pearce-Hall Model
It is important to consider how the salience of the CS () changes
during conditioning
2. The conclusion that extinction destroys the original learning
2. Spontaneous Recovery: the reappearance of a CR to a CS after a period of time following the last extinction trial
Renewal: the reappearance of a CR to a CS due to return to the training environment, instead of the environment used during extinction Reinstatement: the reappearance of a CR to a CS due to a brief presentation of the US
Rapid Reacquisition: rapid return of a CR to a previously extinguished CS
-
Surprise and Prediction Error
US not predicted and therefore very surprising
US somehow predicted and therefore less surprising
US more predicted and therefore much less surprising
V = Prediction Error
-
Neurobiology of Prediction Error
DOPAMINE A neurotransmitter involved in learning, motivation and a variety of psychobiological functions
Agonist: Cocaine, Amphetamine, Methylphenidate
Antagonists: Chlorpromazine, Haloperidol (Anti-Psychotic drugs)
-
US = Drops of juice UR = Lick
CSs
DA neurons VTA
Wolfram Schultz
-
Prior to CS-US conditioning
After CS-US conditioning
During extinction
Dopaminergic neurons in the VTA encode a Prediction Error
-
CS
Auto-shaping
US
-
NAC = nucleus accumbens
-
Homeostatic hypothesis of learning
Optimal level
Disturbance Actual level
Receptors
Activation of learning systems in the brain
Past knowledge
New event
Current knowledge
DA in VTA Error Signal
Response system
-
DA neurons VTA Prediction Error Signal
Learning
Learning
Learning