The basic design:CS ---------> US-------> URbell food salivation

\ | \ | ----> CR: salivation

• important variables:– CS = conditioned stimulus: arbitrary stimulus that does not

automatically evoke the response– UCS or US = unconditioned stimulus: – nonarbitrary stimulus that does automatically evoke the response– UCR or UR = unconditioned response: the response that is automatically

evoked by the US– CR = conditioned response: response that the CR evokes (what learned):

May or may not be identical to UR

• Crucial aspect for learning: Pairing of CS and US predicts an event

Important (critical) things to note about classical conditioning:

• the CS MUST precede the US

• the CS MUST predict the US

• if the CS does not predict the US, no conditioning occurs• the CR does not have to be identical to the UR

– E.g., subtle differences even Pavlov noticed)– may even be opposite: Morphine studies

• Any response is a classically conditioned response if it occurs to a CS after that CS has been paired with a US but does NOT occur to a randomly presented CS-US pairing

Rescorla: 6 types of control groups• CS-alone

– present CS alone with no US pairing– problem: not have same number of US trials as experimental animals do, may actually be extinction effect

• Novel CS group: – looks at whether stimulus is truly "neutral"– may produce habituation- animal doesn't respond because it "gets used to it"

• US-alone – present US aloine with no CS pairing– problem: not have same number of CS trials

• explicitly unpaired control– CS NEVER predicts US– that is- presence of CS is really CS-, predicts NO US– animal learns new rule: if CS, then no US

• Backward conditioning:– US precedes CS– assumes temporal order is important (but not able to explain why)– again, animal learns that CS predicts no US

• Discrimination conditioning (CS+ vs CS-)– use one CS as a plus; one CS as a minus– same problem as explicitly unpaired and backward- works, but

What is it that's important about CC? Rescorla's ideas

• CS-US correlation vs contiguity:– Typically in conditioning arrangements- CS always

followed by the US in a perfect correlation– p(US|CS) = 1.0– p(US|no CS) = 0.0

• but: life not always a perfect correlation

• Problem: how to prove this beyond a reasonable doubt- MUST use truly random control– Must be absolutely no prediction– CS does not either predict or not predict US

Probabilities interact to determine size of CS

• CS = 2 min tone; presented at random intervals (X = 8 minutes)– E.g., for: Group 1:

• p(shock|CS) = 0.4 during 2 min presentation • p(shock|no CS) = 0.2• only information that CS provided = whether probability of shock was high or low

– used 10 groups of rats, all with different values of p(US|CS) and p(US|no US)

• whenever p(US|CS) > p(US|NO CS):– TONE = EXCITATORY CS– that is, response suppression occurred (CER)

• amount of suppression depended on size difference between p(US|CS) and p(US|no CS) and vice versa

• Most predictive stimulus was what attended to

Relation of CS to US

• appears to be the CORRELATION between the CS and US, not the contiguity (closeness in time) that is important

• that is:– correlation (r) carries more information– if r = + then excitatory CS– if r = - then inhibitory CS– if r = 0 then neutral CS (not really even a CS)

Blocking and overshadowing

• Overshadowing:– use one "weak" and one "strong" CS– reaction to weaker stimulus is blotted out by

stronger CS

• Blocking:– One stimulus “blocks” learning to second CS

Kamin’s investigations

• Wanted to study role of attention in classical conditioning

• Usual set up: neutral stimulus becomes CS predictive of a US

• Note: used CER • Wanted to know about “nonneutral” stimuli

– Compound stimuli– Stimuli with a history

How measure in classical conditioning?

• look at change in an operant behavior as a result of a CS-US pairing– teach the rat to bar press for food– shock rat- rat naturally freezes– incompatible response- can't bar press and freeze at same time–  

• suppression ratio:– baseline of A– intro CS condition B

• suppression ratio = B/A+B– no effect = 0.5– complete suppression = 0.0– (disinhibition = 1.0 or oops!)

Kamin’s blocking experiment

• used multiple CS's and 4 groups of rats • the blocking group receives

– series of L+ trials which produce strong CR– series of LT+ trials– then tested to just the T

• control group receives– SAME TOTAL NUMBER OF TRIALS AS BLOCKING

GROUP– no first phase– LT+ in phase 2 (totaling phase 1 and 2 above)

Data are “surprising”!• .prediction:

– since both received same # of trials to the tone- – should get equal conditioning to the tone

• results quite different: – Blocking group shows no CR to the tone- – the prior conditioning to the light "blocked" any more conditioning to the tone

• directly contradicts frequency principle

• Group Phase I Phase II Test Phase Result• Group A LN N Test L L elicits small CR .25• Group B N LN Test L L elicits no CR .45• Group C -- LN Test L L elicits CR .05• Group B2 N ---- Test L L elicits no CR .45

Second experiment:

1st training 2nd 3rd• Group Y: N (16x w/Sr) LN (no sr) (8) N (non sr)• Group Z: N (16x w/Sr) N (no sr) (12)

• Result: – For first 16 trials: identical treatment: 0.02 on average– Group Y: presented with compound, ratio increased to 0.41– Group Z: presented with noise only, ratio = 0.33 (EXT)– Goup Y: noise only slight decrease to about 0.35

• Conclusion: superimposed element provided NEW information – not only notice cue– respond to cue because it carries info!

Things we know about blocking• the animal does "detect" the stimulus:

– EXT of CER with either N alone or with NL is slower thanEXT for compound NL

 • appears to be independent of:

– length of CS– number of trials of conditioning to compound CS

• influenced by:– use of CER measure (not the best)– nature of CS may be important- e.g. modality– intensity of stimuli important– depends on amount of conditioning to blocking stimulus which already occurred

• constancy of US from phase 1 to 2 important. • change in either US or CS can prevent/overcome blocking

– change the intensity of the CS from one situation to another– this is why spent so much time on overshadowing-

• strong vs weak stimulus• is change intensity of the stimulus- presents a different learning situation and no blocking

• same is true if change the intensity of US – (although generally must be stronger, not weaker)– e.g. experiments when changed from 1 ma to 4 ma shock– quickly condition to compound stimulus– little or no overshadowing or blocking

Theoretical Explanations?• Perceptual gating theory:

– tone never gets processed– tone not informative– data not really support this

 • Kamin's Surprise theory:

– to condition requires some mental work on part of animal– animal only does mental work when surprised– bio genetic: prevents having to carry around excess mental baggage

• thus only learn with "surprise"• situation must be different from original learning situation

• Alternative explanation: Rescorla Wagner model:– particular US only supports a certain amount of conditioning– if one CS hogs all that conditioning- none is left over for another CS to be added– question- how do we show this?

Assumptions of R-W model• helpful for the animal to know 2 things about conditioning:

– what TYPE of event is coming– the SIZE of the upcoming event

• Thus, classical conditioning is really learning about:– signals (CS's) which are PREDICTORS for– important events (US's)

• model assumes that with each CS-US pairing 1 of 3 things can happen:– the CS might become more INHIBITORY– the CS might become more EXCITATORY– there is no change in the CS

• how do these 3 rules work? – if US is larger than expected: CS = excitatory– if US is smaller than expected: CS= inhibitory– if US = expectations: No change in CS

• The effect of reinforcers or nonreinforcers on the change of associative strength depends upon:

– the existing associative strength of THAT CS– AND on the associative strength of other stimuli concurrently present

More assumptions

• Explanation of how an animal anticipates what type of CS is coming:– direct link is assumed between "CS center" and "US center": e.g.

between a tone center and food center– assumes that STRENGTH of an event is given and that the conditioning

situation is predicted by the strength of this connection– THUS: when learning is complete: the strength of the association

relates directly to the size or intensity of the CS

• The change in associative strength of a CS as the result of any given trial can be predicted from the composite strength resulting from all stimuli presented on that trial:– if composite strength is low, the ability of reinforcer to produce

increments in the strength of component stimuli is HIGH– if the composite strength is low; reinforcement is relatively less effective

(LOW)

More assumptions:

• Can expand to extinction, or nonreinforced trials:

– if composite associative strength of a stimulus compound is high, then the degree to which a nonreinforced presentation will produce a decrease in associative strength of the components is LARGE

– if composite associative strength is low- nonreinforcement effects reduced

• Yields an equation: Vi =αißj(Λj-VAX) • Here is an easier way to write it:

VT =αißj(Λj-Vsum)

First example:

• rat is subjected to conditioned suppression procedure: – CS (light) ---> US (1 mA shock)– what is associative strength?– 1 = associative strength that a 1mA shock can support at

asymptote ( Λj )

– VL = associative strength of the light (strength of the CS-US association)

• thus: Λ1 = size of the observed event (actual shock)

• VL = measure of the Subjects current "expectation" about the size of the shock

• VL will approach Λ1 over course of conditioning

Second example: Same rat, same procedure but 2CS's:

• CS (light+tone) --> 1 mA shock – Determine associative strength when Λ1 is constant– Vsum = VL + VT = assoc. strength of the 2 CS's– Vsum = αißj(Λ) – if VL and VT equally salient:

• VL = 0.5αißj;

• VT = 0.5αißj

– VT = if not equally salient: VL > VT or VL < VT

• now can restate the 3 rules of conditioning:– Λj > Vsum = excitatory conditioning– Λj < Vsum = inhibitory conditioning– Λj = Vsum = no change

Now have the Rescorla-Wagner Model:

• Model makes predictions on a trial by trial basis

• For each trial: predicts increase or decrement in associative strength for every CS present

• The equation: Vi =αißj(Λj -Vsum)– Vi = change in associative strength that occurs for any CS, i, on a single trial– Λj= associative strength that some US, j, can support at– asymptote– Vsum = associative strength of the sum of the CS's (strength of– CS-US pairing)– αi = measure of salience of the CS (must have value between 0– and 1)– ßj = learning rate parameters associated with the US (assumes– that different beta values may depend upon the particular US employed)

Assumptions of the formal model:

• General Principle: as Va increases with repeated reinforcement of j, the difference between Λa and Va decreases

– increments of Va then decrease– produce negatively accelerated learning curve with asymptote of Λ j

• Reinforcement of compound stimuli: lots of Va trials, then give trials of compound Vax

– Va increases toward Λa as a result of a-alone presentations– Vax then exceeds Λa– result: reinforced aX trial results in DECREMENT to the associative strength

of a and X components

• as a and aX are reinforced:– increments to A occur on the reinforced A trials– increments to A and X occur on reinforced AX trials– result: transfer to A of whatever associative strength X may have

The equation: Vi =αißj(j-Vsum)

• Vi = change in associative strength that occurs for any CS, i, on a single trial

• αi = stimulus salience (assumes that different stimuli may acquire associative strength at different rates, despite equal reinforcement)

• ßj = learning rate parameters associated with the US (assumes that different beta values may depend upon the particular US employed)

• Vsum = associative strength of the sum of the CS's (strength of CS-US pairing)

• Λj= associative strength that some CS, i, can support at asymptote

• In English: How much you learn on a given trial is a function of the value of the stimulus x value of the reinforcer x (the absolute amount you can learn minus the amount you have already learned).

Acquisition• first conditioning trial: CS = light; US= 1 ma Shock

– Vsum = Vl; no trials so Vl = 0– thus: Λj-Vsum = 100-0 = 100 – -first trial must be EXCITATORY

• BUT: must consider the salience of the light: αi = 1.0 and learning rate: ßj = 0.5

• Plug into the equatio: for TRIAL 1– Vl = (1.0)(0.)(100-0) = 0.5(100) = 50– thus: V only equals 50% of the discrepancy between Aj an Vsum for the first trial

• TRIAL 2:– V1 = (1.0)(0.5)(100-50) = 0.5(50) = 25– Vsum = (50+25) = 75

• TRIAL 3:– V1 = (1.0)(0.5)(100-75) = 0.5(25) = 12.5– Vsum = (50+25+12.5) = 87.5

• TRIAL 4:– V1 = (1.0)(0.5)(100-87.5) = 0.5(12.5) = 6.25– Vsum = (50+25+12.5+6.25) = 93.75

• TRIAL 10: Vsum = 99.81, etc., until reach 100 on approx. trial 14

• When will you reach asymptote?

Overshadowing• Pavlov: compound CS with 1 intense CS, 1 weak

– after a number of trials found: strong CS elicits strong CR– weak CS elicits weak or no CR

• Rescorla-Wagner model helps to explain why: assume– αL = light = 0.2; αT = tone = 0.5– ßL = light = 1.0 ; ßt = tone = 1.0

• Plug into equation:– Vsum = Vl + Vt = 0 on trial 1– Vl = 0.2(1)(100-0) = 20– Vt = 0.5(1)(100-0) = 50– after trial 1: Vsum = 70

• TRIAL 2:– Vl = 0.2(1)(100-(50+20)) = 6– Vt = 0.5(1)(100-(50+20)) = 15– Vsum = (70+(6+15)) = 91

• TRIAL 3:– Vl = 0.2(1)(100-(91)) = 1.8– Vt = 0.5(1)(100-(91)) = 4.5– Vsum = (91+(1.8+4.5)) = 97.3 and so on– thus: reaches asymptote (by trial 6) MUCH faster w/2 CS's

• NOTE: CSt takes up over 70 units of assoc. strength CSl takes up only 30 units of assoc. strength

Blocking• similar explanation to overshadowing:

– no matter whether VL more or less salient than Vt, because CS has basically absorbed all the assoc. strength that the CS can support

• give trials of A-alone to asymptote: – reach asymptote: VL = Λj =100 =Vsum– αL =1.0– ß =0.2– First Vt Trial: Vt= αß(Λj-Vsum)

• Vt=0.2*1.0*(100-100)=?

• No learning!

How could one eliminate blocking effect?

• increase the intensity of the US to 2 mA with Λj now equals = 160– then: Vsum still equals 100 (learned to 1 mA shock)

• plug into the equation: (assume Vl and Vt equally salient)– Vt = 0.2(1)(160-100) = 0.2(60) = 12– Vl = 0.2(1)(160-100) = 0.2(60) = 12

• on trial 2:– Vsum = 124– Vt = 0.2(1)(160-124) = 0.2(36) = 7.2– Vl = 0.2(1)(160-124) = 0.2(36) = 7.2– Vsum now = (124+14.4) = 138.

• could also play around with ß

Critique of the Rescorla-Wagner Model:• R-W model really a theory about the US effectiveness:

– says nothing about CS effectiveness– states that an unpredicted US is effective in promoting learning, whereas a well-predicted

US is ineffective

• Fails to predict the CS-pre-exposure effect:– two groups of subjects (probably rats)– Grp I CS-US pairings Control– Grp II CS alone CS-US pairings PRE-Expos

• pre-exposure group shows much less rapid conditioning than the control group

• R-W model doesn't predict any difference, because no conditioning trials occur when CS is predicted alone: Vsum = 0

– BUT: may be that salience for the CS is changing:– habituation to CS

• Original R-W model implies that salience is fixed for any given CS– R-W assume CS salience doesn't change w/experience– these data strongly suggest CS salience DOES change w/experience

• Newer data supports changes salience– data suggest that Si DECREASES when CS is repeatedly presented without consequence– NOW: appears that CS and US effectiveness are both highly important

• Model has stood test of time, now widely used in neuroscience

Can deal with variety of other issues

• Compound CSs:– When two CSs are conditioned together– How much conditioning occurs to one or other

depends on previous exposure and salience of each stimulus.

• Time alone as CS– Time can serve as a CS; as long as it is

predictive!

• Difference between CS and no CS

Can also explain why probability of reward given CS vs no CS makes a difference:

• π = probability of US given the CS or No US given No CS

• can make up three rules:– if πax > πa then Vx should be POSITIVE– if πax < πa then Vx should be NEGATIVE– if πax = πa then Vx should be ZERO

• modified formula: (assume Λ1 =1.0; Λ2 =0; ß1 =.10; ß2=.05; α1=.10; α2=.5)

Va = πaß1 ---------------------- πaß1 - (1-πa)ß2

Vax = πaxß1 ----------------------

πaxß1 - (1-πax)ß2

Vx = Vax - Va

PLUG IN: Probability of CSa then US = 0.2;Probability of CSax then US = 0.8

Va = (0.2)(1.0) --------------------------- = -10 ((.2)(.10)) - (1-.2)(.05)

Vax = (0.8)(1.0) --------------------------- = +11.43 ((.8)(.10)) - (1-.8)(.05)

Vx = Vax - Va or 11.43-(-10) = 21.43

probability of US given AX greater than probability of US given X)

PLUG IN: Probability of CSa then US = 0.8; Probability of CSax then US =0.2

Va = (0.8)(1.0)--------------------------- = 11.43

((.8)(.10)) - (1-.8)(.05)

Vax = (0.2)(1.0) --------------------------- = -10 ((.2)(.10)) - (1-.2)(.05)

Vx = Vax - Va or -10 - 11.43 = -21.43

probability of US given AX is less than probability of US given A

PLUG IN: Probability of CSa then US = 0.5 Probability of CSax then US = 0.5

Va = (0.5)(1.0) --------------------------- = 20 ((.5)(.10)) - (1-.5)(.05)

Vax = (0.5)(1.0) --------------------------- = 20 ((.5)(.10)) - (1-.5)(.05)

Vx = Vax - Va or 20-20 = 0 (probability of AX = A)