conditional learning: switching associations. are there any types of learning that associative...

Conditional learning:

Switching associations

Are there any types of learning that associative theory cannot explain?

"Red Light"

"Red Light"

"Red Light"

sometimes what the CS is associated with depends on the context...

e.g. lexical ambiguity task:

ambiguous word e.g. bank

subject must define based on semantic context

- country walk? going shopping?

e.g. Continuous performance test (CPT)

A x A y B y B y B x A y B y A y A x B x B y

e.g. Continuous performance test (CPT)

A x A y B y B y B x A y B y A y A x B x B y

must respond to x if preceded by A (not B)

i.e. A: x+, y- B: x-

e.g. Continuous performance test (CPT)

A x A y B y B y B x A y B y A y A x B x B y

must respond to x if preceded by A (not B)

must respond to y if preceded by B (not A)

i.e. A: x+, y- B: x-, y+

e.g. Stroop!

e.g. Stroop!

redyellow

green

Patients with schizophrenia have been reported to have difficulty on such tasks:

Lexical ambiguity e.g., Cohen et al, 1988

Continuous performance test e.g., Cornblatt et al, 1989

Stroop e.g., Wysocki & Sweet, 1985

Patients with schizophrenia have been reported to have difficulty on such tasks:

Lexical ambiguity e.g., Cohen et al, 1988

Continuous performance test e.g., Cornblatt et al, 1989

Stroop e.g., Wysocki & Sweet, 1985

... and in rats performance on such tasks disrupted by amphetamine

-- restored by D1/D2 receptor antagonists

e.g. Dunn et al., 2005 Instrumental biconditional task:

Tone: Left lever --> food Right lever --> no food

Click: Left lever --> no food Right lever --> food

0

5

10

15

20

25

30

35

1

Mean Lever presses per trial

CorrectIncorrectSeries3Series4Series5Series6Series7Series8Series9Series10Series11

control 0.5 1.0 1.5

dose of amphetamine

So how are these tasks performed? They cannot be the result of simple associations

Tone: Left lever --> food Right lever --> no food

Click: Left lever --> no food Right lever --> food

Tone and click paired with food

Left and Right paired with food

One interpretation is that they are a type of conditional learning

Conditional cues are interesting because they are not just Pavlovian CSs...

- occasion setters do not extinguish

- get better occasion setting when occasion setter is not a good predictor of the US

- Simultaneous: light+tone-food tone-- Serial: light……tone-food tone-

so what are they? beyond the association...

Outline of lecture:

Consider how conditional cues work

Consider how conditional cues form

Look at what conditional cues can do -- example of therapeutic implications

Illustrate with experiment with pigeons (Bonardi 1996)

Click: red-->food red -->nothing

Flash: green -->food green -->nothing

Birds trained with two occasion setters - 10-sec presentations of diffuse cues - followed by 5-sec presentation of a keylight

Click redfood

red

Question 1:

How do they work?

red food

Click

?

red food

Click

Rescorla’s modulation theory (Rescorla, 1985)

Whenever a CS is presented, it must activate the US representation to get a conditioned response.

If the click is a positive occasion setter, it lowers its activation threshold -- making it easier for the CS to activate.

red food

Click

Holland’s and-gate theory (1983)

The click acts as an and-gate, allowing activation to flow from the CS to the US, and so elicit a conditioned response.

Click: red-->food red -->nothing

Flash: green -->food green -->nothing

These theories make differentpredictions about transfer

- will the clicker elevate responding to the green keylight?

red food

green

These theories make differentpredictions about transfer

- will the clicker elevate responding to the green keylight?

Hollandsays no

Rescorlasays yes

Click

e.g. (Bonardi 1996)

Click: red-->food red -->nothing

Flash: green -->food green -->nothing

Test:

Same: Click: Red Flash : Green

Diff: Click: Green Flash : Red

0

10

20

30

40

50

60

70

80

1

Responses per minute

SameDiffTarget alone

Click does elevate responding to green keylight

......so is Rescorla right?

but this transfer could be generalisation - red/green confusion

.... so transfer not conclusive evidence for Rescorla

0

10

20

30

40

50

60

70

80

1

Responses per minute

SameDiffTarget alone

Plus Rescorla predicts click should be equally good with red and green - and it isn't...

So can Holland's theory do better?

0

10

20

30

40

50

60

70

80

1

Responses per minute

SameDiffTarget alone

Holland predicts incomplete transfer - so is he right?

-- or is this generalisation decrement?

Click: red-->food red -->nothingFlash: green -->food green -->nothing

Test:

Same: Click: Red Flash : GreenDiff: Click: Green!!! Flash : Red!!!

Novel stimulus combination disrupts responding on Diff trials????

(Bonardi 1996) Tests this possibility:

Group OS - Click and Flash are occasion setters

Click: red-->food red -->nothingFlash: green -->food green -->nothing

Test:

Same: Click: Red Flash : GreenDiff: Click: Green Flash : Red

Expect more responding on same than different

(Bonardi 1996) Tests this possibility:

Group pseudo OS - Click and Flash are NOT occasion setters

Click: red-->food red -->foodFlash: green -->food green -->food

Test:

Same: Click: Red Flash : GreenDiff: Click: Green Flash : Red

If previous result generalisation decrement, predict same hereIf previous result occasion setting, do not...

Group OS Group POS

0

10

20

30

40

50

60

70

80

90

1

Responses per minute

SameDiffTarget aloneSeries4Series5Series6Series7

So it's not generalisation decrement...

can you think of a reason why you might get more respondingon same than on different trials? Wagner is a clue....

So occasion setter is specific to the particular CS...

(and Rescorla himself has generated evidence against his theorye.g. Rescorla, 1991a; 1991b) - so probably Holland wins...

But how about the US? is click more effective with CSs pairedwith the same US as red?

red food

Click

So occasion setter is specific to the particular CS...

(and Rescorla himself has generated evidence against his theorye.g. Rescorla, 1991a; 1991b) - so probably Holland wins...

But how about the US? is click more effective with CSs pairedwith the same US as red?

red food

Click

green different food

?

Morell & Davidson, 2002

Light: tone -->suc Light- tone-

Group Same US click -->sucorGroup Diff US click -->oil

Test: Light tone Same CS Same US

Light click (group Same) Diff CS Same US

Light click (group Diff) Diff CS Diff US

0

10

20

30

40

50

60

1

Responses per minute

CompoundFeatureTargetSeries4Series5Series6Series7Series8Series9Series10Series11

Same CS Diff CS Diff CSSame US Same US Diff US

So if you change the US get dramatically reduced effect

So occasion setter is specific to the particular US as well...

Conclude: evidence supports Holland - acts on association

- do get transfer, based on CS-CS and US-US generalisation

- if change anything in association, transfer attenuated

red food

Click

green different food

Question 2:

How do they form?

If this is not an association, then where does it come from?

red food

Click

Where do associations come from? Associative learning explained by Rescorla-Wagner model (among others)

Shows certain characteristics - e.g. blocking:

A-->food AX --->food X? learning about X poor

red food

Click

Does occasion setting show blocking?

Biographie de Al CAPONE :

Ni McGurn, ni Capone ne pensèrent un seul moment que l'assassinat planifié de Bugs Moran serait un événement qui deviendrait notoire pour plusieurs

dizaines d'années. Capone se prélassant en Floride, comment pouvait-on le rendre responsable du meurtre d'un contrebandier ? «Machine Gun» McGurn avait le plein contrôle de l'attaque. Il rassembla une équipe de première classe composée gens de l'extérieur : Fred «Killer» Burke en était le chef et était assisté par un tirreur du nom de James Ray. Deux autres membres étaient John scalise et Albert Anselmi, qui avaient été

utilisés pour le meurtre de Frankie Yale. Joseph Lolordo faisait aussi partie du groupe, tout comme

Harry et Phil Keywell, du Purple Gang de Détroit. Le plan de McGurn était créatif. Un contrebandier invita les membres du gang de Moran à le rencontrer dans un garage afin de leur offrir du whisky de qualité à un prix imbattable. La livraison allait être faite à 10h30, le

14 février. Les hommes de McGurn allaient les attendre vêtus d'uniformes de police et d'imperméables,

donnant l'impression qu'un raid allait se dérouler.

Blocking of occasion setting (Bonardi 1991)

Group Exp

Click: noise-->food noise -->nothing

Rats: 3-min presentation of a click, with embedded reinforced 5-s noise presentations. Outside click noise nonreinforced

Click

noise

+ + + + +

Blocking of occasion setting (Bonardi 1991)

Group Exp

Click: noise-->food noise -->nothing

Group Con

Click: noise / food noise -->nothing

Click

noise

+ + + + +

Blocking of occasion setting (Bonardi 1991)

Group Exp

Click: noise-->food noise -->nothing

Group Con

Click: noise / food noise -->nothing

then allClick+Light: noise-->food noise -->nothing

test all Light: noise ?? noise??

0

2

4

6

8

10

12

14

Responses per minute

noise in ITI

noise infeatureSeries3

Series4

Series5

so blocking of occasion settingoccurred - learningabout L blocked only when Clickan occasionsetter

Group E Group C

And just in case you weren't convinced...

similar experiments have been performed to demonstrate the parallel effect with Sd's

Click

response

+ + + + +

Colwill & Rescorla, 1990

again looks at blocking, but relies on principle of unblocking

mismatch between outcomes in two stages attenuates blocking

e.g. Dickinson Hall and Mackintosh, 1976

A-->sh AX --->sh X? learning poor

A-->sh AX --->sh-sh X? learning restored

A-->sh-sh AX --->sh-sh X? learning poor

A-->sh-sh AX --->sh X? learning restored

Colwill & Rescorla, 1990

noise (Sd) lever

chain

sucrose

foodnoise (Sd)

Trained two further Sd's in compound with the noise:

Stage 2:

Same

Different

noise (Sd) lever

chain

sucrose

foodnoise (Sd)

lever

chain

sucrose

food

lever

chain sucrose

food

noise (Sd)

noise (Sd)

light (Sd)

flash (Sd)

will the Noise block the Flash or the Light ??

Stage 2:

Same

Different

noise (Sd) lever

chain

sucrose

foodnoise (Sd)

lever

chain

sucrose

food

lever

chain sucrose

food

noise (Sd)

noise (Sd)

light (Sd)

flash (Sd)

animals learn the same thing about the noise in Stage 1 as they do about about the light in Stage 2

Same ---> blocking

but animals learn different things about the noise in Stage 1 and about the flash in Stage 2

Different--> unblocking

therefore predict more control by the flash than by the light

Flash

Light

this means occasion setters show blocking

it also demonstrates forcibly that occasion setters convey

information about CS/US combinations

-- if combination changed, learning differs...

this is all consistent with Holland's theory

What does all this mean?

One suggestion -- Hierarchical account: CS-->US association can act as independent associable unit, and occasion setter becomes associated with it (e.g. Bonardi, 1998)

associative learning, but not as we know it...

is there any evidence that CS-->US association can act as an independent unit?

food

Click

red

A x B x C y

D y

C no shock

A shock

D ? Fear CR

B ? Fear CR

Acquired equivalence experiment as a test of this idea

Stimuli become more similar if trained in the same way

x, y food and no food

based on original demonstration by Honey & Hall, 1989

y

C no shock

A shock

x

A x B x C y

D y

An explanation in terms of mediated conditioning:

An association can form between x and shock even though x is only being thought about

y

C no shock

A shock

x

y

D no shock

B shock

x

A x B x C y

D y

An explanation in terms of mediated conditioning:

now when you present B, it makes you think about shock via x

some experiments have tested x and shown this to be true

A x+ y- B x+ y-

C y+ x-

D y+ x-

C no shock

A shock

D ? Fear CR

B ? Fear CR

Now consider a more complex version (e.g Honey & Watt, 1998)

x and y are paired with paired with food or not

y +

C no shock

A shock

x +

A x+ y- B x+ y-

C y+ x-

D y+ x-

A more complex explanation in terms of mediated conditioning:

y +

C no shock

A shock

x +

y +

D no shock

B shock

x +

A x+ y- B x+ y-

C y+ x-

D y+ x-

A more complex explanation in terms of mediated conditioning:

If x+ and y+ associations act as independent associable units, they should act just like x and y in the simple task

0.3

0.35

0.4

0.45

0.5

0.55

0.6

0.65

1 2 3

Suppression ratios

DB

It works.. but is this really because x+ and y- signal shock?

Phase 1

A x+ y- B x+ y-

C y+ x-

D y+ x-

A x

10s

+

10s

A,B,C,Dvisual

x, yauditory

USfood pellet

Bonardi & Jennings 2008

Phase 1 Phase 2

A ---> shock

C ---> no shock

A x+ y- B x+ y-

C y+ x-

D y+ x-

Test 1

x+ ---> ?

y+ ---> ?

If x+ signals shock it should evoke more fear than y+...

x

10s

+

10s

x, yauditory

USfood pellet

10s

P Q

Suppression ratio Q / (P+Q)

Test fear elicited by x+ and y+

Response baseline

432100.2

0.4

0.6

0.8

x+

y+

Trials

Suppression ratios

Effect of trial type p=.020

Effect of Trial type p=.03

432100.4

0.5

0.6

0.7

0.8

0.9

y-

x-

Trials

Suppression ratios

... and y- should evoke more fear than x-

Conclude:

CS-->US association can act as independent associable unit

So is hierarchical account right?

One alternative suggested by Honey & Watt 1998

Another by configural theory

A x+ y- B x+ y-

food

A

x

no foody

B Hierarchical account saysthat A and B both acquire control over the x-->food and y-->food associations

A,B are occasion setters

x,y are CSs

Different things!

A x+ y- B x+ y-

food

A

x

B

Honey and Watt saythat A and B and x become linked to a hidden unit that is associated with food (same for A, B, y and no food)

A and B and x and y play exactly the same roles!

no food

A

y

B

A x+ y- B x+ y-

food

Configural theory says a blend of A and x, and of B and x, becomes associated with food, and blends of Ay and By become associated with no food

Normal classical conditioning with blended representations

no food

A x

B x

B y

food

no food

A y

So which is right?! Jury's out on hierarchical versus Honey & Watt (but we are working on it..) -- issue is whether occasion setters are special or not.

There is some evidence on deciding between hierarchical or configural theory -- e.g. Morell & Davidson 2002

Light: tone -->suc Light- tone-

Group Same US click -->sucGroup Diff US click -->oil

Test Light/tone Light/click(sameUS) Light/click(diff US) 0

10

20

30

40

50

60

1

Responses per minute

CompoundFeatureTargetSeries4Series5Series6Series7Series8Series9Series10Series11

Same CS Diff CS Diff CSSame US Same US Diff US

foodLighttone

Lightclick

?

Lightclick

Train

Test same CS same US

diff CS same US

diff CS diff US

Lighttone

?

?

foodLighttone

Lightclick

?

Lightclick

Train

Test same CS same US

diff CS same US

diff CS diff US

Lighttone

?

?

Configural theory says responding at test depends on similarity of training and test configures

but these do not change with identity of US

so last two conditions will be the same

Train

Test same CS same US

diff CS same US

diff CS diff US

..but what if identity of US encoded in configure?

then configural theory could explain these results --

Lightclicksuc

?

Lightclick oil

?

foodLighttonesuc

?Lighttonesuc

now configures more different if US changes

But how about this - work with rats not yet published...

Jewel: click-->suc click--> Jewel-->

tone --> suc buzz --> oil

Test Jewel tone Jewel buzz

Jewel click

suc

this part like Morell and Davidson

same feature, different target CSconfigural theory predicts less responding when USs differ

Jewel: click-->suc

tone --> suc buzz --> oil

Test Jewel tone Jewel buzz

sucJewelclick suc

Jeweltone suc

Jewelbuzz oil

hierarchical theory predicts the same, because the training andtest associations are more different when USs differ

Jewel: click-->suc

tone --> suc buzz --> oil

Test Jewel tone Jewel buzz

responding higher on same than on different trials

both configural and hierarchical theory can explain these results

6

6.5

7

7.5

8

8.5

9

9.5

1 2 3 4

Test sessions

Responses per minute

SameDifferent

But how about doing converse...same target CS different "occasion setter" (actually just a CS)

Jewel: click-->suc click--> Jewel-->

dark... --> suc flash... --> oil

Test dark click flash click

test compounds still differ by one or two components so configural theory predicts exactly the same as before

Jewel: click-->suc

dark... --> suc flash... --> oil

Test dark click flash click

sucJewelclick suc

darkclick suc

flashclick oil

but hierarchical theory does not predict anything because test compounds do not include occasion setters

Jewel: click-->suc

dark... --> suc flash... --> oil

Test dark click flash click

responding nonsignificantly higher on different trials -

configural theory predicted the opposite result

6

7

8

9

10

11

12

13

1 2 3 4

Test sessions

Responses per minute

SameDifferent

Conclude:

Some evidence against configural theory

But no discrimination yet between hierarchical account and Honey & Watt

A hot topic! Are occasion setters special or not?

Further questions: what else can occasion setters do?

when a CS signals a US animals learn this relation and respond

they also learn when the US will be presented - timing

Tone (20 sec) --> food

432100

10

20

30

40

50

60

Adapted from Roberts (1982)

Five-second blocks

Mean responses per minute

Can occasion setters tell the rat when the US will occur?

Short trials: Click: Light (6)+ Noise: Dark (6+)

Long trials: Click: Dark (30)+ Noise: Light (30+)

Can occasion setters tell the rat when the US will occur?

Short trials: Click: Light (6)+ Noise: Dark (6+)

Long trials: Click: Dark (30)+ Noise: Light (30+)

201816141210864202

4

6

8

10

12

Short trials

Long trials

End of training

Two-second bins

Mean responses

Bonardi & Jennings 2007

Then give probe trials - present Light and Dark for 90 seconds to look for the point of maximum responding

Short trials: Click: Light (6)+ Noise: Dark (6+)

Long trials: Click: Dark (30)+ Noise: Light (30+)

Expect peak responding to be at 6s on short trials and 30s on long trials

40302010010

20

30

40

Short trials

Long trials

Data from probe trials

Two-second bins

Mean responses per bin

Can occasion setters tell the rat when the US will occur?

Apparently yes!

Conditional learning and Drug tolerance (Ramos et al., 2002)

Ethanol induces hypothermia, to which tolerance develops and this conditions to other cues that are present

Investigated effect of extinguishing cues.

Critical fact: CSs extinguish when presented alone

occasion setters do not

Flash (injection -->ethanol) injection -->saline

Tolerance conditions to Flash

Flash

(injection -->ethanol) injection -->saline

When Flash and injection simultaneous (not good for occasion setting) extinction of Flash removed its tolerance effects

Flash (injection -->ethanol) injection -->saline

When Flash and injection serial (good for occasion setting) extinction of Flash had no effect

So understanding occasion setting gives important information on the development of tolerance in drug addiction...

References

Bonardi, C. (1991). Blocking of occasion setting in feature-positive discriminations. Quarterly Journal of Experimental Psychology, 43B, 431-448.

Bonardi, C. (1996). Transfer of occasion setting: The role of generalization decrement. Animal Learning and Behavior, 24, 277-289.

Bonardi, C. (1998). Conditional learning: An associative analysis. In P.C. Holland and N.A. Schmajuk (Eds.) Associative learning and cognition in animals: Occasion setting. Washington, D.C.:APA

Bonardi, C., & Jennings, D. (2007). Occasion setting of timing behaviour. Journal of Experimental Psychology: Animal Behavior Processes, 33, 339-348.

Bonardi, C., & Jennings, D. (2008). Learning about associations: Evidence for a hierarchical account of occasion setting. Journal of Experimental Psychology: Animal Behavior Processes, in press.

Cohen RM, Semple, WE, Gross, M, Mordahl, TE, Holcomb, HH, Dowling, MS, & Pickar, D (1988) The effect of neuroleptics on dysfunction in a prefrontal substrate of sustained attention in schizophrenia. Life Sciences, 43, 1141-1150.

Cornblatt, BA, Lenzenweger, MF, Erlenmeyer,-Kimling, L (1989). A continuous performance tist, idential pairs version. ll Contrasting attentional profiles in schizophrenic and depressed patients. Psychiatry Research, 29, 65-85.

Colwill, R.M., & Rescorla, R.A. (1990). Evidence for the hierarchical structure of instrumental learning. Animal Learning and Behavior, 18, 71-82.

Dickinson, A, Hall, G, & Mackintosh, NJ (1976). Surprise and the attenuation of blocking. Journal of Experimental Psychology: Animal Behavior Processes, 2, 313-322.

Dunn, M., Futter, D., Bonardi, C., & Killcross, A.S. (2005). Attenuation of D-amphetamine-induced disruption of conditional discrimination performance by alpha-flupenthixol. Psychopharmacology, 177, 296-306.

Holland, P.C. (1983). Occasion setting in Pavlovian feature-positive discriminations. In M.L.Commons, R.J. Herrnstein & A.R Wagner (Eds.) Quantitative analyses of behavior: Discrimination Processes (Vol. 4: pp.183-206). New York: Ballinger.

Honey, R.C., & Hall, G. (1989). Acquired equivalence and distinctiveness of cues. Journal of Experimental Psychology: Animal Behavior Processes, 15, 338-346.

Honey, RC, & Watt, A. (1998). Acquired relational equivalence: Implications for the nature o associative structures. Journal of Experimental Psychology: Animal Behavior Processes, 24, 325-334.

Morell, J.V., & Davidson, T.L. (2002). Transfer across unconditioned stimuli in serial feature discrimination training. Journal of Experimental Psychology: Animal Behavior Processes, 28, 83-96.

Ramos, BMC, Siegel, S, & Bueno, JLO (2002). Occasion setting and drug tolerance. Integrative Physiological and Behavioral Science, 37, 165-177.

Rescorla, R.A. (1985). Conditioned inhibition and facilitation. In R.R..Miller & N.E.Spear (Eds.) Information Processing in animals: Conditioned inhibition. (pp.299-326). Hillsdale N.J. Erlbaum.

Rescorla, R.A. (1991a). Combinations of modulators trained with the same and different target stimuli. Animal Learning and Behavior, 19, 355-360.

Rescorla, R.A. (1991b). Transfer of inhibition and facilitation mediated by the original target stimulus Animal Learning and Behavior, 9, 293-303.

Swartzentruber, D. (1995). Modulatory mechanisms in Pavlovian conditioning. Animal Learning and Behavior, 23, 123-143.

Wysocki, MS, & Sweet, JJ (1985). Identification of brain damaged schizophrenic, and normal medical patients using a brief neuropsychological screening battery. International Journal of Clinical Neuropsychology, 7, 40-49.