an information processing perspective on conditioning c. r. gallistel rutgers center for cognitive...
TRANSCRIPT
![Page 1: An Information Processing Perspective on Conditioning C. R. Gallistel Rutgers Center for Cognitive Science](https://reader036.vdocuments.us/reader036/viewer/2022062314/56649e4e5503460f94b451df/html5/thumbnails/1.jpg)
An Information Processing Perspective on Conditioning
C. R. Gallistel
Rutgers Center for Cognitive Science
![Page 2: An Information Processing Perspective on Conditioning C. R. Gallistel Rutgers Center for Cognitive Science](https://reader036.vdocuments.us/reader036/viewer/2022062314/56649e4e5503460f94b451df/html5/thumbnails/2.jpg)
Five Classic Experiments
• Rescorla’s background conditioning experiment (the truly random control)
• Kamin’s blocking experiment
• Reynold’s overshadowing experiment
• Wagner’s relative cue validity experiment
• Conditioned inhibition experiments
![Page 3: An Information Processing Perspective on Conditioning C. R. Gallistel Rutgers Center for Cognitive Science](https://reader036.vdocuments.us/reader036/viewer/2022062314/56649e4e5503460f94b451df/html5/thumbnails/3.jpg)
Truly Random Control
Result: Only Group 1 conditioned
Implication: Contingency not temporal pairing drives conditioning
![Page 4: An Information Processing Perspective on Conditioning C. R. Gallistel Rutgers Center for Cognitive Science](https://reader036.vdocuments.us/reader036/viewer/2022062314/56649e4e5503460f94b451df/html5/thumbnails/4.jpg)
Blocking
Result: No conditioning to the CS that first appears in compound (the blocked CS)
Implications
• Temporal pairing not sufficient
• Conditioning occurs only if US “surprises” subject
![Page 5: An Information Processing Perspective on Conditioning C. R. Gallistel Rutgers Center for Cognitive Science](https://reader036.vdocuments.us/reader036/viewer/2022062314/56649e4e5503460f94b451df/html5/thumbnails/5.jpg)
Overshadowing
• Two pigeons trained with two compound cues: one (+) yielded food when pecked; the other (-) yielded nothing
• Tested with individual elements
• One bird pecked only at triangles, the other, only at red
• Implication: when two predictors (red & triangle) are redundant, one is disregarded
![Page 6: An Information Processing Perspective on Conditioning C. R. Gallistel Rutgers Center for Cognitive Science](https://reader036.vdocuments.us/reader036/viewer/2022062314/56649e4e5503460f94b451df/html5/thumbnails/6.jpg)
Relative Validity
• Implication: only the cue that carries the most information about the US gets conditioned
![Page 7: An Information Processing Perspective on Conditioning C. R. Gallistel Rutgers Center for Cognitive Science](https://reader036.vdocuments.us/reader036/viewer/2022062314/56649e4e5503460f94b451df/html5/thumbnails/7.jpg)
Conditioned Inhibition
Implications:• Temporal pairing neither necessary nor sufficient for
conditioning• Negative contingency just as effective as positive
contingency
![Page 8: An Information Processing Perspective on Conditioning C. R. Gallistel Rutgers Center for Cognitive Science](https://reader036.vdocuments.us/reader036/viewer/2022062314/56649e4e5503460f94b451df/html5/thumbnails/8.jpg)
So what’s the problem?
"We provide the animal with individual events, not correlations or information, and an adequate theory must detail how these events individually affect the animal. That is to say that we need a theory based on individual events.”--Rescorla (1972, p. 10)
![Page 9: An Information Processing Perspective on Conditioning C. R. Gallistel Rutgers Center for Cognitive Science](https://reader036.vdocuments.us/reader036/viewer/2022062314/56649e4e5503460f94b451df/html5/thumbnails/9.jpg)
But we do present correlations!
• We correlate CS and US occurrences so that the CS provides information about the timing of the US
• Rescorla presumably meant that the conditioning process does not operate at the level of information
• Because information does not increment associations, events do (in conventional associative theory)
![Page 10: An Information Processing Perspective on Conditioning C. R. Gallistel Rutgers Center for Cognitive Science](https://reader036.vdocuments.us/reader036/viewer/2022062314/56649e4e5503460f94b451df/html5/thumbnails/10.jpg)
Shannon meets Pavlov
• From an information processing perspective, conditioning is driven by information
• The information an event provides to a subject is measured by the reduction in the subject’s uncertainty--Shannon, 1948
![Page 11: An Information Processing Perspective on Conditioning C. R. Gallistel Rutgers Center for Cognitive Science](https://reader036.vdocuments.us/reader036/viewer/2022062314/56649e4e5503460f94b451df/html5/thumbnails/11.jpg)
Principles
• Subjects respond only to informative CSs• CSs inform to the extent they change uncertainty
about the time to the next US• Bandwidth maximization by minimizing number
of information carrying CSs• Information in a protocol is carried by the
temporal intervals between events and the numbers of events
• Weber’s law: uncertainty (noise) in the representation of intervals and numbers is proportional to their magnitude
![Page 12: An Information Processing Perspective on Conditioning C. R. Gallistel Rutgers Center for Cognitive Science](https://reader036.vdocuments.us/reader036/viewer/2022062314/56649e4e5503460f94b451df/html5/thumbnails/12.jpg)
Tentative Assumption
• Uncertainty about the time to the next US is represented by a cumulative probability function
• Red function is not anchored in time; green one is anchored to the moment of CS onset
![Page 13: An Information Processing Perspective on Conditioning C. R. Gallistel Rutgers Center for Cognitive Science](https://reader036.vdocuments.us/reader036/viewer/2022062314/56649e4e5503460f94b451df/html5/thumbnails/13.jpg)
Can Shannon Information Predict Acquisition?
Preliminaries• The appearance of a
conditioned response in the course of conditioning is abrupt
• As if it were the outcome of an evidence-based decision process
![Page 14: An Information Processing Perspective on Conditioning C. R. Gallistel Rutgers Center for Cognitive Science](https://reader036.vdocuments.us/reader036/viewer/2022062314/56649e4e5503460f94b451df/html5/thumbnails/14.jpg)
Conditioning Protocols
![Page 15: An Information Processing Perspective on Conditioning C. R. Gallistel Rutgers Center for Cognitive Science](https://reader036.vdocuments.us/reader036/viewer/2022062314/56649e4e5503460f94b451df/html5/thumbnails/15.jpg)
Information in Random Rate Change Protocols
€
˙ H = λ log2
e
λΔτ
⎛
⎝ ⎜
⎞
⎠ ⎟
€
H =1
λλ log 2
e
λΔτ
⎛
⎝ ⎜
⎞
⎠ ⎟= k − log 2 λ
€
H B − H CS = k − log2 λ B( ) − k − log2 λ CS( ) = log2 λ CS − log2 λ B
€
log2
λ CS
λ B
⎛
⎝ ⎜
⎞
⎠ ⎟Bits Communicated per Reinfrcd CS
![Page 16: An Information Processing Perspective on Conditioning C. R. Gallistel Rutgers Center for Cognitive Science](https://reader036.vdocuments.us/reader036/viewer/2022062314/56649e4e5503460f94b451df/html5/thumbnails/16.jpg)
Accumulated Bits at Acquisition
€
Na log2λCS
λ B
⎛
⎝ ⎜
⎞
⎠ ⎟
Na is the number of reinforced CS presentations prior to the onset of conditioned behavior
![Page 17: An Information Processing Perspective on Conditioning C. R. Gallistel Rutgers Center for Cognitive Science](https://reader036.vdocuments.us/reader036/viewer/2022062314/56649e4e5503460f94b451df/html5/thumbnails/17.jpg)
Contribution from Fixed CS-US Interval
€
H =1
2log2 2πe
σ 2
Δτ( )2
⎡
⎣ ⎢ ⎢
⎤
⎦ ⎥ ⎥Entropy of a Gaussian
€
12
log2e
2π
⎛ ⎝ ⎜
⎞ ⎠ ⎟− log2 wAdditional info available
€
Na log2λ CS
λ B
⎛
⎝ ⎜
⎞
⎠ ⎟+ 2NaCumulative Bits at Acq
where w = Weber fraction
![Page 18: An Information Processing Perspective on Conditioning C. R. Gallistel Rutgers Center for Cognitive Science](https://reader036.vdocuments.us/reader036/viewer/2022062314/56649e4e5503460f94b451df/html5/thumbnails/18.jpg)
Gibbon & Balsam
• Plotted reinforcements to acquisition as a function of the I/T ratio in continuous reinforcement paradigms, using data from many labs
• CS/B varies directly with I/T
• Slope (on log-log)=-1
• Ergo, bits to acquisition is constant
![Page 19: An Information Processing Perspective on Conditioning C. R. Gallistel Rutgers Center for Cognitive Science](https://reader036.vdocuments.us/reader036/viewer/2022062314/56649e4e5503460f94b451df/html5/thumbnails/19.jpg)
Cumulative Dist of G&B Data
• Cumulative distribution shows the fraction of the subjects that had begun to respond to the CS as a function of the amount of information so far communicated; the longer conditioning lasts, the more bits communicated
• The median amount of information at acquisiton is roughly 100 bits (median = # of bits at which fraction = 0.5)
![Page 20: An Information Processing Perspective on Conditioning C. R. Gallistel Rutgers Center for Cognitive Science](https://reader036.vdocuments.us/reader036/viewer/2022062314/56649e4e5503460f94b451df/html5/thumbnails/20.jpg)
Partial Reinforcement
• Partial reinforcement has no effect on
hence, it should have no effect on reinf to acq, hence no effect on bits cumulative bits conveyed at acquisition
€
CS λ B
![Page 21: An Information Processing Perspective on Conditioning C. R. Gallistel Rutgers Center for Cognitive Science](https://reader036.vdocuments.us/reader036/viewer/2022062314/56649e4e5503460f94b451df/html5/thumbnails/21.jpg)
Poor Positive
• Like the partial reinforcement data, the data from poor positive groups falls right in when plotted as bits to acquisition
![Page 22: An Information Processing Perspective on Conditioning C. R. Gallistel Rutgers Center for Cognitive Science](https://reader036.vdocuments.us/reader036/viewer/2022062314/56649e4e5503460f94b451df/html5/thumbnails/22.jpg)
Conclusion
• For all the protocol variations for which we have data, the cumulative bits conveyed by the CS at the point of acquisition in the median bird is about 100 and the distributions about this median are the same
• The same principles that enable us to understand blocking, overshadowing and relative validity predict the effects of all the manipulations of protocol parameters--with no free parameters!
• This sort of simple, elegant quantitative result is all but unknown in the study of animal learning
![Page 23: An Information Processing Perspective on Conditioning C. R. Gallistel Rutgers Center for Cognitive Science](https://reader036.vdocuments.us/reader036/viewer/2022062314/56649e4e5503460f94b451df/html5/thumbnails/23.jpg)
Collaborators & Support
• The late John Gibbon
• Peter Balsam
• Stephen Fairhurst
• RO1 MH68073 Time and Associative Learning