the hippocampus and neocortex why there are complementary ... · response to a cue...
TRANSCRIPT
Insights from the Successes and Failures of Connectionist Models of Learning and Memory
Why there are Complementary Learning Systems in the Hippocampus and Neocortex
Alessandro StranieriMTAT.03.292 - CNS Seminar - 2018/19
Bruce L. McNaughton University of California Irvine
James McClellandStanford University
Randall C. O’ReillyUniversity of Colorado Boulder
Year of publication: 1995Journal: Psychological review
4278 Citations
Offer an account of how
the neocortex and the hippocampus
support
learning and memory
Hippocampal System
Image generated with BrainFacts.org
Hippocampal System
Image generated with BrainFacts.org
Hippocampal System
Image generated with BrainFacts.org
Lesions to hippocampal system
Consolidation of memories
Inception
How memory seems to work
Connectionist Models
Simulations
Implications
How memory seems to work
Connectionist Models
Simulations
Implications
• Anterograde amnesia (HM subject)• Affected: Memories around the event• Unaffected: Remote memories
• Type of process affected: form and recall associations• Type of memory: declarative
• Episodic• Semantic
Role of Hippocampal System
“In humans an intact hippocampal system is necessary for the formation of an association between arbitrarily-paired words that is sufficiently strong after a single
presentation to have any effect on subsequent performance”
“In animals evidence suggests that damage restricted to the hippocampus impacts on tasks that require the animal to learn responses specific to particular non- spatial
combinations of cues, or to specific contexts”
Role of Hippocampal System
Spared abilities: non declarative (implicit) memory
Subjects can:
• Repetitive activity learned over several sessions• Structure for common items• Non arbitrary associations
Role of Hippocampal System
Complementary Learning Systems
Organization of the memory in the brain
Neocortical Processing System
Hippocampal Memory System
Higher level control of behavior and cognitive processing
Sensory, perception and actuation processes
Task performance through elicitation of pattern of activation in response to a cue pattern
Organization of the memory in the brain
Neocortical Processing System
Assumptions:
Events in the neocortex produce adjustments in the connections
Knowledge is embedded
Changes connections are not enough for learning to happen
Organization of the memory in the brain
Neocortical Processing System
Bidirectional communication between NS and HS
HS receives a compressed version of perceived state
Knowledge is sparse
A pattern may become a stable memory
Activation of part of the pattern might trigger the activation of the whole pattern
Organization of the memory in the brain
Hippocampal System
Reinstatement:
● Recall for task relevant activities● Off-line situations
Hippocampal System
● memory● teacher
Organization of the memory in the brain
Consolidation
The same kind of learning happens not only task relevant situations but for
● episodic memory● encyclopedic● semantic
Organization of the memory in the brain
Consolidation
Organization of the memory in the brain
EvidencePhysiological: pathways
Experimental: specific activations in HS
Associative Long Term Potentiation (LTP)
LTP can last for days
Inhibition of LTP impairs learning
Little evidence of reinstatement from HS to NC
Evidence of reinstatement of HS activity into the HS itself:● Sharp waves (during sleep)● Activity after exploration (same neurons)
Organization of the memory in the brain
Reinstatement
Complementary relationship of
Hippocampal System and Neocortex
Slow Integration of New Information In NS
Key Questions
How memory seems to work
Connectionist Models
Simulations
Implications
Semantic Networks
Recalls hierarchical internal representation of knowledge
Allows generalization (finding the parent)
Do not scale
Mixing hierarchies
Common and uncommon categories
Semantic Networks
Semantic Networks
penguin
“Generalization depends on the internal representation of a concept which captures the similarity to other concepts”
Consistent with psychological evidence
Does away with the problem of partial traits and exceptions
Connectionist Models
Connectionist Models
Connectionist Models
Rumelhart’s experiment
Training data: true propositions
Input is a pair
● concept term● relationship term
Experiment Replication
Connectionist Models
Generalization
Training: only <sparrow, isa>
Test: <sparrow, OTHER RELATION>
The type of learning made possible by connectionist models is not appropriate for all types of learning
Difficulty in learning quickly arbitrary associations
Catastrophic Interference
Connectionist Models
AB-AC paradigm
Subjects trained and tested on AB: arbitrary pairing of words (e.g. Table-Street)
Switch to a new AC list
During training, for some A the subject is asked to recall the B association
Catastrophic Interference
Human subjects
Training on AB
Training on AC
Testing on AB
Catastrophic Interference
Connectionist Model
Catastrophic Interference
Result
Possibility of poorly designed network
Proposals of method to reduce interference
Catastrophic Interference
Connectionist model like Rumelhart’s shows that such a model can integrate different relationships and generalize
Catastrophic Inference shows that such models poorly handle arbitrary association
To address this issue
Interleaved Learning
Modification of Rumelhart’s experiment
Integration of new inconsistent concept: Penguin
2 modalities
● Focused Learning● Interleaved Learning
Interleaved Learning
Interleaved Learning
Results
Interleaved learning is needed for:● discover of structure shared across concepts● incorporation of new material
Neo-cortical system is not capable of interleaved learning ⇒ Hippocampus
Checkpoint
Any relationship between events that can be used to process new events or allow handling of similar events
Structure
Any relationship between events that can be used to process new events or allow handling of similar events
has to relationship in Rumelhart’s network
Structure
First reason comes from the procedures used in weight adjustment in connectionist methods.
Slow Learning
High learning rate means resetting the knowledge
Low learning rate means averaging over large set of events
What discussed so far regarding learning in connectionist model seems sound, granted presence of back propagation procedures
These are not available in biological systems
Structure in Biological Network
Some possibilities are proposed
● Axonal back-projections● Reinforcement signal diffused
Simulation show that both associative RL and BP can discover representation
RL requires even slower learning rate
Structure in Biological Network
The type of learning described is effective but seemingly slow
HS is the solution to fast learning and recall
• Reinstatement can adjust weights in neocortex• Control behavioral responses
Demonstrated through simulation of amnesia
Consolidation
Hippocampus is a black box
● memory decays with time (nature of connections or overwrite of other knowledge)
● learning is proportional to the importance of the event● probability of reinstatement decreases with the intensity of hippocampal trace● probability of reinstatement differs in cases of task relevant or irrelevant
Consolidation
Tone-Shock experiment
Hippocampal and sham lesion
Lesion time distances 1, 7, 14, 28
Measure of recall: percentage of time in fear posture
Consolidation in a period of 28 days
Consolidation
Tone-Shock with Rats
16-16-16 ANN
Normal environment: 20 random input-output pairing
One extra pairing: ETS
After exposure to ETS, only the stimulus would come up in the test
1 training round per simulated day
Performance: reduction in deviation from correct ETS response
Consolidation
Tone-Shock with Rats
Consolidation
Tone-Shock with Rats
Consolidation
Binary Discrimination with PrimatesMonkey trained on binary discrimation
2 objects,of which one was reinforced
100 binary discriminations in sets of 20
11 monkeys had surgery - 7 sham lesions
Normals: performance decrease from 80% to 70%
With surgery: performance increse from 60% to 70%
Consolidation
Binary Discrimination with Primates50-15-1 ANN
3 types of input:
• background trial • training trial• reinstated trials
Consolidation
Binary Discrimination with Primates
How memory seems to work
Connectionist Models
Simulations
Implications
Consolidation
Variation in time-scaleChange across species ⇒ different life expectancy
Change within species ⇒ different task
Change with age ⇒ different optimal learning strategy
No memory of early years
Could be due to high learning rate
Constant change of representations in neocortex
Infantile Amnesia
• Text goes here
• Write catchy header here!