1 episodic memory for soar andrew nuxoll 15 june 2005
Post on 19-Dec-2015
213 views
TRANSCRIPT
1
Episodic Memory for Soar
Andrew Nuxoll15 June 2005
2
Outline
Review Definitions and previous work Improving agent behavior
Improving Performance Two algorithms for memory retrieval Memory usage Processing time
3
What is Episodic Memory?
Memories of specific events in our past Example: Your last vacation
4
Previous Work
Psychology Observations of Humans - Endel Tulving
Cognitive Modeling Soar Model (non-architectural) - Erik Altmann
Artificial Intelligence Continuous CBR - Ram and Santamaría Comprehensive Agents - Vere and Bickmore
5
Long-term Procedural MemoryProduction Rules
Current Implementation
Encoding Initiation?
Storage
Retrieval
When the agent takes an action.
Input
Output Cue
Retrieved
Working Memory
6
Long-term Procedural MemoryProduction Rules
Current Implementation
Encoding Initiation Content?Storage
Retrieval
A portion of working memory is stored in the episode
Input
Output Cue
Retrieved
Working Memory
7
Long-term Procedural MemoryProduction Rules
Current Implementation
Encoding Initiation ContentStorage Episode Structure?Retrieval
Episodes are stored in a separate memory
Input
Output Cue
Retrieved
Working Memory
EpisodicMemory
EpisodicLearning
8
Long-term Procedural MemoryProduction Rules
Current Implementation
Encoding Initiation ContentStorage Episode StructureRetrieval Initiation/Cue?
Cue is placed in an architecture specific buffer.
Input
Output Cue
Retrieved
Working Memory
EpisodicMemory
EpisodicLearning
9
EpisodicMemory
Long-term Procedural MemoryProduction Rules
Current Implementation
Encoding Initiation ContentStorage Episode StructureRetrieval Initiation/Cue Retrieval
The closest partial match is retrieved.
Input
Output Cue
Retrieved
Working Memory
EpisodicLearning
10
Evaluation using Eaters
Pac-Man-like Two types of food
Bonus food (10 pts) Normal food (5 pts)
11
Create a memory cue (input-link + proposed direction)Create a memory cue (input-link + proposed direction)
East
South
North
Evaluate moving in each available directionEvaluate moving in each available direction
An Episodic Memory Eater
EpisodicRetrieval
Retrieve the best matching memoryRetrieve the best matching memory
RetrieveNext Memory
Retrieve the Retrieve the nextnext memory (in temporal order) memory (in temporal order)Use the change in score to evaluate the proposed actionUse the change in score to evaluate the proposed action
Move North = 10 points
12
Working Memory Activation
Used to bias the match at retrieval time Nuxoll, A., Laird, J., James, M. (2004).
Comprehensive Working Memory Activation in Soar. International Conference on Cognitive Modeling.
13
Effects of Memory Activation Bias
Accuracy of Action Evaluation
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 199 397 595 793 991 1189 1387 1585 1783 1981 2179
action
frac
tio
n o
f co
rrec
t ev
alu
atio
ns
Activation-Biased Match
Unbiased Match
14
New Business: Improving Performance Memory Usage
Processing Time
15
Two Algorithms for Retrieval
Instance-Based Store a complete list of each WME in each
memory Interval-Based
Store the duration of each WME (i.e., what cycles during which it existed)
16
Instance-Based Retrieval Algorithm
...
...
...
Working Memory Tree
17
Instance-Based Retrieval Algorithm
Episodic Memories
...
...
...
Working Memory Tree
...
18
Instance-Based Retrieval Algorithm
Episodic Memories
...
...
...
Working Memory Tree
...
...
20
Interval-Based Retrieval Algorithm
...
...
...
Working Memory Tree
21
Interval-Based Retrieval Algorithm
...
...
...
Working Memory Tree
...
16 - 39
51 - 71
74 - 76
107 - 151
776 - 785
Other Data
Range List
O(n2l)
22
Interval-Based: Merging Ranges
+
24
Memory-Bias vs. Cue-BiasEpisodic Memory Eaters Performance
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 130 259 388 517 646 775 904 1033 1162 1291 1420 1549 1678 1807 1936 2065 2194
Action
Fra
ctio
n o
f C
orr
ect
Eva
luat
ion
s
Instance-Based
Interval-Based
25
Memory UsageMemory Usage Comparison
0
1000
2000
3000
4000
5000
6000
7000
8000
0 10000 20000 30000 40000 50000 60000
decision cycles
Mem
ory
all
oca
ted
(ki
lob
ytes
)
Instance-Based
Interval-Based
26
Evaluating Memory Usage
Rough Order of Magnitude Calculation Varies based upon agent and task
One new episode per 150ms (3 cycles) 55MB or 210MB per 24 hours
One new episode per 5-10 seconds <10 MB per 24 hours
27
Processing TimeEpisodic Memory Processing Time
0
0.01
0.02
0.03
0.04
0.05
0.06
0 10000 20000 30000 40000 50000 60000
decision cycles
Pro
ce
ss
ing
Tim
e (
se
co
nd
s p
er
cy
cle
)
Instance-Based
Interval-Based
Nominal CPU time(without Ep. Mem.)
28
Diagnosing “The Spike”Diagnosing the Processing Time Spike
0
0.01
0.02
0.03
0.04
0.05
0.06
0 10000 20000 30000 40000 50000 60000
decision cycles
pro
cess
ing
tim
e (s
eco
nd
s p
er c
ycle
)
System Allocation
Original Routine
Memory Pool
29
Processing Time PotentialEpisodic Memory Processing Time (w/o Allocation)
0
0.01
0.02
0.03
0.04
0.05
0.06
0 10000 20000 30000 40000 50000
decision cycles
Pro
ce
ss
ing
Tim
e (
se
co
nd
s p
er
cy
cle
)
Instance-Based
Interval-Based
Nominal CPU time(without Ep. Mem.)
30
Evaluating Processing Time
Rough Order of Magnitude Calculation Varies based upon agent and task
Worst Case (primitive action level): One new episode per 150ms (3 cycles) Maximum of 50ms allowed for retrieval Result: Limit exceed after four hours (115,000 cycles)
Best Case (human level): One new episode every 5-10 seconds Maximum of 0.1 to 5 seconds allowed for retrieval Result: Limited exceeded in ~1 year
31
Nuggets Coal
Domain independent, architectural implementation
Potential for effective performance
Performance glitches Needs to be tested in
multiple domains