Experimental Design
Background
Stimuli
PRELIMINARY RESULTS (n = 11): Room + Object Representations
PRELIMINARY RESULTS: Segmenting Mental Walkthroughs of Rooms
Summary
Current Directions
Acknowledgements
Mental TrackingKey Questions
Rolando Masís-Obando, Kenneth Norman, Christopher Baldassano
Decoding mental walkthroughs of spatial memories in an immersive virtual reality environment
How do we use spatial context (e.g. rooms of a memory palace) to sca�old episodic memory details (e.g. objects in a memory palace) ?
How do we use spatial context (e.g. rooms of a memory palace) to scaffold episodic memory details (e.g. objects in a memory palace)?
Where in the brain is spatial context represented?
Where in the brain are objects represented?
Can we track people’s mental walks through a memory palace?
Memories need to be accessible and stable over time
Context is one of the most powerful memory cues
Spatial context contributes to the representation of events
Method of loci / memory palace is a memory technique that uses spatial contexts to cue memories
Anterior Temporal (AT) Network : objects
Posterior Medial (PM) Network : context
Hidden Markov Model (HMM) can be used to track mental walkthroughs
Room Representationsvoxels
room group mean
room subject mean
correlate in searchlight analysis
Object Representationsvoxels
object group mean
object subject mean
correlate in searchlight analysis
Single Searchlight Event Boundaries of rooms for a linear path of 5 connected rooms
timepoints (TRs)
Accuracy: 0.70 | Shu�ed: .58 Accuracy: 0.91 | Shu�ed: .61
Room
Ord
er
Obj
ect O
rder
timepoints (TRs)
Single Searchlight Event Boundaries of objects for a linear path of 5 connected rooms
Room-specific patterns in sensory and Posterior Medial Network re-gionsObject specific patterns in sensory and some Anterior Temporal Network regionsRoom-patterns / object-patterns are good predictors for event boundaries
How do the PM and AT network regions work together with the hippocampus to retrieve objects from spatial context cues?
What are the neural correlates of a well-func-tioning (vs. poorly-functioning) memory palace?
learn memory palace
learn room-object associations
map drawing
sleep
post-learning scanpre-learning scan
FREE RECALL
portraits room
chess room
ruins room
clocks room
crystals room
23 rooms | 23 objects | 1.3s TRs | 11 subjects
By comparing brain activity to a pattern template for each room (left) or object (right), the HMM estimates when the subject transitions between recalling room-object pairs. Accuracy for a searchlight was measured as the mean boundary overlap of ’predicted’ vs ‘actual’ (relative to a shuf-fled null) across multiple runs of the 5-room sequence recall task. : real boundaries : predicted boundaries
p < 0.05z = 1.65
p < 0.05z = 1.65
p < 0.05z = 1.65
p < 0.05z = 1.65
Subj 3Subj 3
11+
11+
0
0
0
0 3+ 3+
0 0
1
8040 120
1
2
3
4
5
Funding: MURI to KAN (ONR/DoD N00014-17-1-2961)
For reprints: [email protected]
@Columbia_DPML@appleslicemusic@PtonCompMemLab@HassonLab
80400 0
1
120
1
2
3
4
5
Subject 3 | Sequence Recall #2
tool
class
TV
candy
antiques
carriage
playground
oven
t-rex
puppy
room
sob
ject
s
rooms
objects
