Overall Conclusion & Summary Visual memory for natural scenes is surprisingly long lasting, given the change blindness literature (see also Hollingworth, in press). Good memory for the details of a scene does not require explicit memorization. An active viewing task, such as describing objects, can produce long-lasting memory traces, which preferentially represent the features of the objects that were relevant to the active viewing task. Without a conflicting explicit memorization task, active viewing tasks influence what object features are encoded. An active viewing task does not always influence the details subjects encode about a scene. When subjects can learn what aspects of scenes they need to memorize, the scene describing task has no impact on what details they encode. This suggests that it is possible to allocate memory independently of the demands of the describing task. Further work will be required to determine exactly when an active viewing task can be conducted independently from actively memorizing a scene. In everyday life, we rarely explicitly memorize the scene around us, so our moment-to-moment tasks may play a big role in the visual memories we accumulate.

Central & marginal item effects

0

1

2

3

4

5

6

7

8

Central interest

Marginalinterest

# of

cha

nges

det

ecte

d

appear/disappear color

Error Bars ± 1 Standard Error(s)Change type

Repeated measures ANOVA:Effect of interest: p < .0001Effect of change type: p < .01Interaction: p < .0001

P1 P2 P3 P4 0 1 2 3 4 5 6 7 8 9

9’ 8’ 7’ 6’ 5’ 4’ 3’ 2’ 1’ 0’

1) “Say name (or color) of objects under each crosshair”

2) “Surprise! What changed in these images?”

Fixed orderpractice images

Experimental stimuli,randomized order between subjects

time

Changed images shown inreverse order

Reading instructions

Task Effects!

0

20

40

60

80

100

% c

orre

ct

appear color

Change Type

namecolor

Error Bars: ± 1 Standard Error(s)

Condition: describeP < .007

P < .022

8 different crosshairs displayed on separate objects in the same image

originalimage

500 ms

Subject clicks to advance,timeout after 2 seconds changed

image + crosshairs

Subject clicks to advance,timeout in 10 seconds

Memory capacity for natural scenes: Luck & Vogel, 1997: change detection experiments using simple geometric shapes suggest that short-term memory capacity is about 4 objects, independent of their complexity. But: geometric shape memory may have fairly different properties from scene memory. Rensink, et al., 1997: visual memory is tightly linked to attention; only attended objects are maintained in high-detail form. Attention is limited to a very small number of objects. Hollingworth, in press: memory for computer rendered scenes is quite good if subjects attend to the objects in a controlled order, and are tested with a 2AFC paradigm. Short-term memory (the last 1 or 2 items attended to) is nearly perfect, and longer-term memory, though less accurate, persists for several minutes. Is visual memory for scenes really so limited? How much of a scene do people remember when they are not actively trying to memorize the scene?

Introduction What representations does your brain create as you look around in a scene? To what extent do these representations depend on your current task? How long do these representations persist? We will present evidence that people’s memory capacity for natural scenes is surprisingly good, and that active viewing tasks can influence what aspects of a scene are encoded in memory.

Selectivity of memory for natural scenes: Rensink, et al., 1997: people miss remarkably large changes to natural scenes. Some changes are much easier to notice than others (marginal vs. central interest objects) suggesting that some aspects of the scene are inherently more likely to be encoded in memory. O’Regan, et al., 2000: eye-tracking studies suggest that the “central interest” advantage reduces to a preference to fixate these objects more frequently. Triesch, et al., 2003, Droll et al., 2003: just attending to a single object is insufficient to detect changes to it. In a Virtual Reality experiment where subjects performed two tasks (sorting objects, and monitoring those objects for changes) many changes to the object at the center of attention were missed. Changes were least likely to be missed, however, when they were to the task-relevant features of the object. This suggests that task-relevant features of individual objects are more strongly encoded in memory. Is it sufficient to describe the selection process in terms of which objects are attended and encoded in a scene? OR does a subject’s task also influence which aspects of those objects are encoded?

Experiment 1: Do active viewing tasks influence the contents of memory? • When a person fixates an object, do they encode in visual memory all the details and

features of the object, or do they primarily extract and encode the details necessary for completing their current task?

• To what extent do recently fixated objects have a more detailed representation in memory?

• Does Rensink’s central/marginal object distinction derive exclusively from the increased likelihood that subjects will fixate those objects?

Materials 48 experimental trials (Rensink et al., 1997 natural image set), each with 1 of 3 types of changes:

• Object color • Object location • Object added or removed

14 practice trials with particularly obvious changes of the same types. Example Trial

Procedures

Condition Name objects Say the dominant color of objects Say if the object located is in the foreground or background of the scene Taskless control: just memorize the objects under the crosshairs.

Methods & Subjects have two tasks: 1. Describe 8 objects in each image (designated by displaying a crosshair over each

object one at a time). 2. Detect which of those 8 objects changed in a second image. Subjects are split into 4 between-group conditions that have different versions of task 1:

Expected detection benefit for

color changes object addition or removal

object translation (?)

N/A: baseline condition

15 subjects per condition. Each subject sees the same set of images, crosshair locations, and image changes. Subjects look for the change in each image immediately after describing the 8 objects in

No evidence for any interaction between the describing task and the memory task. • It is somewhat surprising that describing precisely the feature of an object that

will change, just a second or two before the change, confers no detection advantage.

Performance in taskless control condition equivalent to conditions where subjects describe objects.

• It seems that subjects can conduct both tasks in parallel about as well as just focusing on the change detection task.

• Attending to (and even verbally encoding) a specific feature of an object is no better than just looking at the object.

No evidence for any order effect. Change detection performance appears to be driven by long-term memory representations of the scene.

Central/marginal distinction remains. Even after controlling for attention, some aspects of scenes really are more likely to be encoded in a form robust enough to support change detection.

Experiment 2:

Surprise memory test after describing scenes In Experiment 1 subjects were engaged in two tasks at once:

• Only encoding the features relevant to the scene description task would have resulted in lower overall performance on the memory task.

• The describing task required little to no memory use. Perhaps subjects were free to instead focus on encoding the features most helpful for change detection?

Without knowledge of the upcoming memory test to guide their encoding, however, would the describing task influence what features of objects subjects encoded?

• The change detection test is given after all images have been described. Subjects have no warning of the upcoming memory test.

• Change detection performance will depend on features that make it into (long term?) memory as a byproduct of the describing task.

Modifications from Experiment 1

• Only color and name describing conditions. • Uses a minimal number of stimuli to minimize the effect of forgetting. • Uses the most informative stimuli from experiment 1 (those which subjects

detected changes in 50% of the time). • 22 subjects per condition.

Timing results Describing the 14 images: 235 seconds (~17 seconds per image). Reading surprise instructions: 55 seconds. Looking for changes in 10 experimental images: 99 seconds. Average timing between groups differed by 2%.

Subjects still encoded many visual details about the scenes they described. Overall performance was about the same as in experiment 1. Subjects who named objects in the scene were at a disadvantage at detecting when those objects changed color. This suggests they did not encode the color of the objects as strongly because it was of less relevance to their task. Subjects who described the color of objects were at a disadvantage at detecting the addition of objects to the scene. This is somewhat remarkable, given that the color under the crosshair changed by virtue of the object under the crosshair changing. This suggests subjects did not form as robust a representation of the identity of the objects they described, even though they could later recall what color those objects were!

Order effects?

0

1

2

3

4

Early Middle Late

When in trial crosshair visited changing object

num

ber o

f cha

nges

det

ecte

d

add/deletetranslatecolor change

Change Type:

Change detection task starts here X <--

Task effects?

0

20

40

60

80

100

% c

hang

es n

otic

ed

appear/disappear translation colorObject change type

taskless

Error Bars: ±1 Standard Error(s)

namelocationcolorSubject

Condition

Repeated measures ANOVA:Effect of condition: p < .1Effect of change type: p < .0001Interaction: p < .36

Procedure

References Droll, J., Hayhoe, M., Triesch, J., & Sullivan, B. (2003) Task relevance of object features modulates the content of visual working memory. Vision Science Society Meeting,

Sarasota, FL. Hollingworth, A. (in press) Constructing visual representations of natural scenes: The roles of short- and long-term visual memory. Journal of Experimental Psychology: Human

Perception and Performance Luck, S. & Vogel, E. (1997) The capacity of visual working memory for features and conjunctions. Nature, 390, p. 279-281. O’Regan, J., Deubel, H., Clark, J., & Rensink, R. (2000) Picture changes during blinks: Looking without seeing and seeing without looking. Visual Cognition, 7, p.191-211. Rensink, R., O'Regan, J., & Clark, J. (1997) To See or Not to See: The Need for Attention to Perceive Changes in Scenes. Psychological Science, 8, p. 368-373. Triesch, J., Ballard, D., Hayhoe, M., & Sullivan, B. (2003) What you see is what you need. Journal of Vision, 3, p. 86-94.

Acknowledgments We thank Ronald Rensink for donating the images we used in this research, Cherie-Marie Vitartas for running our subjects, and the CogSci HCI Lab for printing this poster. This work was supported by the University of California Academic Senate, San Diego Division, under grant RC142C-TRIESCH.

Poster reprint available online:http://csclab.ucsd.edu/~alan/pub/

Visual memory for natural scenes: automatic + task dependent components

Alan Robinson & Jochen TrieschUC San Diego Department of Cognitive Science

Contact: robinson@cogsci.ucsd.edu

in each scene.