the role of prediction in perception: evidence from ... · jeffrey m. zacks washington university...
TRANSCRIPT
The Role of Prediction in Perception:Evidence From Interrupted Visual Search
Stefania MereuUniversity of Illinois at Urbana-Champaign
Jeffrey M. ZacksWashington University in St. Louis
Christopher A. KurbyWashington University in St. Louis and Grand Valley State
University
Alejandro LlerasUniversity of Illinois at Urbana-Champaign
Recent studies of rapid resumption—an observer’s ability to quickly resume a visual search after aninterruption—suggest that predictions underlie visual perception. Previous studies showed that when thesearch display changes unpredictably after the interruption, rapid resumption disappears. This conclusionis at odds with our everyday experience, where the visual system seems to be quite efficient despitecontinuous changes of the visual scene; however, in the real world, changes can typically be anticipatedbased on previous knowledge. The present study aimed to evaluate whether changes to the visual displaycan be incorporated into the perceptual hypotheses, if observers are allowed to anticipate such changes.Results strongly suggest that an interrupted visual search can be rapidly resumed even when informationin the display has changed after the interruption, so long as participants not only can anticipate them, butalso are aware that such changes might occur.
Keywords: rapid resumption, prediction, perception, attention, visual search
Supplemental materials: http://dx.doi.org/10.1037/a0036646.supp
Prediction has an essential adaptive function, as it is fundamen-tal to many high-level cognitive processes. Prediction is the basisfor learning and decision making (e.g., Bayer & Glimcher, 2005;Schultz, 1998; Schultz, Dayan, & Montague, 1997); counterfactualthinking would be simply impossible without the ability to antic-ipate the consequences of one’s own actions. Furthermore, predic-tion has been demonstrated to be important in language (e.g.,Kamide, Altmann, & Haywood, 2003), time perception (Pariya-dath & Eagleman, 2007), event comprehension (Zacks, Kurby,
Eisenberg, & Haroutunian, 2011), representing visual scenes (e.g.,Enns & Lleras, 2008; Rao & Ballard, 1999), and more generally,in facilitating cognition (e.g., Kveraga, Ghuman, & Bar, 2007).The present study focuses on the role of prediction in visualperception. Specifically, we looked at the influence of goal-drivenbehavior on the early perceptual processes that underlie anticipa-tion in a visual search task.
Both previous knowledge and the evaluation of the currentsituation contribute to the anticipation of future events. Considerthe everyday task of crossing a street: the visual system integratesinformation about the cars’ positions in different moments tocreate expectations about the future positions and coordinate theaction of walking accordingly.
A simple mechanism involving prediction and confirmation isable to account for behavioral choices in humans (see Schultz etal., 1997). It has been suggested that prediction errors—deviationsfrom the predicted outcome—are detected at lower-levels andintegrated into higher-level mechanisms to guide complex behav-ior (Sutton & Barto, 1981; but see Bayer & Glimcher, 2005). EventSegmentation Theory (Zacks, 2004; Zacks, Speer, Swallow,Braver, & Reynolds, 2007), for instance, suggests that the mech-anisms responsible for segmentation of events into meaningfulunits is based on the detection of prediction errors. Recent past andcurrent information is combined into an event model, which guidespredictions about what will happen next. The event comprehensionsystem monitors errors in prediction. When these errors increasetransiently a new event is detected; the model is discarded andreplaced with a new one.
This article was published Online First May 12, 2014.Stefania Mereu, Department of Psychology, University of Illinois at
Urbana-Champaign; Jeffrey M. Zacks, Psychology Department, Washing-ton University in St. Louis; Christopher A. Kurby, Psychology Depart-ment, Washington University in St. Louis and Psychology Department,Grand Valley State University; Alejandro Lleras, Psychology Department,University of Illinois at Urbana-Champaign.
Stefania Mereu is now at TASC, Inc., Chantilly, VA.We thank Dan Simons for his useful comments at an early stage of this
project. Regione Sardegna (T2-MAB-A2008-608) and Sapienza–Universita di Roma (CUN 11–26.64) supported the research of S. Mereu.Grant RO1-MH70674 from the National Institutes of Health (NIH) sup-ported the research of J. Zacks; that grant and NIH T32-AG000030–31supported C. Kurby.
Correspondence concerning this article should be addressed to StefaniaMereu, Department of Psychology, University of Illinois at Urbana-Champaign, 603 E. Daniel Street, Champaign, IL 61820. E-mail:[email protected]
Thi
sdo
cum
ent
isco
pyri
ghte
dby
the
Am
eric
anPs
ycho
logi
cal
Ass
ocia
tion
oron
eof
itsal
lied
publ
ishe
rs.
Thi
sar
ticle
isin
tend
edso
lely
for
the
pers
onal
use
ofth
ein
divi
dual
user
and
isno
tto
bedi
ssem
inat
edbr
oadl
y.
Journal of Experimental Psychology:Human Perception and Performance
© 2014 American Psychological Association
2014, Vol. 40, No. 4, 1372–13890096-1523/14/$12.00 http://dx.doi.org/10.1037/a0036646
1372
A mechanism involving prediction and confirmation has alsobeen suggested to underlie perception (Enns & Lleras, 2008), andhas been formalized into models of visual awareness (Di Lollo,Enns, & Rensink, 2000; Enns & Lleras, 2008). Accordingly,perception is a two-stage process: hypothesis creation and confir-mation. Hypotheses are created through feedforward connections,sent from the lower to the higher areas of the visual system; butonly those that can be confirmed through a feedback signal, sentfrom the higher to the lower areas, result in perception. Hypothesesthat fail to be confirmed are discarded and also fail to be con-sciously represented, triggering a new cycle of hypothesis creationand confirmation.
Convincing evidence for the existence of the hypothesis creationand confirmation cycle comes from the interrupted visual searchtask (Lleras, Rensink, & Enns, 2005; but see Enns & Lleras, 2008).The interrupted visual search task is very similar to a standardvisual search, in which observers have to find, for example, aT-shaped stimulus (target) among L-shaped distractors (see Figure1a). However, instead of giving the observer unlimited time to find
the target, the observer only gets brief 100 ms-long glimpses of thesearch scene, which alternate with blank displays that lasts 900 ms(or longer).1 The key finding is that after an interruption, observersresume the visual search faster than they start a new one, givingcorrect responses as early as 200–300 ms when the display reap-pears after an interruption. According to the perception cyclehypothesis (Enns & Lleras, 2008), on the first glance, the targetneeds an entire cycle of hypothesis creation and confirmation tobecome available for explicit report. This two-stage process typi-cally produces a unimodal reaction time (RT) distribution with apeak around 700 ms. If the observers fail to find the target after thefirst glance, they will need further glances at the display to find it.Thus, for all glances after the first one, the visual system is in oneof two modes when the display reappears. A first possibility is thatthe visual system has no information about the target (no hypoth-esis about it has been created), in which case a hypothesis willneed to be created and confirmed to elicit a response. If both ofthose processes can be completed before the next glance, then RTfor those trials will look like RT on the first glance, with a peakaround 700 ms. A second possibility is that when the displayreappears, the visual system has already created a perceptualhypothesis of the target based on the previous glance. If thathypothesis is available, then it suffices to confirm it against thesensory input on the current glance, a much faster overall process.Lleras, Rensink, and Enns (2005, 2007) proposed that this type oftrial is responsible for a second peak on the RT distribution, withlatency between 200 ms and 300 ms. As a result, RT distributionsin interrupted search tasks are usually bimodal: an initial fast peak(between 200 ms and 300 ms) indexing hypothesis-only confirma-tion in the current glance and a later peak (around 700 ms)indexing hypothesis creation and confirmation in the currentglance. The appearance of the early, fast peak is a phenomenonthat has been referred as to “rapid resumption” (Lleras, Rensink &Enns, 2005) and it has been taken as evidence that implicit mem-ory is involved in visual search (Lleras et al., 2005). In fact, rapidresumption seems to occur because the visual system integratesmemory for past events and the current information to engage in apredictive strategy, used to facilitate visual search. However, analternative hypothesis has been suggested: that rapid resumptionreflects merely a passive accumulation of sensory evidence. VanZoest, Lleras, Kingstone, and Enns (2007) ruled out this possibilityby showing that remembering previous displays is necessary forrapid resumption to occur. In fact, even if gaze distance from thetarget and the probability of giving a fast response are inverselycorrelated, simply landing a saccade on a nearby target’s locationis not sufficient for rapid resumption to occur (Van Zoest Lleras,Kingstone, & Enns, 2007). In their Experiment 2, Van Zoest et al.(2007) used a gaze-contingent target presentation, in which thetarget was presented close to observers’ eye gaze after an inter-ruption. The authors did not observe rapid resumption after asingle contingent look. Instead, visual search was quickly resumedif the target was presented in the same location when the displayreappeared one second later, suggesting that at least two successivepresentations are needed for rapid resumption to occur. This result
1 We will refer to the 100-ms display presentation as a glance or lookand to the duration between one glance and the next as an epoch. Here,epochs last 1 s total.
Figure 1. Schematic representation of the procedure. Across looks, target(“T”) appeared in either (a) one of the distractor locations and never movedor (b) one of the selected locations indicated by dashed circles (notdisplayed in the actual experimental display) and shifted among thoselocations, following the pattern described in the text.
Thi
sdo
cum
ent
isco
pyri
ghte
dby
the
Am
eric
anPs
ycho
logi
cal
Ass
ocia
tion
oron
eof
itsal
lied
publ
ishe
rs.
Thi
sar
ticle
isin
tend
edso
lely
for
the
pers
onal
use
ofth
ein
divi
dual
user
and
isno
tto
bedi
ssem
inat
edbr
oadl
y.
1373PREDICTIVE MINDSET IN PERCEPTION
is taken as evidence that rapid resumption does not depend onpassive accumulation of evidence, but that memory of the previousdisplay and integration of that information with the current displayare necessary to guide perceptual processing. It is this integrationof information gathered from a previous look (which produced aperceptual hypothesis) and information from the current look(which allows for the hypothesis to be confirmed) that allow visualsearch to be quickly resumed after an interruption.
Previous studies have shown that unexpected changes in thedisplay can disrupt the rapid resumption phenomenon (e.g.,Jungé, Brady, & Chun, 2009; Lleras, Rensink, & Enns, 2007).On the one hand, these findings confirm the existence of theperceptual cycle because they offer direct evidence that visualsearch is facilitated only when the perceptual hypotheses can beconfirmed. Indeed, when the prediction cannot be confirmed—that is, when the ongoing stimulation changes before the hy-pothesis is confirmed—a new cycle is necessary to complete thevisual search. On the other hand, they also reveal that percep-tual hypotheses appear to be quite inflexible. Perhaps rapidresumption is more of an automatic stimulus-driven phenome-non, which is completed without much intervention from higher(more executive-level) cognitive functions. In fact, comparingthe display’s features before and after the interruption can bedone without necessarily involving goal-oriented processes.Indeed, the role a person’s goals and intentions on early atten-tional processes is still currently debated (see Lamy &Kristjánsson, 2013, for a recent review). One view suggests thatboth goal-driven and stimulus-driven factors contribute to guidevisual search (e.g., Wolfe, Butcher, Lee, & Hyle, 2003) whilethe opposite view still argues that attentional selection is com-pletely driven by external characteristics of the stimuli (e.g.,Theeuwes, 2010). So it is still not clear whether goal-directedbehavior can affect early attentional processes.
However, there are reasons to believe that the visual systemis able to efficiently incorporate fast changes of the visualscenes to make predictions about future events. Most visualstimuli in real life do, indeed, move in space and time; yet,observers are usually able to quickly integrate informationregarding past and present to infer something about the future.If the visual system were not able to incorporate rapid changesof the visual stimulus into goal-directed behavior, commondaily tasks such as crossing the street would be impossible.Here we show that not only does rapid resumption rely onmemory for previous displays, but that it also adapts to changesin the display, bringing new supporting evidence for goal-oriented modulation of early, attentional processes.
Therefore, the aim of this study was to test whether theperception cycle can incorporate predictable changes in a visualdisplay. In Experiments 1a and 1b we manipulated the locationof the target between looks at a visual search display—varyingwhether changes in location followed a predictable pattern—and we observed rapid resumption when predictable changesoccurred. In Experiment 1c we found that constraining thetarget’s path to a predictable sequence of locations did notfurther facilitate rapid resumption, suggesting that prediction isbased on a coarse representation of the search display. Exper-iment 2 more closely compared the predictable and unpredict-able changes within the same experiment and tested partici-pants’ knowledge of the target sequence change. Lastly,
Experiment 3 suggested that information sources other thanlocation could guide prediction, by showing rapid resumptionwhen the target’s shape changed predictably.
Experiment 1: Changes of the Target’s Location
Experiment 1a
Rapid resumption is disrupted or at least reduced when changesin the display occur between looks (Jungé et al., 2009; Lleras et al.,2005; Lleras et al., 2005). Lleras et al. (2005) first observed thedisruption of rapid resumption after shuffling the identity (i.e.,orientation) of all items in the display, during an interruption.Later, Lleras et al. (2007; Experiment 2) selectively shuffled thelocation of the distractors while leaving the target untouched, andobserved a small reduction of rapid resumption. This result wasconfirmed in a separate study by Jungé, Brady, and Chun (2009),who also observed a significant reduction of fast RT after shufflingthe locations of a subset of distractors between looks, but onlywhen changes involved items located nearby the target.
Critically, in all these studies the display changes were unpre-dictable (Jungé et al., 2009; Lleras et al., 2005, 2007). In natural-istic settings many changes are predictable and entirely unpredict-able changes may in fact be quite rare. Because of inertia, movingobjects often change location and direction smoothly; this renderstheir changes in position predictable. For actions by humans andother intentional agents, constraints from goals render visualchanges yet more predictable. It would therefore be sensible for thevisual system to incorporate predictable changes into its perceptualhypotheses, taking advantage of the dynamics of the perceptualenvironment to guide perceptual processing. People’s ability tointegrate predictable changes into perceptual hypotheses and usethem advantageously in a visual search task would support the ideathat rapid resumption arises when observers are actively investedin an anticipatory strategy. Here we aim to evaluate whetherprediction can be used to modify perceptual hypotheses in situa-tions where the visual display changes between looks.
In Experiment 1a, the target changed locations between consec-utive looks and could appear in a limited subset of locations. Inorder to enable observers to predict the forthcoming change, thetarget moved through a fixed sequence of five target locationsthroughout the trial and throughout the experiment. We comparedthis condition with a condition in which the target never movedfrom the location where it first appeared, which served as abaseline to evaluate rapid resumption. If observers can shape theongoing perceptual hypotheses, rapid resumption should be pre-served despite changes in the visual display as long as thosechanges are predictable. If the ability to anticipate forthcomingevents is inflexible and only depends on low-level comparisonbetween visual stimuli, we expect rapid resumption to be disruptedafter a change in the display.
Method.Participants. Seventeen volunteers (Mage � 20.7 years, SD �
1.8) participated in the experiment. All participants had normal orcorrected-to-normal vision. They signed a consent form before theexperiment and they were compensated $8 or one psychologycourse credit for their participation.
Apparatus and stimuli. All stimuli (see Figure 1) were blackand they were presented on white background, using a 21” color
Thi
sdo
cum
ent
isco
pyri
ghte
dby
the
Am
eric
anPs
ycho
logi
cal
Ass
ocia
tion
oron
eof
itsal
lied
publ
ishe
rs.
Thi
sar
ticle
isin
tend
edso
lely
for
the
pers
onal
use
ofth
ein
divi
dual
user
and
isno
tto
bedi
ssem
inat
edbr
oadl
y.
1374 MEREU, ZACKS, KURBY, AND LLERAS
CRT monitor running at 85 Hz. The experiment was programmedusing Psychophysics Toolbox 2.54 (Brainard, 1997). Responseswere gathered through a keyboard placed on a table in front of theparticipants. The target was a “T” and the distractors were “L”both occupying an area of 0.6° � 0.6° of visual angle. The targetcould be tilted rightward or leftward; the distractors were dis-played in all four possible orientations (rightward, leftward, up-ward, and downward).
The area was divided into 36 virtual locations of 2.3° � 2.3°each. On each trial there was a target and 15 distractors scatteredin an area of 15° visual angle, each randomly located in one of the36 possible locations. A jitter (�1° visual angle) was randomlyadded to each item to avoid collinear alignment.
Procedure. Participants sat on a comfortable chair in a dimlylighted, air-conditioned room. Head position was stabilized with achin rest. Participants read the instructions as they appeared on thedisplay and the experimenter answered questions as they arose.Participants were instructed to report the target orientation bypressing the right arrow if the “T” was tilted toward the right or theleft arrow if the “T” was tilted leftward. Both speed and accuracywere emphasized in the instructions. After the instructions, partic-ipants were presented with a display showing the “T” orientedrightward and a second display showing the “T” oriented leftward.Each display disappeared only after the observer pressed thecorrect key on the keyboard.
Each trial started with the fixation point. After 2,500 ms thesearch display appeared for 100 ms; then, it was replaced by theblank to reappear after 900 ms. The search display/blank sequencewas repeated up to 12 times while the observer produced a re-sponse.
There were two types of trials, which were intermixed: Changeand No-Change trials. On Change trials, the target appeared in oneof the five locations indicated by the dashed circles2 (Figure 1b)and distractors occupied the remaining four chosen locations. Theremaining 11 distractors were randomly placed in one of the other31 possible locations (set size � 16). On any given Change trial,the target “moved” across successive glances, in a clockwisemanner going from one of the five Change locations to the next,and replacing the distractor that had previously occupied thatlocation (see Figure 2a). Once the target left a location, a newdistractor replaced it at that location. This procedure was done tomaintain the overall spatial context of the trial and avoid attentionbeing drawn to locations either by the unexpected appearance ordisappearance of an object (e.g., Theeuwes, 1991). In half of thetrials the “T” was tilted rightward and in the other half of the trialsthe “T” was tilted leftward. Crucially, the “T” did not changeorientation throughout the trial.
In the No-Change trials the target was randomly presented inone of the 31 remaining locations (indicated by the white squaresin Figure 2), and during a trial, it always reappeared at the samelocation after each blank. To avoid contextual cueing3 (Chun,2000), a distractor was placed in each of the five Change locations(gray squares in Figure 2) in the Change as well as in the No-Change trials; so that the subset of five locations selected for thetarget to Change was always occupied by one target and fourdistractors (Change trials) or five distractors (No-Change trials).
Participants completed 16 practice trials, in which only No-Change trials were presented, and then five experimental blocks of
80 trials each. They were instructed to take a small break betweenthe blocks. The experiment lasted for about 40 min.
Results. Two participants were excluded from the analysisbecause of high error rates (� 2 SD above the group mean). Trialsin which the observer did not accurately report the target orienta-tion (5.2%) and RT longer than 10,000 ms (4.1%) were consideredas errors and excluded from the analysis.
Figure 3 shows the normalized distribution of correct responsesfor the first and the following epochs separately, across both typesof trials. For the Change trials, the distribution of RT in the firstepoch (following the first display presentation) was significantlydifferent than the distribution following all the subsequent looks(Epochs 2–9), �2(9) � 328.05, p � .001, Cramer’s V � 0.35(comparison of RT distribution across 10 100-ms bins), as well asfor the No-Change trials, �2(9) � 368.05, p � .001, Cramer’s V �0.36.
The observers’ response pattern during the first display presen-tation did not substantially change across the two types of trials,�2(9) � 6.99, p � .5, Cramer’s V � 0.07. Nevertheless, responsedistributions after the second display presentation differed acrosstrial type, �2(9) � 181.94, p � .001, Cramer’s V � 0.22.
Fast RT were less frequent in the Change than in the No-Changetrials, as revealed by the 2 (Epoch: First, Later) � 2 (Trial Type:Change, No-Change) ANOVA conducted on the proportion of RTfaster than 500 ms.4 The ANOVA showed, as expected, an effectof epoch, F(1, 14) � 138.12, p � .001.
The proportion of fast RT in the first epoch (M � 14.5; SE �3.31) was significantly lower than in the later epochs (M � 53.6;SE � 2.44). There was also an effect of Trial Type, F(1, 14) �8.78, p � .01, with a larger proportion of fast RT in the No-Change(M � 36.2; SE � 2.54) than in the Change (M � 31.9; SE � 2.43)trials; but more important the interaction between trial type andepoch was significant, F(1, 14) � 9.99, p � .01. Post hoc tests ofthe means revealed larger proportion of fast RT in the No-Changethan in the Change trials, but only in Epochs 2–9 (p � .01).
Discussion. The main result observed in the first experiment isthat rapid resumption was preserved when the display changedbetween looks. There was the usual increase of fast (� 500 ms)correct responses observed in Epochs 2–9, and we observed thebimodal distribution that is typical of the rapid resumption phe-nomenon. This result indicates that changes to the search scene canbe incorporated into the perceptual cycle to facilitate rapid vision.There was, however, one odd aspect to the results: whereas weobserved an increase in fast responses in the No-Change condition(indicative of rapid resumption), the bimodal form of the RTdistribution was not evident. This pattern of results is furtherdiscussed below.
2 The dashed circles were not actually displayed during the experimentand are reported here in order to distinguish the Change location from theother locations.
3 Contextual cueing refers to the finding that visual search is facilitatedin displays that were previously presented during the experiment. Thisfinding also extends to small, repeated configurations of stimuli within alarger search array. Thus, we wanted to avoid the presence of items in the5 Change locations to cue observers to the fact that the current trialcontained a target within one of those 5 locations.
4 Such cut-off may seem arbitrary, but it has been previously shown tobe a reliable indicator of the rapid resumption phenomenon (e.g., Jungé etal., 2009).
Thi
sdo
cum
ent
isco
pyri
ghte
dby
the
Am
eric
anPs
ycho
logi
cal
Ass
ocia
tion
oron
eof
itsal
lied
publ
ishe
rs.
Thi
sar
ticle
isin
tend
edso
lely
for
the
pers
onal
use
ofth
ein
divi
dual
user
and
isno
tto
bedi
ssem
inat
edbr
oadl
y.
1375PREDICTIVE MINDSET IN PERCEPTION
Previous studies failed to observe rapid resumption when thedisplay changed between looks (Jungé et al., 2009; Lleras et al.,2005, 2007; Van Zoest et al., 2007). However, it has to be notedthat in all those studies, changes of the display occurred unexpect-edly, randomly and therefore, unpredictably. In sharp contrast, inthis experiment changes were predictable. The target locationschanged between looks, but it was limited to a subset of locations,therefore making the target’s location easy to anticipate. Hence,preprocessing of the target could be combined with online predic-tions regarding possible future target locations to create a dynamicperceptual hypothesis.
In trials where the target’s location did not change, the percent-age of fast RT was higher than that of trials in which the target’slocation changed between display presentations.
Response patterns after the second and following looks alsodiffered across the two types of trial. Visual inspection of thedistributions of RT (see Figure 3) suggests that RT in the Changetrials are bimodally distributed, and the distribution of RT in theNo-Change trials is unusually broad and unimodal, a result thatseems at odds with the results that are typically found in inter-
rupted search tasks. This result was surprising to us because theno-change condition was basically a replication of prior rapidresumption experiments (see Lleras et al., 2005), when no changesto the display are implemented. That said, the observation of aclear and significant proportion of fast RT (faster than 500 ms) inthe No-Change condition is evidence that rapid resumption was atplay. The fact that two peaks in the distribution were not easilyidentifiable implies that there was some temporal smearing in thelocation of the first peak that was likely caused by the presence ofthe intermixed Change trials. Difference in the salience betweenthe two types of trials could explain this unexpected finding.Because the participants noted the target “moving” they mighthave been expecting a change of target’s location on each singletrial, especially when the target appeared within the range of theChange (moving) locations. This hypothesis was directly tested inExperiment 1b.
Finally, as expected, no significant difference was observedbetween the response patterns of Change and No-Change trials inthe first look. This result is important because it argues against thepossibility that differences in rapid resumption rates across the
Figure 2. Simplified structure of the search display. The solid lines delimitate the 36 possible locations for anitem to occur. The gray squares indicate a selected location for the target to appear in the Change trials acrossthe experiments. The arrows in the displays of (a) Experiment 1a; (b) Experiment 1b, indicate the location wherethe target would appear when presented in each of the five locations; and (c) Experiment 1c, the target couldappear on any of the five locations on any given presentation, except that it was never presented on the samelocation for two subsequent looks. The dashed perimeter indicates the central locations (within 10° eccentricity)used in one of the analysis of Experiment 1b.
Figure 3. Normalized distribution of correct responses in Experiment 1a, where the target was presentedin subsequent looks in a nearby location, following a circular pattern around the fixation point. The graphto the left shows the distribution of RT following the first display presentation and the graph to the rightshows the distribution following all following looks (Epochs 2–9). The dashed line shows the distributionof RT in the Change trials and the solid line shows the distribution in the No-Change trials.
Thi
sdo
cum
ent
isco
pyri
ghte
dby
the
Am
eric
anPs
ycho
logi
cal
Ass
ocia
tion
oron
eof
itsal
lied
publ
ishe
rs.
Thi
sar
ticle
isin
tend
edso
lely
for
the
pers
onal
use
ofth
ein
divi
dual
user
and
isno
tto
bedi
ssem
inat
edbr
oadl
y.
1376 MEREU, ZACKS, KURBY, AND LLERAS
Change and No-Change displays were somehow due to incidentaldifferences in difficulty between Change and No-Change displays.For example, one could argue that Change trials were easier thanNo-Change trials. It is the case that target locations in the Changecondition were overall more eccentric than No-Change targetlocations (within 10° eccentricity). It is also possible that observersmay have a central bias, or that monitoring five locations (as in theChange condition) was easier than monitoring 11 locations (asneeded in the No-Change condition). However, the proportion offast RT on No-Change trials was higher than on Change trials. IfChange trials were easier, one would have expected the oppositeresult: a larger proportion of fast RT in the Change (easier)condition.
In sum, results obtained in this first experiment strongly supportthe idea that changes of the display can be included in the percep-tual hypotheses, suggesting that prediction is involved in rapidresumption and, more in general in perception. Indeed, rapidresumption was not disrupted by changes in the display. Also, ourresults suggest that early, fast, nonconscious perceptual processescan be affected by high-level knowledge and goal-oriented strat-egies, actively updated on the basis of memory for previous visualevents.
Experiment 1b
In Experiment 1a we showed that rapid resumption occurredeven when the target’s location changed from one look to the next.Thus, changes can be incorporated in the perceptual hypothesis.Furthermore, the previous experiment added evidence to the ideathat rapid resumption is not the result of passive accumulation ofevidence, confirming the role of prediction in rapid resumption.However, the pattern of locations (circular) chosen for the target toappear after each interruption was particularly noticeable and wewanted to confirm the results with a different pattern of displace-ment in the Change trials. Thus, we ran an experiment very similarto the previous one, with the only difference that the target dis-placement did not take place clockwise. Instead, the target reap-peared in a nearby location, following a noncircular configuration(zig-zag). If the results obtained in Experiment 1a were not due tothe specific pattern we chose for the target’s location change, weexpect to replicate them using a different target location’s changeconfiguration.
Another aim of Experiment 1b was to examine more closely thepattern of RT in the No-Change trials, which was unimodallydistributed in Experiment 1a, contrary to what observed in previ-ous studies that used comparable tasks (e.g., Lleras et al., 2005).One possibility is that observers virtually “divided” the visualdisplay in two broad sectors: a more central one, in which thetargets often moved, and a more peripheral one, in which targetsnever moved. It is likely in fact that the area surrounding theChange location was too broad (10° eccentricity) to allow a fine-grained spatial resolution, which included all the items’ locations.Jungé et al. (2009) showed that only the area immediately close tothe target has such a fine-grained spatial resolution, yet the areasdistant from the target do not. They observed that, changing thelocations of distractors close to the target interferes with rapidresumption, while changing the locations of distractors distantfrom the target does not. In our study, if a search strategy based ona fine-grained spatial resolution was not possible, a strategy based
on coarse subdivisions of the search display might have been moreefficient. If details were not available, they could not obviously beanticipated to predicting forthcoming changes.
We hypothesized that the observer expected the target to moveon each glimpse of the display, when the target appeared roughlyin the area within the Change location eccentricity. If observerscreated a first perceptual hypothesis based on the prediction thatthe target’s location was going to change, such a hypothesis cannotbe confirmed if the target reappears in the same location after theinterruption. Then, another hypothesis has to be created and con-firmed, based on the prediction that the target is not going to move.According to previous studies (e.g., Van Zoest et al., 2007) weshould observe the early phase of RT to arise only after the secondlook, that is, after the target appears in the same location twice.
To test this hypothesis, we compared the RT distributions in theNo-Change trials after the first look with the RT distribution afterthe second and the third looks, considering only the location withinthe same eccentricity as the Change trials (10° visual angle).
Method.Participants. Eleven volunteers (Mage � 19.2 years, SD �
1.1) with the same characteristics as the previous experimentparticipated in the study.
Stimuli and procedure. All stimuli and the procedure wereidentical to the previous experiment, except for the following. Oneach trial, when the search display reappeared after an interruption,the target was presented in one of the remaining four locations,following the pattern depicted in Figure 2b.
Results. Two participants were excluded from the analysisbecause of high error rates (� 2 SD above the group mean). Trialsin which the observer did not accurately report the target orienta-tion (3.2%) and RT longer than 10,000 ms (3.5%) were consideredas errors and excluded from the analysis.
Figure 4 shows the normalized distribution of RT for correctresponses across both trial types for the first and following epochsseparately. RT distributions changed from the first epoch to thelater epochs; this was significant both in the Change trials, �2(9) �175.92, p � .001, Cramer’s V � 0.33, as well as in the No-Changetrials, �2(9) � 206.07, p � .001. Also, the distribution of RT wasdifferent across the two types of trial, in both the first epoch, �2(9)20.81, p � .05, Cramer’s V � 0.16, and the later epochs, �2(9) �90.94, p � .001, Cramer’s V � 0.21.
Again, we conducted a 2 (Epoch: First, Later) � 2 (Trial Type:Change, No-Change) ANOVA on the proportion of RT faster than500 ms. The ANOVA revealed, as expected, an effect of epoch,F(1, 8) � 96.77, p � .001. The proportion of fast RT in the firstepoch (M � 16.9; SE � 4.36) was significantly lower than in thelater epochs (M � 47.3; SE � 3.86). The proportion of fast RT washigher in the Change trials, but the main effect of Trial Type didnot reach statistical significance, F(1, 8) � 4.11, p � .08. Moreimportant, the interaction between epoch and trial type was sig-nificant, F(1, 8) � 5.42, p � .05. Post hoc tests of the meansrevealed a larger proportion of fast RT in the Change than in theNo-Change trials, and such difference was observed only in thefirst epoch (p � .05).
Next, we looked at the normalized distribution of RT, excludingresponses given for targets presented in the peripheral (more than10° eccentricity) locations. Figure 5 shows the distribution of RTfor target presented in the central locations (eccentricity within10°; Figure 2b), after the first, second, and following looks sepa-
Thi
sdo
cum
ent
isco
pyri
ghte
dby
the
Am
eric
anPs
ycho
logi
cal
Ass
ocia
tion
oron
eof
itsal
lied
publ
ishe
rs.
Thi
sar
ticle
isin
tend
edso
lely
for
the
pers
onal
use
ofth
ein
divi
dual
user
and
isno
tto
bedi
ssem
inat
edbr
oadl
y.
1377PREDICTIVE MINDSET IN PERCEPTION
rately. Results are summarized in Table 1. In the No-Change trialsthe distribution of RT in the first epoch differed from the secondone (ps � .001), but more important, the RT distribution of thesecond epoch differed from the third one (p � .05). As for theChange trials, the distribution of RT in the first epoch differedfrom that of the second and the third (ps � .001), but the distri-bution of the second epoch did not differ from the third one (p �.05).
Discussion. The results of this second experiment replicatedthe finding that rapid resumption can be obtained after the changesof the target’s location across looks. Thus, both experimentsstrongly support the idea a change can be incorporated in theperceptual hypotheses, if such change is in-line with the observer’sexpectations.
Both Experiment 1a and 1b also revealed an interesting, unex-pected result. Contrary to the Change trials, when the target waspresented in the same location after an interruption, the distributionof RT after the first epoch was unimodal. As mentioned earlier,this result was surprising because the bimodal distribution istypically observed in experiment using comparable conditions(e.g., Lleras et al., 2005). Also, contrary to Experiment 1a, duringthe first epoch the amount of fast RTs in the Change condition waslarger than in the No-Change condition, confirming that observersmight have been biased to attend to the central locations more thanto the peripheral locations. However, it is worth noting here thatneither the previous nor the following experiments replicated thispattern of results; in fact, except for Experiment 1b, the percentageof fast RT in the No-Change trials was always higher than in theChange trials.
To better understand these results, we looked at the distributionof RT in the No-Change trials, limiting the analysis to the re-sponses given for targets presented within the same eccentricity asthe Change trials—that is, within 10° from fixation. It is possibleindeed that trials in which the target changed were more salientthan those where the target did not change, encouraging observersto restrict their focus of attention on locations closer to the centerand ignoring locations closer to the periphery. Results (see Figure5) revealed that, after restricting the analysis to the central loca-tions, rapid resumption occurred even when the target’s locationdid not change across looks. Furthermore, the observer’s distribu-
tion of RT after one glance was substantially different from the oneobtained after both the second and the following glances. In thethird epoch, the early peak of RT occurred earlier if compared withthe second, and to the first. Moreover, the distribution of RTcollected in the Epochs 3–9 was virtually identical between theChange and No-Change trials, suggesting the same response pat-tern in the two conditions, after the display was presented twice.This result is similar to the one reported in Van Zoest et al. (2007),in which rapid resumption was not observed until the target waspresented twice in the same location.
In sum, the results of the Experiments 1a and 1b of the presentstudy showed that the first, implicit stage of the perception cyclecould be dynamically adapted to ongoing display’s changes.
Experiment 1c
So far, we have seen that a goal-directed strategy could beused to facilitate visual search. An intriguing possibility is thatpredictable movement patterns allow the perceiver to anticipatethe exact location of the target on the subsequent look, andparticipants used this fine-grained location information to im-prove search performance. Experiments 1a and 1b support theidea that observers use a predictive strategy to maximize theefficiency of perception, by showing that changed that can beanticipated preserve rapid resumption. However, they indirectlysuggest the possibility that observers were not anticipating theexact location of the target on the subsequent look, but simplyrestricting the visual search to a subset of central locations afterthe second look (see Figure 5).
In Experiment 1c we wanted to look at the specific strategyused by participants for anticipating aspects of the visual dis-play. It is possible that observers were able to memorize (ex-plicitly or implicitly) the sequence of locations and to use suchinformation to precisely predict the forthcoming target’s loca-tion. This would reveal a fine-grained representation of a largeportion of the visual display. However, it seems unlikely thatviewers can create such a detailed representation of the sceneafter one glance. Furthermore, both theory (e.g., Di Lollo et al.,2000) and experimental evidence (e.g., Jungé et al., 2009)suggest that this might not be the case.
Figure 4. Normalized distribution of correct responses in Experiment 1b, where the target was presented insubsequent looks in a nearby location, following a zig-zag pattern. The graph to the left shows the distributionof RT following the first display presentation and the graph to the right shows the distribution following allfollowing looks (Epochs 2–9). The dashed line shows the distribution of RT in the Change trials and the solidline shows the distribution in the No-Change trials.
Thi
sdo
cum
ent
isco
pyri
ghte
dby
the
Am
eric
anPs
ycho
logi
cal
Ass
ocia
tion
oron
eof
itsal
lied
publ
ishe
rs.
Thi
sar
ticle
isin
tend
edso
lely
for
the
pers
onal
use
ofth
ein
divi
dual
user
and
isno
tto
bedi
ssem
inat
edbr
oadl
y.
1378 MEREU, ZACKS, KURBY, AND LLERAS
Another possibility is that, after one glance, observers areable to create a coarse representation of the area where thetarget was most likely to appear, to subsequently restrict thesearch on the basis of some rough information about the dis-play. To test this hypothesis, we conducted an experiment withthe exact same design as in Experiment 1a and Experiment 1b,
with the only difference that, in the Change trials, after eachinterruption the target reappeared in a randomly selected loca-tion, instead of following a predetermined pattern.
If the ability to rapidly resume a visual search when changesoccurred during an interruption depends on a detailed prediction ofthe forthcoming target’s location, we expect rapid resumption to bedisrupted—or at least decreased—if such location is not preciselypredictable. If rapid resumption relies on a coarse representation ofthe target area, we expect it to be preserved, despite uncertaintyregarding the forthcoming change.
Method.Participants. Seventeen volunteers (Mage � 19.2 years, SD �
1.1), with the same characteristics as the previous experimentparticipated in the study.
Stimuli and procedure. All stimuli and the procedure wereidentical to the previous experiment, except for the following. Oneach trial, when the search display reappeared after an interruptionin the Change trials, the target was presented in one of theremaining four locations, randomly selected from the subset de-picted in Figure 2c. On each trial, the target had the same proba-bility to appear in any of the five selected locations, except for theone where it was prior the interruption.
Results. One participant was excluded from the analysis be-cause of the large error rate (� 2 SD above the group average).Trials in which the observer did not accurately report the targetorientation (3.6%) and RT longer than 10,000 ms (1.7%) wereconsidered as errors and excluded from the analysis.
Figure 6 shows the normalized distribution of RT for correctresponses across both trial types. Observers responded in the firstepoch significantly different from the later epochs in both theChange trials, �2(9) � 448.41, p � .001, Cramer’s V � 0.39, andthe No-Change trials, �2(9) � 502.75, p � .001, Cramer’s V �0.40. The distribution in the first epoch �2(6) � 2.72, p � .8, wasnot different across type of trials. On the contrary, the distributionafter the later epochs differed across trial types, �2(9) � 211.13,p � .001, Cramer’s V � 0.22.
The 2 (Epoch: First, Later) � 2 (Trial Type: Change, No-Change) ANOVA conducted on the proportion of RT faster than500 ms revealed, again, an effect of epoch, F(1, 15) � 302.94, p �.001. The percentage of fast RT in the first epoch (M � 18.8; SE �2.19) was significantly lower than that of the later epochs (M �55.62; SE � 1.63). There was also an effect of the trial type, F(1,15) � 47.21, p � .001, with a larger proportion of fast RT in theNo-Change (M � 42.65; SE � 2.04) than in the Change (M �31.7; SE � 2.19) trials; but more important the interaction wassignificant, F(1, 15) � 50.42, p � .001. Post hoc tests of the meansrevealed larger proportion of fast RT in the No-Change than in theChange trials, but only in the Epochs 2–9 (p � .001).
Discussion. The results of this experiment are straightforward:Rapid resumption was still observed even when precise informa-tion regarding the forthcoming target’s location was not available.Observers seem to be able to create a coarse representation of thearea surrounding the target and to use that information to make aprobabilistic inference on the target’s location, when such infer-ence is needed. Such conclusion is in accordance with models ofvisual perception that hypothesize a feedforward first sweep ofactivation as responsible for the creation of a nondetailed repre-sentation of the visual scene (e.g., Enns & Lleras, 2008). It is alsoin line with previous studies showing that a detailed spatial rep-
Figure 5. Normalized distribution of correct responses in Experiment 1b,considering only the central locations (10° eccentricity). The top panelrepresents the distribution of RT following the first display presentation,the middle panel represents the distribution after the second display pre-sentation and the bottom panel represents the distribution following allfollowing looks (Epochs 3–9). The dashed line shows the distribution ofRT in the Change trials and the solid line shows the distribution in theNo-Change trials.
Thi
sdo
cum
ent
isco
pyri
ghte
dby
the
Am
eric
anPs
ycho
logi
cal
Ass
ocia
tion
oron
eof
itsal
lied
publ
ishe
rs.
Thi
sar
ticle
isin
tend
edso
lely
for
the
pers
onal
use
ofth
ein
divi
dual
user
and
isno
tto
bedi
ssem
inat
edbr
oadl
y.
1379PREDICTIVE MINDSET IN PERCEPTION
resentation is restricted to the area closely surrounding the target(Jungé et al., 2009). However, this experiment does not excludethat a small, but reliable decrease of rapid resumption occurred inthe previous experiments. As previously noted, it is difficult tocompare different populations of subjects because rapid resump-tion varies substantially across observers (see Jungé et al., 2009).We thus ran another experiment, to evaluate whether rapid re-sumption decreased after introducing uncertainty regarding thepossible target’s location after each interruption.
Experiment 2: Implicit Sequence Learning
Experiment 1b revealed that observers roughly divided thesearch display in two regions: central and peripheral. It seemedlike if each time the target appeared in the most central area,observers expected it to appear in a different location after aninterruption. This result suggests that observers were able to inte-grate this prediction into early stages of the visual search. A coreassumption of the perception cycle hypothesis (Enns & Lleras,2008) is the implicit nature of predictions. It is generally acceptedthat perceptual hypotheses are created through the first feedfor-ward sweep of neural activity, triggered by the stimulus onset at animplicit level and therefore, outside of awareness (e.g., Enns &Lleras, 2008; Jungé et al., 2009; Lleras, Rensink & Enns, 2007).There are two possibilities. First, it is possible that observerscreated an implicit attentional setting, which drove their attention,consequently facilitating the resumption of previously interrupted
visual search. That contingencies between cues and targets can belearned implicitly has been demonstrated in other visual domains(e.g., Lambert, Naikar, McLachlan, & Aitken, 1999). Alterna-tively, it is possible that participants in our Experiment 1 devel-oped an explicit strategy, based upon the trials in which theyhappened to see the target in two or more consecutive glimpses.The interrupted search task fails to distinguish between these twopossibilities, even in the original study (Lleras et al., 2005)—where the search display remained untouched between two subse-quent looks. In Lleras et al. (2005), observers most likely assumedthat the display remained unchanged across looks, even in theimplausible circumstance that they never noticed the target in twosubsequent presentations. Thus, implicit perceptual hypotheses inrapid resumption could be the result of explicit conjectures aboutthe visual display. In support of this possibility we observed that,in a postexperimental questionnaire, all the participants reportedthat they had noticed some changes of the display occurring insome of the trials. Thus, observers may have developed an explicitstrategy based on their beliefs about the target’s locations, tofacilitate visual search.
Can rapid resumption be observed when the forthcoming changeis not explicitly known? To evaluate this possibility, we conductedan experiment using the same locations as Experiment 1b (Figure2b) and intermixing trials with predictable and unpredictablechanges into the experiment. The procedure was intended to dis-courage participants from using explicit visual search strategies,
Table 1Summary of the Analysis in Experiment 1b (Zig-Zag), Comparing Across the First, Second, and Third Epochs in Both, Change, andNo-Change Trials
Epochs
Trial TypeThirdEpochChange No-Change
First vs. second First vs. third Second vs. third First vs. second First vs. third Second vs. thirdChange vs.No-Change
�2 188.92�� 147.74�� 16.49 100.22�� 60.08�� 20.86� 15.32Cramer’s V 0.45 0.34 0.13 0.58 0.47 0.26 0.13
� p � .05. �� p � .001.
Figure 6. Normalized distribution of correct responses in Experiment 1c, where the target was randomlypresented in subsequent looks in one of the remaining Change location, so that on each look the possibility forthe target to appear in each of those locations was one out of four. The graph to the left shows the distributionof RT following the first display presentation and the graph to the right shows the distribution following allfollowing looks (Epochs 2–9). The dashed line shows the distribution of RT in the Change trials and the solidline shows the distribution in the No-Change trials.
Thi
sdo
cum
ent
isco
pyri
ghte
dby
the
Am
eric
anPs
ycho
logi
cal
Ass
ocia
tion
oron
eof
itsal
lied
publ
ishe
rs.
Thi
sar
ticle
isin
tend
edso
lely
for
the
pers
onal
use
ofth
ein
divi
dual
user
and
isno
tto
bedi
ssem
inat
edbr
oadl
y.
1380 MEREU, ZACKS, KURBY, AND LLERAS
such that even if observers were aware of display’s changes, theyremained unaware that some of the changes followed a repetitivestructure. In addition, the procedure allowed us to closely comparepredictable and unpredictable trials within the same experiment, toevaluate whether introducing uncertainty regarding the possibletarget’s location after each interruption reduced the amount ofrapid resumption.
Hence, Experiment 2 aimed to evaluate the possibility thatimplicit sequence learning can facilitate an interrupted visualsearch, even if the observer is not engaged in an active searchstrategy.
MethodParticipants.
Seventeen volunteers (Mage � 19.2 years, SD � 1.1), with thesame characteristics as the previous experiment participated in thestudy.Procedure.
The stimuli were identical to those of the previous experiments.The procedure was very similar to the previous experiment withthe only exception that there were three types of trial, which wereintermixed: 50% (160) were No-Change, 25% (80) were Predict-able Change, and 25% (80) were Unpredictable Change trials.Predictable Change trials were identical to Change trials in the inExperiment 1b (Figure 2b). Unpredictable Change trials wereidentical to Change trials in the in Experiment 1c (Figure 2c). Inaddition, immediately after the task, observers were presented witha postexperimental questionnaire containing four questions, inwhich they were asked: (a) whether the target moved betweenlooks, and (b) whether the target location sequence was random.Then, two target’s sequences were presented. The target appearedfor 500 ms on each of five locations, which could be (a) randomlyselected or (b) the same used in the Predictable Change trials.5 Thetarget was the only item in the display and the order in which thetwo sequences were presented was counterbalanced between sub-jects. After watching both sequences the observers were asked twomore questions: () which of the two sequences was used during theexperiment, and (d) to indicate their confidence on a 5-pointLikert-type scale (ranging from not confident at all to extremelyconfident). The answers were used to assess the participants’knowledge regarding the target’s locations sequence.
Results
Two participants were excluded from the analysis because ofhigh error rates. Trials in which the observer did not accuratelyreport the target orientation (12.3%) and RT longer than 10,000 ms(2.1%) were considered errors and excluded from the analysis.
Results obtained from the postexperimental questionnaire indi-cated that eight out of 15 observers correctly recognized thesequence used in the experiment. A binomial test revealed that thispercentage was not higher that expected by chance (p � 1.0).Except for one (confidence rating � 4), all the participants re-ported confidence ratings between 1 and 3 (M � 2.5, SD � 0.9).A logistic regression also revealed that observers’ confidence didnot predict their accuracy in indicating the right sequence (p � .5).
The analysis comparing the distributions in the first epoch withthe distributions of the later epochs revealed significant differencein all three conditions (all ps � .001). The distributions of RT inthe Predictable Change and Unpredictable Change trials were notsignificantly different �2(9) � 11.79, p � .2, Cramer’s V � 0.08;nonetheless, the No-Change distribution was significantly different
from both, the Predictable Change, �2(9) � 81.92, p � .001,Cramer’s V � 0.17, and the Unpredictable Change trials, �2(9) �89.11, p � .001, Cramer’s V � 0.17 (see Figure 7).6
To rule out the possibility that small differences in the amountof fast RT existed between the predictable change and the unpre-dictable change trials, we also performed a 2 (Epoch: First,Later) � 3 (Trial Type: No-Change, Predictable Change, Unpre-dictable Change) ANOVA on the proportion of RT faster than 500ms. The ANOVA revealed, as expected, an effect of epoch, F(1,14) � 56.82, p � .001. The proportion of fast RT in the first epoch(M � 24.7; SE � 3.42) was significantly lower than in the laterepochs (M � 51.9; SE � 3.71). The analysis also showed an effectof the trial type, F(2, 28) � 4.78, p � .05, mostly driven by alarger amount of fast RT in the No-Change condition (M � 41.98;SE � 2.17), with respect to both Predictable Change (M � 37.78;SE � 4.02) and Unpredictable Change (M � 36.17; SE � 3.44)conditions (both ps �.001). Most important, the interaction wasalso significant, F(2, 28) � 13.34, p � .001. Post hoc tests of themeans revealed that the amount of fast RT in the first epoch waslower than in the later epochs in all change conditions (ps � .001).Nonetheless, higher amounts of fast RT were only observed in theNo-Change trials (M � 60.41; SE � 2.82), if compared with bothPredictable Change (M � 47.24; SE � 4.79) and UnpredictableChange (M � 48.06; SE � 4.24) conditions (p � .01 and p � .001,respectively) and only in the later epochs (see Figure 8).
In sum, although evidence for rapid resumption was found in allthree conditions, it was larger for the No-Change condition.
Discussion
In Experiment 2, rapid resumption was again observed formoving targets. This result is in line with the previous experimentsin this study confirming that changes of the display do not neces-sarily disrupt rapid resumption. Critically, rapid resumption wasobserved in both predictable and unpredictable conditions, imply-ing that participants were unable to take advantage of the exactsequence of target locations.
The failure to find an advantage of the predictable change trialshelps to better understand the phenomenon of rapid resumptionand distinguish it from other visual phenomena. Studies on implicitsequence learning (for a review see Cleeremans, Destrebecqz, &Boyer, 1998; but see Shanks & St. John, 1994, for a differentperspective) suggest that long sequences of locations (more than10) can be learned implicitly. In an implicit sequence-learningtask, however, observers are typically exposed to many repetitionsof the entire sequence, before revealing learning through a de-crease of RT in response to a target appearing in the expectedlocation. Assuming that participants in our study did not intention-
5 The target was also presented in the same sequence as Experiment 1b.6 Despite the general lack of confidence, about half of the observers
correctly recognized the sequence used in the experiment. Indeed, some ofthese observers could have been somewhat aware of the target’s sequenceof locations. Thus, we compared the distribution of RT for observers whocorrectly guessed which sequence was used in the experiment (awaregroup) with the distribution of RT for observers who did not recognized theright sequence (unaware group). Neither the responses’ distribution of theaware group, �2(9) � 6.70, p � .7, Cramer’s V � 0.09, nor the responses’distribution of the unaware group, �2(9) � 11.16, p � .2, Cramer’s V �0.12, differed between Predictable and Unpredictable Change trials.
Thi
sdo
cum
ent
isco
pyri
ghte
dby
the
Am
eric
anPs
ycho
logi
cal
Ass
ocia
tion
oron
eof
itsal
lied
publ
ishe
rs.
Thi
sar
ticle
isin
tend
edso
lely
for
the
pers
onal
use
ofth
ein
divi
dual
user
and
isno
tto
bedi
ssem
inat
edbr
oadl
y.
1381PREDICTIVE MINDSET IN PERCEPTION
ally withhold their response voluntarily across multiple displaypresentations,7 the possibility exists that observers never saw theentire sequence at once. Previous studies (e.g., Lleras et al., 2005)suggested that observers might not consciously access the target,before they give a response. If this is true, it is also possible thatobservers never explicitly accessed any of the target’s location,preceding the one in which the target appeared when the observerresponded. Then again, Lambert et al. (1999, Experiment 3) ob-served effects of implicitly learned cue target associations withlow visibility, unconsciously perceived cues. Accordingly, in ourexperiment we should have observed increased rapid resumptionin the predictable trials, compared with the nonpredictable trials,even if the target was not consciously perceived in any of the looksthat preceded a correct response. However, contingency betweencues and targets in Lambert’s experiment was much simpler thanours, involving only two cue-target locations and only four possi-ble cue target combinations. The failure to observe increased rapidresumption in nonconsciously perceived complex sequencechanges does not exclude the possibility that the rapid resumptionphenomenon can arise in response to simpler, unconsciously per-ceived contingency between displays in subsequent looks.
The postexperimental questionnaires suggested that partici-pants were generally unaware of the predictability of the tar-get’s changes— or if they were, they failed to learn the se-quence of events. Interestingly, they also failed to incorporatepredictions about the target’s specific location into a perceptualhypothesis.
It is important to point out that this result does not argue againstthe idea that the display changes were shaping the perceptualhypotheses. In fact they were, even if the prediction was based ona coarse subdivision of the search display rather than on thespecific target locations. Knowing that the target can appear in oneof few possible locations likely allowed participants to update theirperceptual hypothesis and anticipate (i.e., predict) the upcomingtarget location with more confidence that they would have inabsence of such information. Indeed, restricting the search to fewlocations in a complex display represents a huge reduction inuncertainty on where the target will appear next. It is reasonable toassume that participants used such information to facilitate visualsearch.
In sum, even if some forms of implicit learning—such as inimplicit sequence learning (e.g., Destrebecqz & Cleeremans, 2001)or contextual cueing (Chun, 2000)—seem to be able to facilitatestimulus processing, in this situation implicit sequence learningeither did not occur, or if occurred, could not facilitate visualsearch. Instead, this experiment offers evidence in favor of coarselocation prediction (such as predicting that a very small subset oflocations will contain the target) and against precise locationprediction (such as predicting the exact sequence of targetchanges), which was likely impossible or at least very difficult toachieve in the context of our procedure.
Learning Across Experiments
An assumption underlying this study is that the advantagesobserved for predictable search displays were due to learning overrepeated trials. It is reasonable to assume that at the beginning ofthe experiment, during the first few blocks, knowledge about thoselocations was weak, and by the end of the experiment observerswere able to predict them more precisely. Thus, we decided to testthis possibility, by looking at the development of the rapid resump-tion phenomenon, across both trial types. We expected to observean increase of the rapid resumption rate throughout the experiment,but only in the Change trials. We ran a mixed 2 � 5 b� 3 ANOVAwith Trial Type (Change, No-Change) and Block (1–5) as within-subjects variables and Experiment (Experiment 1a, Experiment 1b,and Experiment 1c) as a between subjects factor, to see whetherthe proportion of fast RT increased throughout the experiment, asit is expected in any visual task. Specifically, we expected theproportion of fast RT to increase in the Change more than in theNo-Change trials, which would suggest that learning occurred.Only responses given after the second look (Epochs 2–9) wereincluded in the analysis. One participant from the Experiment 1acould not be included in this analysis because he or she alwaysresponded in the first epoch during the fifth block. TheGreenhouse-Geisser correction of the degrees of freedom was used
7 Most of the correct responses (over 80%) were given within the thirdlook.
Figure 7. Distribution of RT in Epochs 2–9 of Experiment 2, in all threeconditions.
Figure 8. Percentage of RT faster than 500 ms, as a function of the trialtype (No-Change, Predictable, and Unpredictable Change trials) in both thefirst and the later epochs.
Thi
sdo
cum
ent
isco
pyri
ghte
dby
the
Am
eric
anPs
ycho
logi
cal
Ass
ocia
tion
oron
eof
itsal
lied
publ
ishe
rs.
Thi
sar
ticle
isin
tend
edso
lely
for
the
pers
onal
use
ofth
ein
divi
dual
user
and
isno
tto
bedi
ssem
inat
edbr
oadl
y.
1382 MEREU, ZACKS, KURBY, AND LLERAS
when data violated the sphericity assumption. Table 2 shows themeans and standard error for Experiment 1a, 1b, and 1c across alltrial types.
Figure 9 shows the percentage of rapid resumption (RT fasterthan 500 ms) as a function of five blocks (80 trials each) in theexperiment. The ANOVA revealed an effect of Trial Type, F(1,36) � 14.99, p � .001, with the percentage of fast RT in theChange condition (M � 48.4; SE � 1.8) lower than the No-Change condition (M � 56.1; SE � 2.1). The effect of blockapproached significance, F(2.768, 99.66) � 2.54, p � .06,revealing the tendency to a linear trend, F(1, 36) � 3.67, p �.06.
The analyses also revealed a significant effect of experiment,F(2, 36) � 6.38, p � .01 and a significant interaction betweenexperiment and trial type, F(2, 36) � 4.71, p � .01, with alarger percentage of fast RT in the No-Change than in theChange trials for Experiment 1a (p � .01) and Experiment 1c(p � .001) conditions, but not for Experiment 1b.
More important the trial type by block interaction was signifi-cant, F(4, 144) � 3.38, p � .01. Post hoc analysis of the meansrevealed that, for the Change trials, the proportion of fast RTincreased significantly in the third and forth blocks with respect toboth, the first and second blocks (ps � .05); the percentage ofrapid resumption in the first block was also lower than in the fifthblock (p � .05). Such increase was not observed in the No-Changetrials. Post hoc tests also revealed that rapid resumption in theChange trials was smaller than in the No-Change trials in the first(p � .001), second (p � .001), third (p � .01), and fifth (p � .01)blocks, but not in the fourth. This result may reveal a qualitativedifference between the facilitation (fast RT) observed across trialtypes.
Experiment 3: Target Identity Change
So far, and surprisingly in light of previous studies (Jungé et al.,2009; Lleras et al., 2005; Lleras, Rensink, & Enns, 2007), weshowed that changes of the display failed to disrupt rapid resump-tion, if those changes were consistent with the observer’s expec-tations. As mentioned earlier in the article, however, a distinctiveproperty of the changes occurring in the displays of the previousstudies is randomness. If anticipation is driving the rapid resump-tion effect, the preservation of the ability to resume the visualsearch after the interruption in case of predictable changes shouldnot be surprising after all.
However, the results of Experiment 1 and 2 suggest that pre-dictability alone might not be sufficient to activate a predictivestrategy based on all the available information. Both, Experiment1b and Experiment 2 suggest that our participants might have beenusing a strategy based on a coarse representation of the visualsearch display, rather than using fine-grained prediction to locatethe target after the location has changed.
In support of these hypotheses, Lleras et al. (2005) showed thatrapid resumption is disrupted if the item’s identities are scrambledbetween looks, but it is preserved if the distractors’ locations arescrambled (Lleras et al., 2007). Jungé et al. (2009) delved deeperinto the issue and found that the decrease or rapid resumption afterchanges of the item’s locations is mainly due to the items thatclosely surround the target. These results reveal that perceptualhypotheses predominantly contain target’s features; contrary to thefeatural resolution of the target, the featural resolution of nearbysearch items is quite coarse. They also suggest that when relevantaspects of the display do not perfectly match the previous one,observers are not able to rapidly resume the interrupted visualsearch. It follows that irrelevant aspect of the search display mightbe ignored and might fail to be incorporated in the nonconsciousperceptual hypothesis. If that is the case, details regarding thechanges of the visual display that are not relevant for the currenttask/goal might also fail to be incorporated, despite being predict-able.
In all the experiments we presented so far, the participants’ goalwas to report the target’s orientation. Therefore, the target’s loca-
Table 2Mean (M) and Standard Error (SE) of the Amount of RapidResumption, Expressed in Percentage of RTs Faster Than500 ms
Experiment
Trial type Change No-Change
Block Mean Std. error Mean Std. error
Circular 1 38.662 3.762 49.362 3.4532 38.116 4.331 51.104 4.4463 46.247 3.829 56.045 4.3464 52.000 5.587 57.563 4.5125 48.051 5.116 53.342 4.390
Zig-zag 1 43.394 4.692 53.840 4.3062 44.653 5.402 49.513 5.5453 50.196 4.776 45.958 5.4204 48.511 6.968 38.950 5.6285 51.230 6.381 46.966 5.476
Random 1 45.637 3.519 65.609 3.2302 49.559 4.051 63.740 4.1593 56.611 3.582 72.172 4.0654 60.739 5.226 67.156 4.2215 51.870 4.786 70.678 4.107
Note. The means are displayed for Experiments 1a (circular), 1b (zig-zag), and 1c (random) separately in both the change and no-changeconditions.
Figure 9. Percentage of rapid resumption as a function of the block (80trials) in the Experiment. Experiments are collapsed and include Experi-ments 1a, 1b, and 1c.
Thi
sdo
cum
ent
isco
pyri
ghte
dby
the
Am
eric
anPs
ycho
logi
cal
Ass
ocia
tion
oron
eof
itsal
lied
publ
ishe
rs.
Thi
sar
ticle
isin
tend
edso
lely
for
the
pers
onal
use
ofth
ein
divi
dual
user
and
isno
tto
bedi
ssem
inat
edbr
oadl
y.
1383PREDICTIVE MINDSET IN PERCEPTION
tion was relatively irrelevant to perform the experimental task,although it is reasonable to assume that knowing the approximatelocation where the target will appear on each display presentationcould have facilitated the task.
If that is the case, introducing a change that’s relevant forthe task should force observers to incorporate that change into theperceptual hypothesis. The presence of a relevant change in thevisual display also allows us to tests another interesting possibility:that the resolution of the perceptual hypothesis might also change,adapting to the new task demand. In other words, if tracking theexact location of a target might not be relevant to report theorientation of the target, tracking the unceasingly changing orien-tation of a target should be.
In Experiment 3, the target orientation changed sequence wasrepeated so that observers could learn it, and thus predict thetarget’s orientation after few presentations of the display. After thefirst phase, the orientation sequence was disrupted, allowing us toassess the degree to which the predictable sequence was facilitat-ing performance. Contrary to what observed in previous studies(Jungé et al., 2009), changes of the target’s identity should pre-serve rapid resumption, as long as those changes are predictable.To prevent our experimental manipulation from being confoundedwith the temporal sequence of the experimental blocks, the pre-dictable target’s change sequence was resumed in the third blockand for the rest of the experiment.
The aim of this experiment was to confirm that prediction of theforthcoming events plays a fundamental role in rapid resumption.In addition, this study aims to reveal whether fine-grained changescan be also incorporated in the perceptual hypothesis.
If predictable fine-grained feature changes can be incorporatedin the perceptual hypothesis, we expect to observe rapid resump-tion after the target’s orientation changed, as long as such orien-tation is predictable—that is, first bock. We expect rapid resump-tion to be disrupted in the second block, when the target’sorientation was chosen randomly, and therefore, unpredictably, butwe also expect to observe rapid resumption when the target’schange sequence is resumed (e.g., third block).
Method
Participants. Twenty-nine volunteers (Mage � 19 years,SD � 1.2) participated in the experiment. All participants hadnormal or corrected-to-normal vision. They signed a consent formbefore the experiment and they were compensated $8 or onepsychology course credit for their participation.
Stimuli. The stimuli (see Figure 10) were very similar to thoseof the previous experiments, except for a few differences describedbelow. The target was a “T” and the distractors were “L” bothoccupying an area of 0.6° � 0.6° of visual angle. Each distractor(L) was tilted 45° clockwise and there were four possible targets,oriented in four possible directions (see Figure 10): 45° (up-right),135° (down-right), 225° (down-left), and 315° (up-left).
Procedure. The procedure was very similar to that of theprevious experiments, except for the following: Participants wereinstructed to respond “right” or “left,” based on the target direc-tion. If the target was tilted either 45° or 135°, “right” was thecorrect response. If the target was tilted either 225° or 315°, “left”was the correct response.
On each look the target’s orientation changed. During the firstblock, the target changed orientation after each look in steps of 90°clockwise, such that if the orientation in the first look was forexample down-right, in the second look it would be down-left, inthe third look up-left, and so on. During the second block of theexperiment, the target’s orientation was chosen randomly on eachlook, such that the same orientation was never chosen twice in arow. After the second block, the clockwise sequence of targetchanges between looks, such that Blocks 3, 4, and 5 were the sameas Block 1.
Participants completed 16 practice trials, in which the target didnot change orientation, and then five blocks of 80 trials each.
Results
Two participants were excluded from the analysis because theyfailed to complete at least four blocks8 of the experiment. Fiveparticipants were excluded from the analysis because of high errorrates (�57% accuracy).
Trials in which the observer did not accurately report the targetorientation (11.2%) and RTs longer than 10,000 ms (0.3%) wereconsidered errors and excluded from the analysis.
We compared the distribution of RTs in the Blocks 1, 3, and4—in which the target orientation changed predictably movingclockwise—to the distribution of RTs in the second block—inwhich the target’s orientation changed unpredictably being chosenrandomly on each display presentation (see Figure 11).
In all blocks (1–4), the distribution of the first epoch differedfrom the distribution in the later epochs (all ps � 0.001).
The three most relevant caparisons are between Block 1 versusBlock 2, Block 2 versus Block 3, and Block 3 versus Block 4. Asa reminder, we expect the later epochs of the second block to differfrom those in the first and third block, but we do not expect adifference between Blocks 3 and 4.
The distribution of the first epoch (Figure 11, top panel) did notsignificantly differ across blocks, in none of the three relevantcomparisons (all ps � .05).
By contrast, the distribution of the later epochs in Block 2significantly differed from both Block 1, �2(9) � 656.12, p �.001, Cramer’s V � 0.44, and Block 3, �2(9) � 157.30, p � .001,Cramer’s V � 0.22. Importantly, the distribution in Block 3 didnot significantly differ from Block 4 (p � .05; Figure 11, bottompanel).
Three participants were excluded from the analysis of variancebecause they never/always answered in the first epoch on at leastone block of the experiment. That left us with 19 participants onwhom we performed a 2 � 4 ANOVA with block (1–4) and epoch(first, later) as within-subjects variables on the percentage of rapidresumption (RT faster than 500 ms).
Results revealed a significant effect of epoch, F(1, 18) �195.12, p � .001, and a significant effect of block, F(1.97, 35.4) �5.46, p � .01, but more important, it showed a significant inter-action, F(3, 54) � 13.71, p � .01. As expected, post hoc analysisof the means revealed that the proportion of fast RT in the firstepoch was lower than in the later epochs (ps � .001). Moreimportant, it showed that, for the first epoch, the proportion of fast
8 Due to time constraints, not all the participants completed all fiveblocks of the experiments.
Thi
sdo
cum
ent
isco
pyri
ghte
dby
the
Am
eric
anPs
ycho
logi
cal
Ass
ocia
tion
oron
eof
itsal
lied
publ
ishe
rs.
Thi
sar
ticle
isin
tend
edso
lely
for
the
pers
onal
use
ofth
ein
divi
dual
user
and
isno
tto
bedi
ssem
inat
edbr
oadl
y.
1384 MEREU, ZACKS, KURBY, AND LLERAS
RT in the third (M � 13.64; SE � 3.47) and fourth (M � 14.96;SE � 2.87) blocks increased significantly with respect to both, thefirst (M � 5.41; SE � 1.95) and second (M � 8.36; SE � 2.03)blocks (ps � .05). In the later epochs, however, the proportion offast RT in the second block (M � 49.30; SE � 2.90) significantlydecreased (p � .001) with respect to the first block (M � 60.24;SE � 2.93). In addition the proportion of fast RT third (M � 62.01;SE � 3.59) and fourth (M � 57.25; SE � 3.71) blocks signifi-cantly increased with respect to the second block (p � .01 and p �.05, respectively). Importantly, the first block did not significantlydiffer from neither the third, nor the fourth block (ps � .5),although the proportion of fast RT showed a small but significantdecrease (4.7%) in the fourth block, with respect to the third one.
Discussion
The results are clear: The pattern of responses typical of therapid resumption phenomenon developed when the displaychanged between looks, but only when the sequence of changeswas predictable. Indeed, it abruptly disappeared when such se-quence was disrupted and reappeared nonetheless when the se-quence was resumed. This finding has two important implications:First, it shows that observers are able to quickly resume an inter-rupted visual search, by anticipating the forthcoming display, evenin case lower level characteristics of displays do not match; sec-ond, it shows that the resolution of the visual representations canbe fine grained, and can contain details regarding the target’sshape.
In Block 1 we observed rapid resumption—both the bimodaldistribution and a higher proportion of fast responses—interest-ingly, compared with any control condition that we have ever run,the amount of rapid resumption was substantially increased suchthat the first peak is even higher than the second peak in the RTdistribution. In Block 2, the random rotation of the target destroyedrapid resumption: A single later (500–600 ms) peak was observedand the proportion of rapid responses dropped significantly, when
compared with Block 1. Lastly, in Blocks 3 and 4, we observed aunimodal distribution with its peak shifted leftward (400–500 ms)and an increase of fast RTs, comparable with the rates observed inBlock 1.
The disappearance of the bimodal shape in the distribution ofBlocks 3 and 4, when the predictable target’s sequence of changeacross looks was resumed, might seem surprising at first. How-ever, it fits with the general pattern of results we observed in thisstudy (see Experiment 2 and supplemental section): that observers’goals and expectations play a role in the emergence of the phe-nomenon of rapid resumption. Block 1 was technically identical toboth Blocks 3 and 4. However, there is an obvious differencebetween the two: Block 1 came first, and likely shaped ourparticipants’ expectations regarding the experiment. Block 3 and 4however, came after the sequence was disrupted (i.e., in Block 2),and it is reasonable to assume that such disruption might havechanged the participants’ expectations about the task and thepredictability in the experiment.
Experiment 3 also revealed another intriguing aspect of rapidresumption: that the resolution of the nonconscious perceptualhypotheses might change based on the current goals and taskdemands. In fact, Experiment 1c and Experiment 2 in this studyseemed to suggest that details about the visual scene were sacri-ficed in favor of a more effective search strategy based on thecoarse spatial representation of the visual search display. Experi-ment 3 instead suggests that fine-grained details are available, andcan be used to facilitate visual search.
One important difference between the experiments might havecontributed to this result: Although both Experiments 1 and 2required the participants to report the target shape, only in Exper-iment 3 did the shape change between looks. That is, in Experi-ment 1, featural resolution of the target was neither directly relatedto the task, nor strictly necessary to successfully complete the task.However, in Experiment 3, storing details regarding the target’sfeatures was directly relevant to the task. Indeed, we observed that
Figure 10. Sequence of events that occurred on each trial of Experiment 3: (a) Predictable change (Blocks 1and 3–5), in which on each look the “T” appeared rotated by 45° clockwise; (b) Unpredictable change (Block2), in which the “T” appeared rotated by 45° in one of the possible four direction—in which the “T” orientationwas never the same in two consecutive looks. The sequence of display (100 ms–)blank (900 ms) presentationwas repeated for 12 times or until the participant gave an answer by pressing a key on the keyboard.
Thi
sdo
cum
ent
isco
pyri
ghte
dby
the
Am
eric
anPs
ycho
logi
cal
Ass
ocia
tion
oron
eof
itsal
lied
publ
ishe
rs.
Thi
sar
ticle
isin
tend
edso
lely
for
the
pers
onal
use
ofth
ein
divi
dual
user
and
isno
tto
bedi
ssem
inat
edbr
oadl
y.
1385PREDICTIVE MINDSET IN PERCEPTION
changing the orientation predictably increases the amount of rapidresumption and produces the typical bimodal distribution charac-terized by a very early peak of very fast RT (�500 ms). In otherwords, the observers’ task goals might be modulating the level ofdetail stored in the visual representation.
In sum, this last experiment shows that observers can integratefine-grained changes of objects’ visual features in the perceptualhypotheses and even more surprising, that changes in those detailscan be integrated into the perceptual hypotheses as long as they arepredictable.
General Discussion
These results say several important things about visual search.First, rapid resumption was observed even after changing relevantaspects of the search display—such as the target’s location orshape—as long those changes are predictable, confirming thatprediction underlies rapid resumption. Second, goal-driven infer-ences could be used to update nonconscious perceptual processes,such as those hypothesized to be involved in the preprocessing ofthe visual scene—for example, rapid resumption. Third, display’schanges have to be consciously accessible, for observers to be ableto include them into the perceptual hypothesis. Fourth, observers
seem to rely on coarse spatial information rather than exact spa-tiotemporal sequence, when precise information about the displayis not available. However, details about the visual display can bedynamically incorporated into the perceptual hypotheses whenavailable and relevant to the observer’s current goals.
Predictable Versus Unpredictable Changes
Changes of the display have been previously shown to disruptrapid resumption (Jungé et al., 2009; Lleras et al., 2005, 2007).Here, on the contrary, rapid resumption was preserved afterchanges of the target’s location. The crucial difference betweenour study and the previous studies is predictability. In our Exper-iments 1a and 1b, the target’s location changed between looks, butcould be inferred given the previous location. In our Experiment 3,the target shape changed, but it could be predicted given theprevious target’s direction. In both cases, observers used prelimi-nary information regarding the display to facilitate visual searchafter an interruption.
Even a coarse representation of the display is enough to guideprediction in an interrupted search task. In our Experiments 1 and2, observers used their expectation about the target’s location, tocoarsely restrict the visual search to a subset of locations. In ourExperiment 3, observer’s used details about the target direction toprecisely predict the subsequent target shape and facilitate visualsearch. In addition, our Experiment 3 shows that it is possible forobservers to adapt their predictive strategy based on how muchdetail is available and how much is required for the completion ofthe experimental task.
We argue that the knowledge about the previous display andprediction about the forthcoming event were combined into aperceptual hypothesis, which was updated during the interruptionto be confirmed in the next presentation, despite changes in thesearch display.
Goal-Oriented Strategy Over Early Visual Processes
The ability to quickly resume the visual search after aninterruption, during which a change occurred, reveals that high-level inferences, guided by an observer’s strategy, can be usedto update implicit perceptual processes. According to Llerasand colleagues (e.g., Lleras et al., 2005; Enns & Lleras, 2008),perceptual hypotheses are created at a nonconscious level be-cause their content is not available for explicit report if theobserver is only given one presentation of the search display.Yet, our results suggest that implicit perceptual hypothesis canintegrate anticipation of changes occurring in a dynamic visualscene. That is, even when physical appearance of the perceptualhypotheses did not match between looks, the hypotheses couldyet be confirmed.
Other studies reported effects of observer’s implicit inten-tions on early visual processes—for example, unconsciouspriming (e.g., Ansorge, 2004) and metacontrast masking (e.g.,Enns & Oriet, 2007; Gellatly, Pilling, Cole, & Skarratt, 2006).Similar effects on implicit visual processes have been alsoreported on a similar alternating presentation procedure (e.g.,Cole, Kentridge, Gellatly, & Heywood, 2003; Cole, Kentridge,& Heywood, 2004). Cole, Kentridge, Gellatly, and Heywood(2003) for example, reported a series of experiments in which
Figure 11. Normalized distribution of RT observed in the first epoch (toppanel) and for the following epochs (bottom panel) displayed for eachexperimental block in Experiment 3.
Thi
sdo
cum
ent
isco
pyri
ghte
dby
the
Am
eric
anPs
ycho
logi
cal
Ass
ocia
tion
oron
eof
itsal
lied
publ
ishe
rs.
Thi
sar
ticle
isin
tend
edso
lely
for
the
pers
onal
use
ofth
ein
divi
dual
user
and
isno
tto
bedi
ssem
inat
edbr
oadl
y.
1386 MEREU, ZACKS, KURBY, AND LLERAS
participants had to detect the presence of a change in a typicalchange detection task (see Simons & Levin, 1997). The criticalcomparison in Cole et al. (2003) study was between the onset ofa new item in the display and the offset of the same item—thatis, the item disappearing from a location previously occupied.Participants were asked to indicate whether a change had oc-curred between two continually alternating displays. When thechange involved the offset (disappearance) of an item in thesearch display, Cole et al. (2003) observed a unimodal distri-bution with a peak of responses around 500 ms. When thechange involved the onset (appearance) of a new item in thesearch display, they observed a bimodal distribution with peaksaround 300 – 400 ms and 700 – 800 ms. To explain their results,the authors propose a “confirmatory frame hypothesis,” inwhich they argue that sometimes a single presentation might notbe enough for the observer to become aware of a change thatoccur in the visual display. In this case, observers might “sense”that a change occurred, but need to wait for the next frame forthe hypotheses to be confirmed.
It is possible that a similar mechanism is at play in ourexperiments.9 Relying on the idea of “object file” formulated byKahneman, Treisman, and Gibbs (1992) the authors suggest thattemporary visual representations are created for visual objects.When the objects change, the file typically only needs to beupdated. However, some changes (e.g., the onset of a brand newobject in the search display) may trigger the creation of a newfile. Similarly in our study, when the visual display changespredictably, the visual representation only needs to be updated.However, unpredictable changes may require the creation a newfile.
Cole et al. (2003) argue that higher-level representationsseem to be modulating lower-level detection of transient lumi-nance changes. Similarly, in our experiments, higher-level pre-dictive mechanisms seem to be driving lower-level detection ofthe target’s characteristics. In sum, in both studies, the observ-ers’ strategy appears to be affecting early visual processes.
Explicit Versus Implicit Predictions
These results suggest that explicit knowledge regardingchanges of the display is needed for the goal-oriented infer-ences to affect early visual processes. In Experiment 1 observ-ers were aware of the target’s location sequence and were ableto quickly resume the interrupted visual search. In fact, inExperiment 2 (and those reported in the supplemental material)observers were not aware of the repetitiveness of the target’slocation and they were also unable to quickly resume the visualsearch after the interruption.
That said, it is important to note that rapid resumption doesnot seem to be the mere result of changes in the attentionalstrategy. One may argue in fact that rapid resumption in ourexperiments might be a byproduct of the observers’ explicitstrategy to restrict the attentional focus to the central locationsbecause that is were the salient items were presented (seeExperiments 1a–1c). However, only Experiment 1b suggestedthat observers were facilitated after the first look, when thetarget appeared in the central most locations. As previouslynoted, if participants were restricting the focus of attention tothe center, the search in the Change trials should have been
facilitated (higher rate of fast RT). In fact, except for Experi-ment 1b the opposite was observed, with consistently higherrate of rapid resumption in the No-Change trials.
The learning slopes observed across experiments were con-sistent with the idea that Change and No-Change trials devel-oped differently throughout the experiment. Higher rates of RTfaster than 500 ms was always observed when the target’slocation was repeated, and did not substantially increase duringthe experiment. On the contrary, trials where the target wasdisplaced showed a lower rate of rapid resumption, whichsignificantly increased during the experiment.
Taken together, these findings may suggest the existence oftwo qualitatively different types of facilitation in rapid resump-tion: an advantage driven by physical appearance—triggered bylow-level feature similarity—and a goal-driven advantage dueto the observer’s predictive strategy, which does not rely onlow-level features matching.
The original version of the interrupted visual search task (Lleras etal., 2005) does not allow distinguishing between these two types offacilitation. In fact, observers were both, presented with subsequentdisplays that matched regarding their low-level characteristics, andpotentially using an explicit goal-directed strategy, driven by theunderstanding of the static nature of the display. Our Experiment 2,however, allows us to separate the effects of the explicit strategy—based on the explicit knowledge that changes are occurring in thedisplay—from the effect of the implicit facilitation driven by lowlevel similarities between subsequent displays—when the target didnot change. Additional support to this hypothesis comes from Exper-iment 3, where both the bimodal pattern of RT and the high rate of fastRT were observed at the beginning of the experiment (Block 1), butonly the increase in fast RT—not the bimodal distribution—wasobserved in the last part of the experiment (Blocks 3 and 4). The firstand last parts of the experiment were technically identical, except forthe participants’ exposure to an unstructured block of trials betweenthe two, which we argue might have disrupted their motivation toengage in an active predictive search strategy, smearing the RTdistribution slightly rightward. Critically, an increase in the proportionof fast RT was still observed, indicating that the advantage of low-level features matching was still there.
Resolution of the Perceptual Hypothesis
Experiment 1 (especially 1c) and Experiment 2 clearly sup-ports the idea that perceptual hypotheses can be coarse but stillfacilitate perception. This may suggest that limits of the visualsystem prevent fine-grained representations from being createdwith only one glance. To overcome this limit, the visual systemmight choose which details to include in the perceptual hypoth-esis, based on the task demands. It should be noted, however,that this does not exclude the possibility that more details canbe included into the perceptual hypotheses. In fact, Experiment3 suggests that details and changes related to those details couldbe dynamically incorporated into the perceptual hypotheses.
9 Although it is worth noting here that the hypothesis that rapid resump-tion is the consequence of participants’ strategy of voluntarily withholdingtheir response until the next frame has been examined before, no convinc-ing evidence in favor of it has been found (Lleras et al., 2005).
Thi
sdo
cum
ent
isco
pyri
ghte
dby
the
Am
eric
anPs
ycho
logi
cal
Ass
ocia
tion
oron
eof
itsal
lied
publ
ishe
rs.
Thi
sar
ticle
isin
tend
edso
lely
for
the
pers
onal
use
ofth
ein
divi
dual
user
and
isno
tto
bedi
ssem
inat
edbr
oadl
y.
1387PREDICTIVE MINDSET IN PERCEPTION
Previous studies reached the same conclusion (e.g., Jungé etal., 2009; Lleras et al., 2007). Lleras et al. (2007) showed thatthe distribution of RT lacks of the early peak if the target colorchanges but only if observers are responding to the target’scolor (Experiment 4), and the early phase of RT reappears ifobservers are responding to its orientation (Experiment 3).Jungé et al. (2009, Experiment 3) extended this finding byshowing that only changes of the display involving distractorswith the same luminance as the target interfere with the rapidresumption phenomenon. The attentional focus also modulatesthe spatial resolution of the perceptual hypotheses, as suggestedby the finding that only changes in the display involving itemsnearby the target reduce rapid resumption (Jungé et al., 2009).
Our results support and extend this conclusion, showing thatobservers adapted their search strategy based on the task charac-teristics. In our Experiment 1, observers might have found advan-tageous to create a coarse representation of the display, so that allthe salient locations could be included. In fact, a more detailedrepresentation of the target’s features (or location) could not fur-ther facilitate visual search. Nonetheless, we observed that predic-tions based on a coarse spatial resolution were still able to guidevisual search by enabling rapid resumption.
When the details of the target (i.e., its precise shape) becamerelevant to the task, observers were able to integrate changes ofsuch details into the perceptual hypothesis, confirming that thephenomenon of rapid resumption is sensitive to the task de-mands.
In sum, our result strongly support the idea that an activeprediction about both what is present as well as lawfully changingin the world allows visual search to be rapidly resumed after aninterruption and add to the growing number of studies that, moregenerally, attribute to prediction a central role in human cognition.Going back to our initial example about visual perception duringstreet crossing, imagine being about to cross the street and seeinga car on the farther lane start to pass a large truck on the laneclosest to us. Momentarily, the car is occluded by the truck, andthen it reenters our field of view as it overtakes the truck. What ourstudy shows is that seeing even a glimpse of that car before it isobscured by the truck allows us to more readily scrutinize it onceit reappears. But this savings in processing occurs precisely be-cause we have a mind to scrutinize cars in the street and weactively anticipate the occluded car returning into view. Predic-tions about the world are thus an integral part of our perception ofthe world.
References
Ansorge, U. (2004). Top-down contingencies of nonconscious primingrevealed by dual-task interference. The Quarterly Journal of Experimen-tal Psychology, 57, 1123–1148. doi:10.1080/02724980343000792
Bayer, H. M., & Glimcher, P. W. (2005). Midbrain dopamine neuronsencode a quantitative reward prediction error signal. Neuron, 47, 129–141. doi:10.1016/j.neuron.2005.05.020
Brainard, D. H. (1997). The psychophysics toolbox. Spatial Vision, 10,433–436. doi:10.1163/156856897X00357
Chun, M. M. (2000). Contextual cueing of visual attention. Trends inCognitive Sciences, 4, 170–178. doi:10.1016/S1364-6613(00)01476-5
Cleeremans, A., Destrebecqz, A., & Boyer, M. (1998). Implicit learning:News from the front. Trends in Cognitive Sciences, 2, 406–416. doi:10.1016/S1364-6613(98)01232-7
Cole, G. G., Kentridge, R. W., Gellatly, A. R., & Heywood, C. A. (2003).Detectability of onsets versus offsets in the change detection paradigm.Journal of Vision, 3. doi:10.1167/3.1.3
Cole, G. G., Kentridge, R. W., & Heywood, C. A. (2004). Visual saliencein the change detection paradigm: The special role of object onset.Journal of Experimental Psychology, 30, 464–477. doi:10.1037/0096-1523.30.3.464
Destrebecqz, A., & Cleeremans, A. (2001). Can sequence learning beimplicit? New evidence with the process dissociation procedure. Psy-chonomic Bulletin & Review, 8, 343–350. doi:10.3758/BF03196171
Di Lollo, V., Enns, J. T., & Rensink, R. A. (2000). Competition forconsciousness among visual events: The psychophysics of reentrantvisual processes. Journal of Experimental Psychology: General, 129,481–507. doi:10.1037/0096-3445.129.4.481
Enns, J. T., & Lleras, A. (2008). What’s next? New evidence for predictionin human vision. Trends in Cognitive Sciences, 12, 327–333. doi:10.1016/j.tics.2008.06.001
Enns, J. T., & Oriet, C. (2007). Visual similarity in masking and priming:The critical role of task relevance. Advances in Cognitive Psychology, 3,211–226. doi:10.2478/v10053-008-0026-z
Gellatly, A., Pilling, M., Cole, G., & Skarratt, P. (2006). What is beingmasked in object substitution masking? Journal of Experimental Psy-chology: Human Perception and Performance, 32, 1422–1435. doi:10.1037/0096-1523.32.6.1422
Jungé, J. A., Brady, T. F., & Chun, M. M. (2009). The contents ofperceptual hypotheses: Evidence from rapid resumption of interruptedvisual search. Attention, Perception, & Psychophysics, 71, 681–689.doi:10.3758/APP.71.4.681
Kahneman, D., Treisman, A., & Gibbs, B. J. (1992). The reviewing ofobject files: Object-specific integration of information. Cognitive Psy-chology, 24, 175–219.
Kamide, Y., Altmann, G., & Haywood, S. L. (2003). The time-course ofprediction in incremental sentence processing: Evidence from anticipa-tory eye movements. Journal of Memory and Language, 49, 133–156.doi:10.1016/S0749-596X(03)00023-8
Kveraga, K., Ghuman, A. S., & Bar, M. (2007). Top-down predictions inthe cognitive brain. Brain and Cognition, 65, 145–168. doi:10.1016/j.bandc.2007.06.007
Lambert, A., Naikar, N., McLachlan, K., & Aitken, V. (1999). A newcomponent of visual orienting: Implicit effects of peripheral informationand subthreshold cues on covert attention. Journal of ExperimentalPsychology: Human Perception and Performance, 25, 321–340. doi:10.1037/0096-1523.25.2.321
Lamy, D. F., & Kristjánsson, Á. (2013). Is goal-directed attentional guid-ance just intertrial priming? A review. Journal of Vision, 13, 14. doi:10.1167/13.3.14
Lleras, A., Rensink, R. A., & Enns, J. T. (2005). Rapid resumption ofinterrupted visual search. New insights on the interaction between visionand memory. Psychological Science, 16, 684–688. doi:10.1111/j.1467-9280.2005.01596.x
Lleras, A., Rensink, R. A., & Enns, J. T. (2007). Consequences of displaychanges during interrupted visual search: Rapid resumption is targetspecific. Perception & Psychophysics, 69, 980–993. doi:10.3758/BF03193936
Pariyadath, V., & Eagleman, D. (2007). The effect of predictability onsubjective duration. PLoS ONE, 2, e1264. doi:10.1371/journal.pone.0001264
Rao, R. P., & Ballard, D. H. (1999). Predictive coding in the visual cortex:A functional interpretation of some extra-classical receptive-field ef-fects. Nature Neuroscience, 2, 79–87. doi:10.1038/4580
Schultz, W. (1998). Predictive reward signal of dopamine neurons. Journalof Neurophysiology, 80(1), 1–27.
Thi
sdo
cum
ent
isco
pyri
ghte
dby
the
Am
eric
anPs
ycho
logi
cal
Ass
ocia
tion
oron
eof
itsal
lied
publ
ishe
rs.
Thi
sar
ticle
isin
tend
edso
lely
for
the
pers
onal
use
ofth
ein
divi
dual
user
and
isno
tto
bedi
ssem
inat
edbr
oadl
y.
1388 MEREU, ZACKS, KURBY, AND LLERAS
Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate ofprediction and reward. Science, 275, 1593–1599. doi:10.1126/science.275.5306.1593
Shanks, D. R., & St. John, M. F. (1994). Characteristics of dissociablehuman learning systems. Behavioral and Brain Sciences, 17, 367–395.doi:10.1017/S0140525X00035032
Simons, D. J., & Levin, D. T. (1997). Change blindness. Trends inCognitive Sciences, 1, 261–267. doi:10.1016/S1364-6613(97)01080-2
Sutton, R. S., & Barto, A. G. (1981). Toward a modern theory of adaptivenetworks: Expectation and prediction. Psychological Review, 88, 135–170. doi:10.1037/0033-295X.88.2.135
Theeuwes, J. (1991). Exogenous and endogenous control of attention: Theeffect of visual onsets and offsets. Perception & Psychophysics, 49,83–90. doi:10.3758/BF03211619
Theeuwes, J. (2010). Top–down and bottom–up control of visual selec-tion. Acta Psychologica, 135, 77–99. doi:10.1016/j.actpsy.2010.02.006
Van Zoest, W., Lleras, A., Kingstone, A., & Enns, J. T. (2007). In sight, outof mind: The role of eye movements in the rapid resumption of visualsearch. Perception & Psychophysics, 69, 1204–1217. doi:10.3758/BF03193956
Wolfe, J. M., Butcher, S. J., Lee, C., & Hyle, M. (2003). Changing yourmind: On the contributions of top-down and bottom-up guidance invisual search for feature singletons. Journal of Experimental Psychol-ogy: Human Perception and Performance, 29, 483–502. doi:10.1037/0096-1523.29.2.483
Zacks, J. M. (2004). Using movement and intentions to understand simpleevents. Cognitive Science, 28, 979 –1008. doi:10.1207/s15516709cog2806_5
Zacks, J. M., Kurby, C. A., Eisenberg, M. L., & Haroutunian, N. (2011).Prediction error associated with the perceptual segmentation of natural-istic events. Journal of Cognitive Neuroscience, 23, 4057–4066. doi:10.1162/jocn_a_00078
Zacks, J. M., Speer, N. K., Swallow, K. M., Braver, T. S., & Reynolds,J. R. (2007). Event perception: A mind-brain perspective. PsychologicalBulletin, 133, 273–293. doi:10.1037/0033-2909.133.2.273
Received April 30, 2013Revision received February 10, 2014
Accepted February 11, 2014 �
Thi
sdo
cum
ent
isco
pyri
ghte
dby
the
Am
eric
anPs
ycho
logi
cal
Ass
ocia
tion
oron
eof
itsal
lied
publ
ishe
rs.
Thi
sar
ticle
isin
tend
edso
lely
for
the
pers
onal
use
ofth
ein
divi
dual
user
and
isno
tto
bedi
ssem
inat
edbr
oadl
y.
1389PREDICTIVE MINDSET IN PERCEPTION