out (1)
DESCRIPTION
articolTRANSCRIPT
Emotional Modulation of Visual Attention
by
Emma Ferneyhough
A dissertation submitted in partial fulfillment
of the requirements for the degree of
Doctor of Philosophy
Department of Psychology
New York University
September, 2011
_________________________________
Elizabeth A. Phelps, PhD
_________________________________
Marisa Carrasco, PhD
All rights reserved
INFORMATION TO ALL USERSThe quality of this reproduction is dependent on the quality of the copy submitted.
In the unlikely event that the author did not send a complete manuscriptand there are missing pages, these will be noted. Also, if material had to be removed,
a note will indicate the deletion.
All rights reserved. This edition of the work is protected againstunauthorized copying under Title 17, United States Code.
ProQuest LLC.789 East Eisenhower Parkway
P.O. Box 1346Ann Arbor, MI 48106 - 1346
UMI 3482879
Copyright 2011 by ProQuest LLC.
UMI Number: 3482879
© Emma Ferneyhough
All Rights Reserved, 2011
Chapter 1 has been published elsewhere (Psychonomic Bulletin and Review
(2010), Volume 17, Issue 4, p. 529-535). Per the publishing agreement with
PBR, the final version has been included in this dissertation. Excerpt from the
publishing agreement (http://www.psychonomic.org/psp/access.html):
The author retains the right to use his/her article for his/her further scientific career by including the final published journal article in other publications such as dissertations and postdoctoral qualifications provided acknowledgement is given to the original source of publication.
“Fear is the mind-killer.”
- Frank Herbert, Dune (1965)
iv
DEDICATION
For my family and friends.
v
ACKNOWLEDGEMENTS
I would like to thank my advisors Liz Phelps and Marisa Carrasco for
their guidance, support, and generosity, and for being possibly the best role
models for women in science that I can think of. I am extremely grateful for
everything I have learned from being a member of both labs.
To my fellow graduate students, thank you for sharing in the trials and
tribulations of academia, for inspiring me, and for making sure I never had to
drink alone.
To all the members of the Phelps and Carrasco labs, thank you for
keeping things in perspective over the years, for giving me your feedback, and
for being there day in and day out.
Lastly I would like to thank my mom for encouraging me to keep going
when thoughts of dropping everything and becoming a potato farmer clouded
my vision. I could not have done it without your love, phone calls, and
continued commitment to my success.
vi
ABSTRACT
Emotion has been shown to improve perception, and to both facilitate
and impair selective visual attention. The conjoint effect of emotion with
attention has been demonstrated across a range of tasks measuring accuracy
and response speed. Of particular interest in this dissertation are the
behavioral and neural correlates of emotion’s cost to visual attention
allocation, and the individual differences across observers that modulate the
magnitude of this effect. Costs of emotion to visual attention are assessed by
measuring decreases in (1) contrast sensitivity, a low-level visual perceptual
ability, and (2) word identification accuracy. Chapters 1 and 2 utilize a visual
psychophysics spatial cuing paradigm in which emotional or neutral face cues
direct attention prior to an orientation discrimination task dependent on
contrast sensitivity. An incongruent spatial relationship of cues and oriented
targets has previously been shown to alter contrast sensitivity. We show that
observer handedness (Chapter 1), trait anxiety and sex of the observer
(Chapter 2) also modulate this effect. Chapter 3 utilizes a variant of the
attentional blink paradigm to investigate the neural correlates of emotion’s cost
to temporal attention. Emotional distracter words disrupt processing of neutral
target words in a rapid serial visual presentation. We show that brain regions
underlying bottom-up emotional responses, such as the amygdala, may help
vii
direct attention to distracters via the orbitofrontal cortex and intraparietal
sulcus during emotional costs. Emotional costs to attention may be worsened
in individuals who engage the dorsolateral prefrontal cortex less (primarily
observers low in attentional control). As opposed to facilitative effects of
emotion to attention, costs are suggested to occur when bottom-up emotional
responses out-compete top-down attentional control mechanisms. Similar
neural circuitry may underlie emotional costs in both spatial and temporal
attention tasks.
viii
TABLE OF CONTENTS
DEDICATION…………………………………………………………………… iv
ACKNOWLEDGEMENTS……………………………………………………... v
ABSTRACT……………………………………………………………………... vi
LIST OF FIGURES…………………………………………………………….. ix
LIST OF TABLES………………………………………………………………. xi
LIST OF APPENDICES……………………………………………………….. xii
LIST OF ABBREVIATIONS…………………………………………………… xiii
INTRODUCTION………………………………………………………………. 1
CHAPTER 1…………………………………………………………………….. 28
CHAPTER 2…………………………………………………………………….. 47
CHAPTER 3…………………………………………………………………….. 86
CONCLUSION…………………………………………………………………. 128
APPENDICES………………………………………………………………….. 165
BIBLIOGRAPHY……………………………………………………………….. 179
ix
LIST OF FIGURES
Chapter 1
Fig. 1 Experiment 1 trial sequence
144
Fig. 2 (A) Contrast sensitivity data for all observers
(B) Contrast sensitivity data split by handedness group
145
Fig. 3 Contrast sensitivity data split by target visual field
146
Fig. 4 Correlation of handedness score with cue validity effect
147
Chapter 2
Fig. 5 Experiment 1 trial sequence
148
Fig. 6 Experiment 1 cueing effects: all observers
149
Fig. 7 Experiment 1 cueing effects: by anxiety and sex
150
Fig. 8 Experiment 2 trial sequence 152
x
Chapter 2 cont’d
Fig. 9 Experiment 2 cueing effects: all observers and by anxiety 153
Chapter 3
Fig. 10 (A) Trial sequence
(B) fMRI trial events
155
Fig. 11 Experiment 1 behavioral results
156
Fig. 12 Experiment 2 behavioral results
(A) Accuracy
(B) Reaction time
157
Fig. 13 IPS results
(A) Left posterior IPS
(B) Right posterior IPS
158
Fig. 14 Right DLPFC results 159
xi
LIST OF TABLES
Chapter 3
Table 1 (A) Talairach coordinates of ROIs
(B) Self-Report results
161
Table 2 Whole-brain contrasts
(A) All > Baseline
(B) Emotion > Neutral
162
Table 3 Whole-brain contrasts
(A) Late > Early
(B) Early Emotion > Early Neutral
Early Neutral > Early Emotion
(C) Late Emotion > Late Neutral
(D) Early Neutral > Late Neutral
Late Neutral > Early Neutral
163
Table 4 Whole-brain contrasts 164
(A) Early Emotion > Late Emotion
Late Emotion > Early Emotion
(B) Early Emotion Correct > Early Emotion Incorrect
Early Emotion Incorrect > Early Emotion Correct
xii
LIST OF APPENDICES
Appendix A
Study instructions for Chapter 1
165
Appendix B
Study instructions for Chapter 2
170
Appendix C
Study instructions for Chapter 3
Experiment stimuli
173
xiii
LIST OF ABBREVIATIONS
AB attentional blink
cpd cycles per degree
CS contrast sensitivity
ACS Attentional Control Scale
EHI Edinburgh Handedness Inventory
PANAS Positive and Negative Affect Scale
STAI State Trait Anxiety Inventory
fMRI functional magnetic resonance imaging
BOLD blood oxygen level dependent
ROI region of interest
rACC rostral anterior cingulate cortex
OFC orbitofrontal cortex
DLPFC dorsolateral prefrontal cortex
VLPFC ventrolateral prefrontal cortex
IPS intraparietal sulcus
1
INTRODUCTION
Our brains are extremely powerful information processors, but due to
limited skull volume they are likewise limited in size, and consequently, in
processing capacity (Lennie, 2003; Marois & Ivanoff, 2005). Both emotion and
attention can be thought of as strategies the human brain uses to selectively
filter the barrage of information our sensory receptors constantly receive. By
doing so, most brain resources can be dedicated to processing information
based upon not only our goals (top-down; e.g., writing a paper), but also upon
the features of our environment that we learn to associate with survival
(bottom-up; e.g., flashing railroad crossing lights). This efficient allocation of
brain resources allows for the fast extraction of meaning and coordination of
action necessary to survive in a dynamic world.
Humans predominantly depend on the visual system for information
gathering. Much of the research that has been conducted on emotion and
attention has thus focused on how they interactively affect visual processing.
Two ways that emotion and attention interact are to (1) improve, and (2) impair
visual processing. Their interaction can affect visual processing of both very
simple stimuli and very complex stimuli, and can occur both in space and at
different points in time. The outcome of their interaction depends on a
combination of qualities of both task-relevant and -irrelevant stimuli in the
2
external world, as well as the internal state (e.g., anxious) and stable
characteristics (e.g., sex) of an observer. These combined qualities, however,
have not been fully explored, leaving many questions unanswered and
debates unresolved within the emotion and attention literature.
A large part of the difficulty in understanding how visual processing is
affected by emotion and attention interactions is the fact that both emotion and
attention are complex, multi-faceted, psychological constructs on their own.
Within each are a number of subdivisions that have been defined by
generations of researchers, and are being further refined as time goes on. For
decades, emotion and cognitive abilities such as attention had largely been
thought of as separate, non-interacting entities. A consequence of this
independence within scientific research communities is that each field of study
has focused on different issues and has progressed at different rates.
Over the last 25 years, however, mounting evidence has increasingly
suggested that emotion and attention do interact, and their interactions can be
quite extensive across multiple experimental domains and cognitive
processes. This evidence usually comes in the form of studies measuring
reaction time or accuracy while performing a cognitive task. Emotion has been
shown to interact with attention resulting in speeded reaction time or increased
accuracy compared to a baseline condition, but in other instances (some of
3
which will be described later in the introduction) this interaction can also result
in slowed reaction time or decreased accuracy.
More recently, emotion and attention have been investigated in terms of
how they affect visual perception of low-level visual features such as contrast
or spatial frequency. While psychophysicists have made great advances over
the last 10 or more years in our understanding of how attention affects
perception, much less is known about how emotion affects perception. In fact,
at the time of this writing there are only 2 papers that have been published in
the last 5 years on emotion’s independent effects on perception (Bocanegra &
Zeelenberg, 2009; Phelps, Ling & Carrasco, 2006).
The three chapters in this dissertation describe studies investigating
how emotion and attention interact to both improve and impair visual
processing, with the first two focusing on perception of the low-level visual
feature of contrast. Given the short history of emotion and perception
research, these studies will therefore significantly contribute to our
understanding of emotion’s effects, as well as emotion and attention’s conjoint
effect, on visual processing. However, an attempt to understand the interaction
of emotion and attention (both in behavior and in the brain), and how this
impacts the way we see our world, necessitates some preliminary definitions.
Selective attention is the mechanism through which limited processing
resources are allocated to some information in the world at the same time as
4
other information is discarded. This system allows us to focus on our current
goals and minimize distractions (Broadbent, 1954; Cherry, 1953). These
benefits of attention on perception are accompanied by costs in the
unselected, or unattended, parts of the visual world (Pestilli & Carrasco, 2005).
Ignoring irrelevant information allows us to dedicate more energy to what is
important. Put simply, we see better when and where we are attending and we
see worse when and where we are not (for reviews on selective attention see
Carrasco, 2006, 2011).
Emotion is the mechanism that marks the importance or value of events
in our lives as they relate to our own interests and goals (Frijda, 1986). It
influences our learning (LeDoux 1996, 2000), memory for events
(Easterbrook, 1959; Brown & Kulik, 1977; Sharot, Delgado & Phelps, 2004),
and decision-making (e.g., Bechara et al, 1997). At one level, it is a
psychophysiological reaction to external stimuli based on our prior
experiences, and at another level it is our cognitive appraisal of events that
trigger emotional reactions. There is great variability in the range of emotions a
person can experience when faced with an external stimulus. For example,
while one person may unconsciously experience physiological signs of fear in
response to seeing a dog on the street, due to a prior negative experience
with animals, another person seeing the same dog may instead interpret
elevated heart rate as happiness or excitement.
5
As will be reviewed in the following paragraphs, emotion can modulate
attention within both spatial and temporal tasks by (1) enhancing its effects on
visual processing at attended, and (2) reducing its effects on visual processing
at unattended, locations in space and points in time, respectively.
Selective Attention and Visual Perception
Attention may be overt (directing gaze) or covert (keeping gaze fixed),
and covert attention may be deployed in a sustained or transient manner
(Nakayama & Mackeben, 1989). Sustained or endogenous attention is goal-
oriented and completely voluntary. Its effects typically peak around 300 ms
after initial attention allocation and can be sustained indefinitely. Transient or
exogenous attention, on the other hand, is stimulus-driven, occurring
automatically when a sudden change in the environment (e.g., movement or
change in contrast) draws attention to that location. The effects of exogenous
attention typically peak around 100-120 ms (Cheal & Lyon, 1991; Müller &
Rabbitt, 1989) after initial attention allocation, and rapidly decays thereafter.
Brain areas important for endogenous deployment of visual attention
include dorsal frontoparietal cortex, specifically dorsolateral prefrontal cortex
(DLPFC), anterior and posterior intraparietal sulcus (IPS), and putative human
homologue of the frontal eye field (FEF). Exogenous deployment of visual
6
attention depends on ventral frontoparietal cortex, specifically temporal
parietal junction (TPJ) and ventral frontal cortex (VFC), and is weighted more
towards the right hemisphere (Corbetta & Shulman, 2002). Many behavioral,
electrophysiological and imaging studies have shown that attending covertly to
a region in space can improve performance on visual discrimination tasks in
that location via feedback from attention-related brain areas to sensory cortex.
This feedback can either increase the gain or narrow tuning of neurons
sensitive to particular target features (for reviews see Carrasco, 2006, 2011).
Exogenous Covert Attention Modulates Contrast Sensitivity
Exogenous covert attention will be of particular importance in Chapters
1 and 2. When most people think of attention, we think of how we voluntarily
pay attention to different aspects of our environment. We move our bodies, our
heads and our eyes to better position ourselves to look at what we want to
see. However, before we can even “tell” our eyes how to move overtly, we
have to know where we want to look. This is where covert visual attention
plays an important role: it helps us select areas of interest in our visual field
without moving our eyes.
In this dissertation, covert attention will be manipulated spatially in three
ways similar to many other spatial cueing studies (e.g., Pestilli & Carrasco,
7
2005; Posner, 1980), using “valid”, “distributed” and “invalid” cues preceding a
visual discrimination task. Valid cues appear in spatial locations that draw
attention to the target or task location, and visual discrimination or detection of
that target is expected to improve. Invalid cues appear in spatial locations that
draw attention away from the target or task location, and visual discrimination
or detection of that target is expected to be impaired, consistent with the idea
that attention is a limited resource (Lennie, 2003; Pestilli & Carrasco, 2005;
Montagna, Pestilli & Carrasco, 2009). Distributed cues appear in all possible
spatial locations, spreading attention equally. In this case the cues give the
same timing information as the valid and invalid cues, but attention does not
select one location for preferential processing. Thus, there is a benefit of
attention with validly cued targets and a cost of attention with invalidly cued
targets, compared to the baseline attention condition (distributed cues). The
effects of exogenous covert attention on contrast sensitivity will be measured
using these three cuing conditions.
All visual stimuli can be composed of a collection of basic features and
contrast, like spatial frequency, motion or color, is one such low-level visual
feature. Contrast sensitivity is our ability to perceive differences between light
and dark parts of a stimulus. From electrophysiology studies in monkeys and
imaging work in humans, we know contrast is processed in primary visual
cortex, V1, which is the earliest cortical area that processes visual information.
8
It is our contrast sensitivity that, at this early stage, determines what we see
and what we do not see. We discriminate objects, for example, based on
differences in contrast that define the edges and boundaries separating an
object from the surrounding environment. Thus, perception of contrast is one
factor that underlies all of what we see, and as we learn more about contrast
sensitivity we likewise learn more about how we see complex visual objects or
scenes in our everyday life.
It has been previously shown that contrast sensitivity increases at
attended, and decreases at unattended, spatial locations (Pestilli & Carrasco,
2005; Pestilli, Viera & Carrasco, 2007), as if the physical contrast of a stimulus
increase or decreased, respectively. These benefits and costs of attention to
contrast sensitivity have also been found to correlate with changes in striate
and extrastriate visual cortical regions (Liu, Pestilli & Carrasco, 2006). To
investigate these changes in contrast sensitivity, observers participate in an
orientation discrimination task in which exogenous attention cues precede a
number of Gabor stimuli (sinusoidal gratings convolved with a Gaussian
envelope). One of these stimuli is a target while the rest are distracters, and
observers must discriminate the tilt of the target Gabor. On each trial, the
Gabor contrast varies systematically from very low to very high in several log
steps. Performance on the orientation discrimination task is expected to
increase with rising stimulus contrast. Thus, on trials with the lowest contrast
9
Gabors, performance is expected to be at chance, while on trials with highest
contrast Gabors, performance is expected to asymptote.
Gabors are artificial stimuli well suited for use in visual psychophysics
experiments where performance is measured as a function of stimulus
intensity. Although we never come across Gabors in our everyday lives
(unless we are participating in a vision experiment), by using a stimulus that
can be composed of single dimensions of basic visual features, we can learn
about the functioning of the visual system in a systematic manner. For
example, a Gabor is composed of a single spatial frequency, a single contrast
level, a single orientation and a single size. In addition, each of these
dimensions can be modified independently of all the other dimensions, making
it an ideal stimulus to investigate how our visual systems process each of
these basic features. To investigate the effects of exogenous covert attention
on contrast sensitivity, the studies described in the first two chapters will keep
spatial frequency, orientation and size constant while changing contrast from
trial to trial. Performance on an orientation discrimination task that depends on
contrast sensitivity is then measured for each contrast.
Psychometric (e.g., Weibull) functions, which relate observer
performance as a function of stimulus intensity, are fitted to the performance
data across the tested contrast levels. Contrast threshold is then calculated
based on the function fit. This is the contrast needed to perform at a given
10
level of accuracy, and it serves as a measure of contrast sensitivity in our
visual system. It provides a quantitative summary of all the data points
collected in a psychophysics experiment, and allows researchers to assess
differences in sensitivity due to condition at any stimulus contrast level.
Relative to averaging performance over all stimulus intensities and comparing
those averages across conditions, calculating contrast threshold provides a
more precise measurement of perceptual ability and provides information
about how this perceptual ability changes as a function of stimulus intensity. In
addition, there is considerable variation in contrast sensitivity across a subject
population. For example, one person may perform correctly 75% of the time
with a stimulus contrast of 5% whereas another person may need 10%
contrast to perform at the same level. By fitting psychometric functions to data
from individual subjects, we are able to capture and account for these
differences.
The psychophysical task and method of analysis described in the
preceding paragraphs are valuable tools that allow researchers to measure
contrast sensitivity at a given performance level at both attended and
unattended locations, within a single individual. We use these tools to
investigate the effects of exogenous attention cues on contrast sensitivity in
Chapters 1 and 2.
11
The Attentional Blink
In addition to investigating spatial attention in Chapters 1 and 2,
Chapter 3 will investigate the temporal limits of attention using the attentional
blink (AB; Raymond, Shapiro & Arnell, 1992) paradigm. Using the AB
paradigm, one can control which stimuli are accessible to awareness by
inserting task-relevant word(s) or picture(s) into a rapid serial visual
presentation (RSVP) stream of distracter stimuli. Each stimulus is on the
screen for a brief period of time (e.g., 90 ms) so that individual stimuli may not
be fully processed before they are replaced. In a typical AB experiment, two
targets T1 and T2 are inserted in a RSVP stream with a variable number of
intervening distracters. Zero intervening distracters is called “Lag 1”, whereas
six intervening distracters is called “Lag 7”. The observer’s task is to identify
both targets. At short lags, there is a robust decrement in performance in
identifying T2 contingent on correct T1 identification. This decrement lasts for
500 ms post-T1. At longer lags, performance on T2 identification recovers.
The AB shows that there is a brief window of time in which attentional
processing is at capacity, such that only one stimulus may be attended and
encoded at a time.
A number of models have been put forward to explain the AB effect (for
a review see Dux & Marois, 2009). One of the more commonly cited is the
two-stage model proposed by Chun and Potter (1995). In the first stage of
12
processing, stimuli are rapidly recognized and monitored for relevant features
according to the task. In the second stage, this information is encoded and
consolidated into working memory. However, the second stage is capacity-
limited so new information cannot gain entry until the encoding of old
information is finished. The AB is thought to be due to T2 items’ inability to
enter the second stage of processing because T1 is still being encoded. If T2
does not enter Stage 2 processing, its representation in Stage 1 rapidly
decays and is overwritten by flanking distracters. The result is as though one
never saw T2 at all – the AB.
While psychophysics specifically measures performance as a function
of the intensity of an elementary visual feature, such as stimulus contrast, the
AB measures one’s ability to identify words (or pictures), which requires
higher-level computation. Although these two tasks may have different
underlying mechanisms, emotionally salient cues can modify the effects of
attention on visibility in both tasks. This suggests that emotion can modulate
attention at multiple levels of analysis, and can be thought of as a general
strategy used by the brain to prioritize information processing. Throughout this
dissertation, we will show that this is the case, both at the lower level of visual
perception (Chapters 1 and 2) and at the higher level of word identification
(Chapter 3).
13
Emotion Modulates Attention and Perception
Emotion-laden stimuli, and inherently neutral stimuli that become
associated with emotions, are preferentially processed in terms of greater
speed and depth. This is especially true of threat-related stimuli that evoke
emotions such as fear and anger, which are negative in valence and high in
arousal. Fast and exhaustive processing of threat is crucial to an organism’s
survival (LeDoux, 1996), and this is manifested in more efficient use of
resources such as attention (e.g., Notebaert, Crombez, Van Damme, De
Houwer & Theeuwes, 2011; Ohman 2009). Indeed, it has been shown that
emotion’s impact on cognition starts early on in the information-processing
stream with perception and attention (for reviews see Compton, 2003; Stanley,
Ferneyhough & Phelps, 2009).
Emotionally salient stimuli have been shown to modulate perception in
two ways. Emotion can (1) enhance contrast sensitivity (Phelps et al., 2006)
and tilt detection of low spatial frequency stimuli (Bocanegra & Zeelenberg,
2009), or (2) impair tilt detection of high spatial frequency stimuli (Bocanegra &
Zeelenberg, 2009). Furthermore, emotional stimuli can modulate spatial
attention by (1) facilitating the benefits of attention on contrast sensitivity
(Phelps et al., 2006), performance or reaction time (e.g., Koster, Crombez,
Van Damme, Verschuere & De Houwer, 2004), or by (2) capturing attention
and drawing it away from goal-relevant tasks with the result of impaired
14
performance and slowed reaction time (e.g., Fox, Russo, Bowles & Dutton,
2001). What has yet to be shown, however, is whether the interaction of
emotion with attention impairs contrast sensitivity at unattended locations. If it
can, then this means the emotional significance of an object in the world can
not only improve our perception of that and nearby objects, but also impair our
perception of the surrounding environment. Importantly, this detrimental effect
may go beyond the impairment of attention due to non-emotional distractions.
This has real-world implications for things such as highway billboard design,
which in recent years has trended towards more dynamic imagery at the cost
of increasingly more inattentive drivers. The possibility that emotion and
attention will conjointly impair contrast sensitivity at unattended locations will
be tested in Chapters 1 and 2 using visual psychophysics.
Emotion can also modulate attention in temporal tasks such as the AB,
to make target stimuli more, or less, visible. Anderson and Phelps (2001) used
this paradigm with neutral and emotional T2 word stimuli and found that the
AB was attenuated with emotionally arousing compared to low arousal, neutral
T2. This suggested that emotional stimuli have preferential access to the
second stage of Chun and Potter’s (2005) two-stage model of attention.
As mentioned previously, the two-stage model of attention states that a
short-duration initial stage first detects target stimuli before a capacity-limited
second stage takes over to provide more in-depth perceptual and memory
15
encoding. In the traditional AB task, T1 will progress through stage 1 and 2,
but while being processed in stage 2, items that appear after T1 (such as T2)
may be lost due to rapid stage 1 decay. However, it may be the case that
emotionally arousing T2s are able to enter stage 2 processing more easily
than neutral T2. This is supported by data showing that neither bilateral nor left
amygdala patients experienced emotional facilitation in the AB, further
suggesting that the facilitation effect is dependent on the left amygdala. Thus,
the amygdala, a medial temporal lobe structure important for emotional
learning, may have a role in modulating perception of emotional stimuli.
In addition to emotion benefiting attentional engagement with emotional
targets, emotion can impair attentional engagement with neutral targets in the
AB. Modified AB paradigms including a single neutral target preceded by a
neutral or emotional distracter have found that target identification is impaired
with task-irrelevant emotional distracters within the AB window (Arnell, Killman
& Fijavs, 2007; Mathewson, Arnell & Mansfield, 2008; Most, Chun, Widders &
Zald, 2005). The effect seems to depend both on the arousal and valence of
the target stimuli with highly arousing negative stimuli producing the largest
effects (Jefferies, Smilek, Eich & Enns, 2008; Keil & Ihssen, 2004; Most,
Smith, Cooter, Levy & Zald, 2007). Top-down attention towards goal-relevant
stimuli appears to be disrupted by a bottom-up, emotionally modulated
attention component. This results in greater attentional resources being
16
diverted towards emotional, yet task-irrelevant, stimuli in the temporal domain,
leaving insufficient resources for task-relevant target processing.
While the amygdala has been shown to have a crucial role in emotion’s
facilitative effect in the AB, the neural correlates of emotion’s detrimental effect
in this modified AB have not been fully explored. Emotional distractions are
harder to ignore than neutral ones, but for many people they may be nearly
impossible to ignore. This inability to filter out emotional distractions could
indicate under recruitment of brain regions that provide top-down attentional
control, which have previously been associated with vulnerability to anxiety
disorder (e.g., Bishop 2007).
There may also be, however, an interaction between goal-oriented and
stimulus-driven systems of selective attention within the context of this
modified AB task. Goal-oriented attention, for example, can help form an
attentional set which effectively sensitizes stimulus-driven attention to task-
relevant items (e.g., particular target words); however, an attentional set can
also be disrupted if stimulus-driven attention is drawn to an irrelevant location
by a highly salient (i.e., emotional) stimulus. Chapter 3 will address the
behavioral and neural costs of emotion to attention in healthy undergraduates
with functional magnetic resonance imaging (fMRI) using a modified AB task.
17
A Note on Fear Face Stimuli
There is a long and well-established history of research on fear in
animals and in humans. Whereas fear is an emotion that consistently elicits
increased autonomic responses such as freezing in rats, and skin
conductance in humans, it has not been established whether other emotions
such as happiness or anger can be studied in animals. Therefore, parallels
between the human experience of these other emotions with animal models
cannot be drawn. Due to the fact that the neural circuitry of fear has been well-
characterized, we use face stimuli with fearful expressions as spatial cues to
manipulate both exogenous attention and emotional salience in the visual
psychophysics experiments in Chapters 1 and 2. We specifically chose face
stimuli over other possible choices for a variety of reasons, ranging from the
theoretical to the practical.
Humans are a social species and depend on multiple modes of non-
verbal communication. Faces and their underlying musculature have evolved
to express a vast array of emotional states, and this meaningful information
can be conveyed in a short amount of time within the limits of exogenous
attention (e.g., Eimer & Holmes, 2002; Phelps et al., 2006). A fearful face
signals to an observer that threat is present in the environment and that they
should be fearful as well. This fear response may serve to mobilize the
observer’s attentional resources in order to both collect more information and
18
act on that information. The sudden appearance of the face in the visual field
may initially direct exogenous attention to its location for the purpose of
collecting more information about the possible threat; for example, it is likely
that the threat is near the person making the fearful expression. After this
initial attentional process the observer can decide how to react, such as
whether to avoid or engage the threat. All of this occurs before the observer is
able to cognitively evaluate the situation (which can take hundreds of
milliseconds). Thus, we associate fearful expressions with the presence of
threat because this is an evolutionarily beneficial trait that has increased the
chances of our species’ survival.
It can be argued that fearful facial expressions provide a relatively weak
fear signal relative to real threats, such as electric shocks, or even relative to
pictures of evolutionarily “prepared” stimuli that could pose a real threat if we
were to actually face them (e.g., snakes, spiders or tigers). Given that the
studies in this dissertation tests the effects of emotion and attention on
perception using psychophysics methodology (requiring large numbers of
trials), it is not practical or safe to induce fear responses in humans using
shock. Fortunately for our research, it has been shown that seeing another
person’s fearful reaction to threat can induce the same reaction in the
observer without having to subject the observer to the same threat. This is
observational fear learning (for a review: Olsson & Phelps, 2007). Since fearful
19
expressions have been shown to reliably induce a fear response (Olsson,
Nearing & Phelps, 2007; Vaughan & Lanzetta, 1980) using a more prepared
stimulus such as a tiger is not necessary.
Face stimuli have other practical benefits, besides being safe to use
over many hundreds or thousands of trials, that have more to do with the
experimental design. To ensure that any effects of the fearful faces are due to
their emotional expression rather than the fact that they are a face, faces with
neutral expressions provide a natural control. Both fearful and neutral
expressions can be made by the same face, which also controls for other low-
level factors such as coloring, size or shape. Contrast and luminance can also
be equated more easily across a set of face stimuli compared to a set of
stimuli composed of animals of more extreme differences in these features.
These low-level visual differences matter in experiments investigating
exogenous attention because visual salience alone can alter how attention is
deployed. It is also unclear what would serve as a good neutral condition in
the case that, for example, a tiger is used as an emotionally salient cue. While
other striped animals could conceivably be used, this choice lacks the
advantage that faces have – fearful and neutral faces have largely the same
low-level visual characteristics and differ only in emotional valence.
Lastly, face recognition has been studied extensively. Faces have been
shown to modulate activity of specific regions in ventral visual cortex
20
(Kanwisher et al., 1997) and the amygdala (Adolphs, Tranel, Damasio &
Damasio, 1994; Morris, Frith, Perrett, Rowland, Young, Calder & Dolan, 1996).
Attention to faces has also been shown to modulate activity in these emotion-
and face-sensitive regions, and regions underlying shifts of attention (Armony
& Dolan, 2002; Pourtois et al., 2006; Vuilleumier et al., 2003, 2004). By using
stimuli that are commonly used in studies of emotion and attention, predictions
can be made more reliably relative to other stimuli that are not as well-studied.
Individual Differences that Modulate Emotion and Attention Interactions
External stimulus characteristics such as emotional significance or
perceptual salience can modify attention’s effects on perception. What is
relatively understudied are the stable internal characteristics, or the current
state, of individuals that can modulate how emotion and attention interact to
change perception. All of the studies described in this dissertation investigate
individual differences, which will be described below. Chapter 1 has a special
focus on handedness, while Chapter 2 focuses on anxiety and sex. In
Chapter 3 we explore effects of both anxiety and attentional control.
There is a growing literature investigating how subclinically anxious
individuals selectively allocate their attention, especially in cases of potential
threat. In theories of attentional control, anxious individuals are influenced
21
more by bottom-up or stimulus-driven attention, due to an impairment of their
goal-directed attentional system (Eysenck, Derakshan, Santos & Calvo, 2007).
Indeed, attentional control has been found to be inversely related to trait
anxiety (Derryberry & Reed, 2002).
An AB study found that low anxious subjects were better able to avoid
the detrimental effects of emotional distracters on the target discrimination
task than high anxious subjects when they exerted attentional control (Most et
al, 2005). Similarly, Fox and other researchers have used spatial cuing
paradigms to demonstrate that individuals with heightened state and trait
anxiety have difficulty disengaging from threatening stimuli such as negative
emotional words, pictures of angry or fearful faces, or fear-conditioned stimuli,
compared to individuals with low trait anxiety (Fox et al., 2001; Fox 2002;
Koster, Crombez, Verschuere & De Houwer, 2006; Mogg & Bradley, 1999;
Smith, Most, Newsome & Zald, 2006; Yiend & Mathews, 2001). The result is
that performance of highly anxious individuals is impaired in experiment
conditions that require a disengagement and shift of spatial attention from
threat-related stimuli, while individuals with a greater degree of attentional
control are more successful at these tasks.
Studies have also shown an interaction of anxiety with sex. High
anxious females, for example, have greater amygdala activation in response
to unattended fear faces compared to high anxious males (Dickie & Armony,
22
2008). This finding may also be related to the findings showing females are
better at recognizing faces and facial affect than males (McClure, 2000;
Thayer & Johnsen, 2000). Better recognition of fearful expressions on faces
could make them more salient, thereby enhancing the emotional significance
these faces have to the observer.
Individual differences in degree of handedness may affect the
interaction of emotion and attention as well. Previous research has
demonstrated the dominance of the right hemisphere in spatial attention in
right-handers (e.g., Mesulam, 1999). There have also been reports of greater
right hemisphere representation for face (Yovel, Tambini & Brandman, 2008)
and emotion processing (Bourne, 2008). In contrast, less is known about left-
hander functional lateralization, other than it is more inconsistent. Greater
inter-subject variability of left-hander behavior in attention and face perception
tasks (e.g., Bourne, 2008; Dronkers & Knight, 1989; Luh, Redl & Levy, 1994)
has been demonstrated, providing evidence that left-handers and right-
handers may differ in regards to cerebral lateralization of these functions.
As previously mentioned, throughout Chapters 1-3 we will investigate
effects of stable individual characteristics and current states of the observer on
the interaction of emotion with attention. We recruit equal numbers of left- and
right-handers in Chapter 1, and males and females with wide-ranging self-
reported anxiety in Chapter 2 and compare the interactive effects of emotion
23
and attention on their contrast sensitivity scores. For our imaging study of
emotional costs in Chapter 3, all participants provide us with anxiety and
attentional control self-report measures after completing our AB task so we
can investigate how error rate and brain activity changes relate to these
personality traits.
Neural Correlates of Emotion’s Effects on Attention and Perception
There are several neural routes through which emotion may exert its
influence on attention and perception. While they are not mutually exclusive,
the evidence cited to form the basis of these routes have been debated in the
literature (e.g., Bach, Talmi, Hurlemann, Patin & Dolan, 2011; Pessoa 2010;
Pessoa & Adolphs, 2010).
Representations of emotion-laden stimuli or other objects in the vicinity
of emotional stimuli may be boosted in retinotopic visual cortex by feedback
from the amygdala. This is supported by ERP and fMRI research that has
shown enhanced striate and extra-striate activity to emotional compared to
neutral stimuli (e.g., Pourtois, Grandjean, Sander & Vuilleumier, 2004;
Schupp, Markus, Weike & Hamm, 2003; Armony & Dolan, 2002), and this
enhancement is inversely correlated with degree of amygdala sclerosis
(Vuilleumier, Richardson, Armoney, Driver & Dolan, 2004). While these
24
studies provide support for the amygdala feedback hypothesis, they cannot
differentiate among the following three routes. (1) The amygdala receives
feedforward projections from temporal visual cortex, and has feedback
projections throughout ventral visual cortex including area V1 (Amaral,
Behniea & Kelly, 2003). (2) The amygdala receives fast, low-resolution
information regarding the significance of visual stimuli from the superior
colliculus and pulvinar thalamus (Morris, deGelder, Weiskrantz & Dolan,
2001), which then feeds back to V1, bypassing cortex (Adolphs, 2002; LeDoux
2002). (3) In addition to direct feedback from the amgydala, V1 may receive
indirect feedback via frontoparietal attention areas.
Ventral prefrontal cortex (VPFC) has reciprocal connections with the
amygdala (Barbas, 2000) and can regulate its activity in response to emotional
stimuli as well as influence responses of the IPS (Kelly, Uddin, Shehzad,
Margulies, Castellanos, Milham & Petrides, 2010; Taylor & Fragopanagos,
2005). IPS can then feed back to visual cortical areas specific to sensory
processing of the stimuli (e.g., Vuilleumier, Schwartz, Verdon, Maravita,
Hutton, Husain & Driver, 2008).
The neural correlates of emotion’s cost may be, in large part, based on
the correlates of emotion’s benefit, however there are a few important
differences unique to costs. Recent imaging work has revealed that bilateral
IPS activity is reduced in response to contralateral targets cued by invalid fear
25
faces (Pourtois, Schwartz, Seghier, Lazeyras & Vuilleumier, 2006). However,
IPS activity has also been shown to increase in trials with either valid or invalid
fear face cues (Armony & Dolan, 2002). This discrepancy may be explained by
the fact that in the first study, target-related IPS activity was assessed,
whereas in the second study, IPS activity was in response to the whole trial
including cue and target. This suggests that the fearful faces lead to increased
transient focusing of attention to their location, and is evidence for the notion
that emotional cues can capture and hold attention producing a cost when
targets are presented in unattended locations. In addition, both studies report
increased orbitofrontal cortex (OFC) activity in response to fear-invalid trials.
OFC may be important in cases where attention has been involuntarily
captured, or in “breaches of expectation”, by an emotional stimulus. OFC may
redirect attention via top-down signals to IPS (Armony & Dolan, 2002; Nobre,
Coull, Frith & Mesulam, 1999; Pourtois et al, 2006). It has also been
hypothesized that the rostral anterior cingulate cortex (rACC), which receives
input from emotion-sensitive ventral striatum, has a role in gating awareness
to potentially threatening stimuli and may regulate amygdala response in order
to resolve affective interference (DeMartino, Kalisch, Rees & Dolan, 2009;
Most, Chun, Johnson & Kiehl, 2006).
Chapter 3 investigates costs of emotion to attention within the context
of a modified attentional blink task. In addition to the amygdala, based on prior
26
research we are interested in the roles of IPS, OFC, rACC and DLPFC,
regions involved in both bottom-up and top-down attention processes.
Benefits and Costs of Emotion to Visual Attention
The following three chapters explore how external stimulus factors,
stable internal characteristics and current states of individual observers can
modulate the way emotion incurs both benefits and costs to visual attention.
Chapter 1 shows that contrast sensitivity is modulated by exogenous attention
cued by pictures of faces, but this effect depends on observer handedness.
We also find that facial expression effects depend on a specific range of
spatial frequencies. Chapter 2 revisits contrast sensitivity and finds that its
modulation by face cues is further influenced by individual differences in trait
anxiety and sex of the observer. Chapter 3 investigates the neural correlates
of emotional costs to attention in the attentional blink paradigm, revealing a
frontoparietal network of brain areas important in this interaction, as well as
specific regions whose activity is modulated by individual differences in anxiety
and attentional control.
The study of how emotion and attention interact at the perceptual level
has important implications for many cognitive processes that occur
downstream. The particular way we remember an event, make a decision, or
27
perform some action, for example, depends on how we see the world around
us. How we see is intimately linked to individual differences in our selective
attention processes, the emotional significance that each object in our
environment has for us, and the interactions among attention, emotion and
perception.
28
CHAPTER 1
Cueing Effects of Faces are Dependent on Handedness and Visual Field
(2010, Psychonomic Bulletin & Review, Vol. 17, Issue 4, p. 529-535)
Emma Ferneyhough1, Damian A. Stanley1, Elizabeth A. Phelps1, 2 & Marisa
Carrasco1, 2
1New York University Psychology Department
2New York University Center for Neural Science
29
Abstract
Faces are unlike other visual objects we encounter, alerting us to
potentially relevant social information. Both face processing and spatial
attention are dominant in the right hemisphere of the human brain, with a
stronger lateralization in right- than left-handers. Here we demonstrate
behavioral evidence for an effect of handedness on performance in tasks
using faces to direct attention. Non-predictive, peripheral cues (faces or dots)
directed exogenous attention to contrast-varying stimuli (Gabor patches) – a
tilted target, a vertical distracter, or both; observers made orientation
discriminations on the target stimuli. Whereas cueing with dots increased
contrast sensitivity in both groups, cueing with faces increased contrast
sensitivity in right- but not left-handers, for whom opposite hemifield effects
resulted in no net increase. Our results reveal that attention modulation by
face cues critically depends on handedness and visual hemifield. These
previously unreported interactions suggest that such lateralized systems may
be functionally connected.
30
Acknowledgements
We thank Sam Ling and David Carmel, as well as other Carrasco Lab
members, for their helpful comments. This research was funded by grants NIH
R01-EY016200 to M.C. and NIH R01-MH062104 to E.A.P.
31
INTRODUCTION
Faces are special visual objects that we encounter every day. Not only
are they complex and ever-changing, they are a portal into the thoughts and
intentions of others, providing information necessary for navigating our
dynamic social world. Perhaps for these reasons, we are particularly
responsive to faces; we rapidly evaluate them (Haxby, Hoffman & Gobbini,
2002) and use them to make predictions of social outcomes (Oosterhof &
Todorov, 2008). Furthermore, faces have the ability to automatically draw our
attention, more so when they depict a fearful rather than neutral expression
(Phelps, Ling & Carrasco, 2006). This ability is particularly important because
it is one of the first steps necessary to begin the process of evaluation and
prediction formation in our chaotic visual world.
Selective attention can be deployed covertly (without eye movements)
to a region in space and improve performance on visual discrimination tasks in
that location (Carrasco, 2006; Kinchla, 1992). This is true whether attention is
deployed voluntarily (endogenously), or driven involuntarily by a transient
change in the visual field (exogenously) (Nakayama & Mackeben, 1989).
Typically, psychophysicists use peripheral cues consisting of dots or bars to
direct exogenous attention. When cued with dots or bars, not only does
exogenous attention improve performance at cued locations (Carrasco,
32
Penpeci-Talgar & Eckstein, 2000; Ling & Carrasco, 2006a), it also impairs
performance at uncued locations. This is true even though the cues are
uninformative (i.e., they do not predict the target location) and observers are
explicitly told that this is the case (Montagna, Pestilli & Carrasco, 2009; Pestilli
& Carrasco, 2005). These trade-offs in performance have been interpreted as
resulting from the allocation of limited resources. Faces are effective as
exogenous cues and can reflexively draw attention to task-relevant locations,
perhaps because of their ecological validity and social value (Phelps et al.,
2006). However, it is unknown whether there is a corresponding cost, as with
dot cues, at irrelevant locations. How do face cues modulate the benefits and
costs of attention, at attended and unattended locations respectively?
Based on the finding that face cues result in greater attentional benefit
when they depict fearful than neutral expressions (Phelps et al., 2006), in a
pilot experiment we tested if we would find both differential benefits (at cued
locations) and costs (at uncued locations) for fearful and neutral faces.
Although we found no effect of emotion (see Discussion), we did discover an
intriguing pattern of results mediated by handedness: for left-handers the
cueing effect depended on the location of the target in the visual field.
Interestingly, lesion and imaging studies with right-handers have
revealed that face perception and visuospatial attention are hemispherically
lateralized. Face recognition is a specialized process of the right hemisphere
33
(Luh, Redl & Levy, 1994). Consistent with this finding, people are better at
recognizing faces in the left than right visual field (LVF; RVF) (Rhodes, 1985).
Greater face-related activity in the right than left fusiform face area, as
assessed by EEG and fMRI, is thought to underlie this LVF advantage (Yovel,
Levy, Grabowecky & Paller, 2003; Yovel, Tambini & Brandman 2008).
Visuospatial attention is also associated with greater activity in the right
hemisphere (Siman-Tov, Mendelsohn, Schonberg, Avidan, Podlipsky et al.,
2007), with attention benefiting detection (Fecteau, Enns & Kingstone, 2000)
and discrimination (Evert, McGlinchey-Berroth, Verfaellie & Milberg, 2003)
tasks more in the LVF than RVF. Correspondingly, more severe attention
deficits result from lesions to the right- than left- parietal lobe (Mesulam, 1999).
As a group, compared to right-handers, left-handers show more inter-subject
variability in these lateralized brain functions (e.g., Dronkers & Knight, 1989;
Luh et al., 1994).
It is unknown whether brain lateralization differences observed in right-
and left-handers could lead them to exhibit different behavior in experiments
that tap into the lateralized functions of face processing and covert attention.
Hence, in the present study, using an exogenous attention procedure (Pestilli
& Carrasco, 2005), we systematically investigated how the interaction of
handedness and attention cued with faces (Experiment 1) or with dots
(Experiment 2) affects visual performance.
34
METHODS
Experiment 1: Effects of faces as exogenous cues
Participants
Six right-handed (2 males, 20-34 years, M=26.8) and six left-handed (1
male, 24-31 years, M=27.5) observers participated. All had normal or
corrected-to-normal vision and completed the 10-item Edinburgh Handedness
Inventory (Oldfield, 1971). A score of +100 on the inventory indicates complete
right-hand dominance whereas a score of -100 indicates complete left-hand
dominance. Right-handers scored +78 (SD=21) and left-handers scored -83
(SD=14) on average.
Apparatus
Stimuli were presented on a 21” ViewSonic P220f monitor (1600x1200
pixels; 75 Hz) connected to a Power Macintosh G4 computer via an
attenuator. Background luminance was set to 16.5 cd/m2. During the
experiment participants’ heads were stabilized using a chin rest 57 cm from
the monitor.
35
Stimuli
Face stimuli consisted of 22 contrast- and luminance-equated grayscale
pictures of fearful and neutral faces from the Pictures of Facial Affect series
(Ekman & Friesen, 1976). Gabor patches (sinusoidal gratings in a Gaussian
envelope, SD=1˚; 4 cpd) were created using Matlab 5.2.1 and the
Psychophysics Toolbox (Brainard, 1997). The face cues subtended 4x5.3˚,
and were centered 5˚ horizontally and 2.65˚ above fixation. The Gabor
patches subtended 6x6˚ and were centered 5˚ horizontally and 4˚ below
fixation. Gabor patch contrast ranged from 3.4% to 56.7% in 7 log steps.
Gabor tilt ranged from 3 to 6˚, chosen for each observer individually based on
a ~62.5%-correct criterion in pretesting.
Procedure
Observers were seated in a darkened room. On each trial, they fixated
a central cross for 500 ms; then a face cue was presented to the left, right or
on both sides above fixation for 80 ms to manipulate exogenous attention;
following a 53 ms ISI, one tilted (the target) and one vertical Gabor patch were
presented, one on either side below fixation, for 40 ms. Participants indicated
the target location (left or right) and orientation (counterclockwise or
clockwise), with a single button press (Figure 1). Feedback was given after
each trial by a high tone for correct and a low tone for incorrect responses.
36
Cues appeared on the same side as targets (Valid), the opposite side (Invalid),
and on both sides (Distributed) with equal probability (1/3). Observers
completed 3,340 trials on average. See Appendix A for detailed task
instructions.
Analysis
For each condition, we calculated percent correct as a function of
contrast. Psychometric functions were fitted using psignifit 2.5.6 (Weibull;
http://bootstrap-software.org/psignifit/; Wichmann & Hill, 2001). Contrast
threshold was indexed by the stimulus intensity at which observers were
correct 67% of the time (about halfway between chance, 25%, and perfect
performance, 100%). The primary dependent variable was contrast sensitivity
(CS), which is inverse contrast threshold. Observers’ CS scores were
individually normalized by dividing each condition mean by the average of all
conditions to reduce the influence of baseline CS differences across
observers. Normalized CS scores were then averaged across observers in
each handedness group. Reaction times (RTs) were also measured.
Experiment 2: Effects of dots as exogenous cues (Control)
All experimental parameters for Experiment 2 were the same as for
Experiment 1 except for the following: (1) five out of six observers from each
37
handedness group in Experiment 1 participated in Experiment 2; (2) black dot
cues (0.3˚ diameter, 1˚ above and 5˚ horizontally from fixation) were used
instead of face cues; (3) Gabor targets were always tilted ± 4˚; and (4)
observers completed 4,000 trials on average.
RESULTS
Given that there were no differences in performance or RT between
facial expression conditions (fearful vs. neutral face cues, p>0.1), the data
were averaged across both expressions. Here we report detailed statistics for
CS, and note that the RT analyses showed no speed-accuracy trade-offs for
any comparison. For each experiment, there are two within-subject factors:
cue validity (Valid, Distributed, Invalid), and visual field (LVF, RVF); there is
also one between-subject factor: handedness (Left, Right).
To determine whether cue validity interacted with handedness and
visual field and whether this interaction depended on cue type, three-way
mixed factorial ANOVAs were performed for face and dot data separately with
cue validity, visual field, and handedness as factors. There was a significant
interaction of the three factors for faces (F(2,20)=14.349, p<0.001), but not for
dots (F(2,16)=1.652, p>0.10). To better understand how faces are different
from dots, we first present an analysis of the effect of cue type alone. Then,
38
because our pilot study indicated left-hander performance depended on the
target location in the visual field, we investigate the effect of handedness, as
well as the effect of visual field. Lastly, we examine the relation between
degree of handedness and cue validity effect.
Effects of Cue Type
To evaluate the effect of cue type, CS was averaged over both visual
fields and handedness conditions separately for face and dot cue data (Figure
2A). One-way repeated measures ANOVAs performed on cue validity (Valid,
Distributed, or Invalid) indicated that it changed CS marginally when faces
were used (F(2,22)=3.091, p=0.066, !2RM=0.22); however it changed CS
significantly when dots were used (F(2,18)=10.663, p<0.001, !2RM=0.54). The
results replicated previous findings for dots (Carrasco et al., 2000; Ling &
Carrasco, 2006; Pestilli & Carrasco, 2005), but not for faces (Phelps et al.,
2006). Faces decrease the magnitude of the cue validity effect but, at this
point in the analysis, it is unclear why this is the case.
Effects of Handedness
We then split the data to evaluate the effect of handedness. When right-
handers were cued with either faces or dots, cue validity significantly changed
39
CS (face: F(2,10)=5.992, p<0.02, !2RM=0.55; dot: F(2,8)=11.838, p<0.005,
!2RM=0.75; Figure 2B). Although when left-handers were cued with dots cue
validity significantly changed CS (F(2,8)=5.904, p<0.05, !2RM=0.6), this was
not the case when they were cued with faces, (F(2,10)<1). The decreased
magnitude of the initial face-cue validity effect across handedness appears to
be due to the lack of CS modulation in left-handers when cued with faces.
Effects of Handedness and Visual Field
Next, to evaluate the effect of visual field, the data were split based on
whether the target appeared on the left or right side of the screen. Within each
handedness group, two-way repeated measures ANOVAs were performed on
target visual field (LVF, RVF) and cue validity (Valid, Distributed, Invalid)
separately for face and dot cues. Left-hander face data revealed a significant
interaction between VF and cue validity (F(2,10)=6.519, p<0.02; Figure 3A).
When targets were in the LVF, valid-face cues resulted in the highest CS,
followed by distributed and invalid cues (Ms = 1.15, 1.10, and 1.01,
respectively). However, a different pattern was found when targets were in the
RVF: invalid cues led to the highest CS, followed by distributed and valid cues
(Ms = 0.97, 0.90, and 0.86, respectively). In contrast, left-hander dot data
revealed a main effect of cue validity (F(2,8)=5.905, p<0.05) with valid-dot
40
cues leading to the highest CS followed by distributed and invalid cues (Ms =
1.12, 1.02, and 0.86, respectively). VF and cue validity did not significantly
interact (F(2,8)=1.699, p>0.10).
Right-hander face data revealed a significant interaction of VF and cue
validity (F(2,10)=7.93, p<0.01; Figure 3B): CS was higher in the RVF than
LVF, and there were greater differences in CS due to cue validity in the RVF
than LVF (Valid–Invalid CS = 0.5 and 0.18, respectively). Right-hander dot
data revealed a main effect of VF (F(2,8)=18.954, p<0.02) with CS being
higher in the RVF than LVF (Ms = 1.18 and 0.82, respectively). There was
also a main effect of cue validity (F(2,8)=11.837, p<0.005) with valid cues
resulting in the highest CS, followed by distributed and invalid cues (Ms =
1.52, 0.98 and 0.5, respectively). VF and cue validity did not significantly
interact (F(2,8)=1.5, p>0.10).
Lastly, we examined the correlation between each individual’s cue
validity effect (Valid–Invalid CS) and their handedness score (Figure 4).
These two indices were positively and significantly correlated when targets
appeared in the RVF with both face (R2=0.56, p<0.01) and dot (R2=0.45,
p<0.05) cues; however, it appears that these correlations may each be due to
different underlying mechanisms (see Discussion). When targets appeared in
the LVF, this effect was marginal with dot cues (R2=0.38, p=0.058) but no
such correlation emerged with face cues (R2=0.03, p>0.1).
41
DISCUSSION
Does covert attention evoked by face and dot cues have comparable
benefits and costs on contrast sensitivity? Critically, the answer to this
question depends on the observer’s handedness. For right-handers, both
faces and dots are effective at eliciting attention, resulting in a benefit at cued
and cost at uncued locations (Figure 2B, third and fourth triplet from L to R).
Conversely, for left-handers, faces and dots elicit attention differentially:
whereas dot cues result in enhanced CS with attention, face cues have a
different pattern in each hemifield. When faces cued LVF targets, valid cues
increased CS, and invalid cues decreased CS, relative to distributed cues.
However, when faces cued RVF targets, valid cues decreased CS and invalid
cues increased CS relative to distributed cues (Figure 3A, first and third triplet
from L to R). Consequently, averaging over both hemifields resulted in no net
effect of cue validity in left-handers (Figure 2B, first triplet on left).
Previous studies have shown that when exogenous attention is
manipulated via dot or bar cues, it elicits attentional benefits and costs at cued
and uncued locations, respectively (Carrasco et al., 2000; Montagna et al.,
2009; Pestilli & Carrasco, 2005), and that face cues elicit attentional benefits
at cued locations (Phelps et al., 2006). The present study replicated previous
dot cue findings for all observers, and revealed that, for right-handers, the
42
benefits of face cues were accompanied by costs at the uncued locations.
These findings support selective attention’s role in helping to manage limited
resources that result in processing trade-offs (Carrasco, 2006; Kinchla, 1992;
Pestilli & Carrasco, 2005).
Although we had expected both benefits and costs of attention on CS to
be mediated by facial expression (Phelps et al., 2006), no such differences
emerged. A recent study suggests a possible explanation: the valence effect
of facial expression interacts with Gabor spatial frequency. There is no
advantage of fearful faces on the perception of oriented stimuli with spatial
frequency greater than 2 cpd (Bocanegra & Zeelenberg, 2009). These results
suggest that the beneficial effects of emotion are restricted to low spatial
frequencies. Whereas in our previous study we used 2 cpd stimuli, in the
present experiment we used 4 cpd stimuli, which may have resulted in the null
effect.
Could cue complexity rather than “faceness” account for the visual field
and handedness effects? A recent study suggests that the effect is face-
specific. Face cues produce greater differences in RT to detect cued and
uncued targets than equivalently complex phase-scrambled and inverted
faces, but only in the RVF (Elder, Balaban, Kamyab, Wilcox & Hou, 2008).
Consistent with that study, the present results show that performance with
face cues is also affected by visual field asymmetries. One aspect of the data
43
that can be explained by differential cue complexity is that in general, dot cues
result in overall greater cue validity effects and contrast sensitivity than face
cues. To make expressions discriminable, the size of the faces are much
larger than the dots, which may have resulted in a more diffuse attention boost
due to the trade-off between attention field size and spatial resolution (Eriksen
& St. James, 1986). However, the contrast and luminance of the faces were
equated across the whole set, resulting in much lower contrast for faces than
dots, which could also make them less effective exogenous cues (Fuller, Park
& Carrasco, 2009).
Furthermore, the present results show that the cue validity effects are
stronger in the RVF for right-handers, and LVF for left-handers. This is
consistent with the dominant-hand attentional bias seen in the Simon Effect
(Rubichi & Nicoletti, 2006), which reflects an interaction of target location and
the location of the hand used to make the response on RT. Responses are
faster when made with the hand adjacent to the target, compared to the
opposite hand. A larger Simon Effect is observed in the hemifield
corresponding to the dominant hand: for right-handers, the difference in RT
between hands to make an RVF response is larger than the corresponding
difference for an LVF response. Crucially, right-handers have faster RT for
RVF targets when they respond with their right-hand, and slower RT for LVF
targets when they respond with their left-hand (vice versa for left-handers).
44
This effect is thought to be due to spatial attention, which allows a more
efficient response selection for the dominant hand. In the present experiments,
this attentional bias may also explain the increased CS and cue validity effects
in the visual fields corresponding to each group’s dominant hand.
The degree of handedness and the magnitude of the attention effect
were significantly correlated in the RVF for both cue types, whereas the LVF
correlation was insignificant for faces and only marginal for dots (Figure 4).
However, the significant RVF correlations for faces vs. dots may have different
underlying phenomena. The significant face correlation is driven both by a
decrease in left-handers’ and increase in right-handers’ cue validity effect
(Figure 4, top-right panel); the use of face cues seems to affect attentional
deployment to the RVF in opposing ways in these two groups. This pattern of
results is consistent with the difference in degree of lateralization and inter-
subject variability for these two groups (Boles, 1989; Luh et al., 1994),
especially with regard to face processing (Bourne 2008). In contrast, the
significant dot correlation is primarily driven by closer clustering of right-
handers’ cue validity effect, with no real change in left-handers’ cue validity
(Figure 4, bottom-right panel); this finding indicates that attentional
deployment to the LVF results in an increase of CS for everyone but to
different degrees. This pattern of results is consistent with the existence of
attentional asymmetries across the visual field (e.g., Fecteau et al., 2000),
45
which depends in part on handedness (i.e., Simon Effect: Rubichi & Nicoletti,
2006).
Regardless of the differences between visual fields, why might the
effect of faces on covert exogenous attention depend on handedness? It is
possible that in left-hander brains, attention-related signals have to travel
farther to boost the processing of spatially specific locations cued by faces
than by dots. The right hemisphere of the right-handed brain is dominant for
both face and attention processing, allowing for efficient interactions of face
cues and attention signals. However, given their variability in degree of
lateralization, the functions of left-handed brains may be more distributed,
leading to greater distances between face- and attention-related regions. As a
result, left-handers as a group may not experience the same benefits and
costs of attention on CS when cued with faces as right-handers.
Even though left-handers comprise 10% of the population (Raymond,
Pontier, Dufour & M"ller, 1996), they are excluded from most cognitive
psychology and cognitive neuroscience studies because researchers are
concerned with laterality issues. Conversely, visual perception data from right-
and left-handers are usually averaged. However, we show here that
handedness is a critical variable affecting not only higher cognitive processes
but also perception. Our visual systems have evolved to become “face
46
recognition experts”, a specialization that interacts differently with attention in
right- and left-handers.
47
CHAPTER 2
Emotion and Attention Costs on Contrast Sensitivity:
Influences of Anxiety and Sex
(Unpublished, under review at Emotion)
Emma Ferneyhough1, Min K. Kim1, Elizabeth A. Phelps1,2 & Marisa Carrasco1,2
1New York University Psychology Department
2New York University Center for Neural Science
48
Abstract
Emotion and attention affect accuracy and response time in visual
detection and discrimination tasks. Anxiety modulates these effects. Here we
investigate how individual differences in trait anxiety and sex influence the
interaction of emotion and attention on contrast sensitivity, a basic visual
dimension. In two experiments, non-predictive precues directed exogenous
(involuntary) attention to contrast-varying stimuli (Gabor patches). Precues
were faces with either neutral or fearful expressions and were presented to
one or both sides of central fixation along the horizontal meridian (Experiment
1) or at one or four locations along the intercardinal meridians (Experiment 2).
On each trial, a tilted Gabor target was displayed randomly at one of the
possible task locations, concurrently with distracter(s). Attention was thus
randomly cued toward the target (valid cue), a distracter (invalid cue), or
distributed over all locations. Observers discriminated target orientation on
each trial, and completed self-report measures of anxiety. Consistent with
previous research, fear-distributed cues significantly improved contrast
sensitivity compared to neutral-distributed cues (Expt. 1). We also found that
emotion significantly interacted with attention resulting in perceptual benefits
and costs, but this depended on trait anxiety (Expts. 1 & 2) as well as sex
(Expt. 1) of the observer. Specifically, with two task locations high trait anxious
females showed increased contrast sensitivity with fear-valid cues and
49
decreased contrast sensitivity with fear-invalid cues while males showed no
effect (Expt. 1). With four task locations, all high trait anxious individuals
showed costs of emotion (Expt. 2), suggesting sex differences are reduced
with greater attentional demand. These findings are discussed in regards to
known sex differences in facial expression recognition and effects of anxiety
on response to threat-related stimuli.
Acknowledgements
We thank Damian Stanley, Tobias Brosch and David Carmel for helpful
discussions, as well as other Phelps and Carrasco Lab members for
comments on earlier versions of this manuscript. This research was funded by
grants NIH R01-EY016200 to M.C. and NIH R01-MH062104 to E.A.P.
50
INTRODUCTION
Emotion influences many cognitive processes such as learning,
memory, motivation, and decision making, and has been found to have at
least two distinct effects on visual attention and low-level visual perception.
Emotion can improve attention and perception under certain circumstances,
but it can also impair them in others. These two effects are consequences of
the finding that, compared to neutrally-valenced stimuli, emotional stimuli and
stimuli associated with emotions are preferentially processed in terms of
speed and depth (for a review see Compton 2003). This preferential
processing occurs especially for stimuli endowed with negatively arousing and
potentially threatening emotions such as fear and anger. When threat stimuli
attract attention to the location of an experimental task, for example,
performance typically improves; however when threat stimuli distract attention
away, performance is typically impaired. These benefits and costs due to the
interaction of emotion and attention have been shown in studies measuring
reaction time, whereas only the benefits have been demonstrated on
perception, specifically the basic visual feature of contrast sensitivity (Phelps,
Ling & Carrasco, 2006). Here we ask whether there is also a cost of the
interaction of emotion and attention on contrast sensitivity.
51
Fast, exhaustive processing of threat is crucial for survival (LeDoux
1996), but this threat-advantage is manifested differently in our attentional and
perceptual abilities. In the absence of emotion, it is known that covert
exogenous attention (i.e., attending reflexively while gaze is fixed) is a finite
cognitive resource that improves early visual processes at attended locations
but impairs them at unattended locations (for reviews see Carrasco, 2006,
2011). Attention researchers use peripheral cues consisting of dots or bars to
direct exogenous attention, which is driven involuntarily by a transient change
in the visual field. The effect is maximal at about 100-120 ms post cue onset,
and decays shortly thereafter (Nakayama & Mackeben, 1989; Cheal & Lyon,
1991; Fuller, Rodriguez & Carrasco, 2008). When exogenous attention is cued
to a spatial location, performance on visual tasks is improved there (Carrasco,
Penpeci-Talgar & Eckstein, 2000; Ling & Carrasco, 2006a), but this comes at
a cost to performance at uncued locations. These changes in performance
occur even though the cues are uninformative (i.e., they do not predict the
target location). Most relevant for the present study are the benefits and costs
of exogenous covert attention on contrast sensitivity (Pestilli & Carrasco, 2005;
Pestilli, Ling & Carrasco, 2009; Pestilli, Viera & Carrasco, 2007).
The conjoint effect of an emotional stimulus with attention results in
greater benefits and costs in performance compared to a neutral control.
Evidence for these tradeoffs have been mounting steadily, but have been
52
primarily in the form of reaction time (RT) differences in attention tasks, when
either emotional or neutral stimuli were used. As described in detail below,
tasks with emotional stimuli are typically completed faster at attended
locations, but are slower at unattended locations, compared to neutral stimuli.
Whether enhanced emotion processing produces benefits or costs to
attention and perception critically depends on how relevant the emotional
stimulus or cue is to the task at hand. One way to manipulate stimulus/cue
relevance is by changing its spatial location in relation to a target task
stimulus. Using Posner and colleagues’ (Posner & Petersen, 1990) three
components of spatial attention (“shift-engage-disengage”) as a simple model,
researchers have conducted experiments to investigate what effects emotion
can have on the shifting, engagement, and disengagement of attention (e.g.,
Derryberry & Reed, 2002; Yiend & Mathews, 2001). An emotional cue such as
a picture or word, for example, may improve target processing in its vicinity
due to the beneficial effects of attentional shifting to, and engagement with,
task-relevant locations (valid cue). This enhanced attentional engagement with
emotional stimuli, compared to a neutral control, results in greater accuracy
and faster RT on experimental tasks. If that same emotional stimulus is at a
task-irrelevant location, however, it can impair target processing due to costs
of attentional disengagement and shifting from the task-irrelevant back to the -
relevant location (invalid cue). This impaired attentional disengagement from
53
emotional stimuli, compared to a neutral control, results in decreased accuracy
and slower RT on experimental tasks (e.g., Fox, Russo, Bowles & Dutton,
2001).
Benefits and costs to RT have also been demonstrated in a spatial
cueing experiment using fear conditioning. Koster, Crombez, Van Damme,
Verschuere and De Houwer (2004) measured detection RTs for targets cued
with stimuli paired (conditioned stimulus: CS+; predicts threat) and not paired
(CS-; does not predict threat) with an unconditioned stimulus (an aversive
white noise burst). Targets that were validly cued by a CS+ were detected
faster compared to those cued by a CS-. On the other hand, RTs were slower
when these targets were invalidly cued by a CS+ compared to CS-. The
authors concluded that the slowing of RT was due to a delayed
disengagement from the stimulus that predicted threat.
In addition to effects on RT, research has recently focused on
psychophysical investigations of the interaction of emotion and attention on
fundamental dimensions of visual perception. Contrast is a visual feature that
underlies stimulus visibility. Perception of contrast occurs at the earliest levels
of the cortical visual hierarchy, area V1. Thus, contrast sensitivity, unlike RT,
carries important information about the strength of the initial perceptual signal
as it enters primary visual cortex (e.g., Boynton, Demb, Glover & Heeger,
1999; Graham, 2011). As a consequence of its elementary nature,
54
improvement (or impairment) of this signal by emotion and/or attention can
then influence a vast array of perceptual and cognitive processes downstream.
Phelps et al. (2006) used visual psychophysics methodology to
investigate how emotion and attention interact to affect contrast sensitivity. On
each trial, a fearful face precue was briefly presented, reflexively drawing
exogenous, covert (automatic and transient, without eye movements) attention
to its location. When this precue appeared just prior to the onset of a tilted
target Gabor patch (a luminance-defined, sinusoidal grating convolved with a
Gaussian), participants’ orientation discrimination improved, compared to the
presentation of a neutral face precue. This improvement in performance was
more pronounced when the fearful face was a valid cue (spatially informative)
compared to a distributed cue (appearing at all possible locations, therefore
not spatially informative). Given that increased performance on orientation
discrimination tasks depends on increased contrast sensitivity (e.g., Carrasco
et al., 2000; Ling & Carrasco, 2006a; Pestilli et al., 2009), these results
showed that emotion improved contrast sensitivity, and this effect was
facilitated by the beneficial effect of attention in valid trials. There were no
invalid cues, however, leaving open the question of how emotion affects
contrast sensitivity at unattended locations.
The evidence described thus far indicates that emotion and attention
interact to produce benefits and costs for RTs (Koster et al., 2004) and only
55
benefits for contrast sensitivity (Phelps et al., 2006). It remains unclear,
however, whether there is also a cost of emotion for contrast sensitivity in
invalidly cued trials. In other words, does emotion affect the cost of attention
disengagement for the perceptual signal when cueing task-irrelevant (invalid)
locations? Here, we test the hypothesis that emotion, like attention, can
reflexively draw resources resulting in both improved contrast sensitivity at
validly cued locations and impaired contrast sensitivity at invalidly cued
locations compared to a neutral control.
Of particular relevance to our experimental design is a recent study that
showed both benefits and costs of emotion in orientation detection
performance (Bocanegra & Zeelenberg, 2009). Whether there was a benefit or
cost crucially depended on target spatial frequency: the orientation of low
spatial frequency (LSF) targets was detected better, whereas the opposite was
true for high spatial frequency (HSF) targets, when fearful (compared to
neutral) precues were used. Spatial frequency, like contrast, is another basic
feature of vision; unique neural pathways preferentially process low vs. high
spatial frequencies in natural images. This study did not manipulate spatial
attention (two spatially uninformative cues always appeared at the same time
adjacent to both possible target locations), precluding any conclusions
regarding the interaction of emotion and attention. However, the results of this
study underscore the fact that emotion and perception are not independent.
56
Furthermore, these results are consistent with a prior demonstration of the
importance of coarse, LSF information for recognizing emotional facial
expressions, especially threat (Vuilleumier, Armony, Driver & Dolan, 2003).
Within a short time window after fearful face presentation, visual channels
sensitive to LSFs may confer a perceptual benefit to other LSF stimuli in the
vicinity. In light of these results, we hypothesized that the LSF information in
fearful faces would lead participants to be more sensitive, not only to
orientation, but to the contrast of LSF target stimuli as well.
Manipulating externally observable aspects of stimuli in an attention
paradigm, such as cue emotionality or target size and frequency content,
undoubtedly have measurable consequences on the outcome of experimental
results. At the same time, experiment participants’ mental state, personality
characteristics, or sex, should be taken into consideration. In particular, there
exists a wide range of emotional dispositions in the general population. What
is considered “normal” is highly variable. Furthermore, differences between
males and females in regards to brain function (Cahill, 2006) as well as
personality tendencies have been well documented; for example, females are
more likely to have symptoms of anxiety than males (Kessler, Berglund,
Demler, et al., 2005). While investigating costs of emotion on attention, it is
therefore important to look at individual variability in factors known to modulate
emotional effects on attention. Anxiety and sex are two such factors.
57
Non-clinical trait anxiety, for example, correlates positively with both the
benefits (Macleod & Mathews, 1988; Mogg, Holmes, Garner & Bradley, 2008;
Öhman, Flykt & Esteves, 2001) and costs (Fox et al., 2001; Fox, Russo &
Dutton, 2002; Koster, Crombez, Verschuere & De Houwer, 2006; Smith, Most,
Newsome & Zald, 2006; Yiend & Mathews, 2001) of visual attention in tasks
measuring RT and accuracy. Heightened attention, and prolonged
maintenance of attention, to potential threat is thought to underlie anxiety
sufferers’ propensity to dwell on negative thoughts and feelings in the absence
of a trigger, thereby perpetuating anxious symptoms. It remains unknown,
however, whether anxiety modulates the effects of attention on basic visual
dimensions such as contrast sensitivity.
Participant sex may also mediate interactions of emotion and attention
on perception. Females are more sensitive to faces, performing better than
males on tasks involving facial expression discrimination, recognition and
identification (McClure, 2000; Thayer & Johnsen, 2000). Given that the
present study uses fear face stimuli, heightened sensitivity in females to these
threatening facial expressions could result in differences between the sexes in
attentional selectivity for faces. In addition, sex may interact with anxiety-
related attention biases. Anxious females and males are known to cope with
threat differently; while females internalize feelings and are avoidant of threat,
males are more likely to externalize feelings and act out (for a review see
58
Craske, 2003). Different overt coping strategies could be associated with
different covert attention biases in the presence of threat.
The goal of the present study was to investigate whether using fearful
(compared to neutral) faces as attentional cues would lead to benefits and
costs for contrast sensitivity, specifically with low spatial frequency targets. We
hypothesized that fearful face cues will exaggerate both the benefits and the
costs of attention on perception. To investigate whether there is a connection
between the magnitude of emotion’s effects on attention and trait anxiety, we
recruited participants who spanned a wide range on self-report measures of
state- and trait-anxiety. In Experiment 1, equal numbers of male and female
participants were recruited, enabling an investigation into possible interactions
of anxiety and sex. In Experiment 2, we increased the number of possible task
locations to further tax the limits of spatial attention cued with faces.
EXPERIMENT 1: Two task locations
METHODS
Participants
Fifty-six (28 female; age M = 21, SD = 4, range = 18-33) right-handed
observers were recruited. All had normal or corrected-to-normal vision and
completed the 10-item Edinburgh Handedness Inventory (Oldfield, 1971)
59
before participating. Possible scores ranged from -100 to +100 (completely
left- to completely right-handed, respectively). Only observers with
handedness scores ! 40 participated (M = 79, SD = 17). All observers
completed the 20-item Positive and Negative Affect Scale (PANAS: Watson,
Clark & Tellegen, 1988) and the 40-item State-Trait Anxiety Inventory (STAI:
Spielberger, Gorsuch, Lushene, Vagg & Jacobs, 1983) at the experiment’s
conclusion. The PANAS was used to assess the degree to which different
positive and negative emotions were experienced in general over the previous
six months, and scores could range from 10 to 50 within either positive or
negative affect (positive affect: M = 34, SD = 7; negative affect: M = 21, SD =
7). The STAI was used to assess degree of anxiety at the present moment
(state) and in general (trait), and scores could range from 20 to 80 within either
state or trait anxiety (state anxiety: M = 38, SD = 10; trait anxiety: M = 40, SD
= 12). Negative affect and trait anxiety are typically highly correlated with each
other.
Apparatus and Stimuli
Stimuli were presented on a 21” CRT monitor (1600 x 1200 pixels; 75
Hz) connected to a Power Macintosh G4 computer via an attenuator driving
just the green gun (providing a larger possible set of distinct luminance levels).
60
Background luminance was set to 16.5 candelas/m2. During the experiment,
participants’ heads were stabilized using a chin rest 57 cm from the monitor.
Face stimuli consisted of 22 contrast- and luminance-equated grayscale
pictures of fearful and neutral faces from the Pictures of Facial Affect series
(Ekman & Friesen, 1976; same as used in Phelps et al., 2006 and
Ferneyhough, Stanley, Phelps & Carrasco, 2010). Gabor patches (SD = 1
degree (deg); 1.5 cycles per deg (cpd)) were created using Matlab 5.2.1 and
the Psychophysics Toolbox (Brainard, 1997). The face cues subtended 3.5 x
4.6 deg and were centered 8 deg to the left and right of fixation along the
horizontal meridian. The Gabor patches subtended 3 deg and were centered 4
deg to the left and right of fixation. The Gabor target was always tilted 6 deg
right or left from vertical, whereas the distracter was vertical (0 deg; e.g., Liu,
Pestilli & Carrasco, 2005). Seven Gabor patch contrasts were chosen
individually per observer to obtain performance levels that ranged from chance
to asymptotic performance with the goal of having at least three contrasts
within the dynamic range of fitted psychometric functions.
Procedure
Observers were seated in a darkened room and completed an
orientation discrimination task. Performance on orientation discrimination tasks
such as this, in which the target Gabor contrast varies systematically from trial
61
to trial, is commonly used to assess contrast sensitivity (e.g., Carrasco et al.,
2000; Ling & Carrasco, 2006a; Pestilli & Carrasco, 2005; Pestilli et al., 2009).
On each trial, observers fixated a central cross for 500 ms; then a face precue
(fearful or neutral) was presented to the left, right or on both sides of fixation
for 80 ms to manipulate exogenous attention; following a 53 ms ISI, one tilted
(the target) and one vertical Gabor patch were presented, one on either side of
fixation, for 40 ms. Participants were instructed to indicate the target location
(left or right) and orientation (counterclockwise or clockwise), with a single
button press within a 2000 ms response window (Figure 5). Feedback was
given after each trial by a high tone for correct and a low tone for incorrect
responses. Valid cues appeared on the same side as targets, invalid cues
appeared on the opposite side, and distributed cues appeared on both sides.
Each cue type appeared in 1/3 of the trials. Observers completed 672 trials
each (16 trials per condition per contrast level) in one 1-hour session. See
Appendix B for detailed task instructions.
Prior to the 1-hour experimental session with face cues, each observer
completed a half-hour practice session with black dot cues (0.3 deg diameter,
8 deg eccentricity) to avoid habituation to facial expression. From performance
on the practice session, we determined each individual’s contrast range to
allow an average performance of ~67% (about halfway between chance, 25%,
and perfect, 100%, performance) in the distributed dot cue condition.
62
Analysis
For each condition, we calculated percent correct as a function of
contrast. Psychometric Weibull functions were fitted using psignifit 2.5.6
(http://bootstrap-software.org/psignifit/; Wichmann & Hill, 2001). Contrast
threshold was defined as the estimated stimulus intensity at which observers
would be correct 67% of the time. The primary dependent variable was
contrast sensitivity, which is the inverse of contrast threshold. Observers’
contrast sensitivity scores were individually normalized by dividing each
condition mean by the average of all conditions; such normalized scores
reduced the noise introduced by different baseline contrast sensitivity for
different observers (e.g., Ferneyhough et al., 2010). Normalized contrast
sensitivity scores were then averaged across all observers. Reaction times
were also measured as a secondary dependent variable. Here we report
detailed statistics for contrast sensitivity and note that for RT there were no
speed-accuracy trade-offs for any comparison.
RESULTS
One of the six conditions for seven of the observers (~2% of the total
experiment data) could not be reliably fit with a Weibull function (deviance
63
scores, which assess goodness of fit, exceeded #2.05(7) = 14.1; .05 refers to p-
value; 7 refers to number of contrast levels), so their data were discarded. In
addition, the data of three observers whose contrast sensitivity were > 3 SDs
from the mean on one of the six conditions were also discarded. There was no
consistency in which conditions could not be fit or resulted in outliers across
these observers. There were a total of 46 remaining observers for the data
analysis (24 male, 22 female). Self-reported state or trait anxiety were not
different between males and females (ps>.1), however, males reported greater
negative affect (males M = 22, females M = 18, t(44)=2.27, p<.05). All
remaining observers were classified as either low or high trait anxiety via a
median-split. All reported t-tests are two-tailed, unless noted otherwise.
Overall Contrast Sensitivity
A 2x3x2x2 mixed-model ANOVA was conducted on normalized contrast
sensitivity scores of all observers. Facial expression (fearful, neutral) and cue
(valid, distributed, invalid) served as within-subjects factors, and trait anxiety
(low, high) and sex (male, female) served as between-subjects factors. This
ANOVA revealed a marginally significant four-way interaction of face type, cue
condition, sex and trait anxiety (F(2,80)=2.53, p<.1), and a significant three-
way interaction of face type, cue condition and sex (F(2,80)=3.701, p<.05).
Furthermore, there was a marginal two-way interaction between face type and
64
cue condition (F(2,80)=2.551, p<.1). Follow-up paired t-tests comparing fear-
distributed with neutral-distributed cue conditions showed a significant
difference in contrast sensitivity (t(45)=2.76, p<.01) with fear-distributed cues
leading to higher contrast sensitivity (Figure 6). Fear-distributed cues also led
to higher contrast sensitivity than fear-invalid cues (t(45)=3.22, p<.01).
Sex Differences
To investigate the nature of these significant interactions in the overall
ANOVA, two 2x3x2 mixed-model ANOVAs with facial expression and cue as
within-subjects factors, and anxiety as a between-subjects factor were
conducted separately for males and females. Males showed a marginally
significant two-way interaction of face type and cue condition (F(2,40)=2.493,
p<.1). Follow-up paired t-tests revealed that fear-distributed contrast sensitivity
was greater than neutral-distributed (t(23)=1.75, p<.05, one-tailed), and
neutral-valid was greater than both neutral-distributed (t(23)=2.01, p<.05, one-
tailed) and neutral-invalid (t(23)=1.98, p<.05, one-tailed; Figure 7, top right).
Females also showed a marginally significant two-way interaction of
face type and cue condition (F(2,36)=3.128, p=.056). Fear-distributed contrast
sensitivity was significantly greater than both neutral-distributed (t(21)=2.21,
p<.05) and fear-invalid (t(21)=3.12, p<.01; Figure 7, top left). Within females,
there was also a significant three-way interaction of face type, cue condition
65
and anxiety (F(2,36)=3.282, p<.05; note that two females with median anxiety
scores were excluded from this interaction because observers were
categorized as either high or low anxiety based on a median split).
Anxiety by Sex
Interestingly, when the observers were divided into four groups based
on sex and trait anxiety scores (male low, male high, female low, female high),
we observed differences in patterns of contrast sensitivity, especially
comparing high trait anxious females to the other three groups. Figure 7
shows the contrast sensitivity of the fear and neutral expressions in the
distributed cue condition (middle bars) plotted for each of the four groups, as
well as the overall groups. Again, as described above, there was a significant
increase in contrast sensitivity with fear-distributed relative to neutral-
distributed cues across all observers (t(45)=2.76, p<.01). This difference was
significant across all female observers (n=22, t(21)=2.21, p<.05), and was
driven by the high trait anxious females (n=10, t(9)=2.59, p<.05). High trait
anxious males also showed greater contrast sensitivity in the fear-distributed
relative to neutral-distributed condition (n=11, t(10)=1.89, p<.05, one-tailed).
Next we evaluated the female participants’ difference in contrast
sensitivity with fear-valid vs. neutral-valid cues and found that it was
significantly greater than the corresponding difference in contrast sensitivity
66
with fear-invalid vs. neutral-invalid cues (t(21)=1.91, p<.05, one-tailed; Figure
7, top left). This comparison was stronger in high trait anxious females
(t(9)=2.68, p<.05; Figure 7, bottom left). The increase in contrast sensitivity
with fear-valid vs. neutral-valid cues in high trait anxious females was
marginally greater than zero (t(9)=1.72, p=.06, one-tailed). The decrease in
contrast sensitivity with fear-invalid vs. neutral-invalid cues, on the other hand,
was marginally less than zero (t(9)=-1.72, p=.06, one-tailed). Thus, significant
differences found across all females were driven by females with high trait
anxiety. There were no such differences in males.
Correlations
Trait- and state-anxiety, and trait-anxiety and negative affect scores
were significantly correlated with each other across all observers (rs=.75 and
.54, respectively; n=46, ps<.001), validating the self-report measures. Trait-
anxiety was not correlated with the difference between fear-invalid and
neutral-invalid contrast sensitivity as hypothesized, neither across all
observers nor within each sex. However, a correlation between female
negative affect scores and the difference of fear-distributed and neutral-
distributed contrast sensitivity was marginally significant (r=.4, n=22, p<.07).
Higher negative affect scores were associated with greater increases of
67
contrast sensitivity with fear- over neutral-distributed conditions. No other
correlations of self-report measures and behavior were significant.
EXPERIMENT 1 DISCUSSION
We investigated both the benefits and costs of emotion and attention on
contrast sensitivity. Our findings revealed that emotion interacted with
attention in a manner that was dependent on trait anxiety and sex. In addition,
fear-distributed cues significantly improved contrast sensitivity compared to
neutral-distributed cues, replicating a previous study (Phelps et al., 2006).
Although fear-valid and -invalid cues did not consistently modulate contrast
sensitivity across observers compared to neutral cues, group differences in
anxiety and sex indicate that only high trait anxious females demonstrated
both benefits and costs of emotion and attention on perception.
Some results of the present study were not consistent with earlier
research (Phelps et al., 2006). First, we were expecting benefits of emotion
across our whole subject population. However, we found that males,
regardless of anxiety level, and low-anxious females showed no significant
emotion effect. Second, the present research did not replicate the findings that
emotion and attention (fear-valid cues) improve contrast sensitivity above and
68
beyond that of emotion alone (fear-distributed cues). Instead, we found that
the benefits of both were similar in magnitude in high-anxious females.
Possible reasons for these two inconsistencies are the differences in
stimulus and experiment parameters between the two studies. Cue and target
sizes differed, being much smaller in the present study (cues: 5 vs. 3.5 deg
width; targets: 8 vs. 3 deg diameter). Cue and target locations in the visual
field also differed, with both cue and target eccentricity being smaller here as
well (cues: 5 vs. 4 deg eccentricity; targets: 11 vs. 8 deg eccentricity). Could
effects of emotion and attention be more exaggerated further out in the
periphery? This possibility rests on the larger receptive field sizes in peripheral
vision (DeValois & DeValois, 1988), the decreasing ratio of the number of
neurons tuned to high vs. low spatial frequencies with eccentricity (Azzopardi,
Jones, & Cowey, 1999), and the preference of the amygdala for low spatial
frequency information (Vuilleumier et al., 2003). Whereas effects of attention
increase with eccentricity (Carrasco & Yeshurun, 1998; Yeshurun & Carrasco,
1999), another study investigating emotion processing in the amygdala
showed no evidence of eccentricity effects (at -1.68, 5.6, and 11.25 deg;
Morawetz, Baudewig, Treue & Dechent, 2010). Given that trait anxiety data
were not collected in the Phelps et al. (2006) study it is possible that, by
chance, a greater proportion of this small group of observers (n=6) had higher
trait anxiety, which could have contributed to the prior results.
69
In addition, we used two stimulus locations, whereas the 2006 study
used four locations. Attentional resources may not have been sufficiently
taxed with only two locations to see a benefit of emotion and attention over
emotion alone. Anecdotally, many observers said it was easier to do the task
when two (distributed) face cues appeared, as opposed to one (valid or
invalid) face cue, because it made it easier to see both Gabor patches. Being
able to clearly see both stimuli provides an advantage because they can then
be more easily compared to one another. Given that one is always vertical and
one is always tilted, having information regarding both makes the orientation
discrimination task easier.
Experiment 2 addresses the concerns outlined above by using the
same stimulus parameters as Phelps et al. (2006) with four possible task
locations. Differences between the two experiments include: 1) instead of only
the target Gabor being tilted, all four Gabor stimuli were randomly tilted to
prevent any comparisons between target and vertical distracters; and 2) target
identity was revealed with a response cue at Gabor offset (e.g., Ling &
Carrasco, 2006b; Pestilli & Carrasco, 2005).
70
EXPERIMENT 2: Four task locations
METHODS
Participants
Forty-seven new observers were recruited (32 female; age M = 22, SD
= 4, range = 18-34). All observers had normal or corrected-to-normal vision
and were right-handed (Edinburgh Handedness Inventory M = 81, SD = 22;
Oldfield, 1971). All observers completed the PANAS (positive affect: M = 36,
SD = 6; negative affect: M = 21, SD = 7; Watson et al., 1988) and the STAI
(state anxiety: M = 38, SD = 10; trait anxiety: M = 39, SD = 10; Spielberger et
al., 1983).
Apparatus and Stimuli
Stimuli were presented on a 21” CRT monitor (1600 x 1200 pixels; 75
Hz) connected to a Macintosh Intel IMac computer. Background luminance
was set to 57 candelas/m2 (25% of its maximum luminance). The stimuli used
were the same as in Experiment 1, except they were enlarged (faces: 5 x 6.7
deg; Gabor patches: 7.9 deg, 2 cpd, tilted ± 5 deg from vertical).
71
Procedure
On day 1 each observer completed a half-hour training session with
black dot cues (0.3 deg diameter, 5 deg eccentricity). Participants who
performed ! 70% accuracy (about halfway between chance, 50%, and perfect
performance) on average throughout training continued on to the first 4 blocks
of the main experiment in which fearful or neutral faces were used as cues. On
day 2, observers returned to complete the other 8 blocks of the experiment
and to fill out the self-report surveys. In total, observers completed 1,344 trials
(112 trials per block).
Observers performed an orientation discrimination task. The trial
sequence was similar to that of Experiment 1 with the following differences: 1)
the precue was presented to either one (valid or invalid) or four (distributed)
locations along the intercardinal merdians (5 deg eccentricity); 2) four
randomly tilted Gabor patches were presented at each of four intercardinal
locations (11 deg eccentricity); 3) a response cue appeared for 100 ms at
Gabor offset indicating the location of the target Gabor; 4) participants were
instructed to indicate only the target orientation (counterclockwise or
clockwise; Figure 8).
72
RESULTS
Data from seven observers could not be reliably fit with a Weibull
function (see fitting criteria Experiment 1: Results) and four observers had
contrast sensitivity > 3 SDs from the mean, so only data from the 36 remaining
observers (23 females, 13 males) were included in the ANOVAs and t-tests.
Overall Contrast Sensitivity
A 2x3x2 mixed-model ANOVA was conducted on normalized contrast
sensitivity scores of all observers. Facial expression (fearful, neutral) and cue
(valid, distributed, invalid) served as within-subjects factors, and trait anxiety
(low, high) served as a between-subjects factor. This ANOVA resulted in a
marginally significant main effect of cue condition (F(2,68)=2.694, p=.075), in
which contrast sensitivity was highest with valid cues, then distributed and
finally invalid cues (Figure 9, top). Planned paired t-tests across observers
revealed a significant increase in contrast sensitivity in the valid cue relative to
the invalid cue condition (t(35)=1.94, p<.05, one tailed). The main effect was
qualified by a significant two-way interaction of face type and trait anxiety
(F(1,34)=4.278, p<.05).
73
Anxiety Differences
To investigate the nature of the significant interaction of face type and
trait anxiety in the overall ANOVA, planned paired t-tests were conducted
comparing the effects of fear vs. neutral face expression in each cue condition
for both anxiety groups resulting from a median split on the STAI survey. In the
high trait anxious group there was a significant decrease in contrast sensitivity
in the fear-invalid relative to both the neutral-invalid condition (t(17)=-2.741,
p<.05), and the fear-distributed condition (t(17)=2.197, p<.05; Figure 9,
bottom). In the low trait anxious group, there was a significant decrease in
contrast sensitivity in the fear-distributed relative to the fear-valid condition
(t(17)=2.432, p<.05; Figure 9, middle). No other comparisons were significant
(ps >.1).
Correlations
As in Experiment 1, trait- and state-anxiety, and trait-anxiety and
negative affect were significantly correlated with each other across observers
(rs=.66 and .74 respectively; n=38, ps<.001). Trait anxiety and the difference
in contrast sensitivity in the invalid cue condition (fear-invalid minus neutral-
invalid) were significantly correlated (r=-.36, n=36, p<.05). As trait anxiety
increased, the difference in contrast sensitivity between the fear-invalid and
neutral-invalid cue conditions also increased in the hypothesized direction.
74
EXPERIMENT 2 DISCUSSION
In Experiment 2 we modified cue and target parameters and increased
the number of possible task locations, from two to four, to address several
concerns from Experiment 1. Overall there was a significant effect of attention
in which, regardless of emotion, contrast sensitivity was highest at attended
locations (valid), intermediate with diffused attention (distributed), and lowest
at unattended locations (invalid) consistent with previous research
(Ferneyhough et al, 2010; Pestilli & Carrasco, 2005; Pestilli et al., 2007). This
result provides support for our hypothesis that attentional resources were not
sufficiently taxed with only two task locations in Experiment 1 to see benefits
or costs of attention on contrast sensitivity.
In addition, although our results showed no significant differences due
to emotion across all subjects, we did find significant decreases in contrast
sensitivity in the fear-invalid relative to both the neutral-invalid and fear-
distributed cue conditions in the high trait anxiety group across both males and
females. This result is in agreement with the results from Experiment 1,
extending the findings to both high trait anxious males and females. The
significant negative correlation of trait anxiety with the size of the
disengagement cost with emotion (fear-invalid minus fear-valid contrast
75
sensitivity) provides further evidence in support of our hypothesis that anxiety
increases attention disengagement costs with emotion.
Lastly, in this experiment, we found that the low trait anxiety group
showed significantly greater contrast sensitivity in the fear-valid relative to
fear-distributed cue condition, indicating a significant effect of attention with
fear, but not neutral, face cues. This result is consistent with the previously
found facilitation effect of emotion on the benefit of attention to contrast
sensitivity (Phelps et al, 2006).
GENERAL DISCUSSION
Across both Experiments 1 and 2 we showed that trait anxiety
increases the cost of attention disengagement from fearful faces. When a fear-
invalid cue automatically directed attention to a location incongruent with the
target, contrast sensitivity was more impaired at the target location, relative to
a neutral-invalid cue. Moreover, in Experiment 1 we showed that fear-
distributed cues significantly improved contrast sensitivity compared to neutral-
distributed cues, replicating a previous study (Phelps et al., 2006). Further,
only high trait anxious females showed benefits and costs of emotion to
attention when only two task locations were used. Using four task locations in
76
Experiment 2, we additionally replicated previous findings that showed
benefits to contrast sensitivity at attended locations, and costs to contrast
sensitivity at unattended locations (Ferneyhough et al., 2010; Pestilli &
Carrasco, 2005; Pestilli et al., 2007). Importantly, we also replicated previous
findings that showed emotion enhances the beneficial effect of attention on
contrast sensitivity (Phelps et al., 2006) in low trait anxious observers.
Exactly how anxiety biases the allocation of spatial attention is debated.
Some research suggests that those who are more anxious will be more
strongly drawn to threatening stimuli such as faces with angry or fearful facial
expressions, experiencing benefits at these attended locations (Macleod &
Mathews, 1988; Mogg, Holmes, Garner & Bradley, 2008; Öhman et al., 2001).
Other research suggests they will instead be slower to disengage from
threatening stimuli, experiencing costs at unattended locations (Fox et al.,
2001; Fox et al., 2002; Koster et al., 2006; Smith et al., 2006; Yiend &
Mathews, 2001; for reviews: Bar-Haim, Lamy, Pergamin, Bakermans-
Kranenburg & van IJzendoorn, 2007; Weierich, Treat & Hollingworth, 2008).
Experiment 1 suggests females with increased anxiety are both more strongly
drawn to threat and also have greater difficulty disengaging from threat than
other participants. This results in enhanced contrast sensitivity following fear-
valid cues and impaired contrast sensitivity following fear-invalid cues, relative
to their neutral counterparts. Experiment 2 placed more stringent limitations on
77
attentional resources by using four task locations. Within this context, high trait
anxious observers had greater difficulty disengaging from threat, resulting in
impaired contrast sensitivity following fear-invalid cues relative to neutral. In
addition, low trait anxious observers were drawn more to threat resulting in
enhanced contrast sensitivity following fear-valid relative to fear-distributed
cues.
In our task, the rapid presentation of face cues directed exogenous,
bottom-up attention towards or away from target stimuli. We used fearful
faces, commonly used to recruit the amygdala (e.g., Bishop, Duncan &
Lawrence, 2004; Dickie & Armony, 2008; Morris, Friston, Buchel, Frith, Young,
Calder & Dolan, 1998; Morris, deGelder, Weiskrantz & Dolan, 2001; Whalen
1998; Vuilleumier et al., 2003; Vuilleumier, Richardson, Armony, Driver &
Dolan, 2004), which can then strengthen cue representation via feedback
connections throughout the ventral visual pathway (Freese & Amaral, 2005)
enhancing bottom-up attention allocation. Numerous studies have investigated
this possible link between amygdala activity and enhanced signal in visual
cortex (e.g., Amaral, Behniea & Kelly, 2003; Anderson & Phelps, 2001; Morris
et al., 1998, 2001; Vuilleumier et al., 2004). No studies that we know of,
however, have investigated how anxiety might modulate perception.
Neurocognitive theories of anxiety and attention have suggested
amygdala activity is heightened in anxiety in response to sources of potential
78
threat (e.g., Davis & Whalen, 2001). This hyperactivity could in turn bias
bottom-up attention allocation more strongly towards locations of threat,
resulting in both enhanced perception at cued locations and impaired
perception at uncued locations. Recent work investigating the role of frontal
brain regions in the top-down control of emotion have shown that anxious
individuals may have, not only increased amygdala activity (Bishop, Duncan &
Lawrence, 2004; Dickie & Armony, 2008), but decreased recruitment of frontal
control regions as well (Bishop, 2008). This imbalance between bottom-up
emotional response and top-down attention could underlie the difficulty
anxious individuals have in disengaging attention from threat. Consistent with
this imbalance, it has been shown with diffusion tensor imaging that
connections between amygdala and ventral medial prefrontal cortex are
weakened in anxiety (Kim & Whalen, 2009). Furthermore, voxel-based
morphometry research has shown that increased anxiety is associated with
decreased cortical volume in brain regions implicated in anxiety disorders,
such as the amygdala, ventromedial and dorsolateral prefrontal cortex
(Spampinato, Wood, De Simone & Grafman, 2009). These studies provide
support for the idea that the expression of anxiety in an individual is closely
linked to impaired amygdala-frontal cortex interactions, which could result in
increased bottom-up response to threat. In tasks such as ours, in which both
exogenous attention and emotion are manipulated, feedback to V1 from the
79
amygdala and brain regions involved in exogenous attention shifts may
interactively modulate V1 activity resulting in the benefits and costs to contrast
sensitivity described above. With increased anxiety, stronger feedback from
the amygdala may result in greater costs. It is an open question, however, why
the benefits with emotion are less consistently found than the costs, in both
the present study and previous studies (e.g., Fox et al., 2001).
The fact that anxiety can modulate contrast sensitivity, regardless of the
specific direction of these exogenous attention effects, indicates a prioritization
of resources that enhances processing of possibly threatening stimuli in the
environment. Greater sensitivity to differences between light and dark
enhances the perception of borders and outlines of objects, which provides an
advantage in efficiently parsing threat from non-threat. Higher anxiety, at least
to some extent, may impart an even greater advantage in this process, but as
we show here this threat-advantage can come at a cost of performing visual
tasks that do not pose a threat. Evolutionarily, this is often an acceptable cost
in comparison to those from real threats. Today, however, life-threatening
situations are rare, and attentional biases due to anxiety can impair our ability
to focus on task-relevant items.
Whereas high trait anxious females showed the hypothesized benefits
and costs of emotion in Experiment 1, male participants did not. These
differences may have depended on the availability of attentional resources.
80
When attentional resources were more limited, as in Experiment 2, costs of
emotion were evident across all high trait anxious individuals. Although there
were almost twice as many females as males (n=23 and 13, respectively) both
high trait anxious groups showed decreased contrast sensitivity for fear-invalid
than for neutral-invalid trials. What could explain why only females show the
benefits and costs of emotion on attention in Experiment 1? Both females and
males experience the same degree of self-reported anxiety, however, it has
been shown that there are vast differences in their response to anxiety-
provoking stimuli (for a review see Craske, 2003). Females tend to internalize
their feelings and withdraw, avoiding threat, whereas males tend to externalize
their feelings, often resulting in more outwardly aggressive behavior. Here we
showed that there are differences in how anxious males and anxious females
allocate exogenous covert spatial attention in the presence of fearful face
expressions when attention is not severely taxed. Exactly how the
aforementioned differences in overt behavior may be related to these covert
differences cannot presently be determined, however, meriting further
investigation.
As mentioned above, males and females show different patterns of
avoidance of threat. High anxious females tend to overtly avoid threatening
situations, which may have important links with the literature on attentional
avoidance of threat. The ‘vigilance-avoidance’ hypothesis (e.g., Mogg,
81
Bradley, Miles & Dixon, 2004), for example, states that after a fast, initial
orienting to locations of threat, a voluntary avoidance component directs
attention away from the threatening stimulus. This avoidance serves to protect
the individual from further exposure, yet by doing so it tends to maintain
anxious traits because the individual is rarely able to habituate to the threat.
Evidence for avoidance has been shown in many (e.g., Holmes, Nielsen &
Green, 2008; Koster, Verschuere, Crombez & Van Damme, 2005) but not all
(e.g., Bradley, Mogg, Falla & Hamilton, 1998) studies of anxiety. Our contrast
sensitivity results for high anxious females are consistent with reaction time
studies showing vigilance towards threat. However, given that the cue and
target were presented in less than 180 ms our task was not designed to
evaluate avoidance, which has been shown to require more than 1000 ms to
emerge (Koster et al, 2005; Mogg et al., 2004). Instead our paradigm was able
to provide evidence for impaired disengagement of attention from fear cues
relative to neutral cues on high anxious females’ contrast sensitivity, which
was observable in our time frame. This unique finding extends prior work that
has shown benefits and costs of exogenous attention on contrast sensitivity, in
which non-emotional cues direct attention to the location of a target or
distracter (Pestilli & Carrasco, 2005). Here we find that fear cues provide
greater benefits and greater costs than neutral cues in high anxious females.
82
In addition to having different reactions to threat than males, females
have been shown to be more sensitive in recognizing and classifying
emotional facial expressions (McClure, 2000; Thayer & Johnsen, 2000).
Perhaps when attention is not as severely limited, this greater sensitivity is
manifested by females with higher anxiety relative to males with equivalent
anxiety levels, leading to increased attentional allocation to the face cues.
Consistent with this notion, results from an ERP study showed anxious
females had greater processing of stimuli than anxious males at an early stage
(P100, ~100 ms), however this early responding was not modulated by
valence of the stimuli (Sass, Heller, Stewart, Silton, Edgar, Fisher & Miller,
2010). Moreover, fMRI studies have shown that females have greater activity
in primary and secondary visual cortex to unpleasant relative to pleasant
stimuli, with the opposite pattern in males (Lang, Bradley, Fitzsimmons,
Cuthbert, Scott, Moulder et al., 1998), and that high trait anxious females have
increased amygdala response to unattended fearful faces compared to high
trait anxious males (Dickie & Armony, 2008). These studies add to a body of
knowledge demonstrating large differences in brain systems between males
and females (for a review see Cahill, 2006), the most relevant difference in
processing being, in many cases, that of emotional stimuli in anxiety.
It is interesting to note that the stimulus parameters and relative
locations we used in Experiment 1 closely match Bocanegra & Zeelenberg’s
83
(2009) configuration. Though they studied effects of emotion on spatial
frequency resolution and we studied effects of emotion and attention on
contrast sensitivity, we generally observed the same increase in perceptual
sensitivity with emotion alone (i.e., fear-distributed cues). Perhaps this
particular configuration used in both studies is especially well-suited for
demonstrating effects of emotion, but is relatively weak for investigating the
interaction of emotion with attention because attentional resources are not
strongly taxed. Nevertheless, even using this task design we demonstrated
that the emotion effects on attention and perception were driven by high trait
anxious females.
With the addition of two more possible target locations in Experiment 2,
we taxed spatial attention even further. For all observers, regardless of
gender, there was a benefit to contrast sensitivity with fear-valid cues relative
to fear-invalid cues. The same pattern was present with neutral cues but the
valid-invalid difference was not as great, suggesting that fearful faces more
strongly capture attention. Overall, the hypothesized benefits and costs of
attention to contrast sensitivity were augmented with four possible target
locations, and fear-valid cues significantly increased contrast sensitivity
beyond that of fear-distributed cues in low trait anxious observers. However
fear-valid cues did not significantly increase contrast sensitivity above that of
neutral-valid cues. A possible explanation for this is that Phelps et al. (2006)
84
tested fewer, more experienced observers (n=6) and collected significantly
more data from each person, over the course of six days, than the present
study. Collecting more data from a small group of experienced observers
could have lead to a more refined measurement, however, in the present case
more observers were needed in order to investigate the effects of anxiety and
sex.
In our previous work (Ferneyhough et al., 2010) that investigated the
effect of handedness on contrast sensitivity we used a similar experimental
design as Experiment 1, with two task locations. We found a significant
attention effect but no effect of facial expression, even when accounting for
negative affect (we collected PANAS self-report only). A plausible explanation
for this lies in the choice of Gabor target spatial frequency used in each study.
Specifically, we used 4 cpd Gabor targets in the 2010 study, whereas we used
2 cpd targets in the 2006 study, in which there was an interaction of emotion
with attention. Following Bocanegra & Zeelenberg’s 2009 study, we
hypothesized that the amygdala is more sensitive to the low spatial
frequencies in the fearful relative to neutral face cues (e.g., Vuilleumier,
Armony, Driver & Dolan, 2003), which then sends modulatory feedback to
ventral visual areas along magnocellular pathways (Amaral, Behniea & Kelly,
2003). This feedback may then preferentially augment processing of 2 cpd
relative to 4 cpd targets. Future work is needed to better understand the roles
85
of stimulus parameters, configuration and observer factors on how the
interaction of emotion and attention mediates perception.
In conclusion, emotion improves contrast sensitivity at attended
locations and impairs it at unattended locations when attentional resources are
limited. Furthermore, under less stringent attentional limitations, only high trait
anxious females demonstrate both benefits and costs of emotion and attention
on perception. Given that contrast sensitivity is one of the most basic
characteristics of the primary visual cortex, we show that emotion and
attention can modulate the actual perceptual signal representing a stimulus.
86
CHAPTER 3
The role of intraparietal sulcus in the emotional cost to temporal
attention
(Unpublished and not submitted)
Emma Ferneyhough1 & Elizabeth A. Phelps1,2
1New York University Psychology Department
2New York University Center for Neural Science
87
Abstract
The attentional blink (AB) task assesses the temporal limitations of
attention. In a rapid visual serial presentation, identifying a target stimulus
impairs identification of a second target stimulus that follows soon after (early
lag), but not later (late lag). Emotion has been shown to influence temporal
attention in two ways: (1) an emotional second target facilitates attention, and
(2) an emotional first target impairs attention. Previous research exploring the
neural systems of emotion’s facilitation of temporal attention has implicated a
role for the amygdala in driving bottom-up emotional responses. The goal of
the present study was to explore the neural systems linked to the emotional
cost of temporal attention, which may involve competition between bottom-up
and top-down components. Whereas the orbitofrontal cortex (OFC) and the
intraparietal sulcus (IPS) have been implicated in spatial attention costs, the
dorsolateral prefrontal cortex (DLPFC) is important for attentional control
processes. In Experiment 1, we replicate the behavioral emotional cost to
attention. In Experiment 2, we conducted an AB task in the scanner
investigating the role of these regions in the interplay of bottom-up and top-
down attentional processes that may underlie emotional costs. Emotional or
neutral distracter words appeared 3, 4, 7 or 8 lags prior to a single neutral
target. Lags 3 and 4 were within the AB window (early: 270, 360 ms
88
respectively) whereas lags 7 and 8 appeared outside (late: 630, 720 ms
respectively). Participants provided self-report anxiety and attentional control
measures. We observed greater activation in (1) the IPS, OFC and amygdala
for emotional distracter trials, and (2) DLPFC for neutral distracter trials, at
early, but not late, lags. Moreover, amygdala activity differences were driven
by high state anxiety whereas DLPFC activity differences were driven by low
trait anxiety and high attentional control, consistent with prior research. Thus,
IPS and OFC may be part of a frontoparietal network underlying attentional
costs with emotion, not only in spatial, but also temporal domains.
Acknowledgements
We would like to thank research assistants Rita Ludwig and Caroline McClave
for their help in collecting and processing data and all Phelps and Carrasco
Lab members for helpful feedback. This research was funded by Grant NIH
R01-MH062104 to E.A.P.
89
INTRODUCTION
Emotionally arousing information can both facilitate attention as well as
impair it. Being completely engrossed in a Hollywood blockbuster, for
example, can impair your ability to hear the doorbell announcing the pizza you
ordered just minutes before. Thrillers have thus perfected the art of capturing
our attention, melding together fast-paced story arcs full of action, violence
and sex. These elements engage us by eliciting emotions associated with
physiological arousal, like fear, anger and surprise. Greater engagement by
emotional stimuli, however, is accompanied by the exclusion of the low-
arousal, mundane aspects of everyday life. Costs of emotionally arousing
stimuli to attention and perception of the mundane have been studied
experimentally in both the spatial and the temporal domains, but our
understanding of the neural mechanisms underlying this phenomenon is
incomplete (for a review, Stanley, Ferneyhough & Phelps, 2009).
In the current study we use an attentional blink task (AB: Raymond et
al, 1992) to investigate the neural correlates of costs of emotion to attention in
the temporal domain. This task, in which target stimuli are embedded in a
rapid serial visual presentation (RSVP) of masking stimuli, has been used
extensively to study the temporal limitations of attention (for a review, Dux &
Marois, 2009). The AB is an impairment in correctly identifying a second target
90
stimulus (T2) that has appeared within 500 ms of correctly identifying a
preceding first target stimulus (T1). As the amount of time between T1 and T2
increases, lessening attentional demands, performance on T2 identification
progressively improves.
When T2 is an emotionally arousing stimulus, however, the AB is
attenuated at short T1-T2 time intervals relative to when T2 is a neutral
stimulus (Anderson, 2005; Anderson & Phelps, 2001; De Martino, Strange &
Dolan, 2008; Fox, Russo & Georgiou, 2005; Mathewson, Arnell & Mansfield,
2008). In this case, emotion facilitates performance when the task-relevant
item is emotional. For this AB ‘facilitation’ effect, it has been proposed that the
amygdala plays a crucial role in enhancing bottom-up processing (Anderson,
2005; Anderson & Phelps, 2001; Lim, Padmala, & Pessoa, 2009; Schwabe,
Merz, Walter, Vaitl, Wolf & Stark, 2010). The amygdala is important for
emotion processing broadly (for a review, Phelps 2006), and has feedback
connections throughout ventral visual cortex (Amaral et al., 2003) including
both primary visual cortex and ventral occipitotemporal regions involved in
visual word form recognition (e.g., Cohen, Dehaene, Naccache, Lehéricy,
Dehaene-Lambertz, Hénaff & Michel, 2000). Re-entrant processing from the
amygdala could enhance early activity (<300 ms post-stimulus onset) in these
regions (e.g., Kissler, Herbert, Peyk & Junghofer, 2007; Luo, Peng, Jin, Xu,
Xiao & Ding, 2004) improving the perception, and identification accuracy, of
91
emotional T2 stimuli. Consistent with this idea, a recent fMRI study of
emotion’s interaction with spatial attention has shown increased visual cortical
activity following fear-valid relative to fear-invalid face cues (Pourtois et al,
2006). Another fMRI study using fear-conditioned cues found both enhanced
amygdala and visual cortical activity (Armony & Dolan, 2002). Moreover, left
amygdala patients do not experience attenuation of the AB with emotional T2
stimuli (Anderson & Phelps, 2001), providing further support for the idea that
the facilitation effect is due to enhanced bottom-up processing of emotion.
An alternative variant of the AB paradigm requires identifying a single
neutral target preceded by an emotionally arousing, task-irrelevant distracter.
Instead of facilitating performance, these to-be-ignored emotional distracters
impair target identification and produce an AB at short distracter-target
intervals. This ‘capture’ task shows that even ignored emotional stimuli can
automatically capture attention and divert it from neutral, task-relevant targets
(Arnell, Killman & Fijavs, 2007; Keil & Ihssen 2004; Most, Chun, Widders &
Zald, 2005; Most, Smith, Cooter, Levy & Zald, 2007; Smith, Most, Newsome &
Zald, 2006). Neutral distracters, on the other hand, are easily disregarded.
The analogous capture effect in studies of spatial attention occurs when
emotional attention cues direct attention away from target stimuli, resulting in a
delayed disengagement from emotional, relative to neutral, cues before
attention is reoriented to the target. The magnitude of the delayed
92
disengagement effect on task performance has been found to correlate
positively with anxiety (e.g., Fox et al., 2001) and negatively with attentional
control (Derryberry & Reed, 2002), a measure of one’s ability to ignore
distractions and regulate attention allocation (for reviews: Bar-Haim, Lamy,
Pergamin, Bakerman-Kranenburg & van IJzendoorn, 2007; Weierich, Treat &
Hollingworth, 2008). Similarly, the degree to which task-irrelevant emotional
distracters disrupt target processing in the AB have also been shown to
depend on self-reported anxiety (Most et al, 2005). Higher anxiety is thus
associated with a greater inability to suppress processing of emotional
information (Fox et al, 2005).
It is believed that this inability to suppress emotion processing, both in
the spatial and temporal domains, is due to failures in top-down attentional
control, functions of prefrontal and parietal cortices (for a review: Cisler &
Koster, 2010). When emotional distractions automatically capture attention
through facilitative bottom-up mechanisms centered on the amygdala,
reorienting attention back to a task-relevant target likely requires top-down
cognitive processes. These processes may involve both suppressing
emotional responses and engaging regions important for voluntary attention.
Whereas the emotional facilitation effect in the AB is thought to be a result of
bottom-up enhancement of target stimuli, the neural mechanisms underlying
the emotional capture of attention may involve competition among regions
93
important for both bottom-up and top-down information processing (e.g.,
Desimone & Duncan, 1995; Pessoa, Kastner & Ungerleider, 2002). One
imaging study has investigated costs of task-irrelevant emotional distracters in
the AB (Most et al., 2006), however, it leaves the nature of this competition
unclear.
The main question addressed in the study of Most and colleagues’
(2006) concerned how attentional modulation of amygdala activity is correlated
with self-reported harm avoidance. Attention was manipulated with task
instructions that resulted in participants forming either a ‘specific’ or ‘non-
specific’ attentional set as they searched for a target during an RSVP task.
The authors found that decreased amygdala and increased rostral anterior
cingulate cortex (rACC) activity were linked to trials with emotional distracters,
but only among high harm avoidant (similar to anxious) participants who were
maintaining a specific attentional set (Most et al., 2006). Previous research
has suggested that rACC can inhibit the amygdala response (Kim & Whalen,
2009). Consistent with this, Most et al. (2006) suggested that increased rACC
activity is indicative of the extra effort high harm avoidant participants must
exert in order to ignore emotionally distracting stimuli, presumably by down-
regulating the amygdala. In contrast, while maintaining a non-specific
attentional set the same participants showed increased amygdala and
decreased rACC. Moreover, the authors found that the degree to which
94
dorsolateral prefrontal cortex (DLPFC) activation was modulated by attentional
set depended on self-reported attentional control. Individuals with higher
attentional control engaged the DLPFC more during the ‘specific’ relative to
‘non-specific’ attentional set, consistent with DLPFC’s known role in top-down
attentional control functions (e.g., Macdonald, Cohen, Stenger & Carter,
2000).
These findings by Most et al. (2006) confirm the role of the amygdala in
response to emotional distracters, and suggest roles for the rACC in inhibiting
the amygdala’s impact on perception, and the DLPFC in maintaining attention
on task-relevant goals. However, it is important to note that all imaging
analyses were performed on trials without a target, and only early, but not late,
lags were included in the task. As such, these results do not have a direct
bearing on the behavioral emotional capture of attention effect, i.e., reduced
accuracy on emotional relative to neutral distracter trials, at early but not late
lags. In fact, no imaging study to date has specifically investigated the neural
mechanisms underlying costs of task-irrelevant, emotional distracters to
attention at early vs. late lags. Thus, the question of how bottom-up amygdala
signals may interact with brain regions underlying shifts of attention in the AB
has remained unanswered.
Although little is known about the temporal cost of emotion to attention,
studies investigating costs of emotion to spatial attention may provide useful
95
information regarding which brain areas are involved in shifting attention
towards emotional distracters. One study found that both benefits and costs of
emotion to spatial attention increased intraparietal sulcus (IPS) and lateral
orbitofrontal cortex (OFC) activity, and costs alone increased posterior OFC
activity (Armony & Dolan, 2002). The authors conclude that amygdala
responses to emotional stimuli are relayed to the posterior OFC during invalid
trials. The OFC then modulates activity in IPS, which re-directs attention
towards the location of target stimuli. Another study similarly showed
increased lateral OFC activation for costs of emotion, and decreased IPS
activity to invalidly cued targets (Pourtois, Schwartz, Seghier, Lazeyras &
Vuilleumier, 2006). Results from both spatial studies are consistent with OFC’s
role in the re-orienting of attention after unexpected events (Coull, Frith,
Buchel & Nobre, 2000; Nobre, Coull, Frith & Mesulam, 1999). Furthermore,
these results are consistent with the rACC results of Most et al. (2006). OFC
and rACC are highly interconnected brain regions. Both regions additionally
share extensive reciprocal connections with the amygdala (Bush, Luu &
Posner, 2000) and have shown modulatory effects on the amygdala across a
range of tasks (e.g., Kim & Whalen, 2009; for reviews, Hartley & Phelps, 2010;
Quirk & Mueller, 2008). Given these results, IPS and OFC/rACC may be
important brain areas underlying emotional costs to attention, not only in the
spatial domain, but the temporal domain as well, specifically within the AB.
96
In the present study, we use fMRI to explore the neural systems
underlying costs of emotion in the AB to answer the following questions. 1)
How do emotion-sensitive brain regions such as the amygdala modulate
bottom-up and top-down attention systems to result in emotional costs at early
but not late lags? 2) What are the roles of IPS, OFC and rACC in these costs?
3) And how does anxiety or attentional control affect emotional costs to
attention in the brain?
Given the findings from the emotional facilitation of the AB, and the
spatial attention invalidity effects, we are able to make the following
predictions. We hypothesize that goal-driven attention used to search for a
target in the emotional capture of attention task may be disrupted in a bottom-
up fashion by emotional stimuli, resulting in the AB at short, but not long,
distracter-target intervals. For example, while looking for a target word
embedded in an RSVP of distracter words, DLPFC, which is known to play a
role in maintaining task-relevant goals (e.g., Corbetta & Shulman, 2002), may
be engaged. However, when attention is automatically captured by an
emotional distracter word, the amygdala may become more active. While the
amygdala does not have direct projections to DLPFC (Barbas 2000;
McDonald, Mascagni & Guo, 1996) or IPS (Saygin, Osher, Augustinack, Fischl
& Gabrieli, 2011), it can signal the presence of the emotional conflict to
OFC/rACC which may serve as a hub and momentarily alter responses in
97
attentional control regions such as DLPFC or IPS (Yamasaki, LaBar, &
McCarthy, 2002). Depending on the relative strength of the bottom-up
emotional disruption to the top-down attentional control, as well as the
distracter-target lag, the visibility of the target word is determined. At short time
intervals, we expect that emotional distracters will have a greater influence on
this competition and decrease target word visibility during the AB, while at long
time intervals we expect less impairment due to top-down attentional control
mechanisms coming online, refocusing attention on the task at hand.
In addition, we expect that individuals who score high on anxiety
measures will have less attentional control and greater behavioral costs, than
lower scoring individuals. This will be accompanied by greater amygdala
activity in response to emotional stimuli and attenuated activity in regions
important for attentional control such as DLPFC, consistent with research
showing amygdala activity is correlated with anxiety (Bishop, Duncan &
Lawrence, 2004) and DLPFC activity is correlated with attentional control
(Bishop, Duncan, Brett & Lawrence, 2004; Most et al., 2006). This model
includes elements of previously proposed models (Bishop 2007; Taylor &
Fragopanagos, 2004, 2005), however, here we have the opportunity to test it
using the emotional capture of attention AB task in the scanner.
In Experiment 1 we test a variant of the emotional capture of attention
AB task in which participants make a 4-alternative forced choice (4AFC)
98
response with the goal of replicating previous behavioral findings (e.g.,
Mathewson et al., 2008). In Experiment 2 we test the proposed model by
measuring BOLD signal as participants complete the task in the scanner.
EXPERIMENT 1: Task validation
In pilot studies not described here, we replicated the attention capture
by emotion effect previously found by other groups (Arnell, Killman & Fijavs,
2007; Keil & Ihssen 2004; Mathewson et al, 2008; Most, Chun, Widders &
Zald, 2005; Most, Smith, Cooter, Levy & Zald, 2007; Smith, Most, Newsome &
Zald, 2006). To bring the experiment into an fMRI environment without the use
of a full keyboard to identify target words, we changed the word identification
task to be a 4AFC task. The purpose of Experiment 1 was to validate this task
for the scanner.
METHODS
Participants
Twenty undergraduates from the NYU Psychology subject pool
participated for course credit. Two participants were later excluded because
they learned English as a second language. Given that we used English words
99
as stimuli, one of our requirements for participation was that English be the
first language learned to reduce interference of the first acquired language.
Thus, we analyzed data from 18 total participants (13 female).
Apparatus
The experiment was conducted on a Windows PC running MATLAB
(The Mathworks, Natick, MA) and Psychophysics Toolbox (version 3;
Brainard, 1997). Observers made responses by choosing numbers 1-4 on the
top row of the keyboard with their index through pinky fingers.
Stimuli
The same word stimuli as was used in Anderson (2005) were used here
with supplemental words (see Appendix C). Highly arousing words that could
be either negative or positive in valence were used as emotional distracters,
and words with neutral valence and low arousal were used as neutral
distracters, targets, and maskers in our RSVP. The target was printed in one
of 16 shades of green by changing the RGB levels to manipulate ease of
visibility, which was adjusted for each individual observer to keep performance
between 65 and 75% correct on average. After each block, accuracy was
assessed and the visibility was adjusted to maintain a challenging level. The
distracter and filler words were printed in black. The background was mean
100
gray. All words were in Arial font, size 38. The distracter word was either
neutral or emotional with equal probability, and the target word was always
neutral. The black masking words were 10 letters in length on average. Both
the distracter and target words were 5 letters in length on average.
Procedure
Participants were seated in front of the computer with an approximate
viewing distance of 40 cm. After 2 short practice blocks where only neutral
words were presented, participants completed 4 blocks of 56 trials each.
Within each trial, a fixation point appeared for 500 ms, followed by an RSVP
stream of 15 items. The RSVP stream contained 13 filler words, 1 distracter
word inserted in position 2, 3, 4 or 5, and 1 target word inserted 1, 2, 3, 4, 5, 6,
or 7 positions after the distracter. Each word was on the screen for 90 ms and
immediately replaced by the following word with no intervening blank. After the
RSVP stream completed, there was a 500 ms blank screen, then participants
were shown a list of 4 words and they had to choose which word matched the
target. There was 1500 ms to make a response by pressing 1 of 4 buttons with
their right hand (Figure 10, top panel).
101
Analysis
The data were divided by condition (emotion vs. neutral distracter) and
lag (positions 1-7 after distracter). Accuracy was averaged over every trial
within a condition at every lag. Average accuracy in the emotion condition was
compared to the average accuracy in the neutral condition for all 7 lags. A 2 x
7 repeated measures ANOVA was conducted on the data with distracter type
and lag as the two factors, and individual t-tests comparing the emotional and
neutral conditions for each lag was performed.
RESULTS
The ANOVA revealed significant main effects of distracter emotion
(F(1,17)=17.55, p<0.001), and lag (F(6,17)=25.67, p<0.001) as well as a
significant interaction of the two factors (F(6,17)=3.592, p< 0.01; Figure 11).
The difference in performance for lags 3, 4, 5, and 6 were all individually
statistically significant (ps<.01 at the two-tailed level, except lag 5 p<.05 at the
one-tailed level) with neutral distracter trials leading to greater accuracy
(Figure 11, left panel). The strongest effect was found at lag 4 (t(17)=4.35,
p=0.0004), or 360ms after presentation of the distracter. In addition we
averaged target accuracy separately for neutral and emotional distracter trials
102
over lags 3-4. Performance for these two types of trials was significantly
different: neutral distracter trial performance was 81%, while for emotion
distracters performance was 68% (t(17)=4.58, p<0.001). Importantly,
performance for lag 7 was the same for emotion and neutral distracter trials
(77%; p>0.1; Figure 11, right panel).
EXPERIMENT 1 DISCUSSION
This task successfully produced a strong capture effect at early lags,
but not late lags. These results provided the information we needed regarding
the efficacy of using a 4AFC task, so we felt free to further modify the
experiment for use in the scanner in a second pilot experiment not described
here. To maximize the amount of data we could collect in the shortest amount
of time in the scanner, we tested a simplified version of Experiment 1, which
included only 2 distracter positions (3 and 5) and 4 distracter-target lags (3, 4,
7, and 8). We used lag 8 as a second “late” lag, given the results showing lag
6 still showed a cost of emotion. In addition, we introduced a jittered ITI in a
rapid event-related design, which allowed us to include trials closer together in
time without having to wait for the brain’s hemodynamic response to return to
baseline between every trial. The results of this second pilot replicated
103
Experiment 1, and showed enhanced attention capture by emotion with high
trait anxiety relative to low trait anxiety, indicating that participants with
elevated trait anxiety should be preferentially recruited for Experiment 2 in the
scanner.
EXPERIMENT 2: fMRI component
METHODS
Participants
31 naïve observers (19 female) from the NYU community participated in
this study. They were screened to ensure they are native English speakers
with corrected or normal vision, no history of psychiatric disorder, not on any
psychotropic medication, and met all criteria for safe scanning. In addition,
given pilot results indicating that individuals high in trait anxiety and negative
affect show the strongest emotion capture effect, we mainly recruited
volunteers from an Introduction to Psychology course whose trait anxiety
scores on the 40-item State-Trait Anxiety Inventory (STAI: Spielberger,
Gorsuch, Lushene, Vagg & Jacobs, 1983), and whose negative affect scores
on the 20-item Positive and Negative Affect Scale (PANAS: Watson, Clark &
104
Tellegen, 1988), were in the top half of the total range (collected several
months in advance of this study in a general battery of surveys).
Apparatus
A 3-Tesla Siemens head-only scanner housed in the NYU Center for
Brain Imaging was used for collecting functional (T2*-weighted EPI) and
anatomical (T1-weighted) data. For each participant, 830 volumes of functional
data were collected (166 volumes/run x 5 runs), with each volume consisting
of 34 interleaved 3mm slices oriented approximately parallel with the
anterior/posterior commissure (inplane resolution = 3mm2, interslice gap =
0mm, flip angle = 82°, TE = 15ms, TR = 2s) providing whole-brain coverage in
most participants. Anatomical data had a resolution of 1mm3. A Dell PC
computer running MATLAB 7.5 (The MathWorks, Natick, MA) and the
Psychophysics Toolbox (version 3; Brainard, 1997) controlled timing of
stimulus presentation. The display was back-projected into the bore of the
magnet via an Eiki LC-XG250 projector approximately 57 cm from the
observers’ eyes. Observers made their choices using 1 of 4 possible buttons
on a button-box in their right hand.
105
Stimuli
Same as in Experiment 1 (see Appendix C).
Procedure
On the day of the scan participants completed 2 short practice blocks
(20 trials each) using only neutral distracter words outside the scanner to
familiarize themselves with the task. Participants then completed 5 functional
runs of the task containing 32 trials each (4 trials per condition per run) in the
scanner. See Appendix C for detailed task instructions. The trial structure was
identical to that of Experiment 1, except for (1) an additional 8-second initial
fixation period and 12-second ending fixation period appended to the
beginning and end of each run, (2) the length of the inter-trial-intervals (ITIs)
were jittered and could range from 2 to 16 seconds long (Figure 10, bottom;
ITI duration and trial ordering was optimized using optseq2:
http://surfer.nmr.mgh.harvard.edu/optseq/), and (3) only lags 3, 4, 7 and 8
were tested. There was a ~10 minute T1-MPRAGE structural scan after
completion of the 5 runs, to which functional data were aligned in order to
localize active brain regions.
All observers filled out the following self-report measures at the
experiment’s conclusion: the PANAS (Watson et al., 1988), the STAI
106
(Spielberger et al., 1983), the 20-item Attentional Control Scale (ACS:
Derryberry & Reed, 2002), and the Edinburgh Handedness Inventory (EHI;
Oldfield, 1971). The PANAS was used to assess the degree to which different
positive and negative emotions were experienced in general over the previous
six months, and scores could range from 10 to 50 within either positive or
negative affect. The STAI was used to assess degree of anxiety at the present
moment (state) and in general (trait), and scores could range from 20 to 80
within either state or trait anxiety. The ACS was used to assess the degree of
attentional control participants have over distractions, and scores could range
from 20 to 80. EHI scores could range from -100 (completely left-handed) to
+100 (completely right-handed) and were collected in the case of differences
in functional lateralization due to handedness.
Behavioral Analysis
Each participant completed 160 experimental trials, equally divided
among 8 conditions, created by crossing 2 distracter types (emotional, neutral)
with 4 lags (3, 4, 7, 8). The emotional experiment conditions will be referred to
as E3, E4, E7, and E8. The neutral experiment conditions will be referred to as
N3, N4, N7 and N8. Since distracter position is not a variable of interest, data
were averaged over both distracter positions 3 and 5. Lags 3 and 4 comprised
the “early” component (the AB period) and Lags 7 and 8 comprised the “late”
107
component. Accuracy and RT was averaged over every trial within a condition
at both early and late lags.
Functional and Anatomical Data Preprocessing
The first 2 TRs (4 s) of each functional run were discarded before
preprocessing. Data quality of each functional run was visually inspected in a
time course movie to note any major movement. Functional runs then
underwent the following preprocessing steps: (1) slice scan time correction, (2)
temporal filtering (linear trend removal and high pass filter of 3 cycles), and (3)
motion correction to the last image closest to the T1-MPRAGE. No participants
were excluded due to excessive (>3 mm) movement. For group whole-brain
analyses, the functional data were spatially smoothed (Gaussian kernal spatial
smoothing of 6 mm full width half maximum).
The T1-MPRAGE for each participant was cleaned, AC-PC aligned,
and then morphed into Talairach space (Talairach & Tournoux, 1988). The
anatomical data were then co-registered with the corresponding functional
data.
BOLD Response Analysis
Based on our a priori hypotheses regarding the brain regions involved
in the emotional cost to attention, we defined regions of interest (ROIs) in the
108
DLPFC, OFC and rACC using a whole-brain contrast of all task-related activity
greater than baseline. IPS was defined individually for each subject using the
same contrast (with a separate anterior and posterior region on the left side)
due to the variability in the location of IPS activation across subjects. Given
that we used a contrast that revealed regions involved in the task generally,
ROI definition was orthogonal to our specific contrasts of interest. The
amygdala was defined two ways: (1) anatomically based on each subject’s T1
scan, and (2) based on a contrast of all emotional trial-related activity greater
than all neutral trial-related activity. The mean Talairach coordinates and
number of voxels for each region are listed in Table 1A (Table 2B for
amygdala ROIs based on E>N contrast). We performed convolution analyses
using these ROIs, in which we estimated betas for each trial type. The
resultant betas from conditions E3 and E4, as well as E7 and E8, were
averaged together to form the “early” and “late” components, respectively. The
same was done for the corresponding neutral conditions.
Whole-brain contrasts were conducted across all participants to
observe global brain activity correlated with main effects and interactions of
the 2 experimental factors (distracter type and lag) to serve as confirmation of
the ROI results. Multi-subject design matrices containing stick predictors
109
convolved with a hemodynamic response function (HRF)1 were constructed to
fit a random effects general linear model to the 3D functional data using
BrainVoyager 2.0 (Brain Innovation, Maastricht, the Netherlands). The stick
predictors’ onsets were placed at the beginning of the RSVP for each trial. The
first 8 predictors in the design matrix corresponded with the 8 conditions (as
defined under Behavioral Analysis), and the last predictor was a constant2.
The fixation/ITI periods served as baseline. The following whole-brain
contrasts were conducted:
Contrast Effect 1 E3+E4+E7+E8+N3+N4+N7+N8 task-related activity 2 E3+E4+E7+E8 > N3+N4+N7+N8 main effect of emotion 3 E3+E4+N3+N4 > E7+E8+N7+N8 main effect of attention 4 E3+E4 > N3+N4 simple effect of emotion (early lags) 5 E7+E8 > N7+N8 simple effect of emotion (late lags) 6 N3+N4 > N7+N8 simple effect of attention (neutral) 7 E3+E4 > E7+E8 simple effect of attention (emotional) 8 E3correct + E4correct >
E3incorrect + E4incorrect simple effect of accuracy (early emotion lags)
1 The HRF used in the convolution of the design matrices had a peak response at 10 seconds post-stimulus-onset due to a data processing error, however, the standard HRF has a peak response at 6 seconds. New analyses will be conducted using a 6 s HRF, and the results are expected to be similar, and may even be stronger. 2 An additional analysis was conducted including reaction time as a regressor of no interest to ensure that any activity revealed in whole-brain contrasts was due to our contrasts of interest rather than a main effect of RT. Contrasts using this GLM revealed overlapping regions of activation compared to when RT was not entered in the model, suggesting that differences in brain activity are not due only to systematic differences in RT from trial to trial.
110
RESULTS
Data from three participants (1 female) were discarded. The first
participant failed to make a response on over 25% of the trials; the second
scored more than 2 SDs away from the mean on both the PANAS and STAI,
and 2 out of 8 experiment conditions; and the third scored more than 2 SDs
away from the mean on 2 out of 8 conditions. 1 out of 28 participants had a
negative EHI score indicating left-handedness, however, their behavioral and
imaging data were no different from the other 27 participants so they were not
excluded. Behavioral and imaging data of the remaining 28 participants were
analyzed.
Self-Report
Scale means and standard errors are listed in Table 1B. ACS and
STAI-T were negatively correlated (r(28)=-.38, p<.05), confirming prior
research showing an inverse relationship between attentional control and trait
anxiety. Positive affect (PA) and negative affect (NA) were positively
correlated (r(28)=+.44, p<.05), and PA and STAI-S were negatively correlated
(r(28)=-.39, p<.05).
111
Target Identification Accuracy
A 2x2x2 repeated measures ANOVA on participants’ accuracy data
with distracter-target lag (early, late) and distracter type (emotional, neutral) as
within-subjects factors, and trait anxiety (low, high) as a between-subjects
factor was conducted. (Attentional control was not included as a factor
because it was significantly correlated with trait anxiety, but for the imaging
data both were analyzed separately.) A main effect of lag was revealed
(F(1,27)=5.98, p<.05) in which early lags resulted in decreased accuracy
compared to late lags. The main effect of distracter type was marginally
significant (F(1,27)=3.63, p<.1), with emotional distracter trials resulting in
decreased accuracy compared to neutral distracter trials. In addition there was
a main effect of anxiety (F(1,26)=5.13, p<.05) in which high trait anxious
individuals had greater accuracy overall than low trait anxious. Furthermore,
lag and distracter type significantly interacted (F(1,27)=5.66, p<.05). At the
early lag, neutral distracters resulted in greater target identification accuracy
than emotional distracters (t(27)=2.7, p<.05), however at the late lag, there
was no difference in accuracy (Figure 12, left panel). Additionally, early
emotion trials resulted in significantly decreased accuracy relative to late
emotion trials (t(27)=-3.14, p<.01). Across all four tested lags (3, 4, 7 and 8),
the difference between neutral and emotional distracter trials was greatest at
lag 4, whereas the magnitude of the difference at lags 7 and 8 were
112
equivalent.
Although neither STAI-T nor ACS was correlated with the size of the
emotional capture of attention effect, only high trait anxious individuals
showed the significant effects described above. In addition, PA was
significantly and negatively correlated with the behavioral effect (r=+.48, n=28,
p<.05). Thus, a greater behavioral cost was associated with lower self-
reported positive affect.
Reaction Time
A 2x2 repeated measures ANOVA on participants’ reaction time data
showed a main effect of lag (F(1,27)=6.63, p<.05) in which early lags resulted
in increased response latency compared to late lags. There was a marginal
main effect of distracter type (F(1,27)=3.79, p<.1) in which emotional
distracters resulted in slowed response times. Lag and distracter type
interacted as well (F(1,27)=6.12, p<.05) with early emotion trials resulting in
slower reaction time than early neutral trials (t(27)=2.53, p<.05) and late
emotion trials (t(27)=3.01, p<.01; Figure 12, right panel). Across all four tested
lags (3, 4, 7 and 8), the difference between neutral and emotional distracter
trials was greatest at lag 3, whereas the magnitude of the difference at lags 7
and 8 were equivalent.
113
Regions of Interest Analysis
In addition to examining the beta values derived from our ROI analyses
across the entire sample, we also divided the sample based on median splits
of the self-report measures. Presented below are the mean results across all
subjects, and analysis for both anxiety and attentional control.
Amygdala
Given that the amygdala is sensitive to emotionally salient information,
and may be the source of bottom-up emotional facilitation effects in the AB, we
hypothesized that amygdala activity would increase in response to emotional
distracter trials relative to neutral. Greater amygdala activity to emotional
distracter trials may then signal the presence of emotional stimuli to OFC and
rACC. Consistent with this hypothesis, a contrast of all emotional distracter
trials vs. neutral trials (Contrast 2; Table 2B) revealed activation in bilateral
amygdala. No other contrasts involving interactions with attention resulted in
amygdala activation, indicating that the amygdala response was specific to
emotional distracters regardless of lag.
Two ROI analyses were conducted on bilateral amygdala. The first
used anatomically defined ROIs, based on each subject’s T1 scan. The
second used functionally defined ROIs, based on the above contrast of E>N
across all subjects. Betas for each condition were extracted from each ROI for
each subject, and these betas were then averaged together. The results of
114
both ROI analyses yielded similar results. Both showed a main effect of
emotion, confirmed in repeated measures ANOVAs on the betas with
distracter type, E or N, and lag, 3, 4, 7 or 8, as within-subjects factors
[functional ROIs: R amygdala: F(1,27)=5.11, p<.05; L amygdala:
F(1,27)=11.96, p<.001] [anatomical ROIs: R amygdala: F(1,27)=4.88, p<.05; L
Amygdala: F(1,27)=4.9, p<.05].
Upon closer inspection with paired t-tests, it was found that right
amygdala activity was greater for early emotional relative to neutral trials
(functional ROI: t(27)=1.83, p<.05 one-tailed; anatomical ROI: t(27)=2.54,
p<.05). This difference was driven by lag 3 trials in which there was greater
activity in lag 3 emotion compared to neutral trials (functional ROI: t(27)=2.73,
p<.05; anatomical ROI: t(27)=3.15, p<.01).
Intraparietal Sulcus
Consistent with its role in top-down shifts of attention, we expected IPS
activity to be differentially modulated during early emotional vs. neutral
distracter trials. IPS may direct attention towards emotional distracters via top-
down signals from OFC and rACC.
Bilateral posterior IPS (left n=28, right n=27; right IPS could not be
defined in one subject even at very low thresholds) showed significantly
greater activity for emotional distracter trials relative to neutral at early lags
115
(left t(27)=2.36, p<.05; right t(26)=2.47, p<.05), but not at late lags (Figure 13).
In addition, early emotional distracter trials resulted in greater activity than late
emotional distracter trials (left t(27)=3.24, p<.01, right t(26)=2.79, p<.01). This
was not the case for the neutral distracter condition. Left anterior IPS (n=24)
showed the same pattern of results. Early emotion was greater than early
neutral (t(23)=2.06, p=.05) and early emotion was greater than late emotion
(t(23)=2.97, p<.01). In both right and left (posterior and anterior) IPS, this
pattern was driven by greater activity for emotional distracter trials at lag 3 (all
ts>2.45, ps<.05), but not lag 4 (all ps>.1).
When the data were examined based on median splits of the anxiety
and attentional control self-report measures, no obvious pattern emerged,
suggesting that the activity differences were not driven by any particular group
in any of the 3 IPS ROIs.
Dorsolateral Prefrontal Cortex
DLPFC has been shown to be broadly involved in attentional control
functions in the absence of emotional stimuli. Given that greater attentional
control is needed during early lag trials within the AB, we expected greater
DLPFC activity in early vs. late neutral distracter trials. For emotional distracter
trials, we expected that DLPFC activity would decrease due to inhibitory
connections with OFC/rACC. Based on previous research showing increased
116
DLPFC activity with increased self-reported attentional control (Most et al.,
2006), we also predicted that DLPFC activity would be modulated by degree of
self-reported attentional control.
Consistent with our hypotheses, right DLPFC (n=28) showed
decreased activity in emotional distracter trials at early lags relative to neutral
(t(27)=-2.38, p<.05; Figure 14). In addition, early neutral distracter trials
resulted in decreased activity relative to late neutral trials (t(27)=2.51, p<.05).
These differences were driven by increased activity for neutral distracter trials
relative to emotional at lag 4 (t(27)=2.92, p<.01), but not lag 3 (p>.1), though
the comparison is in the same direction at lag 3. The difference between lag 3
and lag 4 emotion trials was marginally significant (p<.1), with a greater
reduction in DLPFC activity at lag 4. A whole-brain contrast of early emotional
greater than early neutral revealed negative activation in right DLPFC,
corroborating these ROI results.
When the data were examined based on median splits of the anxiety
and attentional control self-report measures, these differences were only
significant for individuals low in trait (early lags, emotional vs. neutral: t(13)=
4.28, p<.001; early neutral vs. late neutral: t(13)=3, p<.05) and state (early
lags, emotional vs. neutral: t(13)= 3.66, p<.01) anxiety, and high in attentional
control (early lags, emotional vs. neutral: t(13)=2.53, p<.05; early neutral vs.
late neutral: t(13)=2.55, p<.05). High positive affect led to greater differences
117
between emotional and neutral at early lags, while low positive affect led to a
greater difference between early and late neutral trials. No significant
differences between emotional vs. neutral, or early vs. late, conditions were
found within the left DLPFC ROI.
Orbitofrontal Cortex
The OFC receives input from the amygdala and was expected to play a
role in relaying emotional signals to DLPFC and IPS during early emotional
distracter trials. No significant differences between emotional vs. neutral, or
early vs. late, conditions were found within the OFC ROI. However, the main
whole-brain contrast of interest comparing emotional distracter trials vs.
neutral trials at early lags only (Contrast 4; Table 3B) revealed activation in
the OFC, consistent with our hypothesis.
Rostral Anterior Cingulate Cortex
Given that the rACC has been implicated in the processing of emotional
stimuli in cases of attentional conflict, we hypothesized that rACC would be
preferentially active for early emotional distracter trials. Partially inconsistent
with this notion, our ROI analyses revealed rACC activity was marginally less
active in early vs. late trials (p<.1, one-tailed), but was not differentially active
for emotional vs. neutral conditions. Furthermore, no whole-brain contrast
comparing emotional vs. neutral distracter trials showed rACC activity.
118
Comparing all late vs. early lags (Contrast 3; Table 3A), on the other hand,
revealed a single activation in right rACC. Restricting this contrast to only
neutral (Contrast 6; Table 3D) or only emotional (Contrast 7, Table 4A) trials
revealed very similar regions of right rACC. Overall, right rACC was
preferentially active in late lag trials regardless of whether the distracter was
emotional or neutral.
Early Emotion Performance Analysis
We had hypothesized that during early emotional distracter trials,
emotion signals from the amygdala would activate OFC, which would then
inhibit DLPFC activity while exciting IPS activity. Given DLPFC’s role in
attentional control and IPS’s role in attention shifts, we predicted less DLPFC
activity would lead to more errors and greater IPS activity would lead to
greater orienting towards emotional distracters, also leading to more errors. To
test this hypothesis, a GLM was fit to the data including predictors for only
early emotional distracter trials differentiating between correct and incorrect
responses on each of these trials (E3correct, E3incorrect, E4correct,
E4incorrect).
Consistent with the hypothesized role of IPS activity increasing during
trials in which emotional distracter words captured attention leading to
incorrect responses, bilateral IPS is shown to be more active during early
119
emotional incorrect trials (Table 4B). In addition, a region that we did not
previously report, medial PFC, is shown to be more active for correct vs.
incorrect early emotional trials. This is consistent with prior research showing
that medial frontal cortex can regulate amygdala responses, suggesting that in
the early emotional distracter trials when this region is successfully recruited,
participants are able to correctly identify target words.3
EXPERIMENT 2 DISCUSSION
Our aim was to investigate the neural mechanisms underlying the
emotional cost to temporal attention in the attentional blink task. Based upon
previous research on the emotional facilitation effect in the AB, and on imaging
studies investigating emotional costs to spatial attention, we proposed a model
in which bottom-up emotional signals compete with top-down attentional
control for processing resources during early emotional distracter trials. We
hypothesized that the amygdala provides the bottom-up emotional signal. This
3 Previously reported DLPFC and OFC regions were not active in this contrast, suggesting that these regions play no role in whether a correct response is made during emotional trials but may be responding to the emotion itself. However, it could also be a result of insufficient power: only 18% of all trials resulted in an incorrect response.
120
signal may disrupt DLPFC control processes while engaging IPS to shift
attention towards distracters, through mediating OFC and rACC
interconnections. Given that the amygdala may be important for the bottom-up
emotional signal, whereas the frontoparietal regions mentioned above may be
important for the top-down response to task demands, we might expect
different patterns of brain activity underlying these respective functions. For
example, while the bottom-up response may not necessarily be modulated by
task demands, the top-down response may differ depending on distracter-
target lag, with early lags requiring observers to exert greater attentional
control to perform as well as in late lags. We also hypothesized that self-
reported anxiety and attentional control would modulate behavioral costs and
corresponding brain activity. Our results show that both the behavioral and
imaging results are largely consistent with these predictions.
We replicated the behavioral emotional capture of attention effect in
which emotional distracters produced a cost in target identification at early, but
not late, lags relative to neutral distracters (Mathewson et al, 2008; Keil &
Ihssen, 2004). In addition, we found that emotional costs to target identification
accuracy were greatest in high trait anxious individuals, suggesting that they
processed emotional stimuli to a greater extent than low trait anxious
individuals (e.g., Fox et al., 2005; Most et al., 2005). Furthermore, RT was
slowed in early emotional, relative to early neutral, trials. This indicates that
121
subjects were less confident of their responses as well as less accurate.
Consistent with our proposed model, we found that the amygdala was
maximally active for emotional distracter trials relative to neutral, regardless of
attentional demands. Although simple effects analysis of extracted beta values
from both anatomical and functional ROIs revealed stronger amygdala activity
to early vs. late emotional trials, these results did not survive whole-brain
contrasts or overall ANOVAs. In other words, significant amygdala activity was
not revealed in any contrast of early and late lags. Greater amygdala activity
thus facilitates the processing of emotional distracter stimuli at both early and
late lags. However, we propose that whether the emotional distracter impedes
target visibility is determined through the amygdala’s connections with regions
involved in shifts of attention under increased task demands.
Mirroring the amygdala results, we found that IPS was more active for
emotional distracter trials relative to neutral, however this was only true at
early lags. In addition, IPS activity was greater for early emotional, relative to
late emotional, trials. This suggests that the role of IPS in emotional costs to
spatial attention (Pourtois et al., 2006) generalizes to attentional selection in
time. Importantly this also provides evidence for the notion that IPS activity is
specifically modulated by emotion at early but not late lags, suggesting that
the availability of attentional resources determines the IPS response to
emotion. In other words, IPS activity in this task is not a general arousal
122
response. Studies of emotional costs to spatial attention have shown
decreased IPS activity in response to targets invalidly cued by fear faces
(Pourtois et al., 2006), and increased IPS activity in response to the emotional
cues themselves (Armony & Dolan, 2002; Pourtois et al., 2006). Together,
these results combined with the present findings demonstrate that IPS is
preferentially responsive to emotional relative to neutral stimuli under
increased attentional demands, whether they are spatial attention cues or
distracters in the AB task.
Emotional modulation of amygdala and IPS was accompanied by a
relative increase of right DLPFC activity for early neutral relative to both early
emotional and late neutral distracter trials. In contrast, there was no difference
in activity for early emotion vs. late emotion. Given the role of DLPFC in
attentional control (e.g., Corbetta & Shulman, 2002), it appears that this region
was successfully recruited during early neutral, but not emotional, distracter
trials, in order to maintain attention on the target identification task. In addition,
the DLPFC response during neutral trials was specific to early lags, indicating
that increased top-down control was required to meet the increased demands
of attention at early distracter-target lags. Although all early lag trials generate
a challenge for limited attentional resources, bottom-up emotion signals seem
to preferentially disrupt the ability of DLPFC to exert top-down control over
distracters.
123
How might bottom-up emotional signals from the amygdala interact with
top-down attentional signals from IPS and DLPFC? We show that IPS is more
responsive to early emotional relative to late emotional trials, which suggests
that its response is temporally specific. The amygdala is known to respond
quickly to emotionally arousing stimuli (e.g., LeDoux 1996) and this is
confirmed in our results. Hence, the initial IPS response may be driven by
bottom-up signals from the amygdala. Although the amygdala does not project
directly to IPS (Saygin et al., 2011) or DLPFC (Barbas 2000; McDonald et al.,
1996), it has reciprocal connections with OFC (Barbas 2000; Carmichael &
Price, 1995), a region that was more active for emotional vs. neutral trials
specifically at early lags. Modulation of activity in IPS and DLPFC regions may
thus be mediated by the OFC. Specifically, OFC may inhibit DLPFC
(Yamasaki et al., 2002) while engaging IPS activity (Morecraft, Geula &
Mesulam, 1993; Cavada & Goldman-Racik, 1989; Cavada, Compañy, Tejedor,
Cruz-Rizzolo & Reinoso-Suárez, 2000). During early emotional distracter
trials, increased IPS and reduced DLPFC-mediated attentional control could
lead to greater attentional orienting to emotional distracters and, ultimately, to
more errors in the word identification task. During late lag trials regardless of
distracter type, demands on attention are reduced, and less DLPFC-mediated
attentional control may be needed to perform accurately.
Two further analyses were conducted based on behavioral
124
performance. An analysis of accuracy was conducted to test whether our
regions of interest were differentially activated by correct vs. incorrect trials. A
contrast of early emotional incorrect vs. correct trials showed that bilateral IPS
was preferentially active, suggesting greater capture by emotional distracters
(Table 4B). In the opposite contrast, bilateral medial PFC activity was
revealed, suggesting that this region is recruited during correct trials, and may
influence activity in other regions to reduce the influence of emotional
distracters, such as IPS or amygdala. These results should be taken
cautiously however, as only 18% of the trials across all participants resulted in
an incorrect response. Another analysis, this time on RT, was conducted to
ensure that IPS activity was not solely driven by RT. A GLM with RT as a
regressor of no interest showed that the same regions (e.g., IPS) were active
in our contrasts of interest. This suggests that RT alone is not driving the
activity in these regions, however it may very well be the case that there is a
common underlying process, such as emotional capture of attention, that
drives both the RT and IPS activity differences. For example, when attention is
shifted away from a target word due to an emotional distracter, RT to identify
that target slows down due to a noisier perceptual signal.
Although the data are largely consistent with our proposed model, the
amygdala and IPS may also interact through an alternative visual cortical
route. Representations of emotional distracter words may be strengthened by
125
feedback from the amygdala (Freese & Amaral, 2005), consistent with the
greater visual cortex activity revealed in the early emotion vs. early neutral
whole-brain contrast (Table 3B). IPS may receive relatively greater
feedforward input from visual cortex in response to emotional vs. neutral
distracters. Greater attention may consequently be diverted to the emotional
distracter via either amgydala-OFC-IPS connections, or amygdala-occipital-
IPS connections, or both, resulting in poorer perceptual encoding of the
neutral target stimulus.
Somewhat inconsistent with our proposed model, we found that rACC
activity increased in all late compared to early lag trials, but was not
differentially modulated by emotional relative to neutral distracters (similar to
Yamasaki et al., 2002). Previous research has linked rACC to increased target
identification accuracy in the face of emotional and attentional conflict (De
Martino et al., 2009; Most et al., 2006; Schwabe et al., 2010). These studies,
however, did not include late lag trials, precluding a comparison of early vs.
late lags. It may be the case that rACC is preferentially active when attentional
control is exerted in order to ignore processing of distracter stimuli (regardless
of valence). This view is in accordance with previous research demonstrating
rACC’s general involvement in response conflict (e.g., Carter, Braver, Barch,
Botvinivk, Noll & Cohen, 1998; MacDonald, Cohen, Stenger & Carter, 2000).
Differences in self-reported attentional control or anxiety have been
126
associated with differences in frontoparietal cortex or amygdala function
respectively (e.g., Bishop 2007). How do anxiety and attentional control affect
brain activity in the present results? We found that activity differences in the
DLPFC were primarily driven by individuals low in trait anxiety and high in
attentional control. This is consistent with other prior research demonstrating
that the activity in DLPFC has been linked to attentional control (Corbetta &
Shulman, 2002; Corbetta, Patel & Shulman, 2008, Most et al., 2006). Although
we were expecting IPS activity to be modulated by trait anxiety and attentional
control, this was not consistently the case across IPS ROIs or conditions. It is
possible that because of its placement between bottom-up and top-down
attention processes, IPS activity is modulated by a mix of signals via the
amygdala and via DLPFC that, with the current experiment, cannot be
disentangled.
The cost of emotion to attention in the AB task is likely due to failures of
top-down attention to control strong bottom-up responses towards emotional
distracters. This competition for attentional resources (e.g., Desimone &
Duncan, 1995; Pessoa, Kastner & Ungerleider, 2002) indicates that there is
some degree of automaticity in emotional stimulus processing (e.g., Carretie,
Hinojosa, Martin-Loeches, Mercado & Tapia, 2004; Ortigue, Michel, Murray,
Mohr, Carbonnel & Landis, 2004), but we show that the degree to which this
disrupts target processing may also be dependent on the relative strength of
127
top-down attentional control over bottom-up distractions (Fox et al, 2005; Most
et al, 2005). Indeed, individual differences in the degree to which emotional
distracter words disrupt neutral target identification are associated with
differences in self-reported attentional control and anxiety.
Consistent with its involvement in emotional costs to spatial attention,
we show that IPS is involved with emotional costs to temporal attention,
specifically within the attentional blink task. The early amygdala response to
emotion may facilitate the early IPS response through common links with
OFC. However, IPS is maximally responsive to emotion at early, but not late,
distracter-target lags, indicating that the pattern of IPS activity described here
is not just an arousal response. Instead we provide evidence for the notion
that IPS is a site underlying the temporally-dependent capture of attention by
emotion.
128
CONCLUSION
The interactive effects of emotion with attention can both improve and
impair visual processing. Across three chapters I have shown how this
interaction modulates our perceptual experience, with an emphasis on how
emotion can incur attentional costs to visual processes. We have also
explored how external characteristics of visual stimuli and differences among
individual observers affect this interaction.
Whereas individual differences in how we experience emotion, and how
we respond to emotionally-evocative stimuli, are accepted and often expected
in psychological research, individual differences in how we perceive, and how
we pay attention to, visual stimuli is a relatively new idea. Not only do we show
emotion modulates attention and perception, we also show that individual
variability across a number of factors can affect the way we pay attention.
Handedness, a marker of cerebral lateralization (e.g., Bourne, 2008;
Hellige, Bloch, Cowin, Eng, Eviatar & Sergent, 1994), interacts with the type of
stimulus used to cue exogenous attention, resulting in different effects of
attention across the visual field. Anxiety exacerbates costs of emotion to
attention, and under some circumstances affects females more than males.
Finally, the degree to which we can exert control over distracters while
performing a task can determine whether we see a target stimulus. By
129
changing the way we pay attention, these factors critically influence which
information is processed at higher cognitive levels.
Handedness and Attention
Chapter 1 describes a study that was our initial attempt at investigating
costs of emotion to attention and contrast sensitivity. Instead, we found that
observer handedness modulated the degree to which attention cued with
pictures of faces affected contrast sensitivity, regardless of the emotional
expression of the cues. No significant differences between left- and right-
handers were found when simple dots were used to cue exogenous attention:
both groups demonstrated the same pattern of results of increased contrast
sensitivity following valid dot cues, and decreased contrast sensitivity following
invalid dot cues, relative to a distributed dot condition. When face cues were
used to cue attention, right-handers had the expected benefits of valid face
cues and the expected costs of invalid face cues relative to a distributed face
cue condition. Notably, using the same face cues to direct attention in left-
handers resulted in no change of contrast sensitivity across cueing conditions,
suggesting that the handedness effect is specific to faces.
While asymmetries in visual attention, face and emotion recognition
have been previously found to interact with handedness (Bourne, 2008;
Dronkers & Knight, 1989; Luh, Redl & Levy, 1994; Rubichi & Nicoletti, 2006),
130
our study was the first to demonstrate that handedness affects the way in
which attention alters sensitivity to contrast, a low-level visual feature which
modulates cortical (V1) activity at early levels of visual analysis.
We hypothesized that these behavioral differences are due to
differences in the functional organization of brain regions supporting spatial
attention and face processing between handedness groups. Previous laterality
research reports that left-hander brains as a whole are less functionally
lateralized than right-handers (e.g., Boles, 1989; Luh et al., 1994). Significantly
more right-handers, for example, are right-hemisphere dominant for face
processing (e.g., Badzakova-Trajkov, Häberling, Roberts & Corballis, 2010;
Bourne 2008) and for spatial attention (e.g., Flöel, Buyx, Breitenstein,
Lohmann & Knecht, 2005) than left-handers. It may be the case that in order
to elicit the beneficial effects of attention using face cues, signals must travel
between face and attention regions within the time limits of exogenous
attention (100 to 120 ms; Nakayama & Mackeben, 1989; Cheal & Lyon, 1991).
Given that left-handers have a lesser degree of functional laterality these
signals may, on average, travel farther, take a longer amount of time to
interact, and result in no effect. Regardless of the true explanation, Chapter 1
shows that individual differences in handedness can affect attention and
perception at very early levels of processing, indicating that researchers with
interests in these areas should adopt appropriate controls.
131
Emotion, Anxiety, Gender and Attention
We revisited the question of whether emotion produces a cost to
attention and contrast sensitivity in Chapter 2. Although we were initially
puzzled with regard to the lack of a significant emotion effect in Chapter 1, an
article was recently published that proved very influential for our later
experiments. Bocanegra & Zeelenberg (2009) found that emotion’s effects on
orientation perception depend on spatial frequency. Emotion improved
perception of low spatial frequency targets (<3 cpd) and impaired perception
of high spatial frequency targets (>3 cpd). The authors suggest this is due to
the amygdala’s preference for the low spatial frequency components in
pictures of fearful faces, which may boost processing of these spatial
frequencies in retinotopic cortex. Rather than using 4 cpd Gabor stimuli as we
did in Chapter 1, this information led us to use low spatial frequency Gabor
stimuli (1.5 and 2 cpd) in the experiments reported in Chapter 2.
Our main unique finding was that costs of emotion to attention and
contrast sensitivity were modulated by trait anxiety (Experiments 1 and 2) and
sex (Experiment 1). Relative to neutral face cues that invalidly cued attention
to a distracter location, invalid fearful face cues resulted in decreased contrast
sensitivity for target stimuli. In Experiment 1, in which we tested two task
locations, this cost was found most strongly in high trait anxious females,
whereas in Experiment 2, in which we tested four task locations, this cost was
132
found across all high trait anxious individuals regardless of sex. We also
replicated previous research (Phelps et al., 2006) showing that fear cues
increased contrast sensitivity beyond that of neutral cues when they (1)
directed attention in a distributed fashion across all task locations, and (2)
directed selective attention to the target’s location.
The detrimental effects of anxiety on attention have been studied for
decades, with reaction time in dot probe or Stroop tasks serving as the main
dependent measure of attention allocation (e.g., Macleod, Mathews & Tata,
1986; Mathews & Macleod, 1985; Richards & Millwood, 1989; Fox 1993). The
present research is the first to show perceptual consequences of anxiety’s
influence on attention. In addition, our results suggest that known differences
in behavioral and neural responses to emotional or anxiety-provoking stimuli
between males and females, such as better recognition of facial affect in
females and greater amygdala activity in response to fearful faces in high
anxious females (Cahill, 2006; Craske, 2003; Dickie & Armony, 2008; Kemp,
Silberstein, Armstrong & Nathan, 2004; Lang et al., 1998; McClure, 2000;
Thayer & Johnsen, 2000) are extended to differences at the perceptual level.
Neural Correlates of Emotion’s Cost to Attention
Chapter 3 explored the neural correlates of emotion’s cost to attention
within the attentional blink. We tested a model in which bottom-up amygdala
133
responses to emotion influence activity in brain regions involved in the top-
down control of attention such as the OFC, IPS and DLPFC. We found that the
amygdala was more active for all emotion vs. neutral distracter trials,
consistent with its broad role in emotion processing (e.g., Phelps 2006), and
its specific role in the emotional facilitation effect in the AB (Anderson &
Phelps, 2001). We also had a specific interest in the IPS based on previous
reports of this region’s involvement in spatial attention invalidity effects, in
tasks where attention has been directed to non-target locations by emotional
face cues (Armony & Dolan, 2002; Pourtois et al., 2006). Consistent with these
accounts, we found that IPS activity increased in response to emotional
distracters within the early, but not late, AB window. Rather than being
responsive based solely on stimulus-evoked arousal, this time-dependent
activity suggests IPS is sensitive to emotion in the context of increased task
demands.
A specific contrast comparing early emotional vs. early neutral
distracter trials further revealed a region in OFC, while the opposite contrast
revealed DLPFC. We propose that DLPFC activity is reduced during early
emotional distracter trials via inhibitory connections from OFC (Yamasaki et
al., 2002). These events involving both bottom-up emotion and top-down
attention processes result in greater attentional resources being funneled into
emotional distracter processing, at the cost of neutral target processing. Our
134
results are consistent with research on the frontoparietal attention network as
reviewed by Corbetta & Shulman (2002). Ventral frontoparietal regions, which
respond automatically to visually salient and/or behaviorally relevant objects in
the environment, can disrupt ongoing goal-directed attention under the control
of dorsal frontoparietal regions.
Furthermore our results extend prior imaging work on the AB (De
Martino et al., 2009; Most et al., 2006; Schwabe et al., 2010) by showing that
differences in brain activity correspond to differences in self-reported anxiety
and attentional control. Increased DLPFC activity, which we hypothesized is
related to control over early neutral distracters, was driven by individuals who
self-reported high attentional control and low trait anxiety. The impact these
traits had on brain activity is consistent with earlier research on the
neurocognitive mechanisms of anxiety and attentional control (Bishop et al.,
2004; Bishop 2009).
Individual Differences in Perception
The first two chapters of this dissertation have shown that there are
significant variations in our ability to perceive contrast. While individual
differences in perception have been studied throughout the history of modern
psychology, they have mostly pertained to our high-level conceptualization of
objects or scenes. For example, one study found that gender influenced
135
interpretations of ambiguous drawings (Coren, Porac & Ward, 1978) and
another found that mood (happy, critical or anxious) affected the way
individuals interpreted a scene (Leuba & Lucas, 1945).
More recently, there has been an increased interest in inter-subject
variability of low-level visual perception. The individual differences that are
reported in low-level perception research, however, are usually described and
explained in terms of how they are associated with other visual perceptual
abilities (e.g., Peterzell & Teller, 1996; Simpson & McFadden, 2005) rather
than innately non-perceptual qualities such as personality or gender, or non-
visual cognitive processes such as emotion. Psychological research in general
is becoming more open to research at the intersections of multiple “distinct”
sub-fields that for most of the last century were traditionally studied separately.
Our investigations of how attention affects perception, and how emotion
affects both perception and attention, make an important contribution to this
interdisciplinary movement. In particular, emotion’s effects on perception, for
which there are currently only two papers published that I’m aware of (spatial
frequency: Bocanegra & Zeelenberg, 2009; contrast sensitivity: Phelps et al.,
2006), is relatively unexplored territory. Given the importance of both attention
and emotion in filtering large amounts of information for what is most relevant,
there is sure to be many new advances in this area in the near future.
136
Other Relevant Issues
Observer expertise in visual psychophysics
Studies of attention and perception traditionally recruit a small number
of well-trained psychophysical observers. The main reason being that
experienced observers provide less noisy data, and data are analyzed per
individual. For a given stimulus intensity, experienced observers give more
consistent responses, whereas completely naïve observers may at first
answer randomly during perceptually difficult trials. In addition, naïve
observers are more likely to press the wrong button even if they know the
correct answer, adding even more variance to the data. Given that we cannot
know which trials were answered incorrectly due to finger-error, it is impossible
to remove these trials from analysis.
Observer expertise may have influenced the results of Experiment 2 in
Chapter 2. Fear-valid and fear-distributed cuing conditions were expected to
produce greater contrast sensitivity than their neutral counterparts across all
observers, replicating the results of Phelps et al. (2006). Instead the
differences we found were a result of the interaction of cuing condition and
anxiety. Specifically, low trait anxious observers demonstrated greater contrast
sensitivity following fear-valid cues, but this was relative to the fear-distributed
rather than the neutral-valid cuing condition. Furthermore, high trait anxious
observers demonstrated greater contrast sensitivity following fear-distributed
137
cues, but this was relative to the fear-invalid rather than the neutral-distributed
cueing condition. We hypothesize that our large and inexperienced observer
population, made necessary by the fact that we were also interested in
individual differences in anxiety and sex, underlies the lack of statistically
significant differences between fear and neutral conditions in these cases.
Costs to spatial vs. temporal attention
Given that we did not scan our spatial attention studies we cannot say
for sure that the spatial disengagement cost due to invalid cuing with fearful
faces has common underlying brain regions as the emotional capture of
attention in the AB task. That is, it is unclear whether the spatial
disengagement cost is a result of enhanced amygdala activity driving OFC and
IPS relative to attentional control brain regions such as DLPFC. Moreover, an
important point to consider is that the attentional systems engaged during
exogenous spatial cuing, and those engaged in the attentional blink task, may
be subserved by different underlying neural networks. However, given the IPS
and OFC results of Pourtois et al. (2006) and Armony and Dolan (2002), which
were both spatial attention studies, and our current imaging study of the
attentional blink, we hypothesize that similar neural circuitry may underlie
emotional costs in space as well as at different points in time.
138
If we were to scan Study 2 Experiment 2 (the four locations task), we
would need to test ~6-10 observers at their distributed cue condition contrast
threshold only, due to the fact that changes in contrast modulate V1 activity on
their own (Boynton, Demb, Glover & Heeger, 1999). This would necessitate
pre-testing outside of the scanner in order to obtain reliable estimates of each
person’s contrast threshold. We would also need to create retinotopic maps for
all observers, from data collected in a second scanning session, and localize
the four possible target locations in visual cortex.
To separate cue- vs. target-related activity, we would utilize two types
of trials: one in which the cue precedes the target, and another in which the
target precedes the cue (similar to Liu et al., 2005). While the overall visual
stimulation is equivalent, there will only be cue-related effects on the target
during the trials where the cue precedes the target. What we might expect is
that visual cortical activity would increase at validly cued target locations, and
decrease at invalidly cued target locations, relative to when there were
distributed cues. These changes may be accompanied by similar changes in
IPS as attention is transiently directed towards cued locations. We might also
expect graded activity changes across the four locations. For example, if a
target in the upper left quadrant was validly cued, we might see the greatest
increase in activity in the corresponding brain region, the greatest decrease in
activity in the diagonal region, and intermediate changes in the other two. As
139
for frontal cortex, it may be difficult to tease apart responses to, say, the upper
left vs. the lower left quadrant of the visual field. A general contrast of fear-
invalid vs. fear-valid, however, may reveal OFC activity with decreases in
DLPFC, whereas a contrast of neutral-invalid vs. neutral-valid may reveal
increases in DLPFC due to the attentional control required in shifting from
task-irrelevant back to task-relevant locations.
Current Projects and Future Directions
The results of Chapter 3 confirmed that the amygdala is involved in
emotional costs to attention, consistent with other imaging studies of the AB
(Most et al., 2006; Schwabe et al., 2010). We also show that the right
amygdala may play more of a role than the left amygdala given that only the
right side had differential activity in response to emotional distracters at early
compared to late lags. What Chapter 3 is not able to answer, however, is
whether the (right) amygdala is necessary for these costs in behavior to occur.
Following Anderson and Phelps’ (2001) patient study showing that the left
amygdala was necessary for emotion’s facilitative effect during the AB, we
now are interested in investigating whether the amygdala is necessary for
emotion’s detrimental effect during the modified AB.
To answer this question, we are currently collecting data from anterior
temporal lobectomy (ATL) volunteers as they participate in the same modified
140
AB task as described in Chapter 3. ATL patients elect to have surgery to
excise the anterior temporal lobe on one side as a treatment for intractable
epilepsy. It includes removal of the amygdala, hippocampus and anterior
temporal cortex. At the moment we have a participant pool of 13 left ATL and
9 right ATL patients, with the goal of having 15 in each group. In addition, we
have data from a bilateral amygdala patient, who had a right ATL and a more
focal left amygdala lesion. Our control group consists of 14 healthy age-
matched volunteers.
Thus far, our results indicate that the left ATL patients, and the bilateral
amygdala patient, show the emotional capture of attention effect (same as
controls). While the results are trending in the same direction for the right ATL
patients, they are not significant. These results are tentative, but results
suggest that the left amygdala is not necessary for the capture effect to occur,
while the right amygdala is necessary. However, the bilateral results appear to
be contradictory. Given that we are still collecting data, it is possible that there
is not enough power in the right ATL group to show the effect.
If after collecting more data, the right ATL results become significant,
this would indicate that the emotional capture effect is not dependent on the
amygdala. This would raise an interesting conundrum. While the emotional
capture of attention effect may be more dependent on failures of top-down
attentional control rather than bottom-up emotional responses, the effect still
141
requires a strong emotional response to distract attention in the first place. If
the amygdala is not the source of bottom-up emotional prioritization of
attentional resources, which brain region(s) could it be? A recent patient study
showed that two bilateral amygdala patients showed the emotional facilitation
effect (Bach, Talmi, Hurlemann, Patin & Dolan, 2011), in direct contrast to
Anderson & Phelps (2001) and consistent with another study showing the
amygdala is not necessary for non-conscious processing of fearful faces
(Tsuchiya, Moradi, Felsen, Yamasaki & Adolphs, 2009). These authors
suggest the pulvinar nucleus of the thalamus and visual cortex could be the
source of this automatic relevance detection, and the amygdala modulates
other cognitive processes after considerable cortical computation has already
been completed. Although this is a debate that cannot currently be settled, it
demonstrates that more research is needed to understand the role of the
amygdala in this process of attentional prioritization.
Concluding Remarks
Emotion and attention are highly interactive processes that, for the most
part, allow us to dedicate limited metabolic resources to the important events
in our lives, while simultaneously discarding irrelevant events. At times,
however, emotionally significant information can disrupt ongoing attentional
processes. We have all experienced the distraction caused by the aftermath of
142
a car accident, for example, while trying to concentrate on driving safely down
the freeway.
The consequences of emotion and attention’s interaction, both the
beneficial and the costly, are far-reaching and occur at multiple levels of visual
stimulus processing. We show that it impacts both contrast sensitivity at a
lower level, and word identification at a higher level, of object recognition.
Furthermore, we show that a number of factors not traditionally associated
with visual perception have significant effects on the perception of contrast via
their interactions with attention. Whereas handedness interacts with the type
of stimulus that directs spatial attention, anxiety and sex interact to produce
different outcomes depending on cue validity, cue valence, and the availability
of attentional resources.
One of the themes common to Chapters 2 and 3 is our demonstration
of a significant positive relationship between the magnitude of emotion’s cost
to attention and trait anxiety. The fact that we show this for both contrast
sensitivity and word identification accuracy in the AB suggests that common
underlying mechanisms may be responsible. Weakened top-down attentional
control processes that are associated with anxiety may result in enhanced
sensitivity to bottom-up stimulation, especially under conditions of task-
irrelevant emotional distraction. This emphasis on processing distractions may
divert attentional resources from processing task-relevant targets, resulting in
143
decreased contrast sensitivity. Contrast processing, impoverished or not, is an
important first step in the recognition of complex visual stimuli such as words.
The outcomes of the three chapters in this dissertation just begin to hint
at some of the other perceptual and cognitive processes that are likely
impacted by emotional and attentional selection. Our results also emphasize
the importance of taking into account the significant individual variability in a
given subject pool, because it may indicate not only behavioral differences but
also functional differences at the neural level. If we can link differences in
function to differences among individuals, then we are a step closer to
understanding emotion and attention’s conjoint effects on how we see.
144
FIGURES
Figure 1. Chapter 1: Trial sequence for Experiment 1 (face cues) and
Experiment 2 (dot cues). Cues preceded Gabor stimuli in this exogenous
cuing task. Participants indicated both the location and the orientation of the
tilted target Gabor using a single button press on each trial. Images not to
scale; contrast and target Gabor tilt emphasized for clarity.
145
Figure 2. Chapter 1: (A) Contrast sensitivity data for all observers (face n=12;
dot n=10), averaged over handedness group and target visual field. (B)
Contrast sensitivity data split by handedness group (each group: face n=6; dot
n=5). Error bars are ± 1 standard error of the mean. *p=.05. **p=.01.
***p=.001.
146
Figure 3. Chapter 1: Contrast sensitivity data from Figure 2B, split by target
visual field. (A) Left-hander face (n=6) and dot (n=5) data. (B) Right-hander
face (n=6) and dot (n=5) data. Error bars are ± 1 standard error of the mean.
*p=.05.
147
Figure 4. Chapter 1: Correlation of handedness score with cue validity effect.
For RVF targets cued with faces, this significant correlation is driven by both a
decrease in attention effect in left-handers and an increase in attention effect
in right-handers, whereas for dots, the significant correlation is driven mostly
by closer clustering of attention effect in right-handers.
148
Figure 5. Chapter 2: Experiment 1 trial sequence. Face cues preceded Gabor
stimuli in this exogenous cuing task. Participants indicated both the location
and the orientation of the tilted target Gabor using a single button press on
each trial. Images not to scale; contrast, target Gabor tilt and spatial frequency
emphasized for clarity.
149
Figure 6. Chapter 2: Experiment 1 cueing effects: all observers. The Y-axis is
normalized contrast sensitivity. The X-axis is spatial cueing condition. ($$)
indicates a significant two-tailed comparison. Error bars are ± 1 SE of mean.
150
151
Figure 7. Chapter 2: Experiment 1 cueing effects: by anxiety and sex. The Y-
axis is normalized contrast sensitivity. The X-axis is spatial cueing condition.
Top row: all females and all males; middle row: low trait anxious females and
males; bottom row: high trait anxious females and males. ($$) indicates a
significant two-tailed comparison, ($) indicates a significant one-tailed
comparison, and (!) indicates a marginal one-tailed comparison. Error bars
are ± 1 SE of mean. Note: two female and two male observers with median
trait anxiety scores were not included in either the low or high groups.
152
Figure 8. Chapter 2: Experiment 2 trial sequence. Face cues preceded Gabor
stimuli in this exogenous cuing task. Participants indicated the orientation of
the tilted target Gabor using a single button press on each trial. Images not to
scale; contrast, target Gabor tilt and spatial frequency emphasized for clarity.
153
154
Figure 9. Chapter 2: Experiment 2 cueing effects: all observers and by
anxiety. The Y-axis is normalized contrast sensitivity. The X-axis is spatial
cueing condition. Top row: all observers; middle row: low trait anxious
observers; bottom row: high trait anxious observers. ($$) indicates a significant
two-tailed comparison, ($) indicates a significant one-tailed comparison. Error
bars are ± 1 SE of mean.
155
Figure 10. Chapter 3: Top: Trial sequence. Bottom: fMRI trial events.
156
Figure 11. Chapter 3: Behavioral results of Experiment 1 (n=18). Left:
Accuracy data across all 7 lags. Right: Data split by early vs. late lags (note:
late lags consist only of lag 7). Error bars are ± 1 SEM. (*) indicates significant
comparison.
157
Figure 12. Chapter 3: Behavioral results of Experiment 2 (n=28). Left:
Accuracy data across early and late lags. Right: Reaction time data across
early and late lags (note: late lags consist of both lags 7 and 8). Error bars are
± 1 SEM. (*) indicates significant comparison.
158
Figure 13. Chapter 3: IPS results. Top: Bilateral IPS region active for all task-
related activity across the whole group. ROIs were defined individually per
subject using this contrast (Table 1A), and beta values were extracted per
subject; this figure is for illustrative purposes only. Bottom left: left posterior
IPS. Bottom right: right posterior IPS. Error bars are ± 1 SEM. (*) indicates a
significant comparison.
159
Figure 14. Chapter 3: Right DLPFC results. Top: Right DLPFC region active
for all task-related activity across the whole group. The group ROI was defined
160
across the whole group using this contrast (Table 1A), and beta values were
extracted individually per subject; this figure is for illustrative purposes only.
Error bars are ± 1 SEM. (*) indicates a significant comparison.
161
TABLES
Table 1A. Chapter 3: Mean Talairach coordinates of a priori defined ROIs. All
n=28 except: L Ant. IPS (n=24), R Post. IPS (n=27). Coordinates of center of
gravity in Talairach space. ‘Nr of Voxels’ refers to voxels interpolated into
1x1x1 space.
Region x y z Nr of Voxels
L Posterior IPS -27 -65 38 1655 L Anterior IPS -34 -52 39 1208 R Posterior IPS 28 -60 38 1181 L DLPFC -24 25 43 738 R DLPFC 24 21 48 810 rACC -2 30 7 594 OFC -11 36 -8 1266 L Amygdala (anat.) -18 -5 -13 1067 R Amygdala (anat.) 17 5 -12 1128 Table 1B. Self-report scores. n=28. Scale Mean SE Min Max Positive Affect 35.04 1.35 13 46 Negative Affect 25.07 1.20 14 42 State Anxiety 41.18 1.68 24 60 Trait Anxiety 44.79 1.62 27 64 Attentional Control 51.14 1.17 43 66 Handedness 78.46 5.54 -25 100
162
Table 2. Chapter 3: Whole-brain Contrasts. n = 28. Coordinates of center of gravity in Talairach space. ‘Nr of Voxels’ refers to voxels interpolated into 1x1x1 space. t-scores are of peak voxel within cluster. 300 voxel threshold. Table 2A. Contrast 1 All > Baseline Region x y z t(27)-score Nr of
Voxels p (corr)
R Post Occ Ctx 5 -92 -3 8.44 4752 <.05 L Post Occ Ctx -19 -95 1 10.61 1215 <.05 Calcarine Sulcus -4 -79 -2 8.98 1728 <.05 L Fusiform Gyrus -42 -48 -15 8.78 1890 <.05 L Post IPS -24 -65 36 8.21 351 <.05 L Ant IPS -35 -47 36 8.26 1404 <.05 R white matter (thal) 24 -24 2 7.72 378 <.05 L white matter (thal) -25 -26 2 9.92 1134 <.05 L white matter (caud) -20 -12 16 8.48 486 <.05 L Sup Frontal Gyr -45 -1 33 8.79 1026 <.05 Baseline > All PCC -2 -52 27 11.75 12825 <.05 rACC / PFC -2 34 11 12.21 19467 <.05 R DLPFC 24 21 48 8.02 810 <.05 L DLPFC -24 25 43 7.98 756 <.05 R Ant Temp Lobe 50 -6 -10 9.92 1539 <.05 L Ant Temp Lobe -55 -12 -13 8.59 621 <.05 L Sup Occ Ctx -43 -77 27 10.06 2052 <.05 L Frontal Pole -20 56 21 7.64 324 <.05 Sup Colliculus -2 -34 -9 7.94 432 <.05 Table 2B. Contrast 2 E > N p (unc) R Middle Temp Gyr 60 -37 -1 4.66 405 <.001 R Ant Temp/Post
OFC 28 14 -17 4.64 324 <.001
L Ant Temp Lobe -50 10 -20 6.15 1053 <.001 L VLPFC -48 38 -3 4.44 324 <.001 L Amygdala -16 -3 -16 3.31 847 <.01 R Amygdala 12 -3 -16 3.06 269 <.01
163
Table 3. Chapter 3: Whole-brain Contrasts. n = 28. Coordinates of center of gravity in Talairach space. ‘Nr of Voxels’ refers to voxels interpolated into 1x1x1 space. t-scores are of peak voxel within cluster.
Table 3A. Contrast 3 Late > Early Region x y z t(27)-score Nr of
Voxels p (unc)
R rACC 7 43 14 5.04 918 <.001 Table 3B. Contrast 4 Early E > N
R Post Lat Occ 36 -86 -2 4.39 405 <.005 R Cing Ctx 11 -33 18 3.72 189 <.005 Medial OFC 0 35 -20 4.46 162 <.005 L DPFC -13 58 31 4.08 162 <.005 L VLPFC -27 51 -12 3.81 108 <.005 L Post OFC/Ant Temp -45 18 -4 3.26 108 <.005 L Ant Temp Lobe -53 11 -18 4.27 297 <.005 Early N > E R DLPFC 24 18 46 3.40 216 <.005 L TPJ -44 -40 29 3.34 135 <.005 Table 3C. Contrast 5 Late E > N
R Inf Temp Gyr 46 -22 -10 5.45 405 <.001 R Ant Temp Lobe -48 0 -23 5.54 405 <.001 L Ant Temp Lobe -52 1 -24 5.67 189 <.001 Table 3D. Contrast 6 Early N > Late N
L Posterior Caudate -10 -27 18 4.79 216 <.001 Late N > Early N R rACC 3 36 15 4.33 135 <.001
164
Table 4. Chapter 3: Whole-brain Contrasts. n = 28. Coordinates of center of gravity in Talairach space. ‘Nr of Voxels’ refers to voxels interpolated into 1x1x1 space. t-scores are of peak voxel within cluster. Table 4A. Contrast 7 Early E > Late E
Region x y z t(27)-score Nr of Voxels
p (unc)
R Posterior Occ Ctx 22 -93 7 4.97 1836 <.01 L Posterior Occ Ctx -29 -96 2 4.08 1296 <.01 Late E > Early E R rACC 7 43 15 4.11 810 <.01 Table 4B. Contrast 8 Early Emotion Correct > Incorrect
R MPFC 6 58 11 3.99 432 <.005 L MPFC -7 48 1 3.9 945 <.005 Early Emotion Incorrect > Correct L IPS -28 -57 32 4.63 1593 <.005 R Ant IPS 24 -51 33 3.71 297 <.005 R Post IPS 21 -60 36 4.17 270 <.005
165
APPENDICES
Appendix A: Chapter 1
Study 1 Instructions
Thanks in advance for your participation. The programs for this
experiment are “do-it-yourself” so you may run at your convenience without
having to change settings or filenames.
Experiment Set-up: Located in the first room on the left (testing room “L1”) in
the Carrasco Lab, room 970 on the 9th floor of Meyer Hall.
When you come in to run yourself, please go over this checklist first to ensure
the experiment set-up is correct:
1) The monitor must be 57 cm from the chinrest and the edge of the
table (there is a piece of tape on the table marking 57 cm).
2) The monitor video attenuator must be attached (the little metal box
on the video cable that makes the screen green).
3) The monitor resolution must be 1600 x 1200 at 75Hz – please check
this every time you run by going to the Monitor Control Panel under
the apple menu (top left).
166
If these settings are not correct and you run in the experiment, I will have to
discard your data for that session.
You will have your own experiment file in the folder “EmoAttention” on the
Desktop that you will use each time you run.
To run the experiment:
1) Put up the “Experiment in Progress” sign on the door so no one will
interrupt you.
2) Turn out all the lights, and allow your eyes to adapt to the darkness
for a minute.
3) Double click on the folder “EmoAttention” on the Desktop.
4) Find your experiment file (it will have your initials at the end of the
filename) and double click it. This will open the file in Matlab. Do not
modify the file.
5) With the file open, type “Apple-E”. This saves the file and executes
the program.
6) Enter your subject ID in addition to the year, month and day,
followed by the run number for that day. For example, if I ran myself
on June 20th, 2008 I would use this ID: “ef080620_r1”.
167
7) Click OK and you’re ready to run. The program will ask you to press
the spacebar to begin.
Experiment Details: Each run of the program consists of 672 trials in 6 blocks
of 112 trials each. Please feel free to take a break in between blocks. After 3
blocks I would ask that you force yourself to take a break so your eyes don’t
get overtired. Each run takes about 45 minutes to 1 hour to complete. In all, I
ask that you complete 6 sessions over the next 2 weeks. You may do multiple
sessions in one day but please leave a few hours in between each session.
Task Instructions: This experiment requires that you make visual
discriminations about images that appear on the screen. On each trial a
fixation point appears, followed by a brief presentation of 1 or 2 faces located
on either side of the fixation point. (Your eyes must be looking at the fixation
point at all times, unless you are taking a break.) The faces are followed by a
very brief presentation of two tilted gratings in positions just underneath the
faces. One of the gratings will be tilted slightly to the left or the right. Your task
is to indicate which grating is tilted, and which direction it is tilted.
There are 4 response keys: ‘x’, ’c’, ’<’, and ’>’. If you position your pinky
fingers on the shift keys, your middle and index fingers will fall on the correct
168
keys. Use your left hand to indicate if the left grating is tilted and your right
hand if the right grating is tilted. Indicate the direction of tilt using the key that
corresponds to that direction.
Response Keys
X = grating on left, left tilted
C = grating on left, right tilted
< = grating on right, left tilted
> = grating on right, right tilted
Following the presentation of the gratings, you have 2 seconds to make
a response. Try your hardest to push the correct button. We are also
measuring reaction time so try to answer correctly as quickly as possible, but
not at the cost of making more errors. The contrast of the gratings will vary
from trial to trial so they may actually be quite difficult to see on some trials. If
you are unsure which grating was tilted in which direction, please guess.
On each trial you will receive feedback in the form of a tone. A high
tone means you were correct, while a low tone means you were incorrect. No
tone means that the response period ended before you responded. PLEASE
DO YOUR BEST TO GUESS BEFORE THE RESPONSE PERIOD ENDS.
Trials in which we don’t get a response in time will have to be discarded.
169
When the experiment is over, please close all program windows and
quit Matlab. Remove the “Experiment in Progress” sign and, if you’re the only
one in the lab, please close the lab door ensuring it is locked.
Troubleshooting:
- If the monitor will not display anything (it’s black) but the computer is
on, this is probably a problem with the video attenuator. Sometimes if
the computer is started up with the attenuator in place it allows only one
choice for video resolution, but not the one we want. The computer
should be started up again without the attenuator attached, and then
the video resolution should be changed to 1600 x 1200 in the Control
Panel. Only then should the attenuator be attached.
- If the experiment freezes or crashes, please exit the program by
pressing “Apple-period, Apple-0” and type “clear all” in the command
window. Then just re-start the program by typing “Apple-E”.
170
Appendix B: Chapter 2
Study 2, Experiment 2 Instructions
During this experiment, you will be presented with a picture of a face or
four faces, followed by four gratings that are all tilted randomly either left or
right. After the gratings disappear, a line indicating the location of the target
will appear. Your task is to indicate the tilt of the target grating. To indicate
your answer, press either the ‘<’ button for a left tilted grating, or the ‘>’ button
for a right tilted grating. If you get the answer right, you will hear a high tone,
and if you get the answer wrong, you will hear a low tone. You will not be
penalized for wrong answers so make your best guess if you are not sure of
your response.
This experiment will take about an hour on day 1 and an additional hour
on day 2 (two days in the same week).
Schedule for each day:
On day 1, you will arrive at the Carrasco Lab, room 970, at your
designated appointment time. The experiment for this day will take about an
hour. Before we begin the training session, we will go over the task
instructions, and then ask if you have any questions about procedures,
remuneration, time, etc. Then you will begin the training session, which
consists of 3 six-minute blocks of 112 trials each. Instead of pictures of faces
171
you will see black dots, but your task is still to indicate the tilt of the target
grating. After this ~20 minute training session we will check your data and
choose appropriate contrast levels for the main experiment. Next the
experimental session will begin. There will be four blocks and each block will
take about 5-6 minutes each. Thus, if you do not take any breaks in between
each block, you will finish the experiment in about 40 minutes.
On day 2, you will complete the second half of the experiment. There
will be eight blocks total. One block will take around 5-6 minutes. After you are
finished with the experiment, we will ask you to fill out a short questionnaire
about your emotions during and after the experiment. You will then be
debriefed as to the purpose of the experiment and will be paid twenty dollars.
Important:
1. You MUST choose either left or right for every trial. If you do not answer,
the trial will have to be discarded. It is also important to make your
response within two seconds after the gratings disappear from the
screen.
2. Keep your eyes focused on the black cross in the middle of the screen. It
is very important that your eyes don’t wander around to look for the
target grating.
3. Please don’t move the position of the chin rest. We will adjust the chin
172
rest before the experiment so that it is comfortable for you. The chin
rest ensures that you stay the same distance from the screen
throughout the experiment.
4. If, during the training, you cannot get more than 70-75% right, we cannot
have you continue to the main experiment. For that reason, we will not
be able to give you the full $20 remuneration. However, we will
remunerate you with $7 for coming to our lab and doing the training
session part of the experiment. If you do qualify, the $20 remuneration
will be paid on the second day.
5. If your eyes are getting tired, you may utilize a short, less than two-minute
break after each 5-6 minute block.
173
Appendix C: Chapter 3
Study 3 Instructions
Please read the instructions along with the experimenter.
During this experiment you will view a stream of rapidly changing words
in the center of the screen. Each word will immediately replace the one that
came before it. All the words will be printed in black, however there is one
word which will be printed in green. This is the target word. After the word
presentation is completed, you will be presented with a list of 4 possible
choices.
Your task is to view the words and to pick the one word out of the list of
four words that matches the target word. To do so, place your fingers along
the top number row on the keyboard, with your index finger lightly resting on
the 1, middle finger on the 2, ring finger on the 3 and pinky on the 4. We ask
you to rest your hand there in order for you to make your responses as
efficiently as possible.
You won’t get any feedback when you make a response. The choices
will be on the screen for a full 2 seconds (3 seconds during practice). The next
trial will begin automatically even if you don’t make a response.
There will be 5 blocks with breaks in between. Each block is about 5
minutes long. After each block ends, you must notify the experimenter who will
then start you on the next block. They will record how well you do and adjust
174
the difficulty of the task to keep it challenging but not impossible to do fairly
well.
Things to remember:
It is very important that you make a response on every trial. Even if
you are unsure of your response, which is highly likely, you must make a
guess. If you don’t answer too many trials (more than 8) we cannot use your
data.
You must make your response while the choices are on the screen
(approximately 2 seconds). If you don’t, your answer will not be recorded.
Please do not try to lean closer to the screen. Stay a constant
distance so you do not have an unfair advantage over other participants.
After all 5 blocks have been completed there are 3 short surveys to fill
out. If you are interested, the experimenter will then debrief you on the
purpose of the experiment and answer any questions you might have.
To begin, we’ll start with 2 short practice blocks.
175
Neutral Distracters (80)
ART FUSSY PEAR SNAP BAKED GEAR PERCOLATE SOLID
BIN HIKERS PINKY SOLVENT BOARDING HILL PISTON SON
BOOTH HITCH POETIC SPEND CAPE ICICLES PURSE SPIN
CARGOS INSECT RATE STORIED CENT JAZZ REPORT SUM
COLUMN LEASHES RIGGED TACTICS COMPUTER LONER RIVETED THAWS
CORN LOWBROW RUDDER THIS CUPID LUCK RUMOR THORN DATES LUNCH SANDBAR TOCK DOCK MASTERMIND SAY TOGGLE
ENABLES MELODY SEEMS TOURISM ETHICS MILL SHERPA TREADS EXALTS MOOD SHORE VARIOUS FENCE MUST SHUT WAGON FLATS NUN SLAB YOGA FLOOD PAVING SLIPPED ZITS
176
Emotional Distracters (56)
AGONY FAGGOT PENIS ANUS FART PISS
AROUSAL FECES PUBIC ASSHOLE FETISH PUSSY
BARF FONDLE RAPE BASTARD FUCK SCROTUM
BITCH HERPES SEMEN BLOWJOB HORNY SEX
BONER INCEST SHIT BOOBS KILL SHRIEK BREAST KINKY SLAVE
CLITORIS LESBIAN SLUT COCK LEWD TESTICLE
CONDOM LUST TITS CUM LYNCH TUMOR CUNT MASTURBATE VAGINA DICK NIPPLES VIBRATOR
DILDO ORGASM WHORE EJACULATE ORGY
177
Neutral Targets (160)
ABOVE EITHER JAW ORDER TAIL ACADEMY EMBASSY JOIN OWNER TAN
ACTOR EON JOKE PADDLE TASTE AROUND EVERY JUDGE PAGE THINK
AVID FADE JUMP PALE TICKET BALCONY FATIGUED JUNE PATROL TRIVIALITY
BASIC FERN KARMA PLAIN TROPHY BELIEVE FIELD KAYAK PLASTER TUITION
BEND FLAG KEY POINTY TURNED BINDER FLEW KIN QUEEN UNDER BRAVE FOLDER KIND QUICK UPON
BUFFER FOREIGN KNOWN QUIETLY USUAL BUILT FORK LATER RADIO UTTER CALM GARDEN LAZY RAINBOW VACATION
CANVAS GENTLE LEARN RAMBLING VALET CARRIER GIVES LED RELAXATION VALIANT CASUAL GRAIN LUXURIOUS REPEAT VEND CEDAR GRAY MAGNETIC RULE VERSE CHAPEL GROUP MANAGE RYE VIEW CHEEK HALF MEAL SADDLE VOICE CHIN HANGAR MEND SCARCE WAS COP HEAR MONK SECURE WHILE
CUBIC HEAVY MOVIE SIT WIND DEALT HOME NATIVE SOFT WOULD
DEEPEST HOW NEEDLE SOLAR YARD DISCUSS HUNT NERVE SOLD YELLOW
DISK INFER NIECE SPEAKER YESTERDAY DIVIDE INSIDE NOTE SPILL YOUNG
DREARY INVENT OATMEAL SPONSOR ZANY DRYING IRATE OBOE STABLE ZEBRA
DUG IRONED OCCUPANT STAR ZERO EAST JACKET OKAY STRETCHY ZONING
178
Maskers (120)
ABBREVIATION GOVERNMENT RHODODENDRON ALBUQUERQUE HALLUCINATION RIGAMAROLE AMPITHEATER HANDKERCHIEF RIGHTEOUSLY
ANTHROPOLOGY HIEROGLYPHIC RITUALIZATION BEATIFICATION HOUSEKEEPER SIMULTANEOUS
BIOGRAPHY HOUSESITTER SINUSOIDAL BOOTSTRAPPING HUMMINGBIRD SOUNDLEVEL
BUSINESSMAN HYPOTHETICAL TABLEWARES CHRYSANTHEMUM ILLUMINATION THOROUGHBRED CIRCUMFERENCE INCOMPREHENSIBLE THOROUGHLY CLASSIFICATION INTELLECTUALISM THREADBARE
COMEDIAN INTERNALIZATION TIDDLYWINKS CONDENSATION INVESTIGATION TOPOGRAPHY
CONFEDERATION IRREPROACHABLE TOURNAMENT CONGREGATION IRRESPECTIVE TRANSPORTATION CONTORTIONIST JOURNEYMAN ULTRASONIC
DELETERIOUS JUSTIFICATION UNQUESTIONABLY DELICATESSEN JUXTAPOSITION UNSYSTEMATIC
DEMOGRAPHICS KALEIDOSCOPE UPHOLSTERY DEMYSTIFY KINDERGARTENER UTILITARIAN
DESCRIPTION KNOWLEDGEABLE VEGETATION DICTIONARY LEGISLATORSHIP VERBALIZATION
DIFFERENTIATION LOGROLLING VIDEOCASSETTE DISAMBIGUATE LOQUACIOUSNESS VIRTUOSITY
DISAPPEARANCE LUNCHEONETTE VOCALIZATION DISINTEGRATE MERETRICIOUS VOLUMINOUS
EMULATION MOTORCYCLE WINDSHIELD ENCYCLOPEDIA NOTEWORTHINESS WINTERLAND ENTREPRENEUR NOTWITHSTANDING WOODPECKER
ERADICATE NULLIFICATION WOOLGATHERER ESTABLISHMENT NUTRITIONIST WORKSTATION
EXCEEDINGLY OBSERVATORY WRAPAROUND EXIGENCIES ORGANIZATION XYLOPHONIST
EXTRAPOLATION OSCILLATION YARDMASTERS FERTILIZER PERTURBATIONS YESTERYEAR
FORESHADOW PHOTOGRAPHED YOUTHFULLY FUNCTIONALITY PONTIFICATION YOUTHFULNESS FURTHERMORE PURIFICATION ZESTFULNESS
GENERALIZATION QUADRICEPS ZILLIONAIRE GLOBALIZATION RATIONALISTIC ZOOLOGICALLY
179
BIBLIOGRAPHY
Adolphs, R. (2002). Neural systems for recognizing emotion. Current Opinion
in Neurobiology, 12, 169-177.
Adolphs, R., Tranel, D., Damasio, H., & Damasio, A. (1994). Impaired
recognition of emotion in facial expressions following bilateral damage to the
human amygdala. Nature, 372, 669-672.
Amaral, D. G., Behniea, H., & Kelly, J. (2003). Topographic organization of
projections from the amygdala to the visual cortex in the macaque monkey.
Neuroscience, 118(4), 1099-1120.
Anderson, A. K. (2005). Affective influences on the attentional dynamics
supporting awareness. Journal of Experimental Psychology: General, 134(2),
258-281.
Anderson, A. K., & Phelps, E. A. (2001). Lesions of the human amygdala
impair enhanced perception of emotionally salient events. Nature, 411(6835),
305-309.
Armony J. L. & Dolan, R. J. (2002). Modulation of spatial attention by fear-
conditioned stimuli: an event-related fMRI study. Neuropsychologia, 40(7),
817-826.
180
Arnell, K. M., Killman, K. V., & Fijavs, D. (2007). Blinded by emotion: target
misses follow attention capture by arousing distractors in RSVP. Emotion,
7(3), 465-477.
Azzopardi, P., Jones, K. E., & Cowey, A. (1999). Uneven mapping of
magnocellular and parvocellular projections from the lateral geniculate nucleus
to the striate cortex in the macaque monkey. Vision Research, 39(13), 2179-
2189.
Bach, D. R., Talmi, D., Hurlemann, R., Patin, A, & Dolan, R. J. (2011).
Automatic relevance detection in the absence of a functional amygdala.
Neuropsychologia, 49(5), 1302-1305.
Badzakova-Trajkov, G., Häberling, I. S., Roberts, R. P., & Corballis, M. C.
(2010). Cerebral asymmetries: complementary and independent processes.
PLoS ONE, 5(3), e9682.
Barbas, H. (2000). Connections underlying the synthesis of cognition,
memory, and emotion in primate prefrontal cortices. Brain Research Bulletin,
52, 5, 319-330.
Bar-Haim, Y., Lamy, D., Pergamin, L., Bakermans-Kranenburg, M. J., & van
IJzendoorn, M. H. (2007). Threat-related attentional bias in anxious and
nonanxious individuals: a meta-analytic study. Psychological Bulletin, 133(1),
1-24.
181
Bechara, A., Damasio, H., Tranel, D. and Damasio, A. R. (1997). Deciding
advantageously before knowing the advantageous strategy. Science,
275(5304), 1293-1295.
Bishop, S. J. (2007). Neurocognitive mechanisms of anxiety: an integrative
account. Trends in Cognitive Sciences, 11(7), 307-316.
Bishop, S. J. (2009). Trait anxiety and impoverished prefrontal control of
attention. Nature Neuroscience, 12(1), 92-98.
Bishop, S. J., Duncan, J., & Lawrence, A. D. (2004). State anxiety modulation
of the amygdala response to unattended threat-related stimuli. Journal of
Neuroscience, 24(46), 10364-10368.
Bishop, S. J., Duncan, J., Brett, M., & Lawrence, A. D. (2004). Prefrontal
cortical function and anxiety: controlling attention to threat-related stimuli.
Nature Neuroscience, 7(2), 184-188.
Bocanegra, B. R., & Zeelenberg, R. (2009). Emotion improves and impairs
early vision. Psychological Science, 20, 707-713. doi:10.1111/j.1467-
9280.2009.02354.x
Boles, D. B. (1989). Do visual field asymmetries intercorrelate?
Neuropsychologia, 27, 697-704. doi:10.1016/0028-3932(89)90114-0
182
Bourne, V. J. (2008). Examining the relationship between degree of
handedness and degree of cerebral lateralization for processing facial
emotion. Neuropsychologia, 22, 350-356. doi:10.1037/0894-4105.22.3.350
Boynton, G. M., Demb, J. B., Glover, G. H. & Heeger, D. J. (1999). Neuronal
basis of contrast discrimination. Vision Research, 39, 257-269.
Bradley, B., Mogg, K., Falla, S., & Hamilton, L. (1998). Attentional bias for
threatening facial expressions in anxiety: Manipulation of stimulus duration.
Cognition and Emotion, 12(6), 737-753.
Brainard, D. H. (1997). The Psychophysics Toolbox. Spatial Vision, 10, 433-
436. doi: 10.1163/156856897X00357
Broadbent, D.E. (1954). The role of auditory localization in attention and
memory span. Journal of Experimental Psychology, 47(3), 1-6.
Brown, R. & Kulik, J. (1977). Flashbulb memories. Cognition, 5, 73-99.
Bush, G., Luu, P. & Posner, M. I. (2000). Cognitive and emotional influences in
anterior cingulate cortex. Trends in Cognitive Sciences, 4, 6, 215-222.
Cahill, L. (2006). Why sex matters for neuroscience. Nature Reviews
Neuroscience, 7(6), 477-484.
183
Carmichael, S. T., & Price, J. L. (1995). Sensory and premotor connections of
the orbital and medial prefrontal cortex of macaque monkeys. The Journal of
Comparative Neurology, 363, 642-664.
Carrasco, M. (2006). Covert attention increases contrast sensitivity:
Psychophysical, neurophysiological and neuroimaging studies. Progress in
Brain Research, 154, 33-70.
Carrasco, M. (2011). Visual attention: The past 25 years. Vision Research,
doi:10.1016/j.visres.2011.04.012.
Carrasco, M., Penpeci-Talgar, C., & Eckstein, M. P. (2000). Spatial covert
attention increases contrast sensitivity across the CSF: support for signal
enhancement. Vision Research, 40, 1203-1215. doi:10.1016/S0042-
6989(00)00024-9
Carrasco, M. & Yeshurun, Y. (1998). The contribution of covert attention to the
set-size and eccentricity effects in visual search. Journal of Experimental
Psychology: Human Perception & Performance, 24(2), 673-692.
Carretie, L., Hinojosa, J. A., Martin-Loeches, M., Mercado, F. & Tapia, M.
(2004). Automatic attention to emotional stimuli: neural correlates. Human
Brain Mapping, 22, 290-299.
Carter, C. S., Braver, T. S., Barch, D. M., Botvinivk, M. M., Noll, D., & Cohen,
J. D. (1998). Anterior cingulate cortex, error detection, and online monitoring of
performance. Science, 280, 747-749
184
Cavada, C. & Goldman-Racik, P. S. (1989). Posterior parietal cortex in rhesus
monkey. II. Evidence for segregated corticocortical networks linking sensory
and limbic areas with the frontal lobe. Journal of Comparative Neurology, 287,
422-445.
Cavada,, C., Compañy, T., Tejedor, J., Cruz-Rizzolo, R. J., & Reinoso-Suárez,
F. (2000). The anatomical connections of the macaque monkey orbitofrontal
cortex: a review. Cerebral Cortex, 10, 220-242.
Cheal, M. & Lyon, D. R. (1991). Central and peripheral precuing of forced-
choice discrimination. Quarterly Journal of Experimental Psychology A., 43(4),
859-880.
Cherry, E. C. (1953). Some experiments on the recognition of speech, with
one and with two ears. The Journal of the Acoustical Society of America,
25(5), 975-979.
Chun, M. M. and Potter, M. C. (1995). A two-stage model for multiple target
detection in rapid serial visual presentation. Journal of Experimental
Psychology: Human Perception & Performance, 21(1), 109-127.
Cisler, J. M. & Koster, E. H. W. (2010). Mechanisms of attentional biases
towards threat in anxiety disorders: An integrative review. Clinical Psychology
Review, 30, 203-216.
185
Cohen, L., Dehaene, S., Naccache, L., Lehéricy, S., Dehaene-Lambertz, G.,
Hénaff, M-A., & Michel, F. (2000). The visual word form area: spatial and
temporal characterization of an initial state of reading in normal subjects and
posterior split-brain patients. Brain, 123, 291-307.
Compton, R. (2003). The interface between emotion and attention: A review of
evidence from psychology and neuroscience. Behavioral and Cognitive
Neuroscience Reviews, 2(2), 115-129.
Corbetta, M. & Shulman, G. L. (2002). Control of goal-directed and stimulus-
driven attention in the brain. Nature Reviews Neuroscience, 3, 201-215.
Corbetta, M. Patel, G. & Shulman, G. L. (2008). The reorienting system of the
human brain: from environment to theory of mind. Neuron, 58, 306-324.
Coren, S., Porac, G. and Ward, L.M. (1978). Sensation and Perception. New
York: Academic Press.
Coull, J. T., Frith, C. D., Buchel, C., & Nobre, A. C. (2000). Orienting attention
in time: behavioral and neuroanatomical distinction between exogenous and
endogenous shifts. Neuropsychologia, 38, 808-819.
Craske, M. G. (2003). Origins of phobias and anxiety disorders: Why more
women than men? Oxford: Elsevier.
Davis, M. & Whalen, P. J. (2001). The amygdala: vigilance and emotion.
Molecular Psychiatry, 6(1), 13-34.
186
De Martino, B., Strange, B. A., & Dolan, R., J. (2008). Noradrenergic
neuromodulation of human attention for emotional and neutral stimuli.
Psychopharmacology, 197(1), 127-136.
De Martino, B., Kalisch, R., Rees, G., & Dolan, R. J. (2009). Enhanced
processing of threat stimuli under limited attentional resources. Cerebral
Cortex, 19(1), 127-133.
Derryberry, D., & Reed, M. A. (2002). Anxiety-related attentional biases and
their regulation by attentional control. Journal of Abnormal Psychology, 111(2),
225-236.
DeValois, R.L. & DeValois, K. K. (1988). Spatial Vision. New York: Oxford
University Press.
Desimone, R. & Duncan, J. (1995). Neural mechanisms of selective visual
attention. Annual Review of Neuroscience, 18(1), 193-222.
Dickie, E. W., & Armony, J. L. (2008). Amygdala responses to unattended
fearful faces: Interaction between sex and trait anxiety. Psychiatry Research,
162(1), 51-57.
Dronkers, N. F., & Knight, R. T. (1989). Right-sided neglect in a left-hander:
Evidence for reversed hemispheric specialization of attention capacity.
Neuropsychologia, 27, 729-735. doi:10.1016/0028-3932(89)90118-8
187
Dux, P. E., & Marois, R. (2009). The attentional blink: a review of data and
theory. Attention, Perception & Psychophysics, 71(8), 1683-1700.
Easterbrook, J. A. (1959). The effect of emotion on cue utilization and the
organization of behavior. Psychological Review, 66(3), 183-201.
Eimer, M. & Holmes, A (2002). An ERP study on the time course of emotional
face processing. NeuroReport, 13, 427-431.
Ekman, P., & Friesen, W. (1976). Pictures of Facial Affect. Palo Alto, CA:
Consulting Psychologists Press.
Elder, J. H., Balaban, D. Y., Kamyab, A., Wilcox, L., & Hou, Y. (2008, May).
Selectivity for faces as exogenous attentional cues [Abstract]. Journal of
Vision, 8, 685a.
Eriksen, C.W., & St. James, J.D. (1986). Visual attention within and around the
field of focal attention: A zoom lens model. Perception & Psychophysics, 40,
225-240.
Evert, D. L., McGlinchey-Berroth, R., Verfaellie, M., & Milberg, W. P. (2003).
Hemispheric asymmetries for selective attention apparent only with increased
task demands in healthy participants. Brain and Cognition, 53, 34-41. doi:
10.1016/S0278-2626(03)00207-0
Eysenck, M. W., Derakshan, N., Santos, R., & Calvo, M. G. (2007). Anxiety
and cognitive performance: attentional control theory. Emotion, 7(2), 336-353.
188
Fecteau, J. H., Enns, J. T., & Kingstone, A. (2000). Competition-induced visual
field differences in search. Psychological Science, 11, 386-393.
doi:10.1111/1467-9280.00275
Ferneyhough, E., Stanley, D. A., Phelps, E. A., & Carrasco, M. (2010). Cuing
effects of faces are dependent on handedness and visual field. Psychonomic
Bulletin & Review, 17(4), 529-535.
Flöel, A., Buyx, A., Breitenstein, C., Lohmann, H., & Knecht, S. (2005).
Hemispheric lateralization of spatial attention in right- and left-hemispheric
language dominance. Behavioral Brain Research, 158(2), 269-275.
Fox, E. (1993). Attentional bias in anxiety: selective or not? Behavioral
Research and Therapy, 31(5), 487-493.
Fox, E. (2002). Processing emotional facial expressions: the role of anxiety
and awareness. Cognitive & Affective Behavioral Neuroscience, 2, 1, 52-63.
Fox, E., Russo, R., Bowles, R. J., & Dutton, K. (2001). Do threatening stimuli
draw or hold visual attention in subclinical anxiety? Journal of Experimental
Psychology: General, 130(4), 681-700.
Fox, E., Russo, R., & Dutton, K. (2002). Attentional Bias for Threat: Evidence
for Delayed Disengagement from Emotional Faces. Cognition & Emotion,
16(3), 355-379.
189
Fox, E., Russo, R., & Georgiou, G. A. (2005). Anxiety modulates the degree of
attentive resources required to process emotional faces. Cognitive & Affective
Behavioral Neuroscience, 5(4), 396-404.
Freese, J. L., & Amaral, D. G. (2005). The organization of projections from the
amygdala to visual cortical areas TE and V1 in the macaque monkey. The
Journal of Comparative Neurology, 1-23.
Frijda, N. H. (1986). The Emotions. New York: Cambridge University Press.
Fuller, S., Park, Y. & Carrasco, M (2009). Cue contrast modulates the effects
of exogenous attention on appearance. Vision Research, 49, 1825-1837.
doi:10.1016/j.visres.2009.04.019
Fuller, S., Rodriguez, R. Z., & Carrasco, M. (2008). Apparent contrast differs
across the vertical meridian: visual and attentional factors. Journal of Vision,
8(1), 1-16, http://journalofvision.org/8/1/16/, doi:10.1167/8.1.16.
Graham, N. (2011). Beyond multiple pattern analyzers modeled as linear filters
(as classical V1 simple cells): useful additions of the last 25 years. Vision
Research, doi:10.1016/j.visres.2011.02.007.
Hartley, C. A. & Phelps, E. A. (2010). Changing fear: the neurocircuitry of
emotion regulation. Neuropsychopharmacology Reviews, 35, 136-146.
190
Haxby, J. V., Hoffman, E. A., & Gobbini, M. I. (2002). Human neural systems
for face recognition and social communication. Biological Psychiatry, 51, 59-
67.
Hellige, J. B., Bloch, M. I., Cowin, E. L., Eng, T. L., Eviatar, Z. & Sergent, V.
(1994). Individual variation in hemispheric asymmetry: multitask study of
effects related to handedness and sex. Journal of Experimental Psychology:
General, 123, 3, 2353-256.
Holmes, A., Nielsen, M. K., & Green, S. (2008). Effects of anxiety on the
processing of fearful and happy faces: an event-related potential study.
Biological Psychology, 77(2), 159-173.
Jefferies, L. N., Smilek, D., Eich, E., & Enns, J. T. (2008). Emotional valence
and arousal interact in attentional control. Psychological Science, 19(3), 290-
295.
Keil, A. & Ihssen, N. (2004). Identification facilitation for emotionally arousing
verbs during the attentional blink. Emotion, 4(1), 23-35.
Kelly, C., Uddin, L. Q., Shehzad, Z., Margulies, D. S., Castellanos, X., Milham,
M. P. & Petrides, M. (2010). Broca’s region: linking human brain functional
connectivity data and non-human primate tracing anatomy studies. European
Journal of Neuroscience, 32, 3, 383-398.
191
Kemp, A. H., Silberstein, R. B., Armstrong, S. M., & Nathan, P. J. (2004).
Gender differences in the cortical electrophysiological processing of visual
emotional stimuli. Neuroimage, 21(2), 632-646.
Kessler R. C., Berglund P., Demler O., et al. (2005). Lifetime prevalence and
age-of-onset distributions of DSM- IV disorders in the National Comorbidity
Survey Replication. Archives of General Psychiatry, 62, 593-602.
Kim, M. J., & Whalen, P. J. (2009). The Structural Integrity of an Amygdala-
Prefrontal Pathway Predicts Trait Anxiety. Journal of Neuroscience, 29(37),
11614-11618.
Kissler, J., Herbert, C., Peyk, P. & Junghofer, M. (2007). Buzzwords: early
cortical responses to emotional words during reading. Psychological Science,
18, 6, 475-480.
Kinchla, R. A. (1992). Attention. Annual Review of Psychology, 43, 711-743.
doi: 10.1146/annurev.ps.43.020192.003431
Koster, E. H. W., Crombez, G., Van Damme, S., Verschuere, B., & De
Houwer, J. (2004). Does imminent threat capture and hold attention? Emotion,
4(3), 312-317.
Koster, E. H. W., Verschuere, B., Crombez, G., & Van Damme, S. (2005).
Time-course of attention for threatening pictures in high and low trait
anxiety. Behaviour Research and Therapy, 43, 1087-1098.
192
Koster, E. H. W., Crombez, G., Verschuere, B., & De Houwer, J. (2006).
Attention to threat in anxiety-prone individuals: Mechanisms underlying
attentional bias. Cognitive Therapy and Research, 30(5), 635-643.
Lang, P. J., Bradley, M. M., Fitzsimmons, J. R., Cuthbert, B. N., Scott, J. D.,
Moulder, B., et al. (1998). Emotional arousal and activation of the visual
cortex: an fMRI analysis. Psychophysiology, 35(2), 199-210.
LeDoux, J. E. (1996). The Emotional Brain: The Mysterious Underpinnings of
Emotional Life. Simon & Schuster: New York.
LeDoux, J. (2000). Emotion circuits in the brain. Annual Review of
Neuroscience, 23, 155-184.
LeDoux, J. (2002). Synaptic self: How our brains become who we are. New
York: Viking Penguin.
Lennie, P. (2003). The cost of cortical computation. Current Biology, 13(6),
493-497.
Leuba, C. & Lucas, C. (1945). The effects of attitudes on descriptions of
pictures. Journal of Experimental Psychology, 35, 517-524.
Lim, S-L., Padmala, S., & Pessoa, L. (2008). Affective learning modulates
spatial competition during low-load attentional conditions. Neuropsychologia,
46(5), 1267-1278.
193
Ling, S., & Carrasco, M. (2006a). Sustained and transient covert attention
enhance the signal via different contrast response functions. Vision Research,
46, 1210-1220. doi:10.1038/nn1761
Ling, S. & Carrasco, M. (2006b). When sustained attention impairs perception.
Nature Neuroscience, 9,1243-1245.
Liu, T., Pestilli, F., & Carrasco, M. (2005). Transient attention enhances
perceptual performance and fMRI response in human visual cortex. Neuron,
45(3), 469-477.
Luh, K. E., Redl, J., & Levy, J. (1994). Left- and right-handers see people
differently: Free-vision perceptual asymmetries for chimeric stimuli. Brain &
Cognition, 25, 141-160. doi: 10.1006/brcg.1994.1028
Luo, Q., Peng, D., Jin, Z., Xu, D., Xiao, L. & Ding, G. (2004). Emotional
valence of words modulates the subliminal repetition priming effect in the left
fusiform gyrus: an event-related fMRI study. Neuroimage, 21, 414-421.
Macdonald, A. W., Cohen, J. D., Stenger, A. & Carter, C. S. (2000).
Dissociating the role of the dorsolateral prefrontal cortex in cognitive control.
Science, 288, 1835-1838.
Macleod, C., Mathews, A. & Tata, P. (1986). Attentional bias in emotional
disorders. Journal of Abnormal Psychology, 95, 1, 15-20.
194
Macleod, C., & Mathews, A. (1988). Anxiety and the allocation of attention to
threat. Quarterly Journal of Experimental Psychology A, 40(4), 653-670.
Marois, R. & Ivanoff, J. (2005). Capacity limits of information processing in the
brain. Trends in Cognitive Sciences, 9, 6, 296-305.
Mathews, A., & Macleod, C. (1985). Selective processing of threat cues in
anxiety states. Behavioral Research & Therapy, 23, 5, 563-569.
Mathewson, K. J., Arnell, K. M., & Mansfield, C. A. (2008). Capturing and
holding attention: the impact of emotional words in rapid serial visual
presentation. Memory & Cognition, 36, 1, 182-200.
McClure, E. B. (2000). A meta-analytic review of sex differences in facial
expression processing and their development in infants, children, and
adolescents. Psychological Bulletin, 126(3), 424-453.
McDonald, A. J., Mascagni, F. & Guo, L. (1996). Projections of the medial and
lateral prefrontal cortices to the amygdala: a Phaseolus vulgaris
leucoagglutinin study in the rat. Neuroscience, 71, 55-75.
Mesulam, M. M. (1999) Spatial attention and neglect: Parietal, frontal and
cingulate contributions to the mental representation and attentional targeting
of salient extrapersonal events. Philosophical Transactions of the Royal
Society London B Biological Sciences, 354, 1325–1346.
195
Mogg, K. & Bradley, B. P. (1999). Some methodological issues in assessing
attentional biases for threatening faces in anxiety: A replication study using a
modified version of the probe detection task. Behavioral Research & Therapy,
37, 6, 595-604.
Mogg, K., Holmes, A., Garner, M., Bradley, B. P. (2008). Effects of threat cues
on attentional shifting, disengagement and response slowing in anxious
individuals. Behavioral Research & Therapy, 46(5), 656-667.
Mogg, K., Bradley, B.P., Miles, F., and Dixon, R. (2004). Time course of
attentional bias for threat scenes: testing the vigilance—avoidance hypothesis,
Cognition & Emotion, 18, 689–700
Montagna, B., Pestilli, F., & Carrasco, M. (2009). Attention trades off spatial
acuity. Vision Research, 49, 735-745. doi:10.1016/j.visres.2009.02.001
Morawetz, C., Baudewig, J., Treue, S., & Dechent, P. (2010). Diverting
attention suppresses human amygdala responses to faces. Frontiers in
Human Neuroscience, 4, 226.
Morecraft, R. J., Geula, C. & Mesulam, M. M. (1993). Architecture of
connectivity within a cingulo-fronto-parietal neurocognitive network for directed
attention. Archives of Neurology, 50, 3, 279-284.
Morris, J. S., de Gelder, B., Weiskrantz, L., & Dolan, R. J. (2001). Differential
extrageniculostriate and amygdala responses to presentation of emotional
faces in a cortically blind field. Brain, 124(Pt 6), 1241-1252.
196
Morris, J. S., Friston, K. J., Büchel, C., Frith, C., Young, A. W., Calder, A. J., et
al. (1998). A neuromodulatory role for the human amygdala in processing
emotional facial expressions. Brain, 121(Pt 1), 47-57.
Morris, J. S., Frith, C. D., Perrett, D. I., Rowland, D., Young, A. W., Calder, A.
J. & Dolan, R., J. (1996). A differential neural response in the human
amygdala to fearful and happy facial expressions. Nature, 383, 812-815.
Most, S. B., Chun, M. M., Widders, D. M., & Zald, D. H. (2005). Attentional
rubbernecking: cognitive control and personality in emotion-induced blindness.
Psychonomic Bulletin & Review, 12(4), 654-661.
Most, S. B., Chun, M. M., Johnson, M. R., & Kiehl, K. A. (2006). Attentional
modulation of the amygdala varies with personality. Neuroimage, 31, 2, 934-
944.
Most, S. B., Smith, S. D., Cooter, A. B., Levy, B. N., & Zald, D. H. (2007). The
naked truth: positive, arousing distractors impair rapid target perception.
Cognition and Emotion, 21, 5, 964-981.
Müller, H. J. & Rabbitt, P. M. A. (1989). Reflexive and voluntary orienting of
visual attention: time course of activation and resistance to interruption.
Journal of Experimental Psychology: Human Perception & Performance, 15, 2,
315-330.
197
Nakayama, K., & Mackeben, M. (1989). Sustained and transient components
of focal visual attention. Vision Research, 29, 1631-1647. doi: 10.1016/0042-
6989(89)90144-2
Nobre, A. C., Coull, J. T., Frith, C. D., & Mesulam, M. M. (1999). Orbitofrontal
cortex is activated during breaches of expectation in tasks of visual attention.
Nature Neuroscience, 2, 1, 11-12.
Notebaert, L., Crombez, G., Van Damme, S., De Houwer, J. & Theeuwes, J.
(2011). Signals of threat do not capture, but prioritize, attention: A conditioning
approach. Emotion, 11, 1, 81-89.
Öhman, A. (2009). Of snakes and faces: an evolutionary perspective on the
psychology of fear. Scandinavian Journal of Psychology, 50, 6, 543-552.
Öhman, A., Flykt, A., & Esteves, F. (2001). Emotion Drives Attention:
Detecting the Snake in the Grass. Journal of Experimental Psychology:
General. 130(3), 466-478.
Oldfield, R. C. (1971). The assessment and analysis of handedness: The
Edinburgh Inventory. Neuropsychologia, 9, 97 - 113. doi: 10.1016/0028-
3932(71)90067-4.
Olsson, A., Nearing, K. I., & Phelps, E. A. (2007). Learning fears by observing
others: the neural systems of social fear transmission. Social Cognitive and
Affective Neuroscience, 2, 3-11.
198
Olsson, A. & Phelps, E. A. (2007). Social learning of fear. Nature
Neuroscience, 10, 9, 1095-1102.
Oosterhof, N. N., & Todorov, A. (2008). The functional basis of face
evaluation. Proceedings of the National Academy of Sciences, 105, 11087-
11092. doi: 10.1073/pnas.0805664105
Ortigue, S., Michel, C. M., Murray, M. M., Mohr, C., Carbonnel S. & Landis, T.
(2004). Electrical neuroimaging reveals early generator modulation to
emotional words. Neuroimage, 21, 4, 1242-1251.
Pessoa, L. (2010). Emotion and attention effects: is it all a matter of timing?
Not yet. Frontiers in Human Neuroscience, 4, 172, 1-5.
Pessoa, L. & Adolphs, R. (2010). Emotion processing and the amygdala: from
a ‘low road’ to ‘many roads’ of evaluating biological significance. Nature
Reviews Neuroscience, 11, 11, 773-783.
Pessoa, L., Kastner, S. & Ungerleider, L. G. (2002). Attentional control of the
processing of neutral and emotional stimuli. Cognitive Brain Research, 15, 1,
31-45.
Pestilli, F., & Carrasco, M. (2005). Attention enhances contrast sensitivity at
cued and impairs it at uncued locations. Vision Research, 45, 1867-1875.
doi:10.1016/j.visres.2005.01.019
199
Pestilli, F., Ling, S., & Carrasco, M. (2009). A population-coding model of
attention's influence on contrast response: Estimating neural effects from
psychophysical data. Vision Research, 49(10), 1144-1153.
Pestilli, F., Viera, G., & Carrasco, M. (2007). How do attention and adaptation
affect contrast sensitivity? Journal of Vision, 7(7), 1-12.
Peterzell, D. H. & Teller, D. Y. (1996). Individual differences in contrast
sensitivity functions: the lowest spatial frequency channels. Vision Research,
36, 19, 3077-3085.
Phelps, E. A. (2006). Emotion and cognition: insights from studies of the
human amygdala. Annual Review of Psychology, 57, 1, 27-53.
Phelps, E. A., Ling, S., & Carrasco, M. (2006). Emotion facilitates perception
and potentiates the perceptual benefits of attention. Psychological Science,
17, 292-299. doi:10.1111/j.1467-9280.2006.01701.x
Posner, M. I. (1980). Orienting of attention. The Quarterly Journal of
Experimental Psychology, 32, 1, 3-25.
Posner, M. I., & Petersen, S. (1990). The attention system of the human brain.
Annual Review of Neuroscience, 13(1), 25-42.
Pourtois, G., Grandjean, D., Sander D., Vuilleumier, P. (2004).
Electrophysiological correlates of rapid spatial orienting towards fearful faces.
Cerebral Cortex, 14, 6, 619-633.
200
Pourtois, G., Schwartz,S., Seghier, M. L., Lazeyras, F., & Vuilleumier, P.
(2006). Neural systems for orienting attention to the location of threat signals:
an event-related fMRI study. Neuroimage, 31, 2, 920-933.
Quirk, G. J., & Mueller, D. (2008). Neural mechanisms of extinction learning
and retrieval. Neuropsychopharmacology, 33 1, 56-72.
Raymond, J. E., Shapiro, K. L., & Arnell, K. M. (1992). Temporary suppression
of visual processing in an RSVP task: an attentional blink? Journal of
Experimental Psychology: Human Perception and Performance, 18, 3, 849-
860.
Raymond, M., Pontier, D., Dufour, A., & M"ller, A. P. (1996). Frequency-
dependent maintenance of left-handedness in humans. Proceedings of the
Royal Society of London, B, 263, 1627-1633.
Rhodes, G. (1985). Lateralized processes in face recognition. British Journal
of Psychology, 76, 249 – 271.
Richards, A. & Millwood, B. (1989). Colour-identification of differentially
valenced words in anxiety. Cognition & Emotion, 3, 2, 171-176.
Rubichi, S., & Nicoletti R. (2006). The Simon effect and handedness: Evidence
for a dominant-hand attentional bias in spatial coding. Perception &
Psychophysics, 68, 1059-1069.
201
Sass, S. M., Heller, W., Stewart, J. L., Silton, R. L., Edgar, J. C., Fisher, J. E.,
et al. (2010). Time course of attentional bias in anxiety: Emotion and gender
specificity. Psychophysiology, 47(2), 247-259.
Saygin, Z. M., Osher, D. E., Augustinack, J., Fischl, B. & Gabrieli, J. D. E.
(2011). Connectivity-based segmentation of amygdala nuclei using
probabilistic tractography. Neuroimage, 56, 1353-1361.
Schupp, H. T., Markus, J., Weike, A. I., Hamm, A. D. (2003). Emotional
facilitation of sensory processing in the visual cortex. Psychological Science,
14, 1, 7-13.
Schwabe, L., Merz, C. J., Walter, B., Vaitl, D., Wolf, O. T., & Stark, R. (2010).
Emotional modulation of the attentional blink: the neural structures involved in
capturing and holding attention. Neuropsychologia, 49, 3, 416-425.
Sharot, T., Delgado, M. R., & Phelps, E. A. (2004). How emotion enhances the
feeling of remembering. Nature Neuroscience, 7(12), 1376-1380.
Siman-Tov, T., Mendelsohn, A., Schonberg, T., Avidan, G., Podlipsky, I.,
Pessoa, L., Gadoth, N., Ungerleider, L. G., & Hendler, T. (2007).
Bihemispheric leftward bias in a visuospatial attention-related network. Journal
of Neuroscience, 27, 11271-11278. doi: 10.1523/JNEUROSCI.0599-07.2007
Simpson, W. A. & McFadden, S. M. (2005). Spatial frequency channels
derived from individual differences. Vision Research, 45, 2723-2727.
202
Smith, S. D., Most, S. B., Newsome, L. A., & Zald, D. H. (2006). An emotion-
induced attentional blink elicited by aversively conditioned stimuli. Emotion,
6(3), 523-527.
Spampinato, M. V., Wood, J. N., De Simone, V., & Grafman, J. (2009). Neural
correlates of anxiety in healthy volunteers: a voxel-based morphometry study.
Journal of Neuropsychiatry & Clinical Neuroscience, 21(2), 199-205.
Spielberger, C.D., Gorsuch, R.L., Lushene, R., Vagg, P.R. and Jacobs, G.A.
(1983). Manual for the State-Trait Anxiety Inventory. Consulting Psychologists
Press: Palo Alto.
Stanley, D.A., Ferneyhough, E., & Phelps, E.A. (2009). Neural Perspectives
on Emotion: Impact on Perception, Attention and Memory. In G.G. Berntson &
J.T. Cacciopo (Eds.), Handbook of Neuroscience for the Behavioral Sciences.
New York: John Wiley & Sons.
Talairach, J. & Tournoux, P. (1988). Co-planar stereotaxic atlas of the human
brain: 3-dimensional proportional system; an approach to cerebral imaging.
Stuttgart: George Thieme Verlag.
Taylor, J. G. & Fragopanagos, N. F. (2004). Modelling human attention and
emotions. Proceedings of the International Joint Conference on Neural
Networks.
Taylor, J. G. & Fragopanagos, N. F. (2005). The interaction of attention and
emotion. Neural Networks, 18, 353-369.
203
Thayer, J. F., & Johnsen, B. H. (2000). Sex differences in judgement of facial
affect: a multivariate analysis of recognition errors. Scandinavian Journal of
Psychology, 41(3), 243-246.
Tsuchiya, N., Moradi, F., Felsen, C., Yamazaki, M., & Adolphs, R. (2009).
Intact rapid detection of fearful faces in the absence of the amygdala. Nature
Neuroscience, 12, 10, 1224-1225.
Vaughan, K. B. & Lanzetta, J. T. (1980). Vicarious instigation and conditioning
of facial expressive and autonomic responses to a model’s expressive display
of pain. Journal of Personality and Social Psychology, 38, 6, 909-923.
Vuilleumier, P., Armony, J. L., Driver, J., & Dolan, R. J. (2003). Distinct spatial
frequency sensitivities for processing faces and emotional expressions. Nature
Neuroscience, 6(6), 624-631.
Vuilleumier, P., Richardson, M., Armony, J. L., Driver, J., & Dolan, R. J.
(2004). Distant influences of amygdala lesion on visual cortical activation
during emotional face processing. Nature Neuroscience, 7(11), 1271-1278.
Vuilleumier, P., Schwartz, S., Verdon, V., Maravita, A., Hutton, C., Husain, M.,
& Driver, J. (2008). Abnormal attentional modulation of retinotopic cortex in
parietal patients with spatial neglect. Current Biology, 18, 1525-1529.
204
Watson, D., Clark, L.A. and Tellegen, A. (1988). Development and validation
of brief measures of positive and negative affect: The PANAS scales. Journal
of Personality and Social Psychology, 54, 1063–1070.
Weierich, M., Treat, T., & Hollingworth, A. (2008). Theories and measurement
of visual attentional processing in anxiety. Cognition & Emotion, 22(6), 985-
1018.
Whalen, P. J. (1998). Fear, vigilance, and ambiguity: initial neuroimaging
studies of the human amygdala. Current Directions in Psychological Science,
7, 177-188.
Wichmann, F. A., & Hill, N. J. (2001): The psychometric function: I. Fitting,
sampling and goodness-of-fit. Perception & Psychophysics, 63, 1293-1313.
Yamasaki, H., LaBar, K. S. & McCarthy, G. (2002). Dissociable prefrontal
brain systems for attention and emotion. Proceedings of the National Academy
of Sciences, 99, 11447-11451.
Yeshurun, Y. & Carrasco, M. (1999). Spatial attention improves performance
in spatial resolution tasks. Vision Research, 39, 293-306.
Yiend, J., & Mathews, A. (2001). Anxiety and attention to threatening pictures.
The Quarterly Journal of Experimental Psychology Section A, 54(3), 665-681.
Yovel, G., Levy, J., Grabowecky, M., & Paller, K. A. (2003). Neural correlates
of the left-visual-field superiority in face perception appear at multiple stages of
205
face processing. Journal of Cognitive Neuroscience, 15, 462–474. doi:
10.1162/089892903321593162
Yovel, G., Tambini, A., & Brandman, T. (2008). The asymmetry of the fusiform
face area is a stable individual characteristic that underlies the left-visual-field
superiority for faces. Neuropsychologia, 46, 3061-3068. doi:
10.1016/j.neuropsychologia.2008.06.017