neurociencia the cognitive electrophysiology of mind and brain - a. zani, a. proverbio (ap, 2002) ww

Contributors

C. J. Aine New Mexico VA Health Care Systemand Departments of Radiology, Neurology,and Neuroscience, University of New MexicoSchool of Medicine, Albuquerque, NewMexico 87018

Gabriele Biella Institute of Neuroscience andBioimaging, Consiglio Nazionale delleRicerche, 20090 Segrate (Milan), Italy

Elvira Brattico Cognitive Brain Research Unit,Department of Psychology, FIN-00014University of Helsinki, Finland

Roberto Cabeza Center for Cognitive Neuro-science, Duke University, Durham, NorthCarolina 27708

Francesco Di Russo Department of Neuro-sciences, University of California, San Diego,La Jolla, California 92093; and IRCCSFondazione Santa Lucia, 306 00174 Rome, Italy

Kara D. Federmeier Department of CognitiveScience, University of California, San Diego,La Jolla, California 92093

Steven A. Hillyard Department of Neuro-sciences, University of California, San Diego,La Jolla, California 92093

Robert Kluender Department of CognitiveScience, University of California, San Diego,La Jolla, California 92093

Marta Kutas Departments of Cognitive Scienceand Neurosciences, University of California,San Diego, La Jolla, California 92093

Phan Luu Electrical Geodesics, Inc., Universityof Oregon, Eugene, Oregon 97403

George R. Mangun Center for CognitiveNeuroscience, Duke University, Durham,North Carolina 27708

Teresa V. Mitchell Brain Imaging andAnalysis Center, Duke University MedicalCenter, Durham, North Carolina 27710

Risto Näätänen Cognitive Brain ResearchUnit, Department of Psychology, FIN-00014University of Helsinki, Finland

Helen J. Neville Department of Psychology,University of Oregon, Eugene, Oregon 97403

Lars Nyberg Department of Psychology,Umeå University

Alice Mado Proverbio Department of Psy-chology, University of Milano-Bicocca, 20126 Milan, Italy; and Institute ofNeuroscience and Bioimaging, ConsiglioNazionale delle Ricerche, 20090 Segrate(Milan), Italy

Helen Sharpe School of Psychology, CardiffUniversity, Cardiff CF10 3YG, Wales, UnitedKingdom

Wolfgang Skrandies Institute of Physiology,Justus-Liebig University, 35392 Giessen,Germany

J. M. Stephen New Mexico VA Health CareSystem and Department of Radiology,University of New Mexico School ofMedicine, Albuquerque, New Mexico 87018

xi

Wolfgang A. Teder-Sälejärvi Department ofNeurosciences, University of California, SanDiego, La Jolla, California 92093

Mari Tervaniemi Cognitive Brain ResearchUnit, Department of Psychology, FIN-00014University of Helsinki, Finland

Don M. Tucker Electrical Geodesics, Inc.,University of Oregon, Eugene, Oregon 97403

Rolf Verleger Department of Neurology,University Clinic, D23538 Lübeck, Germany

Edward L. Wilding School of Psychology,Cardiff University, Cardiff CF10 3YG, Wales,United Kingdom

Alberto Zani Institute of Neuroscience andBioimaging, Consiglio Nazionale delleRicerche, 20090 Segrate (Milan), Italy

xii CONTRIBUTORS

Acknowledgments

We express our heartfelt gratitude toJohannes Menzel, Senior Publishing Edi-tor, Neuroscience, of Elsevier Science/Academic Press, who from the very begin-ning believed in the project and encouragedus strongly. His unflagging good spiritsmade him a joy to work with, and hisconcern for quality kept us on our toes.

Also many thanks are due to CindyMinor for supervising the gathering ofchapter manuscripts and for paying carefulattention to detail. Without her assistance,this complex project would never havegotten off the ground. Both JohannesMenzel and Cindy Minor were staunchsupporters for the realization of the book.

To Mike Posner, a peerless, unflaggingpioneer in indicating new tracks for theuncovering of how mind and brain arestrictly intermingled, and who willinglywrote the foreword to the book, we offerour great respect and admiration. We alsothank the three anonymous reviewers, whogave such a positive appraisal of our bookproposal and its table of contents, and thedistinguished panel of international con-tributors. We are deeply indebted to all thecontributors who, we feel, played a mostrelevant role in making this book a reality.We thank them for contributing the out-standing chapters gathered in the book.

We are also indebted for the generousfinancial support given by the Institute ofNeuroscience and Bioimaging (INB) of the

National Research Council (CNR), Milan(Italy), and the Department of Psychologyof the University of Milano-Bicocca. Wethank Minna Huotilainen and Fred Previc,who willingly gave their permission for thereproduction of their illustrations. In addi-tion, we thank Kathy Nida, ProductionEditor for Academic Press, and Shirley Tan,of Best-Set Typesetters, as well as all thoseof the staff of Academic Press who helpedus, in any way at all, during the develop-ment of the project itself.

For the photos of the ERP Lab we thankLuciano Chiumento for the shooting andLuisa Aquino, who kindly volunteered topose during the shooting itself. Thanks arealso due to Ian McGilvray for his help inre-editing some chapters of the book.

Special thanks to Dr. Rachel C. Stennerwho, through her creative rewriting, con-tributed greatly to the clarity and fluency ofa significant part of our writings.Furthermore, the constant support of ourcollaborators and the kindness they showedat all times, despite their being somewhatneglected, has been much appreciated.

Finally, we acknowledge, with our mostheartfelt and affectionate thanks, our fami-lies, who gave of themselves unstintinglywhen we needed them and who showedgreat patience, especially little Alessandrofor his understandable lack of it given histender years and for the quality time thatwas inevitably lost to him now and then.

xiii

Foreword

ELECTRICAL PROBES OF MIND AND BRAIN

The Cognitive Electrophysiology of Mineand Brain makes explicit that scalp electri-cal recordings have joined other methodsas a means of understanding the connec-tions between brain and mind. Only sevenyears ago, I wrote a foreword to a newvolume, Electrophysiology of Mind (Ruggand Coles, 1995). That book summarizedthe use of electrical recording as a chrono-metric tool to describe the time course ofmental operations, but no explicit effortwas made to relate these findings to otherapproaches to neuroimaging. In that fore-word, I suggested that the future of scalpelectrical recording lay in firm connectionsto hemodynamic imaging methods such asPET and fMRI. Acceptance of the fullimport of these connections is still inhib-ited. However, in my view and those ofmost of the authors of this volume, it istime to face the consequences of localiza-tion of generators in neural tissue, bymaking efforts to use electrical recordingmethods to probe the time course ofanatomical areas recruited in performingcognitive functions.

The effort to understand the origins andsignificance of the brain’s electrical andmagnetic signals is detailed in Chapter 2.

The methods for linking them to underly-ing generators in the brain are described inseveral of the chapters; these efforts areactive areas of research (e.g., Dale et al.,2000). A number of algorithms are alreadyavailable as commercial packages, and newideas, like those described in Chapters 5and 11, are being developed. Within thevisual system, there has been very detailedvalidation linking these generators toretinotopic maps found in several visualareas. For complex skills, the evidence isless complete, but because tasks like visualimagery, reading, and number processinghave yielded widely separated generators,it has proven possible to provide detailedanalysis of their time course from scalpelectrical recordings (Abdullaev andPosner, 2000; Dehaene, 1996; Posner andMcCandliss, 1999; Raij, 1999).

Even if researchers reading this book areconvinced that electrical recordings canplay a role in probing the organization ofneural networks involved in cognition, itdoes not mean that all controversies aresettled. In fact, the controversy may bemore severe, because if scalp electrodes areto be integrated with lesion studies, cellu-lar recording, and hemodynamic imagingas a means of probing both mind and brain,we will have to be serious about reasoningfrom the combination of these methods.

xv

Chapter 3 provides a very usefulsummary of many findings that indicatedifferent cognitive function (e.g., workingmemory and attention) often recruit thesame brain area. The authors are surelycorrect that cognitive psychology textbookchapter titles are not an appropriate guideto brain localization. However, before weconclude that different operations activatethe same brain area, we need to be moreclear about what makes a difference inmental operations. For example, theories ofworking memory assume the involvementof attentional networks, so it would be sur-prising not to find attention areas active inworking memory tasks, but it is rather easyto design an attention task that does notinvolve working memory. We also need tobe more explicit about what the same brainarea means (i.e., the extent of overlapneeded to assume identity). Finally weneed to know when in the task a particulararea is active. Both perception and imagerytasks may activate prestriate visual areas,but the latter may do so only after activa-tion of higher association areas.

The use of electrical recordings is impor-tant for tracing the time course of brainactivity and for indexing communicationbetween neural areas. This book showshow such recordings can be useful in ana-lyzing generators of the electrical signals inreal time as is done in chapters on lan-guage (Chapter 6), memory (Chapter 7),executive function (Chapter 8), and atten-tion (Chapters 10, 11, and 12). These chap-ters also discuss event related potentials,while steady state electrical potentials arediscussed in Chapters 4 and 11, and someconcerns with the use of oscillations andcorrelations within particular frequencybands as a means of probing communica-tion between neural areas are discussed inChapters 5 and 8.

The editors have also made a significantattempt to give new readers the back-ground necessary to understand thematerial contained in the volume. Chapter1 deals with general theoretical issues and

Chapter 2 reviews how electrical and mag-netic signals arise from neural tissue andget conducted to the sensors from whichthey are recorded. Appendixes A—F pro-vide a primer of brain recording techniquesas applied to normal persons and thosesuffering from neurological disorders.

The visual system, including visual atten-tion (Chapters 4, 5, 8, 10, 11, and 12), hasbeen the best area for the close integrationof hemodynamic, lesion, and EEG work. Inmy view, the results have been very impres-sive. A few years ago, it was puzzling thatdifferent areas of the parietal and occipitallobes were active during attention tasks.However, by use of event related fMRImethods it now seems clear that the supe-rior parietal lobe is most related to orienting(e.g., voluntary shifts of attention), while thetemporal parietal junction is most importantfor processing novel or unexpected events.Lesions of the TOJ and surrounding areasare also closely related to the neurologicalphenomena of extinction and neglect. Theoccipital sites, which are related to the pro-cessing of target identity, while not a part ofthe attention systems per se, can, like mostbrain areas, be amplified during an atten-tive act. Detailed analysis of the orientingnetwork tends to bring into harmony thestudy of lesions, hemodynamic imaging,and electrical recording.

Cognitive neuroscience involves func-tional anatomy, circuitry, plasticity, andpathology. All these topics are well repre-sented within the volume. Although mostchapters deal with circuitry (i.e., timecourse of processing), chapters on memory,vision, development, and self-regulationprovide substantial backgrounds in howthe brain changes with experience andmaturation. Human brain development isbecoming an increasingly important fieldof research (see Chapter 9 and Posner,Rothbart, Farah and Bruer, 2001). Forexample, new methods are now availablefor examining the development of whitematter pathways in the human brain byuse of diffusion tensor MRI. This could

xvi FOREWORD

open up the prospect of using measures ofthe development of coherence betweendistant electrode sites as a means of prob-ing the earliest functional use of particularwhite matter pathways. In addition toChapter 9, which deals with some forms ofatypical development, a whole section ofthe volume is devoted to applications toneurological patients (Chapter 13) and clin-ical application of mismatch negativity.

This volume sets research with thebrain’s electrical and magnetic signalssquarely within the large and growing toolkit of methods that have opened up theblack box and made the human brainaccessible to detailed investigation. What isthe next step? A goal must be to movebeyond the box score data summariesfound in Chapter 3, to reveal the principlesthrough which brain areas are assigned tofunctions and get assembled into circuits.We are starting to have the requisite cluesto do this for visual attention and somehigh level skills like reading and numeracy.It will be a great challenge, but reading this book and absorbing its many lessonsshould give the researchers of the nextgeneration a good start.

ReferencesAbdullaev, Y. G., and Posner, M. I. (1998). Event-

related brain potential imaging of semantic encoding during processing single words.Neuroimage 7, 1–13.

Dale, A. M., Liu, A. K., Fischi, B. R., Ruckner, R.,Beliveau, J. W., Lewine, J. D., and Halgren, E.(2000). Dynamic statistical parameter mapping:Combining fMRI and MEG for high resolutioncortical activity. Neuron 26, 55–67.

Dehaene, S. (1996). The organization of brain activation in number comparison: Event relatedpotentials and the additive factors method. J. Cog. Neurosci. 8, 47–68.

Posner, M. I., and McCandliss, B. D. (1999). Braincircuitry during reading. In “Converging Methodsfor Understanding Reading and Dyslexia” R. Klein and P. McMullen, eds.), pp. 305–337. MIT Press, Cambridge, MA.

Posner, M. I., Rothbart, M. K., Farah, M., and Bruer, J.(eds) (2001). Human brain development. Dev. Sci.4/3, 253–384.

Raij, T. (1999). Patterns of brain activity during visualimagery of letters. J. Cog. Neurosci. 11(3), 282–299.

Rugg, M. D., and Coles, M. G. H. (eds.) (1995).“Electrophysiology of Mind.” Oxford Univ. Press.

Michael PosnerSackler Institute

University of Oregon

FOREWORD xvii

3 Copyright 2002, Elsevier Science (USA). All rights reserved.The Cognitive Electrophysiology of Mind and Brain

C H A P T E R

1

Cognitive Electrophysiology ofMind and Brain

Alberto Zani and Alice Mado Proverbio

INTRODUCTION

The event-related potentials (ERPs) ofthe brain are wave forms reflecting brainvoltage fluctuations in time. These waveforms consist of a series of positive andnegative voltage deflections relative tosome base line activity prior to the onset ofthe event. Under different conditions,changes may be observed in the morphol-ogy of the wave forms (e.g., the presence orabsence of certain peaks), the latency, dura-tion, or amplitude (size) of one or more ofthe peaks, or their distribution over thescalp. ERPs are useful measures for study-ing mind and brain functions because theyare continuous, multidimensional signals.Specifically, ERPs give a direct estimate ofwhat a significant part of the brain is doingjust before, during, and after an event ofinterest, even if this is prolonged. ERPs canindicate not only that two conditions aredifferent, but also whether, for example,there is a quantitative change in the timingand/or intensity of a process or a qualita-tive change as reflected by a different mor-phology or scalp distribution of the waveforms. For all these reasons, ERPs are wellestablished as powerful tools for studyingphysiological and cognitive functions ofthe brain.

ERPs AND COGNITIVE THEORY

The so-called cognitive revolution(Baars, 1986) that has permeated researchon the mind in psychology and the neuro-sciences has led to widespread recognitionthat cognition and the knowledge thatderives from it, rather than being anaccumulation of sensory experiences, is aconstructive process that requires theverification of hypotheses influenced byprevious knowledge, past experience, andcurrent aims, as well as emotional andmotivational states. Cognitive theory lednot only to the rejection of the mind–braindualism (Mecacci and Zani, 1982; Finger,1994), but also to firm establishment of thenotion that the nature of the mind is deter-mined to a large extent by the neuro-functional architecture of the brain. Animportant corollary of this concept is theidea that in order to understand the mindit is essential to study and understand thebrain (Gazzaniga, 1984, 1995; Posner andDiGirolamo, 2000).

Understanding the mind and brain does not in any way mean understandingconscious processes—quite the contrary,because to a large extent it means investi-gating nonconscious neural processes. Thisfact suggested to researchers of the stature

of Le Doux (1996) that the unconscious isreal, and the renown Gazzaniga (1998)stated that “many experiments highlighthow the brain acts earlier than we realize.”This occurs at different hierarchical levelswithin the complex entity of themind–brain, ranging from intra- and inter-cellular ion exchanges at the microcellularlevel to the flow of information, at themacrosystem level, along the differentfunctional circuits underlying the veryfunction of the brain and the mind. On theother hand, at a macrosystem level, uncon-scious function is manifested throughoutalmost all spheres of the mind, startingfrom the basic operations of analyzingphysical characteristics of stimuli by oursensory system, to recording past events ormaking decisions.

We do not believe that this is surprisingif we consider results of modern researchon the brain; contemporary studies demon-strate the existence of processes of uncon-scious or subliminal knowledge andperception that influence a manifestedbehavior, or the capacity of the brain to“filter” or suppress the processing ofstimuli (this argument is dealt with morefully in Section III of this volume, onprocesses of attention). This capacity tofilter, studied by Freud, who used the term“repression” to describe it, allows us to beconcious of specific thoughts and percep-tions, but not others, apparently under freewill and by choice.

In consideration of the relevance of thissubstantial unconscious component of themind and, indeed above all, the emotions,it can only be concluded that a heuristi-cally valid cognitive theory of the mind isone that considers the mind’s rational andcognitive aspects, which are maintained bythe activity of the neocortex, inseparablefrom the emotive and irrational aspects,expressed by the amygdala and the limbicanterior cingulate cortex (Bush et al., 2000;see also Chapter 8, this volume). Thisconclusion is also supported by the closerelationship existing between thought pro-

cesses and emotional processes, suggestedby authoritative researchers of the brainsuch as Le Doux (1996) and Damasio(1994). According to this logic, the brain isseen as a so-called living system. A system,despite being the sum of various parts,each with its specific function, acts as awhole in which each function inevitablyinfluences the other.

Furthermore, it must be rememberedthat whatever conception of the mind isadopted, it is not heuristically correct toconsider this latter as an immutable entity.In fact, the mind must be considered indynamic terms, that is, as undergoing con-tinuous variations on the basis of evolutiveprocesses and experience (Berlucchi andAglioti, 1997). It is essential to rememberthat the functional processes that distin-guish the mind vary as a function of theontogenetic development of the individual—depending, as a consequence, on thediversified maturation of cerebral structure—and as a function of the individual’slearning processes and specific experiencegained for the stage of developmentreached (Nelson and Luciana, 2001; seealso Chapter 9, this volume).

Cognitive electrophysiology is a verywell-established field of science (Heinze etal., 1994; Kutas and Dale, 1997). The newtechnologies used to pursue the investiga-tion of mind and brain, with the theoreticalbacking of the cognitive sciences, havedeveloped at a dizzying speed over therecent “decade of the brain.” As a researchtool, cognitive electrophysiology mayprovide relevant contributions to both cog-nitive and brain sciences, putting togethernew knowledge about humans as inte-grated sociobiological individuals. Thisambitious task implies an integration ofneurofunctional concepts and basic ormore complex cognitive concepts, such asthose proposed in cognitive sciences(Wilson and Keil, 1999). Unlike most elec-trophysiological research, mired down bydata collection and “correlation state-ments,” to the detriment of theorization,

4 1. COGNITIVE ELECTROPHYSIOLOGY

I. A COGNITIVE FRAMEWORK

ERPs AND COGNITIVE THEORY 5


The relationships between these “tools”and cognitive processing are deduced bymeans of an “assumed” criterion thatlocates these physiological responses inaccordance with hypothesized constraintsabout their position and function withinongoing activity. These constraints aremediated by well-defined theories ofhuman cognition and information process-ing (Donchin, 1982, 1984a).

In seeking to clarify these proceduralsteps, let us take a concept such as learn-ing, viewed from the psychological orbehavioral level, and let us try to showhow this concept may fruitfully drive elec-trophysiological experiments. Both at thelevels of cognition and brain neurofunctionthree different, major principles of learninghave been coherently identified: (1) know-ing what is out in the world, to be used inlater recognition and recall (2) knowingwhat goes with, or follows, what, and (3) knowing how to respond or what to do,given the drive and the situation. Theinterweaving of these three kinds ofknowledge is manifested as complex vol-untary action and skilled performance. Inmany respects these principles underliethe acquisition and deployment of proce-dures that manipulate the knowledgestructures (Bransford et al., 1999). A veryrelevant topic relative to these proceduresis the distinction between so-called con-trolled and automatic procedures. Skilllearning is thought to be characterized bya slow transition from dominance by con-trolled processes to dominance by auto-matic processes. However, this transitionhas been shown to take place only fortasks for which consistent—i.e., repetitiveand predictable—information is available.Taking this theoretical framework as astarting point for psychophysiologicalresearch on learning, it may be predictedthat any spatiotemporal changes in ERPcomponents (amplitude and latency) thatmay occur with learning should only beobserved in tasks providing such consis-tent information (Kramer and Strayer,

the main assumption of cognitively ori-ented electrophysiological research is thatcognition is implemented in the brainthrough physiological changes. An implicitcorollary of this assumption is that electro-physiological measures, i.e., ERP compo-nents, may be taken as manifestations, andnot simply as correlates, of these interven-ing processes of the flow of informationprocessing (McCarthy and Donchin, 1979).

Indeed, arguments may be, and indeedoften are, raised against this theoreticalview in the name of “physiological objec-tivity.” However, we are aware that thesestatements arise from questionable adher-ence, in many cases without any aware-ness, to operationa1 meaning theory. Theprocedure of giving meaning to conceptsinductively on the basis of measures pro-vides an outmoded brand of operationismthat may have functioned well for theor-etically developed sciences, such as phy-sics, proceeding in the framework of thePopperian view of scientific progress, but which has been only detrimental toatheoretical electrophysiological research.Indeed, the difficulties often met indefining any intrinsic and immutable prop-erty of a physiological response, changingas a function of the conditions of its occur-rence, make the latter loosely defined inconceptual terms. This failure to find aspecific response definition is a problem-atic criterion for delineating a psychologi-cal process for the correlational approach.

To cope with the spatiotemporal overlapin scalp-recorded manifestations of under-lying cerebral processes, and with theproblems in determining their physiologi-cal generators, cognitive electrophysiolo-gists identify ERP components, i.e., thecerebral responses, as the portions of arecorded wave form that can be inde-pendently changed by experimental variables—task condition, state, subjectstrategy, etc. ERP components are notviewed as “structural markers” per se, butas “psychological tools,” as is any otherpsychological measure, e.g., reaction times.

1988; Sirevaag et al., 1989). In the pastdecades, evidence strongly supporting thisprediction has been accumulated in ERPliterature relative to all the best knowncomponents, especially the contigent nega-tive variation (CNV) and the so-called latepositive complex (LPC), i.e., N2, P300, andslow wave (see, Proulx and Picton, 1980;Kramer et al., 1986). Thanks to its inconsist-ent features, the “oddball” task is a simplebut flexible experimental task that hashelped to provide evidence, either as suchor in the context of a probe-based dual-taskparadigm, for the limited capacity of con-trolled processes and the spatiotemporalstability of ERP components.

ERPs AND THE BRAIN

Traditionally, for more than 100 yearscognitive and neurophysiological pro-cesses in humans have been studied bypsychophysical and behavioral methods.Modern neurosciences offer several hemo-dynamic, anatomofunctional, and elec-trophysiological methods to furtherinvestigations of the mind and brain.Nevertheless, only noninvasive whole-system procedures can be used to examinehumans (see Appendix A, this volume, fora synopsis of molecular and systemicresearch methods). Because neurophysio-logical processing takes place in fractionsof a second, one of the most feasible toolsis to record brain electrofunctional activity(see, e.g., Heinze et al., 1994; Rugg andColes, 1995). The advantages of electro-physiological signals, or ERPs, lie in theirvery high time resolution—in the order ofmilliseconds—and their reliable sensitivityin detecting functional changes of brainactivity. The high temporal resolution andnoninvasiveness of this method privilegeits use over brain imaging techniques suchas computed tomography (CT), positronemission tomography (PET), or functionalmagnetic resonance imaging (fMRI), aswell as over the behavioral measures most

used in traditional neuropsychologicalstudies. Thanks to these advantages, event-related brain potentials may reveal steps insensory–cognitive information processingoccurring very rapidly within the brain.Furthermore, unlike behavioral and neu-roimaging techniques, ERPs may revealdetails of functional organization, andtiming of the activation, of regional areasof anatomically distributed functionalsystems of the brain involved in cognitiveskills as well as in executive capacities.

Volume conduction and lack of three-dimensional reality do, however, mean thatthese brain signals are of more limited usethan neuroimaging techniques for examin-ing where in the brain processes take place.Nevertheless, localization processes carriedout using these signals may be made more sound through source-modelingalgorithms.

There is no doubt that modern neuro-imaging techniques have dramaticallyincreased our knowledge of the brain andthe mind (Posner and Raichle, 1994; Rugg,1998; Cabeza and Kingstone, 2001). Aswith ERPs, studies carried out with thesetechniques focus on an individual’s brainwhen it is involved in carrying out a partic-ular mental task: memorizing a list ofwords, distinguishing some objects fromothers that are similar but not the same,directing attention toward objects pre-sented in a particular part of the visualfield, etc. The theory underlying all thesestudies is that the areas of the brain that arefound to be most active during the tasksare those that are crucial for the varioustypes of mental activity.

However, simple mapping of the sites ofmental processes can indicate only wherein the brain a given functional activationtakes place but, at present, can in no wayexplain the mechanisms of the mind. Howdo we recognize objects and faces, how dowe recall the memory of experiences andthings, how do we direct our attention to objects and the surrounding space, etc.? These complex and extraordinary



mental mechanisms still remain unchartedterritory.

No spatial or temporal resolution,however good, can localize something ofwhich we have only superficial know-ledge. In fact, in order to be able to “local-ize” a given cognitive state or mentalprocess (for example, “remembering some-thing”) in the brain we must know clearlywhat the state or process is and what thefunctional subprocesses are that invariablylead to one cognitive state and not another.If we do not know what these subprocessesare, or whether they vary in different con-ditions, we cannot reliably localize them inthe brain.

It is not at all difficult to find examplesin the literature to illustrate what we mean.The reader is referred to Chapter 7 in thisvolume for an impressive review showinghow different the cerebral localizations ofactivity can be during episodic mnemonicanalysis of figurative and linguistic infor-mation, according to the different type/state of analysis carried out by subjects(e.g., familiarity, encoding effort, recog-nition). Furthermore, there is no lack ofexamples of different localizations in dif-ferent studies by different scientists for asimilar form of mental activity, such asspatial attention. For example, Mangunand colleagues repeatedly reported activa-tion of the fusiform gyrus with PETimaging, and of this same gyrus togetherwith the medial occipital gyrus, whenimaged with fMRI, during attention to arelevant space location (see the exhaustivereview by Mangun in Chapter 10), whereasCorbetta and colleagues (see Corbetta,1998) reported a localized activity in theparietal lobe, a region of the cortex classi-cally associated with the control of spatialattention, in addition to the more dorsolat-eral occipital regions. It is not inconceiv-able that to cope with differences in thespatial tasks across these studies, differentcognitive processes, and thus, differentregions of the volunteers’ brains, musthave been activated during what was

reported by the authors as apparently thesame mental activity.

The difficulty in differentiating cog-nition from brain localization is not, however, unique to neuroimaging andelectrophysiological studies. Unfortun-ately, it is also difficult in most traditionalclinical neuropsychological research. Con-sider, for instance, research on hemineglector cortical blindness, or any other clinicalsyndrome. Although robust, direct post-mortem and neuroimaging evidence isavailable for the anatomical localization ofbrain lesions from which these syndromesderive, only controversial theories can beadvanced to explain which processes arelacking, compared to normal cognition, inthese patients’ cognitive processing andthus to explain their symptomatology.Examples of opposing theories can befound in Köhler and Moscovitch’s (1997)outstanding review on unconscious visualprocessing.

To complicate the picture further, local-ization research is often pushed to anextreme, frequently without being soundlybased on the theory of the mind or thefunctional architecture of the brain. Thereare now many authoritative investigatorsspeaking out against this approach toresearch, and it will probably emerge asmore of a hindrance than a help for under-standing the mind and brain. For example,according to Frith and Friston (1997), mostneuroimaging studies concentrate exclu-sively on subtraction techniques and onfunctional segregation to associate a givenarea with a given function. However,according to Frith and Friston, in order tobuild an accurate map of the mind, it iscrucial to understand the functional inter-connectivity of the centers and pathwaysof the brain by investigating the correla-tions between these different anatomo-functional entities.

This problem is felt, shared, and cre-atively developed in the excellent reviewby Cabeza and Nyberg in Chapter 3. Notby chance did they give their chapter the

ERPs AND THE BRAIN 7


title “Seeing the Forest through the Trees:The Cross-Function Approach to ImagingCognition”; they identify “the trees” as thesingle cognitive functions on which manyimaging studies focus their concern, withloss of sight the whole—“the forest”––rep-resented by the fact that, on the one hand,many brain areas are involved in manycognitive functions, and, on the other, thatcognition is not actually subdivided intodistinct modular cognitive processes, asartificially proposed in cognitive sciencetextbooks for explanatory purposes.

Fuster (2000) is of the same opinion, andauthoritatively reports that “commonsense, psychophysics, and experimentalpsychology provide ample evidence thatall cognitive functions are interdependent.…Also interdependent must be, of course,their neural foundations.” And, cautioningthe reader about some of the problemswith the neuromodular principle of cog-nition, Fuster advances the concept of a“distributed cortical network” according towhich performance in cognitive tasks, or,more specifically, tasks of executive con-trol functions, is not solely mediated vialocalized areas of the brain, but by manyregional brain areas that are dispersedthroughout the brain, although beingstrictly linked to each other, and activatedin a divergent and convergent way at dif-ferent times. Again in Fuster’s (2000)words, “practically any cortical neuron orneuronal assembly, or module, can be partof many networks. A network can serveseveral cognitive functions, which consistsof neuronal interactions within andbetween cortical networks.” The closeresemblance between this carefully wordedand articulate definition and the nowadaysforgotten “functional system” theory ofbrain neurofunctional architecture, firstadvanced by the great Russian neuro-psychologist Alexander Lurija (Lurija,1962, 1976), will, we believe, have hardlyescaped anyone.

In the light of these considerations, webelieve that it is correct to think that the

moment has returned for researchers todedicate more of their forces to studyingthe mechanisms inherent to human cogni-tion in order to reach a fuller understand-ing not merely of the brain, of which in abroad sense we know quite a lot, but ratherthe mind, that is, its higher and morearcane product, of which we are still pro-foundly ignorant.

For decades, aware of the limited capac-ity of ERPs to localize intracerebralprocesses of cognition, cognitive electro-physiologists have continued their researchin the firm belief that the brain’s electro-magnetic signals spread over the scalpduring electrofunctional activation are pre-cious for understanding the ways withwhich the brain changes with experienceand knowledge. Furthermore, they haveshared the belief that the nature and mech-anisms of the neural processes of cognitiveand emotional reorganization are objec-tively and reliably codified by the differentcomponents of the ERPs and event-relatedfields (ERFs) (Donchin, 1979, 1984b;Hillyard and Picton, 1979; Zani, 1988;Hillyard, 1993; Näätänen and Ilmoniemi,1994; Rugg and Coles, 1995; Kutas andDale, 1997).

It was in this conceptual “framework”that the idea was advanced that ERPscould make an important contribution toour understanding of the cerebral mecha-nisms of knowledge (Kutas and Hillyard,1984; Heinze et al., 1994). And it is follow-ing this idea that the ERPs, rather thanbeing considered a now obsolete methodin comparison with the currently availabletechniques, are still used as a direct,quantifiable measure of processes ofknowledge, both conscious and uncon-scious, and as such are still used toproduce and validate models of the mindrather than to provide generic “correlates”of poorly defined psychological constructs.This will become extremely clear in the prestigious articles written byrenowned researchers collected together inthis book.



In conclusion, it seems that using ERPs,in combination with other available tech-niques, as quantifiable measures of cog-nitive and affective processes of the brain,the cognitive electrophysiologist can helptest existing theories on the human mindand also can propose newer and moreheuristic ones. In order to be efficient inthis task of identifying mental processesarising from the brain, it is essential towork in the context of well-founded theor-ies and with sophisticated methodologycapable of distinguishing between thesetheories.

THIS BOOK—OVERVIEW

The chapters of this book—prepared bya panel of international neuroscientists and electrophysiologists—provide state-of-the-art reviews of the latest develop-ments in the study of the relationshipsbetween mind and brain as investigated byevent-related potentials and event-relatedfields. Some indications are explored ofhow these signals may be combined withthe high spatial resolution of the hemo-dynamic signals of the brain, such as thoseacquired through positron emission tomo-graphy and functional magnetic resonanceimaging, in order to come closer to the goalof localizing cognition within the brain.

The book is systematically organizedinto thematic sections. The three chaptersin the first section cover the theoretical andmethodological framework of investigat-ing the human mind through the recordingof electrical, magnetic, and hemodynamicsignals of the brain. In Chapter 1 we haveraised the point that the study of cognitioncan benefit enormously from the use of brain electrical and magnetic activity.Efforts are being made to demonstrate thatthese benefits will derive mostly fromtheoretically oriented electrophysiologicalresearch in the framework of cognitivesciences and neurosciences. Chapter 2focuses on the morphology of visual, audi-

tory, and somatosensory wave forms ofelectric potentials and magnetic fields ofthe brain, and also on the functionalsignificance of these electrophysiologicalindices in relation to the basic and higherdomains of cognition. Furthermore, theintracranial electro ionic origins of thesescalp-recorded physiological measures aredescribed, with indications for solving the“direct” and “inverse” problems of localiz-ing their electromagnetic dipoles withinthe brain.

Chapter 3 (Cabeza and Nyberg) offersan original theoretical cross-function frame-work for guiding hemodynamic functionalimaging of brain and cognition. Thisframework provides the foremost con-straints to functional interpretations, par-ticularly when assuming the so-calledsharing view, that is, the view that thesame brain region is recruited by differentcognitive functions. In the authors’ words,these constraints “help us overcome func-tion-chauvinism and see the ‘big picture.’In other words, cross-function compar-isons allow us to see the forest [what manyfunctional studies have in common]through the trees [the single cognitivedomains investigated by single studies].”

The second section (Chapters 4–9) sys-tematically covers electromagnetic researchon a representative sample of the neuraldomains of human cognition. Chapter 4(Skrandies) illustrates how the recordingof brain electrical activity in combinationwith knowledge on the human visualsystem may be employed to study visualinformation processing in healthy volun-teers as well as in patients with selectivevisual deficiencies. Data are presented ondifferent experimental questions related tohuman visual perception, including con-trast and stereoscopic vision as well asperceptual learning.

Chapter 5 (Aine and Stephen) dealswith magnetoencephalography (MEG)mapping of the ventral and dorsal streamsin human visual cortex. MEG cues, prov-ing that isoluminant, central field stimuli

THIS BOOK—OVERVIEW 9


preferentially excite the ventral streamstructures and, alternatively, that periph-eral stimuli alternating at high rates prefer-entially activate the dorsal stream, aresystematically addressed. Furthermore, thefocus is on present progress in our under-standing of brain cortical areas involved inhigher visual processing, such as recogni-tion memory, as investigated by means ofMEG, and the future direction of MEGresearch in this field is discussed.

Chapter 6 (Federmeier, Kluender, andKutas) reviews ERP studies on language.Rather than simply presenting a collectionof various processes, throughout the chap-ter the authors illustrate the viewpoint thatthe goals of electrophysiological investiga-tions of language, as well as the goals ofresearch exploring language processingwith other tools, are to fashion an under-standing of how the various processesinvolved in language comprehension andproduction are coordinated to yield themessage-level apprehension we attain fromreading or listening to speech. As stated inChapter 6, “linguists, psycholinguists, andneurolinguists alike strive to understandhow the brain ‘sees’ language––because, inturn, language is such an important facetof how humans ‘see’ their world.”

Chapter 7 (Wilding and Sharpe) isdevoted to memory processes that con-stitute another very important domain ofhuman cognition. Indeed, memory hasbeen a subject of fascination to psycholo-gists and other brain scientists for over acentury. Recently, the study of the role ofdifferent brain areas in memory hasreceived a boost from new techniques andchanging pretheoretical orientations. Theauthors offer an original review of the bulkof electrophysiological studies on retrievaland encoding processes underlying epi-sodic memory. Commendably, they do notsimply share knowledge from ongoingresearch, but identify some acute out-standing problems in this field of inves-tigation, indicating its likely futuredevelopments.

Chapter 8 (Luu and Tucker) is intendedto close the misleading gap that exists inthe “cognitive” approach to brain functionand architecture between a pure cognitivefunctional processing of the brain and itsemotional counterpart, which is of suchimportance in producing thinking andbehavior, both in normal and emotionallydisordered people. With such a goal, theauthors deal specifically with mentalprocesses involved in emotion, motivation,and reward, reviewing studies in this field from a modern neuroscience-basedviewpoint.

Chapter 9 (Mitchell and Neville)addresses “neuroplasticity,” a dominantresearch theme in neuroscience at present.Neuroplasticity usually refers to somechange in the nervous system as a functionof age and/or experience. This contribu-tion reviews studies on the effects of ageand experience on the development of neu-rocognitive systems. A broad survey andsynthesis are provided of essential data onnormal brain and cognitive development,as well as on development after early deaf-ness, blindness, or following delays in lan-guage acquisition. The authors provideinsights into and ideas on the complexityand diversity of contemporary brain neu-roplasticity research in humans.

The three chapters gathered in the thirdsection are concerned with visual attention.Chapter 10 (Mangun) and Chapter 11 (DiRusso, Teder-Sälejärvi, and Hillyard)mostly address neural mechanisms ofspatial attention. Chapter 10 reviewsfindings indicating how human spatialattention involves top-down processes thatinfluence the gain of sensory transmissionearly in the visual cortex. Chapter 11 dealswith steady-state cortical processing of thebrain that reveals slow-rising changes incortical reactivity to the outer world. First,the authors provide an overview of thisprocessing mode in the visual modality.Then they present a review of experimentalfindings of modulations of this processingmode with selective attending of spatial



and, to a lesser extent, nonspatial features(color, shape, etc.) of visual information, inline with the previous chapter. UnlikeChapters 10 and 11, Chapter 12 (Proverbioand Zani) concentrates on feature-based andobject-based selection mechanisms of thebrain as investigated with ERPs. An over-view is provided of studies showing theclose interconnections across the anteriorand posterior attention systems. In addition,a review is made of studies reporting the dif-ferential activation of the “Where” and“What” systems of the visual brain in con-ditions in which stimulus attributes have tobe separately and/or conjointly attended.Efforts are made to demonstrate the task-related relative segregation and complexinteractions of the aforementioned systemsduring the separate or conjoint processing ofstimulus attributes.

The final two chapters comprising thefourth section are concerned with clinicaland applied perspectives of ERP research.Chapter 13 (Verleger) provides an exhaus-tive overview of ERP studies on neuro-psychological syndromes. The detaileddescription given of these syndromes is sub-divided into three main categories. Theresult is a unique, up-to-date, and wide-ranging discussion of these disorders thatdraws on biology, genetics, neuropsychol-ogy, clinical presentation, and treatment.Chapter 14 (Näätänen, Brattico, and Tervan-iemi) introduces the mismatch negativity(MMN), a component of auditory ERPsreflecting the brain’s automatic response toany discriminable change in auditory stimu-lation. Because the MMN can be measuredeven in the absence of attention and withoutany task requirements, it is particularly suit-able for investigating several clinical popula-tions as well as infants. Moreover, the MMNprovides a unique index of the subject’saccuracy in the processing of speech andmusical sounds. It can be used, for example,to unravel the neural determinants of lan-guage skills and musical expertise.

In addition to the specialist review chap-ters, a fifth section of this book collects

together a number of appendixes contain-ing the primers of the theoretical andmethodological matters––including somesimple-level mathematical material—treated in the specialist chapters. Theseappendixes are intended for the benefit ofnonexperts (such as psychology andmedical students), as well as experts inother neighboring fields. These appendixeshave been included in order to clarify, insimple but detailed terms, the basics ofmolecular and systemic methods of inves-tigating the nervous system (Appendix A),as well as neuropsychological clinical prac-tice (Appendix B). They also provide thefundamentals of electromagnetic recordingand data analysis and laboratory setup(Appendixes C and D), and topographicand dipole mapping methods (Appendix E).Last but not least, the invasiveness and thespatial and temporal resolution of electro-magnetic techniques, as compared to othertechniques, are given (Appendix F).

ReferencesBaars, B. J. (1986). “The Cognitive Revolution in

Psychology.” Guilford, New York.Berlucchi, G., and Aglioti, S. (1997. The body in the

brain: Neural bases of corporeal awareness. TrendsNeurosci. 20, 560–564.

Bransford, J. D., Brown, A. L., and Cocking, R. R.(1999). “How People Learn. Brain, Mind,Experience, and School.” National Academy ofSciences, New York.

Bush, G., Luu, P., and Posner, M. I. (2000). Cognitiveand emotional influences in anterior cingulatecortex. Trends Cogn. Sci. 4, 215–222.

Cabeza, R., and Kingstone, A. (2001). “Handbook ofFunctional Neuroimaging of Cognition.” MITPress, Cambridge, Massachusetts.

Corbetta, M. (1998). Functional anatomy of visualattention in the human brain: Studies withpositron emission tomography. In “The AttentiveBrain” (R. Parasuraman, ed.), pp. 95–122. MITPress, Cambridge, Massachusetts.

Damasio, A. R. (1994). “Descartes’ Error: Emotion,Reason, and the Human Brain.” Avon Books, NewYork.

Donchin, E. (1979). Event-related brain potentials: Atool in the study of human information process-ing. In “Evoked Brain Potentials and Behavior”(H. Begleiter, ed.), pp. 13–88. Plenum Press, NewYork and London.

REFERENCES 11


Donchin, E. (1982). The relevance of dissociations andthe irrelevance of dissociationism: A reply toScwartz and Pritchard. Psychophysiology 19, 457–463.

Donchin, E. (1984a). Dissociation between electro-physiology and behavior––A disaster or a challenge? In “Cognitive Psychophysiology.Event-related Potentials and the Study ofCognition” (E. Donchin, ed.), pp. 107–118.Lawrence Erlbaum Assoc., Hillsdale, New Jersey.

Donchin, E. (1984b). Cognitive psychophysiology. In“Event-related Potentials and the Study ofCognition” (E. Donchin, ed.), pp. 107–118.Lawrence Erlbaum Assoc. Hillsdale, New Jersey.

Finger, S. (1994). “Origins of Neuroscience.” OxfordUniversity Press, New York.

Frith, C. D., and Friston, K. J. (1997). Studying brainfunction with neuroimaging. In “CognitiveNeuroscience” (M. Rugg, ed.), pp. 169–195.Psychology Press, London.

Fuster, J. M. (2000). The module: Crisis of a paradigm.Neuron 26, 51–53.

Gazzaniga, M. S. (ed.) (1984). “Handbook of CognitiveNeuroscience.” Plenum Press, Cambridge.

Gazzaniga, M. S. (ed.) (1995). “The Cognitive Neuro-sciences.” MIT Press, Cambridge, Massachusetts.

Gazzaniga, M. S. (1998). “The Mind’s Past.”University of California Press, Berkeley and LosAngeles.

Heinze, H.-J., Münte, T. F., and Mangun, G. R. (1994).“Cognitive Electrophysiology.” Birkhäuser, Boston,Basel, and Berlin.

Hillyard, S. A. (1993). Electrical and magnetic brainrecordings: Contributions to cognitive neuro-science. Curr. Opin. Neurobiol. 3, 217–224.

Hillyard, S. A., and Picton, T. W. (1987). Electro-physiology of cognition. In “Handbook ofPhysiology: Section 1, The Nervous System: HigherBrain Functions” (V. B. Mountcastle, ed.), Vol. 5, Part2, pp. 519–584. American Physiological Society,Baltimore.

Köhler, S., and Moscovitch, M. (1997). Unconsciousvisual processing in neuropsychological syn-dromes: A survey of the literature and evaluationof models of consciousness. In “CognitiveNeuroscience” (M. Rugg, ed.), pp. 305–373.Psychology Press, London.

Kramer, A. F., and Strayer, D. L. (1988). Assessing thedevelopment of automatic processing: An applica-tion of dual-task and event-related brain potentialmethodologies. Biol. Psychol. 26, 231–267.

Kramer, A. F., Schneider, W., Fisk, A., and Donchin, E.(1986). The effects of practice and task structure oncomponents of event-related brain potentials.Psychophysiology 23, 33–47.

Kutas, M., and Dale, A. (1997). Electrical and magneticreading of mental functions. In “Cognitive Neuro-science” (M. D. Rugg, ed.), pp. 197–242. Psy-chology Press, Taylor & Francis Group, Hove, EastSussex, UK.

Kutas, M., and Hillyard, S. A. (1984). Event-relatedpotentials in cognitive science. In “Handbook ofCognitive Neuroscience” (M. S. Gazzaniga, ed.),pp. 387–409. Plenum Press, New York.

Le Doux, J. E. (1996). “The Emotional Brain.” Simonand Schuster, New York.

Lurija, A. R. (1962). Vyssie korkovye funkcii celoveka[The Superior Cortical Functions in Man]. MoscowUniversity (MGU), Moscow.

Lurija, A. R. (1976). “The Working Brain. An Introduc-tion to Neuropsychology.” Penguin Books,Harmondworth.

McCarthy, G., and Donchin, E. (1979). Event-relatedpotentials––manifestations of cognitive activity. In“Bayer Symposium VII, Brain Function in OldAge” (F. Hoffmeister and C. Muller, eds.), pp. 318–335. Springer-Verlag, New York.

Mecacci, L., and Zani, A. (1982).”Theories of the Brain.Since the Ninth Century Up to Today” [in Italian].Loescher, Torino.

Näätänen, R., and Ilmoniemi, R. J. (1994). Magneto-encephalography in studies of human cognitivebrain function. Trends Neurosci. 17, 389–395.

Nelson, C. A., and Luciana, M. (eds.) (2001).“Handbook of Developmental Cognitive Neuro-science.” MIT Press, Cambridge, Massachusetts.

Posner, M. I., and DiGirolamo, G. J. (2000). Cognitiveneuroscience: Origins and promise. Psychol. Bull.126, 873–889.

Posner, M. I., and Raichle, M. E. (1994). “Images ofMind.” W. H. Freeman, New York.

Proulx, G. B., and Picton, T. W. (1980). The CNVduring cognitive learning and extinction. In“Motivation, Motor and Sensory Processes of theBrain: Electrical Potentials, Behaviour and ClinicalUse” (H. H. Kornhuber, and L. Deecke, eds),pp. 309–313. Elsevier, Amsterdam.

Rugg, M. D. (1998). Functional neuroimaging in cog-nitive neuroscience. In “The Neurocognition ofLanguage” (C. M. Brown and P. Hagoort, eds.), pp. 15–36. Oxford University Press, Oxford.

Rugg, M. D., and Coles, M. G. H. (1995). The ERP andcognitive psychology: Conceptual issues. In “Elec-trophysiology of Mind: Event-related Brain Poten-tials and Cognition” (M. D. Rugg and M. G. H.Coles, eds.), pp.27–39. Oxford University Press,Oxford.

Sirevaag, E. J., Kramer, A. F., Coles, M. G. H., andDonchin, E. (1989). Resource reciprocity: An event-related brain potential analysis. Acta Psychol. 70,77–97.

Wilson, R. A., and Keil, F. C. (1999). “The MITEncyclopaedia of the Cognitive Sciences.” MITPress, Cambridge, Massachusetts.

Zani, A. (1988). Event-related brain potentials as apoint of entry into the integrated analysis of thecognitive and affective bases of perceptual andaesthetic experiences. In “X International Collo-quioum on Empirical Aesthetics,” pp. 90–100.Corda Fratres, Barcelona (ME).




C H A P T E R

2

Electromagnetic Manifestationsof Mind and Brain

Alice Mado Proverbio and Alberto Zani

ELECTROENCEPHALOGRAM AND MAGNETOENCEPHALOGRAM

The cognitive activity that accompaniesfunctional activation of the brain is reflectedin a series of physiological transformations;these can be recorded by a variety of tech-niques of varying invasiveness, ranging fromsingle-unit recordings to hemodynamic andelectromagnetic techniques (Hugdhal, 1993).

From an electrofunctional point of view,the activity of the brain essentially trans-lates into (1) wave-formed electromagneticfields or potentials, which constitute theelectroencephalogram and the magnetoen-cephalogram and (2) transient changes inthe electromagnetic fields caused by nerveimpulses induced by external stimuli orindependent mental events, which consti-tute the event-related potentials (ERPs) andevent-related fields (ERFs), respectively. It isnot the aim of this review to discuss theelectroencephalogram (EEG). The readerinterested in the EEG and the rhythmicalwaves that distinguish it, as well as itsorigins in the cerebral cortex or inprocesses regulating these waves (pace-maker) in the thalamic nuclei, is referred tothe excellent works by Buzsaki (1991) andSilberstein (1995a,b), as well as the impres-sive review by Nunez et al. (2001).

In this chapter we discuss the principlesunderlying the nature of the electromag-netic signal, including a description of theintracortical sources of the potentialsrecorded from scalp surface, as well as themethods of analyzing and identifying thebest known components of these signalsbased on their functional properties. Forfurther information on neurofunctionalchanges other than the strictly electrophys-iological ones (namely, ERPs), and on thetechniques by which these can be recorded,the reader is referred to Chapters 5 and 14,for a description of magnetoencephalo-graphic studies on the mechanisms of ana-lyzing visual and auditory information,respectively. Chapter 3, on the other hand,carefully examines the cross-functionalapproach to hemodynamic neuroimagingof cognition.

Electroionic Origins of Electromagnetic Signals

The changes in electrical potentialrecorded on the scalp are generated by thesum of the excitatory and inhibitory post-synaptic potentials (EPSPs and IPSPs) ofthe nerve cells, which must be oriented in acertain way. The EEG technique canmeasure only the potentials of cellsarranged in organized layers and whose

apical dendrites are oriented perpendicu-larly to the surface of the cortex (as are, forexample, pyramidal cells or the hyper-columns of the visual cortex). The bioelec-tric potential that can be recorded at thesurface is nothing other than the differencein potential between the basal part and theapical part of the active neurons that areoriented in that direction, producing aninfinitely low-intensity flow of current. Inthe example shown in Fig. 1a, the EPSPsthat converge on the pyramidal neuronsthrough the direct afferent fibers that endin the upper part of the apical dendritescause a flow of charged ions betweenpoints at different potentials within andoutside the neurons. In other words, posi-tive ions entering the cell produce a trans-membrane electrical current (as shown in

Fig. 1b). Once the positive ions haveentered the cell, following the concentra-tion and electrical charge gradient, theypropagate from the subsynaptic area to therest of the neuron (Fig. 1c). When the EPSPhas involved the distal part of the apicaldendrite, as shown in Fig. 1, the flow ofcurrent is greater starting from the apicalpart nearest to the synapse toward the cellbody, rather than in the opposite direction,because the resistance to this flow is less.The flow of positive ions entering the cellprogressively neutralizes the negative ionsinside the extracellular somatodendriticmembrane (Fig. 1d). Outside, on the otherhand, the positive ions that have enteredthe cell are substituted by ion flow directedtoward the synaptic region along the extra-cellular space (cases c and d in Fig. 1).

14 2. ELECTROMAGNETIC MANIFESTATIONS OF MIND


FIGURE 1 Electrofunctional activity of a pyramidal neuron: the relative transmembrane ion flows and electro-magnetic induction phenomena. (a–e) Subsequent stages of electroionic alterations between the cell body and theapical dendrites of the neuron induced by postsynaptic excitatory potentials and the generation of reverberatingflows of current, which generate the fields of potential or equivalent electromagnetic dipoles. The large vertical,cylindrical arrow pointing downward indicates the electromagnetic dipole (Ed) with its relative sink and source.

These flows particularly involve the ionsreleased by the neutralization of the chargeof negative ions arranged along the inter-nal surface of cell membrane. The flow ofthe current perpendicular (or radial) to theapical dendrite is accompanied by a mag-netic field that propagates orthogonally (ortangentially) to the flow of current alongthe extracellular somatodendritic mem-brane (Fig. 1e). This set of electroionicfunctional alterations thus generates theso-called fields of electromagnetic potentials orelectromagnetic dipoles. Alternatively, theyare defined as single equivalent dipoles. Incorrespondence with the intracortical areawhere the excitatory (or generator) post-synaptic potential is generated, a negativepotential (the so-called sink, or minimum)is recorded, whereas in correspondencewith the “outflow” of the current on thescalp a positive potential (the so-calledsource, or maximum) is recorded (Fig. 1).The outflow indicates the dispersion of theflow of intracerebral current on the surfaceof the scalp after it has crossed the cere-brospinal fluid, the meninges, and thebony structures of the skull (Fig. 2). In thecase of inhibitory postsynaptic potential,the relationship between the site of thesynapses and the polarity of the recordingis inverted.

As can be seen in Fig. 2, electrofunc-tional activation of the pyramidal cells ori-ented perpendicularly to the surface of theskull—that is, those located in the dorsalgyri of the encephalus—can be recorded onthe scalp in the form of an EEG or ERPs,because it generates so-called open fields. Incontrast, because of the morphology andthe directional arrangement of the den-drites of the cell assembly units (some neu-ronal assembly units in the brain stem andtelencephalon, e.g., the hippocampus, areorganized concentrically), some intracere-bral potentials finish by canceling eachother or by being too weak to be recordedby electrodes far away from them. This isthe reason that they are called closed fields(Fig. 2).

For further details on the neuronal elec-troionic bases of cerebral electromagneticfields the reader is referred to theinfluential articles by Katznelson (1981)and Pilgreen (1995), and the outstandingand more recent surveys by Kutas andDale (1997) and Kutas et al. (1999).

The Electromagnetic Dipoles

Given that the electroencephalogram(EEG) is generated by electrical activitythat derives from neuronal interactions,the distribution of the EEG signals on thescalp contains information on the localiza-tion of the underlying electrical sources. Itmust not, however, be thought that theelectrical source is more or less exactlyunderneath the electrode that records thestrongest intensity signal. Even in the casesin which this is in fact the situation, noinformation is given on the depth of thesource within the skull. In many cases,however, the electrode is not recordingelectrical sources from directly below itslocation, because the electrical sources ofthe brain generate a dipolar distribution ofpotentials, thus the minimum (sink) andthe maximum (source) of the distributionof the potential do not necessarily coincidewith the localization of the source. As afurther complication, in the cases in whichmore than one source is activated (multi-ple equivalent dipoles), the maximum ofone source can be cancelled by theminimum of another. A similar logic alsoapplies in the case of magnetoencephalo-graphy (MEG), which measures the veryweak magnetic fields produced by neu-ronal electrical activity of the brain. Thesesignals are in the order of femtoteslas (10–15 T) (Peters and De Munck, 1990;Scherg, 1992). In order to be able to appre-ciate just how weak these signals are, con-sider that 1 T, which measures the strengthof magnetic induction, is equivalent (ingauss) to about 0.6 G, and that the Earth’smagnetic field is in the order of 0.3–0.6 G(depending on the latitude). On the basis

ELECTROENCEPHALOGRAM AND MAGNETOENCEPHALOGRAM 15




FIGURE 2 Simplified diagram of the cortical generators of electromagnetic fields and both intracerebral andsurface recordings of the signals derived from them. Top: Enlarged detail of a section below the scalp, skull, andmeninges, depicting a cerebral cortical convolution with one of its characteristic sulci formed by the folding of thecortical mantle. The section also shows some pyramidal neurons with their apical dendritic prolongations, ori-ented perpendicularly or horizontally, depending on their position in the folds of the cortex. These neuronsproduce open fields. The thick, dark arrows represent the direction of propagation of the electrical counterpart ofthe dipole; the smaller arrows indicate the direction of propagation of its magnetic counterpart. In the superficialareas of the cortex, the propagation of the electrical dipoles of the pyramidal neurons is radial to the brain andhead. An electroencephalogram (EEG) electrode (shown on the right of the scalp) is therefore able to pick up thispropagated current and reflect it in the form of an electroencephalogram. Only part of the electrical activity of the neurons arranged perpendicularly deep in the bottom of the sulci is picked up by the electrode. Thus, thispart of the cerebral electrofunctional activation is not appropriately represented in the EEG. The same electrode

of this rough equation, the magnetic fieldsof the brain are about one billionth lessstrong than the Earth’s magnetic field.

The Direct and Inverse Problem

We have so far seen that the neuronsthat transmit signals to the synapses act aselectromagnetic dipoles that constituteintracortical generators of the EEG and theMEG. The reason why these sources arerepresented by dipoles is that, in an acti-vated synapse, electrical current flowsfrom a localized source to a nearbyminimum. In mathematical terms, a dipoleis the simplest description of this geomet-ric flow. It can be described by a vector,indicated by an arrow that points frommaximum to minimum. The length of thearrow indicates the intensity (or strength)of the source.

Overall, the dipole is characterized bysix parameters: three of position (x, y, andz) on a three-dimensional plane, two thatfix the orientation, and one that representsits intensity (De Munck et al., 1988). Thesedipoles are situated in a conductor (braintissue) and set into movement the freecharges (electrons and ions) present in theconductor. The amount of movement of thecharges (that is, the electrical current)depends on the conductivity of themedium. When many neurons are simulta-neously activated with a unidirectional ori-entation, these produce measurable signalsin the form of EEG and MEG activity.

It is important to note that even if theelectrical activity of the brain starts at onepoint, the resulting EEG can be measuredall over the scalp. In other words, the con-ductive medium (generally defined, thevolume conductor) diffuses the localizedelectrical activity throughout its volume(DeMunck et al., 1988). To a certain extent,this can be likened to seismic activity inthat an earthquake occurring at its epicen-ter can be recorded on the Earth’s surface aconsiderable distance away from the epi-center, with ever decreasing intensity(Allison, 1984); this same analogy links the activity of electrophysiologists andseismologists. If there is more than onesource activated simultaneously in thebrain, each electrode measures a con-tribution from all the sources. In order toseparate these sources it is fundamentallyimportant to study the spread of theconductance on the basis of the volume—or in other words, to resolve the so-calleddirect problem.

This problem consists in providing thedistribution of the potential (or of the mag-netic field outside the head) on the scalpfor a given electrical source located withinthe head. The reason why this problemcannot be ignored is that the head consistsof different compartments, each with dif-ferent conductance and geometry. Eachcompartment affects the electrical andmagnetic fields generated by the source.The most important compartments are thewhite matter (nerve fibers), the gray



FIGURE 2 (continued) is, however, insensitive (i.e., “blind”) to the dipole flows occurring in the neurons in thevertical parts of the walls of the cortical sulci. This is because, as indicated by the thick, horizontal arrows, theseflows move in parallel or tangentially to the surface of the cortex. The situation for the magnetic component ofthe dipoles is, normally, the reverse. The magnetic dipoles of the neurons of the superficial areas of the corticalconvolutions are not, in fact, picked up by the sensor of magnetic activity (indicated in the figure as “MEG 1storder gradiometer”) because, as can be seen, the dipole and the magnetic flow are tangential to the skull. Thegradiometer does, however, efficiently pick up magnetic fields, even of infinitesimally small strength, comingfrom neurons arranged horizontally on the vertical area of the opposing walls of the cortical sulci. It is not,however, able to pick up the magnetic fields generated by neurons arranged perpendicularly to the base of thesulci. This set of signals can be efficiently recorded by an intracortical microelectrode placed near the neuronalmembrane. Bottom: An example of neuronal units arranged concentrically () and a closed field () thus derivedbecause of the convergent dipolar flows.

matter (neuronal bodies), the cerebrospinalfluid, the skull, and the skin. Most of thesetissues are anisotropic conductors. Thismeans that the electrical current dependsnot only on the conductivity and theamplitude of the applied potential, but alsoon the direction of the gradient of thepotential. This type of current conduction,which has been demonstrated to occur inthe skull, cortex, and white matter, furthercomplicates calculations of electrical poten-tials and magnetic fields (Peters and DeMunck, 1990).

In any case, it is important that the headis roughly spherical and that the compart-ments form a series of concentric sphericallayers. With the geometry it is possible tocalculate the distribution of potentialsusing a fast mathematical algorithm, evenif some of the conduction is anisotropic.This model is normally called the multi-sphere model. It is completely characterizedby the external radius, the radial conduc-tivity, and the tangential conductivity ofeach layer.

The three-spheres model, often used andreported in literature, is a special case ofthe multisphere model. In this model thehead is taken to be a homogeneous sphere.It is considered to have an external spheri-cal layer of poor conductive capacity (inpractice, being isotropic), and an interior,which is the brain. The model is character-ized by three parameters: the internal andexternal radii of the cranium, and its con-ductivity (relative to the homogeneoussphere). This is, therefore, a much simplermodel than the general spherical model.However, disregarding the anisotropy ofthe skull and the conductivity distorted byother parameters could lead to large errorsin estimating the parameters of a source.Furthermore, the simplification does notprovide a faster algorithm for calculatingthe potential. It is, therefore, better to usethe multisphere model.

It is important to remember that there isa marked difference between MEG andEEG as far as concerns the effect of conduc-

tivity changes of the signal measured. Ifthe head were a perfect sphere, made up ofperfect concentric spheres, the magneticfield generated by the electrical sources ofthe conductor would be independent of therays that intersect the various layers of thesphere and of the conductivity of theselayers. Thus MEG has the advantage overEEG of eliminating possible systematicerrors in source localizing caused by theuse of inaccurate parameters of conductiv-ity. Another advantage is the amount ofdiffusion produced by the conductor,which is much less with MEG than withEEG, thus resulting in a better spatial reso-lution. The problem, however, is that theMEG is insensitive to radial dipoles. Inother words, a dipole directed along a linethat crosses the center of the sphere (that is,the radius) does not spread a magneticfield outside the sphere (see Fig. 2), andthus this type of source can never berecorded using MEG equipment (Petersand De Munck, 1990; Del Gratta andRomani, 1999). Strictly speaking, however,these characteristics of the MEG are validwithin the class of models based on exactspheres.

In practice, it can be emphasized thatthe MEG predominantly provides a repre-sentation of dipoles of neural activationarranged perpendicularly in the corticalsulci, whereas EEG provides a profile ofcortical activation above all by measuringthe dipoles radial to the cortical gyri. Itwill, therefore, be clear that only combinedrecordings of electrical and magnetic flowswill be able to provide an exhaustive andaccurate localization of functional activityof the cortex of the brain.

Independently of this, as has been seen,the source of a dipole can be considered tobe described by six parameters, of whichthree denote position and three denoteorientation. There are various coordinatesystems to translate the series of values ofthese parameters into a three-dimensionalvector in space. All the coordinate systemsare derived from Cartesian coordinates,



consisting of three perpendicular axes: x, y,and z. In all these systems the angles aremeasured in degrees and the distances aremeasured in fractions of the radius of thehead, or in centimeters. A diagram of thedipole, with its identifying characteristics,is shown in Fig. 3.

From a practical point of view, electro-physiologists and magnetophysiologiststrying to localize neural processes findthemselves working in the difficult situa-tion in which the cerebral potentials andmagnetic fields are known, because theyhave been recorded, but their intracerebralorigins are still to be determined. In orderto gain this information, they must resolvethe so-called inverse problem. It is clear thatthis is easier to say than to do because inorder to find these sources, exactly what isbeing searched for must be described asprecisely as possible. Fortunately, giventhat, as seen above, there are good reasonsto assume that the brain’s electrical sourcescan be described efficiently by mathemati-cal dipoles (De Munck et al., 1988), theseries of possible responses is considerablyreduced. This is because each dipole hasonly six unknown parameters.

If it is assumed that a specific peak ofthe potential that occurs in all recordings

can be attributed to a dominant source,only six parameters need to be estimatedin order to resolve the spatiotemporalmodeling of this source. Normally, the esti-mate is based on the mathematical crite-rion of least squares. This requires collectingthe group of parameters of the dipole thatbest explain the measurements obtained—that is, the group of parameters that mini-mize the sum of the squared deviationsbetween the real measurements and thoseof the model.

In practice, with this procedure, resolv-ing the inverse problem corresponds, moreor less, to nothing other than resolving thedirect problem by applying a mathemati-cal equation that allows a satisfactoryminimum to be found starting from themany different groups of parameters. Thegroup of parameters yielding the mini-mum value is the solution to the inverseproblem and constitutes the source of theinvestigated dipole (Peters and De Munck,1990).

Principal Component Analysis

In the context of spatiotemporal model-ing, in order to estimate the minimumnumber of sources and speed up thecomputations, a mathematical techniqueknown as principal component analysis(PCA) is applied. The minimum number ofsources depends largely on the estimate ofthe relationship between signal and noise,which can be inaccurate (for more detailson the advantages and limitations of thismathematical technique, see Appendix E,this volume).

Localization of the Dipole and theReference Electrode

In order to localize the sources, it is notnecessary to have a site hypothetically“silent” from an electrical point of view toact as a reference electrode. Given that thehead can be considered as an isolated con-ductor with finite dimensions and a group



FIGURE 3 Equivalent dipoles. Diagram of thehypothesized localization of the electrical dipoles ofvisual ERPs in three planes. The temporal functionsof the sources, calculated by principal componentanalysis, are shown on the left. The vertical lines overlying the wave forms indicate the latency (~110 msec) with which the two different dipoles arecalculated. On the right, the localization of the twodipoles within the three-sphere model of the head canbe observed in the different axial, coronal, and sagittalplanes. The white circles in these sections indicate thesite of the dipole. The lines extending from themindicate the direction of the flow of the dipole.

of sources that produce potentials on thesurface of the conductor, the potentialsrecorded consist of the differences inpotentials between active electrodes andthe reference electrode and not absolutevalues. This has an effect on the morphol-ogy of the signals (for more information onthis subject see Appendixes D and E).

Indeed, the reference electrode could beamong the grid of electrodes used, as longas its position is known with the same pre-cision with which the positions of the otherelectrodes are known. In the inverse modelthis condition is taken into considerationby treating the measurements not asabsolute potentials but rather as differ-ences in potentials between active elec-trodes, and the reference electrode prior tothe computation of the sum of the squarederrors. This does not mean that the choiceof reference does not affect the results. Infact, for each possible reference, a differentsum of the squares is obtained, which, inprinciple, could lead to a differentminimum of the dipole. Great care is,therefore, required on this point (Petersand De Munck, 1990).

EVENT-RELATED POTENTIALS

ERPs are based on the electrophysiologi-cal recording of brain potentials synchro-nized with the presentation of externalsensory stimuli such as light flashes,words, faces, etc., as well as the occurrenceof internal cognitive events such as deci-sion making or selective attentionprocesses (Hillyard et al., 1978; Hillyardand Kutas, 1983; Picton and Hillyard,1988). In the first theoretical treatise theformer potentials were called exogenousbecause they were produced or evoked byexternal stimuli, whereas the latter weredefined endogenous because they were“emitted” even in the absence of any phys-ical stimulus, and were linked to cognitiveprocessing of information in which theindividual was occupied. The potentials

intermediate between these two types,more of a perceptive type than a sensorytype (such as visual N1), were calledmesogenous (Hillyard and Kutas, 1983). Inreality this distinction between exogenousand endogenous potentials is no longerconsidered to be valid because it has beendiscovered that sensory potentials are alsostrongly modified by higher cognitiveprocesses such as attention (for furtherdetails on this topic refer to Chapters10–12, this volume). Naturally, the firstsensory components of the ERPs (exoge-nous) are much more strongly influencedby sensory-physical factors (such as stimu-lus luminance, spatial frequency, eccentric-ity) than are the later components, whichreflect higher order mental processeslinked to thought and emotions.

Unlike the electroencephalogram, madeup of “spontaneous” fluctuations of electri-cal activity in the brain—linked, in a broadsense, to variations in the state of generalactivation (e.g., sleep–wake, differentarousal states)—ERPs represent transientchanges of potentials in the form of a seriesof negative and positive deflections mea-surable only with the technique of syn-chronization between the event and thepotential that it produces. Besides thepotentials synchronized with a stimulusadministered or omitted, it is also possibleto record potentials synchronized with theindividual’s motor response, using thislatter as the synchronization event andaveraging the EEG traces relative to acertain number of trials with a backwardtrend for a certain period of time withrespect to it. In this way brain activityoccurring during decisional processes andmotor programming preceding theresponse can be studied.

In general, evoked potentials, whetherthese are visual, auditory, or somatosen-sory, are wave forms characterized by aseries of positive or negative deflectionswhose polarity is often indicated by theletters P and N, respectively, followed byincreasing numbers denoting the temporal



progression of their appearance. Althoughto some extent incorrect, these deflectionsare sometimes called “components.” Inreality, it is correct to speak of componentsonly when the functional properties of acertain response are known together withthe other characteristics, such as the topo-graphic distribution. For example, accord-ing to Donchin (1979) it is appropriate tospeak of the (“endogenous”) ERP compo-nent when this varies systematically as afunction of the cognitive context. Each bio-electrical potential recorded on the surfacecan be considered the manifestation ofnervous activity associated with specificstages of transmission and processing ofinformation within the nervous system.Measuring the various parameters of theERP components, numerous indicationscan be obtained on the velocity andefficiency with which the processing ofinformation occurs, on the processingresources required by each of the stages ofthe processing, and on the levels of mentalload and attention invested in each ofthese. But, above all, compared with theother neuroimaging techniques, the ERPsare able to provide very precise informa-tion, with a resolution of milliseconds, onthe time course of the various processingstages. In order to measure the amplitudeof the evoked potentials in microvolts (µV),the so-called base line, or isoelectric line,must be calculated by measuring the meanvalue of the EEG in a given time window(normally 100 msec), which precedes theonset of the stimulus and which representsthe absolute amplitude of the nervoussystem response in neutral conditions orduring rest. This value is hypotheticallyconsidered as a zero amplitude responseand every variation in positive or negativepotential is measured with respect to thiszero amplitude.

Alternatively the peak-to-peak ampli-tude can be obtained by calculating the dis-tance between one positive peak and thesubsequent negative one. In both cases, theamplitude is considered, in a broad sense,

to be the expression of the entity of theactivation of the cell assembly units andthe processing carried out on the stimulus.In other words, the greater the amplitude,the greater the number of activated neu-ronal units, the more elaborate the process-ing carried out on the stimulus at thatparticular stage. The exception to this ruleis the so-called probe technique used bysome investigators in research paradigmsbased on recordings of the ERPs inresponse to irrelevant stimuli administeredto volunteers while they perform primarytasks. While the volunteer is occupied intasks that are increasingly difficult toperform, the stimuli presented have thefunction of “probing” the level of mentalwork required by that task. It is probablyobvious that in this case the implicitassumption is the opposite of the pre-ceding: the greater the amplitude of thecomponents recorded in response to theirrelevant stimuli, the greater the state ofprocessing being used on these latter, theless the “cost” of the mental work requiredto carry out the “primary” task. It isreasonable to think that this method can be used to obtain information onprocessing resources, be they controlled or automated, “saved” by the individualduring the experimental protocol to tackleconcomitant tasks that might be adminis-tered to the individual (Isreal et al., 1980;Papanicolau and Johnston, 1984).

In linear terms, on the other hand, thetime interval between the stimulus (or themoment in which this should have beenpresented, because in reality it has been“omitted”) and the appearance of each ofthe different components indicates thelatency of these latter, normally expressedin milliseconds. The longer this interval,the later the activation of a particular stageof reflex information processing of themeasured component is considered to be.For example, a component with a positivepolarity appearing about 300 msec afterthe delivery or omission of a stimulus isdefined P300.

EVENT-RELATED POTENTIALS 21


The functional properties of the majorERP components so far known aredescribed in the following discussions.This description is divided according to thedifferent sensory modalities. As will beseen, there are considerable differencesbetween the modalities in the componentsthat reflect early sensory processing.

Auditory Evoked Potentials

Different wave forms generated byauditory stimuli can be distinguished onthe basis of the latency of their appearance.Each potential reflects the response of dif-ferent districts of the auditory pathways,has a different latency of appearance, andoffers different possibilities for the study ofinformation processing (Picton andHillyard, 1988; Kutas and Dale, 1997) andfor clinical and diagnostic aims (Gibson,1980; McCandles, 1987).

Brain Stem and Middle Latency Potentials

Brain stem potentials, a group of compo-nents with a short latency, appear between1 and 10 msec after administration of anauditory stimulus. These components areexclusive to acoustic stimuli and consist of

six positive waves recorded at the centralregion of the head or the vertex (Cz); theyare identified by Roman numerals ofincreasing size (I–VI), as can be seen in Fig.4. They are surface reflections of sequentialactivation evoked by the acoustic stimuli indifferent centers and auditory pathways ofthe brain stem, that part of the centralnervous system between the spinal cordand the encephalon, and are thus definedbrain stem potentials (BSPs). In detail, these“exogenous” potentials are an expression ofthe sequential activity of the auditorynerve, the cochlear nuclei, the superiorolives, the lateral lemniscus, the inferiorcolliculus, and the medial geniculatenucleus. The first, the third, and the fifth ofthese wave responses can be measuredwithout problems or ambiguity in themajority of subjects examined and thereproducibility of the wave forms is con-stant. Because of this relationship betweenthe wave form and the underlying nervestructures, the brain stem evoked potentialsare without doubt the method of choice for(1) studying the mechanisms of processingauditory information at a very early level(e.g., Zani, 1989; Giraud et al., 2000;Galbraith et al., 2000) and (2) establishing



FIGURE 4 Auditory evoked potentials in an adult, distinguished roughly according to the poststimuluslatency and the neural source of the different components.

the functional integrity of the auditoryapparatus or the level of functional disorderin uncollaborative subjects (such as youngchildren). Given the noninvasive nature ofthe technique of evoked potentials, the BSPsare very useful in ascertaining the degree ofmaturation of nerve pathways at variousages and, above all, in neonates, as can beseen in Fig. 5. The latencies of these compo-nents, and in particular that of the V wave,do in fact change with age and consequentlywith the degree of biological maturation.Starting from 1 to 1 1–2 months of age, the rela-tionship between the intensity of the stimu-lus and the latency of the peak is similar tothat found in adults. It is, therefore, under-standable how these BSPs constitute a valu-able method for early diagnosis of auditorydefects and congenital deafness, thus allow-ing correspondingly early intensive rehabili-tation interventions.

The so-called middle latency potentialsappear between about 10 and 50 msec after

the stimulus onset (as shown in the secondbox in Fig. 4) (Picton et al., 1974) and prob-ably reflect neural activity of the thalamicnuclei, as well as the initial stages ofarrival in the temporal region of the audi-tory cortex. These potentials, too, areuseful for audiological analyses, butknowledge on their specific functions isscarce. It seems that they are an expressionof the response of the auditory system tospecific frequencies of presentation of thestimulus (that is, the number of repetitionsof the stimulus in a given unit of time).They can be correctly identified in thewave forms recorded from children, buteven if they can be used efficiently toanalyze the auditory capacity of children,the reliability of the results obtained is lessthan that afforded by the brain stem poten-tials (McCandles, 1987; Gibson, 1980).

Cortical Auditory Potentials

The cortical components of auditorypotentials form part of the class of ERPcomponents, and apart from the firstresponses of sensory/perceptive typeamong which the N1 and the P2 areprominent (Picton and Hillyard, 1988),they are very similar to those elicited in thevisual modality, and are schematically pre-sented in the third box in Fig. 4. The firsttwo components appear between 60 and250 msec after the stimulus has beenadministered and reflect the activity gener-ated in the cortex by the primary and asso-ciative areas of the temporal and parietallobes of the cerebral hemispheres. Thesepotentials seem to reflect analysis of thephysical characteristics of a stimulus—forexample, its intensity, frequency, pitch,timbre. In situations in which the individ-ual is involved in cognitive tasks that, forexample, require processes of allocation ofattention or discrimination of stimulusmaterial, these components become super-imposed by negative potentials that modu-late their amplitude. These oscillations arecalled attention-related negativity or pro-cessing negativity (Hansen and Hillyard,



FIGURE 5 Brain stem evoked potentials in re-sponse to tones of 60 dB intensity, recorded fromneonates of various ages. The latency of the V com-ponent (expressed in milliseconds) changes with age,reflecting the degree of biological maturation.

1980; Näätätnen, 1982; Näätätnen et al.,1985). The so-called negative difference(Nd) is the negative difference waveobtained by subtracting ERPs to irrelevantstimuli from those to task-relevant stimuli.There is evidence that Nd contains an earlyphase (Nde) with a central maximum(sensory specific), and a later phase (Ndl)with a more anterior distribution.

One very characteristic component ofthe auditory ERP, because it is not presentin the other sensory modalities, is the so-called mismatch negativity (MMN), identi-fied by Risto Näätänen (Näätänen et al.,1978), to which a whole chapter of thisbook is dedicated. The MMN generallyrepresents the response of the brain to anydetectable difference in a repetitiveacoustic stimulus and has a very widerange of uses in both clinical and experi-mental settings (see Ch. 14).

Late Components and the P300

The so-called late or cognitive com-ponents of the ERPs appear between 250 msec and 1 sec after the stimulus(Picton and Hillyard, 1988). Although theyhave a fairly supramodal character, theyare not completely nonspecific. There is, infact, experimental evidence of topographi-cal differences according to the sensorymodality considered. These componentsinclude the so-called P300, a family of com-ponents with positive polarity that appearfrom about 250 msec onward after thestimulus. They include a P3a, prominenton the frontal area of the brain in responseto an unexpected novelty, which suggestsan orientation reflex by the individual, andalso a P3b, prominent on the parietal areasof the brain in response to stimuli that,even if known, are of unlikely probabilityof being presented, requiring reiteration of the process of categorization (Ruchkin et al., 1987). Furthermore, they include aslow positive component, the slow wave,appearing at about 600–700 msec and con-tinuing sometimes even beyond 1 sec.

These components reflect processes of cog-nitive updating, of activation of workingmemory, and of comparison with modelsstored in the memory. They are, therefore,strongly dependent on past experience, theindividual’s expectations, and semanticevaluation of the stimulus. It is preciselyfor this reason that they also appear in thecase of a lack of presentation (omission) ofan expected stimulus, thus justifying theirdesignation as “endogenous” components(Ruchkin et al., 1988). In more detail, if, asthe result of the process of categorizing astimulus, a discordance with the individ-ual’s own interior “model” is perceived, aprocess of “contextual cognitive updating”takes place, reflected by the P300. The hypothesized “subroutine” (Donchin,1981) or the so-called dedicated processor(Donchin et al., 1973), of which these com-ponents are thought to be the surfaceexpression, would therefore be invoked bythe system any time that it obtains infor-mation that requires a revision of hypo-theses or models of the surroundingenvironment. These are, therefore, formedonly after the individual clearly recognizesto which category to attribute the stimulus.An obvious extension of this is that thesecomponents are also expressions of theability to discriminate the stimulus. Inpractice they are a function of the a posteri-ori certainty of having correctly receivedand interpreted the stimulus.

Speaking of processes of contextualupdating or memory recovery is not anti-thetical to the distinctions proposedbetween automated and controlled processesin information processing (Hoffman, 1990;Rösler and Manzey, 1986). This conceptual-ization is based on the assumption thatthere are two qualitatively different modal-ities of processing that characterize ournervous system. Automated processes arecarried out without the individual’s atten-tion and indeed the individual is not con-scious of performing them. Furthermore,they do not interfere with other automatedactivities carried out at the same time by



the nervous system. For example, all thoseprocesses that synthesize a percept startingfrom the sensory input or that transform aplan of action into a real movement can beautomated. Examples of automated pro-cesses in daily life are all those complexmotor actions that we carry out withoutany apparent effort in adult life (such aseating spaghetti, unbuttoning a shirt,brushing our teeth, driving a car) and thatallow us to speak or carry out other activi-ties at the same time without any difficulty.Nevertheless, these are very difficult andcomplex to reproduce for a small child oranyone who has not had adequate training.Another characteristic of the automatedprocesses is that they are carried outwithout a control that we could define asconscious or voluntary, to the point thatthey can be unintentional—for example,when we dial one telephone numberinstead of another when our mind is occu-pied with other things. In contrast, con-trolled processes are voluntary and indeedcannot take place without direct control ofthe conscience. These interfere with othercontrolled cognitive activities because theyare dependent on a central system (pre-sumably frontal) with a limited capacity.The controlled processes take longer tocarry out compared to the automatedprocesses. On the other hand, controlledprocesses can be changed and altered andapplied in new and different situations thatcould not be efficiently managed by theautomated processes. Examples of con-trolled processes are learning new materialor difficult motor sequences. For example,dialing a telephone number never previ-ously used requires attention to the taskand does not allow other tasks requiringattention to be carried out simultaneously.

Analyzing the properties of the P300 inthe framework of this theory, it can beshown that the P300 is always evoked witha notable amplitude when the situationrequires controlled processing, whereas ithas a minimal amplitude or is absent whenthe situation can be managed automati-

cally. Contextual updating, therefore, is anexample of a situation that requires adop-tion of a controlled strategy in a situationin which the processing of information,which proceeded up to that point in anautomated way, no longer permits auto-mation because the resulting stimuli aredifferent from the model stored in thememory.

This explains some contradictory resultsin the scientific literature. Some studiesfound a correlation between P300 andreaction times (Roth et al., 1978), whereasothers, with rigorously controlled studies,found no association between the two (e.g.,Karlin et al., 1971). This contradiction wasresolved by the repeated demonstrationthat the latency of the P300 depends onlyon the processes of classifying the stimulusand is, to a large extent, independent of theselection of the motor response and thetime to carry it out. For example, it wasshown (Kutas et al., 1977) that in a situationrequiring precision and accuracy in evalu-ating a stimulus, the selection of theresponse depends on the process of catego-rizing the stimulus, and, thus, the reactiontimes (RTs) and the latency of the P300appear to be strongly correlated. In con-trast, if the speed of response is privileged,the response can be made before the stimu-lus has been completely evaluated andthus the two measures result as only beingweakly correlated or indeed not correlatedat all. Thus, generally speaking, althoughthe RTs depend on many processesbetween the stimulus and the response,including selection of a plan of action andprocesses of performing the response, thelatency of the P300 depends largely on theprocesses of evaluating the stimulus.Valuable data supporting this theoreticalmodel were provided by studies based onSternberg’s paradigm of additive factors. Insome of these (McCarthy and Donchin,1981; Magliero et al., 1984), both the dis-criminability of the stimulus (easy ordifficult) and the response processes (com-patible or incompatible with the stimulus)



were made to vary. The result was thatalthough the latency of the P300 dependedonly on the discriminability of the stimulus,the RTs depended in an additive manner onboth the discriminability of the stimulusand the compatibility between the stimulusand response.

Besides categorization processes, theP300 has been strongly associated withworking and short-term memory pro-cesses. For example, in an electrophysio-logical study by Kleins et al. (1984) itemerged that this component representedprocesses of access to the representation ofsounds in the auditory working memory.This study compared the ERPs recorded intwo groups of musicians, one withabsolute pitch, the other without thisability. The ear with absolute pitch allowsits owner to identify the absolute pitch of pure tones in the absence of refer-ence stimuli. The results showed that both groups had a positive component(between 300 and 600 msec poststimulus)over the parietal area during a visual“oddball” task. During an auditory odd-ball task, however, this component waspresent only in the control group and itsamplitude was greatly reduced or indeednot present in the group of musicians withabsolute pitch. The authors’ hypothesiswas that these latter have a permanentreference of all tones and do not need tomake continual comparisons of informa-tion and thus activation of short-termmemory mechanisms expressed by theP300 component.

In another electrophysiological study byNielsen-Bohlman and Knight (1994), theP300 was associated with processes ofshort-term maintenance of mental repre-sentation. This study consisted of a task ofrecognizing pictorial images of familiarobjects that could have been presented inthe previous 1.2 sec (immediate interval),in the previous 1.2 to 4 sec (short interval),or more than 5 sec earlier (5–158 seconds,long interval). The ERP results showed afrontal P3a and a parietal P3b in concomi-

tance with the recognition of stimuli pre-sented after a short interval, and a mesialtemporal N400 for stimuli presentedimmediately or after a long interval.According to the authors, the P300 couldindicate activation of mechanisms of accessto the working memory guided by thefrontal area, whereas the N400 could reflectactivation of the mesial temporal cortexinvolved in long-term memory.

Alternatively, and also on the basis ofERP and RT data that demonstrate thedecreasing P300 and the prolongation of theRT with the increasingly long interval, itcould be hypothesized that the P300 ob-served in this study reflects in any case thecertainty of the recognition and thus pro-cesses of categorization rather than ofmemory.

Motor Potentials

Stimulus-locked motor potentials areformed from variations of brain electricalpotentials that occur when an individual’sattention is directed to planning an action inrelation to a sequence of stimuli. This actioncan be a decision process, a motor action, orinhibition of a possible motor action. Inthese situations a large expectation negativ-ity develops, which is called contingent nega-tive variation (CNV) [see also Fabiani et al.(2000) and Brunia and Van Boxtel (2000) fora description of motor potentials]. Thisdeflection is considered to be the expressionof processes of preparation and orientationin view of performing a required response tothe task (see an example in Fig. 6).

A series of two negative and two posi-tive components, defined as a whole as“motor and premotor potentials” distin-guished by CNV and direct expression ofthe performance of motor actions, belongto the same family. In fact, the motor andpremotor potentials are obtained by syn-chronizing the EEG with the onset of theelectromyographic activity. By recordingthe EEG and EMG simultaneously andwithout interruptions, the start of the first



muscle spike is used as the synchronizationevent to average the EEG trace, both in thetime preceding and subsequent to it for agiven interval. It is, therefore, understand-able that these can be recorded only for anintentional act carried out by the individual,completely independently of any stimulus.The first negative component that precedesthe start of muscle activity is known as readi-ness potential (Bereitschaftpotentiale, in Ger-man). This deflection, which evolvessymmetrically over both cerebral hemi-spheres for up to 500 msec before the start ofthe motor activity, becomes asymmetricallygreater over the motor areas of the hemi-sphere contralateral to the limb to be movedfrom 500 msec onward. It appears to reflectthe general preparation, or motivation, of avoluntary motor action. It is followed by asmall positive deflection, called P1 anddefined as “premotor positivity,” whichdevelops starting from 150 msec before thestart of the movement (or rather the onset ofEMG, because between this and the motoraction there is a delay of 250 msec, onaverage). It has been suggested that thisexpresses the sending of the order of the

motor program. The wave form then contin-ues with a negative deflection, called N2,defined as the “motor potential,” and is par-ticularly pronounced over the motor areas.Finally, this is followed by the so-called reaf-ference potential, or P2 (Deecke et al., 1984;Kristeva et al., 1990). This descriptionshould, however, be considered a simplifica-tion because there is a much richer andcomplex division of the potentials [seeBrunia and Van Boxtel (2000) for an exhaus-tive review].

Visual Evoked Potentials

The visual evoked potentials (VEPs) aredistinguished from the auditory evokedpotentials predominantly by their mor-phology and by the fact that it is not poss-ible, or only to a very limited extent, toobserve the evoked responses derivedfrom subcortical structures such as thesuperior colliculus or lateral geniculatenucleus of thalamus. Therefore, theyconsist of fewer waves, namely, thosederiving from the activity of the cerebralcortex.



FIGURE 6 Grand-average auditory potential evoked by a 1000-Hz pure tone of 100-msec duration, andrecorded from the midline central site Cz. In this experiment (Proverbio, 1993), the auditory stimulus acted as awarning tone, alerting the subject to the appearance of a visual target stimulus to which the subjects had torespond. Note the prominent cortical N1 and P2 followed by an increasing negativity lasting for the more than600 msec; this is the contingent negative variation.

The study of VEPs has elicited greatinterest since the very earliest electrophysi-ological techniques were realized to beinstruments to identify anomalies orpathologies of the visual system (e.g.,Celesia and Daly, 1977). These techniquesremain an irreplaceable tool for studyingalterations in the mechanisms of visualprocessing. For the past 25 years, thanks tothe development of cognitive electro-physiology and cognitive neuroscience,researchers have not merely concentratedon the first stages of sensory analysis, tra-ditionally studied by VEPs (Jeffreys, 1970),but also on the subsequent stages of infor-mation processing and on the way inwhich higher cognitive processes (such asattention, emotions, motivation, memory,and cognition) can affect visual processing(e.g., Neville et al., 1982; Duncan-Johnson

and Donchin, 1982). Therefore, in modernstudies the term “VEPs” has been replacedby or combined with the concept of visualERPs (e.g., Friedman et al., 1978; Shulman-Galambos and Galambos, 1978).

Here we briefly describe the mainsensory components of visual ERPs, whichmodern literature has shown to be affectedby higher cognitive processes. This topic isdealt with specifically in Chapters 10 and12, which are devoted to visual selectiveattention mechanisms. Therefore, the olddistinction between exogenous and endo-genous components (e.g., Picton and Hill-yard, 1988) will not be referred to in thisdiscussion.

Figure 7 shows human VEPs elicited bya black and white luminance-modulatedgrating with a spatial frequency of 6 cyclesper degree. The grating was presented to



FIGURE 7 Grand-average ERPs recorded at the right lateral occipital electrode in response to gratings of 6 cycles/degree presented to the fovea during a task of selective attention for spatial frequency. The two super-imposed lines indicate different conditions of attention of the observer, depending on the type of stimulus seen:the solid line represents cerebral activity evoked by gratings of the relevant spatial frequency, whereas the dottedline represents the cerebral response to the same stimuli when irrelevant.

the central visual field with a pattern onset1

modality, randomly intermixed with grat-ings of different spatial frequencies. Thesolid line represents the cerebral activityevoked by the same grating when itsspatial frequency was relevant, whereasthe dotted line represents the evokedresponse to the same grating when irrele-vant. The potentials are grand averagesand were recorded at the right lateraloccipital site. At this location a characteris-tic series of negative and positive deflec-tions can be seen, reflecting the differentstages of analysis of the visual information.The first peak around 40 msec is notalways seen because its amplitude ismodest and it emerges only when thesignal-to-noise ratio is optimal. In this caseit has a negative polarity and reflects astage of subcortical analysis (probably thal-amic). The second peak is called N80, orP80 when it presents with the oppositepolarity, and reflects the earliest stages ofprocessing of visual inputs in the primaryvisual area (striate cortex or area 17). It isalso known as C1, which stands for thefirst component of VEPs (Jeffreys, 1971,1977; Drasdo, 1980). This early component,together with the subsequent P1, has avariable amplitude and latency accordingto the spatial frequency of the stimulus. In

particular, low spatial frequencies (e.g., 1.5 cycles/degree) elicit a bigger and fasterP1, whereas higher spatial frequencies(e.g., 12 cycles/degree) elicit a moremodest and later P1 and large early nega-tivities (Bodis-Wollner et al., 1992; Hudnellet al., 1990; Previc, 1988; Proverbio et al.,1996; Skrandies, 1984; Zani and Proverbio,1995, 1997). These two components arevery sensitive to physical parameters ofthe stimulus (Lesevre, 1982) and varygreatly in function of the quadrant of stim-ulation, its luminance, contrast, retinallocation (Clark and Hillyard, 1995), widthof the visual field, orientation (Proverbio et al., 2002b), and color, as well as spatialfrequency (Aine et al., 1990). It has beenhypothesized that the C1 (or PN/80),which is topographically and functionallyseparable from the P1, represents activityof the parvocellular visual system,2 devotedto analysis of color and high-frequencypattern (Livingstone and Hubel, 1988),whereas P1 represents the activity of themagnocellular system, devoted to analysis of achromatic, low-frequency patterns(Kulikowski et al., 1989; Paulus and Plendl,1988; Zani and Proverbio, 1995). One isoverrepresented in the fovea, thus confer-ring the visual acuity necessary for photopicvision; the other constitutes the majority ofperipheral ganglia cells and allows, forexample, immediate appreciation of ashadow moving at the observer’s shoulder(scotopic vision). In an ERP study showinghemispheric asymmetries in evoked activ-ity during passive viewing of sinusoidalspatial frequency gratings (Proverbio et al.,1996), scalp current density mapping (seeAppendix D for details on this procedure)provided evidence of a progressive shift offocus of the maximum amplitude of theN80 from the right hemisphere to the left



2 In particular, reflecting activity of the ganglia cellsfrom the lateral geniculate body of the thalamus andafferents to the striate visual cortex (area 17).

1 The recording of an evoked potential using thepattern onset modality occurs in synchrony with the onset(or start) of the stimulus, generally followed by its dis-appearance (or pattern offset). This gives rise to a transientcerebral response, which is a contingent response from thebrain followed by a return to prestimulus base line “activ-ity.” When the stimuli are presented very closely togetherin time, thus not allowing a return to base line, a so-calledstable (or steady-state) evoked response is produced. Thesame type of response occurs when the configuration ofstimuli remaining fixed on the screen alter over time [forexample, a checkerboard with black and white squares ordifferently colored squares, whose constituent elementsrhythmically, at a preestablished velocity (e.g., 3 Hz) takeon the opposite color: white–black, black–white, etc.], atechnique defined as pattern inversion (or pattern rever-sal). For a fuller treatment of evoked potentials inresponse to pattern reversal the reader is referred toSilberstein (1995b) and to Chapter 11, this volume.

one as the spatial frequency increased.These data support the hypothesis of ahemispherical asymmetry in spatial fre-quency processing independent of higherorder cognitive factors (Hellige, 1993;Hughes et al., 1996; Proverbio et al., 1997a,1998). Other properties of this component,with particular reference to the neural gen-erators and modulation by superior cogni-tive factors such as attention, are describedin Chapter 12 (this volume).

The third positive peak is the P1 compo-nent of VEPs, which has a latency of about100 to 140 msec. This is supposed to derivefrom the evoked activity of the extrastriatevisual cortex (Lesevre and Joseph, 1979;Heinze et al., 1994; Mangun et al., 2001;Ossenblock and Spekreijse, 1991; Proverbioet al., 1996; Zani and Proverbio, 1995, 1997)and often shows a right hemispheric asym-metry. P1 component analysis is crucial forstudies on space-based selective attentionmechanisms. Its amplitude is stronglyaffected by attention and expectancyfactors. For example, if an observer expectsthat a certain stimulus will appear at agiven space location (for example, that amouse will come out of a mousehole) thevisual system processes all the stimuli rela-tive to that retinal location with moreemphasis and accuracy. This activates alarger number of neurons that fire withgreater strength as soon as they receive thesubcortical inputs, producing larger bio-electrical potentials with a shorter latency.The greater efficiency of the sensory analy-sis, as well as of the classification processesand motor programming, mean that thereaction times to the stimulus are faster(the mouse will be caught). Observing theamplitude of the visual P1, it is possible toestablish a gradient of activation in func-tion of the different levels of attention andexpectation of the observer.

Returning to the ERP in Fig. 7, the P1sensory component is followed by a seriesof negative deflections, N1 and N2, bothstrongly affected by attention. As can beseen in the figure (see also Chapters 10–12,

on attention), selective attention toward agiven category of visual stimuli is mani-fested by an increase in the negativityrecorded over the posterior region of thescalp between 150 and 300 msec after thestimulus; this is called selection negativity(Harter and Aine, 1984) or processing nega-tivity (Näätänen, 1982). This large negativ-ity is followed by the P300 positivecomponent, which we have already dis-cussed thoroughly—this, too, stronglyinfluenced by attention, cognitive, andmotivational factors of various types.

Linguistic Potentials

On the basis of the rich literature on thissubject (e.g., Kutas and Van Petten, 1994;Kutas et al., 1999; Osterhout, 1994;Osterhout and Holcomb, 1995; Van Petten,1995), various different ERP componentshave been identified, each with their owntopography and functional characteristics.The observation and analysis of these incertain experimental paradigms allow usto investigate the different processingstages of mechanisms of understandinglanguage in humans. Given the obviousproduction of muscle artifacts duringspeech, oral production of language is notgenerally used in studies with this method.Analysis of the different components of theERPs, described extremely well in Chapter6, has allowed identification (roughlyspeaking) of a stage of grapheme/ortho-graphic (in reading) or phonological (inspeech processing) analysis, a stage ofsemantic/lexical analysis, a stage of analy-sis of the structure of the phrase, and amore sophisticated stage of syntactic analy-sis. The technique of ERPs enables a certaintemporal course to be established for thevarious processes, but naturally many ofthese occur in parallel and are accom-panied by numerous integration processesthat put together the information in a compatible way, piece by piece, as itbecomes available on the basis of contextand previous knowledge.



On the basis of event-related brainpotential studies we are able, for example,to establish that semantic analysis andintegration processes take place about400 msec poststimulus, as reflected by theN400 component, a centroparietal nega-tivity whose properties have been widelycharacterized (Kutas and Hillyard, 1980;Kutas and Van Petten, 1994). On the otherhand, phrase structure assignment andsyntactic integration are assumed to bereflected by an early left anterior negativity(ELAN) with a latency of about 100–300 msec, a left anterior negativity (LAN)with a latency of about 300–500 msec, anda late centroparietal positivity (P600), also called syntactic positive shift (Osterhoutand Holcomb, 1992, 1993; Hahne andFriederici, 1999; Friederici et al., 1999;Münte et al., 1998). The ELAN is thought toreflect a first-pass parsing process and tobe very sensitive to word category, beingguided by phrase structure rules. The laternegativity (LAN), overlapping in time withsemantic N400, is thought to reflect mor-phosyntactic analysis. This late positivitymight reflect relatively controlled lan-guage-related processes (Hahne andFriederici, 1999) sensitive to inflectionalinformation (Gunter and Friederici, 1999)and associated with secondary syntacticprocesses such as sentence reanalysis andrepair (Friederici, 1997), or processes toinhibit incorrect representation due todifficulty with syntactic integration (Kaanet al., 2000).

Figure 8 shows an example of these lin-guistic components derived from dataobtained in our laboratory in an experimentin which volunteers were required to evalu-ate the correctness of visually presentedphrases (Proverbio et al., 2002a). In theexample shown below, the ERPs wererecorded at the onset of the last word, whichcould be correct or incorrect and/or incon-gruous with respect to the previous context:

Correct sentence: She was frightened by his NASTINESS

Semantically The structure of the city incongruent: was too ENVIOUS

Syntactically All the windows were incongruent: CONVICTION

In the upper part of Fig. 8, grand-averageERPs recorded from the frontal sites are dis-played as a function of correctness of theterminal word. We can see that semantically+ syntactically incongruent words elicit alarge negativity around 400 msec poststimu-lus, which is the N400 component originallyidentified by Marta Kutas (Kutas andHillyard, 1980; Kutas and Van Petten, 1988).This negative response generally indicatesword recognition and semantic processing,as well as the brain response to contextuallyunexpected items (Kutas and Kluender,1994). It is generated by semantically anom-alous words, or at any rate words notexpected by the subject—that is, at low clozeprobability (probability that a given wordwill be used to finish the sentence); for amore in-depth discussion of the functional



FIGURE 8 Examples of grand-average (N = 9)ERPs recorded at left and right frontal and posteriortemporal sites in response to correct (solid line) orsemantically + syntactically incongruent terminalwords (dashed line). Note the large anterior negativ-ity (N400) in response to semantic incongruence, andthe delayed left temporal positivity (P615) in responseto syntactic incongruence (adapted and modifiedfrom Proverbio et al., 2002a).

significance of this see Chapter 6 (thisvolume).

The lower part of Fig. 8 shows grand-average ERPs recorded from the posteriortemporal sites, where it is possible to seethe onset of a late positive component(P600) for the phrases that are also syntac-tically incorrect. A late centroparietal posi-tivity with a latency varying between 600and 900 msec has been described in the lit-erature as the response to syntactic incon-gruence, syntactic positive shift (SPS)(Hagoort et al., 1993), and/or processes ofreviewing the interpretation of a phrase(Mecklinger et al., 1995). The two compo-nents N400 and P600 indicate, also in termsof temporal latency, that the linguistic pro-cessing is occurring at two levels: a seman-tic level with analysis of significance on thebasis of context and a syntactic level withanalysis of the relationships between theparts of the phrase on the basis of thegrammatical rules of the language.Semantic analysis thus seems to take placeabout 400 msec after auditory or visualpresentation of a word, but it can be seenhow the N400 peak in reality represents theculmination of a process (access to thelexicon) that has begun gradually severaltens of milliseconds earlier.

The first stage of processing visuallypresented words consists of a sensoryanalysis of their physical features, such asluminance, orientation, size, color, contrast,and spatial frequency. Identification of thecharacters as having linguistic significance(orthographic analysis) takes place about150 msec after the onset of the stimulus inthe occipitotemporal area (lingual gyrusand left and right fusiform gyri), as shownin a magnetoencephalographic study byKuriki et al. (1988). On the other hand, witha PET study, Petersen and colleagues(1988) identified a visual area specializedin recognizing the graphic form of words(visual word form system) in the left medialextrastriate cortex of the occipital lobe. Intheir study, strong activation of this regionwas seen during the reading of words or

pseudowords (nonexistent words, but onesthat obeyed the orthographic rules of thegiven language), but this activation wasabsent for nonwords (ill-formed letterstrings) or false font strings. These firstfindings were confirmed by subsequentstudies that identified the area devoted torecognition of real letters in the posteriorfusiform gyrus (Allison et al., 1994) and inthe occipitotemporal and inferior occipitalsulci (Puce et al., 1996). The left hemi-sphere’s specialization of the regiondevoted to letter analysis was very recentlydemonstrated by neuroimaging studies(Polk et al., 2002).

The first stages of graphemic analysisand recognition of the above-describedwords correspond to the appearance of theN1 evoked component with a latencybetween 150 and 200 msec (Bentin et al.,1999). Subsequently, the N2 indicates astage of phonological type analysis of theinformation—that is, analysis related to thesounds of the language, both with theauditory and visual modalities of presenta-tion. For example, in the ERP study byConnolly and Phillips (1994), auditory–phonemic type linguistic violations pro-duced a negative response called phono-logical mismatch negativity (MMN) around270–300 msec after the onset of the word.This component in reality responds auto-matically to any discriminable differencebetween two auditory stimuli (e.g., puresounds, chords, linguistic stimuli) even inthe absence of attention (Näätänen, 1992,1995) and is described in depth in Chapter14.

In a study in our laboratory (Proverbioet al., 1997b, 1999), subjects performed aphonological decision task on parts ofwords or syllables without sense (trigrams)briefly presented to the central visual field.The ERPs showed a large negative res-ponse to phonological incongruence (non-targetness) starting from about 250 msecpoststimulus. Figure 9 shows grand-average ERPs to targets (syllables whosegrapheme/phoneme conversion produced



the phonological form expected by sub-jects) and nontargets (syllables incompati-ble with the expected phonological form)recorded at frontal (FZ) and parietal (PZ)sites. It is interesting to note that over the anterior area there was a negativedeflection in response to phonologicallyincorrect syllables, with two peaks around350 and 600 msec, whereas over the poste-rior area correct syllables produced a hugepositivity, with two peaks around 340(P300) and 520 msec (late positivity). Giventhat the participants had to respond toboth targets and nontargets by pressingone of two response keys with either theindex or middle finger, any differencebetween the two wave forms can beattributed to recognition of a phonologicalincongruence that generates a phonologicalmismatch negativity similar to that found inthe auditory–phonetic modality.

SOMATOSENSORY EVOKED POTENTIALS

The somatosensory evoked potentials(SEPs), reflect the activity of corticalregions of somatosensory projections, andthus, in the initial stages, of the functional-

ity of the somatesthetic systems. Most ofthe components that indicate the differentstages of both sensory and cognitive pro-cessing of the somatesthetic inputs areshown in Fig. 10a. The positive compo-nents P10, P12, P13, and P14, not repre-sented in the figure and indicating the veryearly latency of the first phases of the pro-cessing, are thought to reflect, respectively,discharge of the peripheral nerve, arrivalin the dorsal root of the spinal cord, andthen arrival in the mesencephalon (13–14msec). On the other hand there are indica-tions that the P15 (see Fig. 10b) could begenerated in the medial lemniscus, or inthe thalamus (Allison, 1984). The N20, theP20, and the P25 are thought to reflectactivity of the primary somatosensorycortex. All of the other components up toP100 reflect potentials generated in theprimary somatosensory cortex or sur-rounding areas. Desmedt and Robertson(1977) demonstrated that the N140 and theP190, also called vertex potentials, arestrongly influenced by attention. Indeed,during tasks of selective attention, a large somatosensory processing negativity(Näätänen and Michie, 1979) develops,which is very similar to that described forthe auditory modality. More recent electro-

SOMATOSENSORY EVOKED POTENTIALS 33


FIGURE 9 Examples of grand-average ERPs elicited by visually presented syllables during a phonologic deci-sion task, and recorded at frontal (Fz) and parietal (Pz) midline sites. The solid and dashed lines represent thepotentials elicited, respectively, by the syllables that included or did not include the target sounds (adapted fromProverbio et al., 1999).

physiologic studies have provided evi-dence of attention-related modulation ofsomatosensory potentials already at thelevel of the early cortical P40 component.This component is, in fact, more positivefor relevant stimuli than for irrelevantstimuli over the areas of the scalp con-tralateral to the stimulated limb (Desmedtand Tomberg, 1989; Garcia-Larrea et al.,1991; see also Näätänen, 1992).

EVENT-RELATED FIELDS

The notable development of neuromag-netometry, which is the recording of brainmagnetic fields, has led to the identi-fication of a set of components of differentprominence and polarity in wave forms,which distinguish the magnetic responses,or event-related fields, elicited by stimuliin the various sensory modalities. In the



FIGURE 10 Diagram of somatosensory evoked potentials. (a) Theoretical example of a wave form recordedover the left parietal area (P3) in response to electric shocks to the right median nerve in a young adult; (b) promi-nent early components of the wave form of the somatosensory evoked potential (SEP) obtained recording at Erb’spoint, or the midclavicular point, from the cervical vertebrae, and from the contralateral cortex, respectively.

following discussion we provide somebrief descriptions of the nature and func-tional characteristics of these components.We are, of course, completely aware thatthese descriptions are far from beingexhaustive.

Auditory Event-Related Fields

Similar to the findings of the auditoryERP, the first deflection shown by an audi-tory ERF wave form is an early positivecomponent defined as P50m (see Fig. 11).This component appears in particularwhen a click is used instead of tones. It isfollowed by a prominent N100m, whosegenerator seems to lie in the supratemporalauditory cortex. The amplitudes of bothare greater over the auditory cortex of thehemisphere contralateral to the stimulatedear (Hari et al., 1987). It was found that thesource of the N100m has different locationswithin the auditory cortex; the locationsare a function of the frequency of the tran-sient tones administered. These resultsindicate that the auditory cortex has atonotopic organization (Romani et al.,1982). This type of functional organizationwas also suggested by sustained (steady-state) auditory stimuli (Elberling et al.,1982).

The N100m component is followed by aP200m. Rif et al. (1991) reported thatpaying attention to tones administered at aconstant interval of 405 msec modifies thewave forms starting precisely from theP200m. Indeed, the authors found anotable reduction in the amplitude of theP200m caused by the superimposition ofthe magnetic analog (i.e., SNm) of selectionnegativity (Näätänen, 1982). However, in amore recent study, Woldorff et al. (1993)demonstrated that attention modulates thebrain auditory magnetic responses muchearlier than indicated by the results citedabove. Using an attention-requiring task inwhich the participants had to concentrateon tones presented rapidly to one ear whileignoring those presented to the other ear,

they found that when the tones were rele-vant, they elicited fields of a greater ampli-tude than when they were irrelevant, asearly as 20 msec after the stimulus. On thebasis of the measurement of fMRI-recorded brain activity in this same study,the authors were also able to indicate thatthe generators of these very early auditoryattention effects were localized in thesupratemporal plane of the temporal lobe.

Visual Event-Related Fields

As can be seen in Fig. 11, the wave formof the visual magnetic fields (visual evokedfields, VEFs) is characterized by a promi-nent P1m component, the peak of whichappears after a latency between 100 and150 msec. This is followed by an N1m anda P2m. A full treatment of the functionalproperties of these components is beyondthe scope of this chapter. For this thereader is referred to the impressive review

EVENT-RELATED FIELDS 35


FIGURE 11 Diagram of the magnetic wave formsillustrating the components most discussed in theliterature as a function of sensory modality. Note thedifferent scales [in femtoteslas (fT)] of the wave forms.

in Chapter 5, on MEG studies of visualcognition.

We restrict discussion here to a couple ofshort and very sketchy notes on the P1mcomponent. There are experimental resultsin the literature demonstrating that thiscomponent varies as a function of thethreshold of contrast, and of the temporaland spatial frequencies of the visual stimu-lus. For example, there is evidence thatwith increased temporal frequency, lowspatial frequency gratings elicit magneticP1m of greater amplitude than do highspatial frequency gratings (e.g., Okada etal., 1982). Furthermore, and most impor-tantly, there are indications that payingselective attention to a point in space or tothe physical characteristics of the stimulus(e.g., spatial frequency) increases theamplitude of this early magnetic response(Aine et al., 1995), analogous to the findingsof Zani and Proverbio (1995, 1997) for itselectrical counterpart, the P1, of the ERPs.

Somatosensory Event-Related Fields

In neurologically healthy individuals,electrical stimulation of the median nervein the wrist induces an early magneticresponse in the contralateral primarysomatosensory cortex, SI, detectable by 20msec after the stimulation. This responsehas a negative polarity and is thereforeindicated as N20m (Fig. 11). It should beemphasized, however, that the polarity ofthis response changes as the recording elec-trode is moved from the perisylvian areaup to the vertex of the parietal postcentralgyrus. This early component is followed bythe P30m.

The latency of appearance of both thesecomponents is much delayed in patientswith multiple sclerosis. Furthermore, insuch patients the responses recorded witha latency of about 50–80 msec for the arearepresenting the hand, within the SI, areabnormally large (Karhu et al., 1982).

The responses on the ipsilateral somato-sensory cortex are very different and vary

according to whether an electric shock isused to stimulate the median nerve (upperlimb) or the peroneal nerve (lower limb). Inthe former case, the first magnetic responseobservable in the SEF wave forms has apoststimulus latency of about 90–95 msec.In the latter case, the peak latency of thefirst response is later at about 110–125 msec(Hari et al., 1984; Hari, 1994). It has beensuggested (Hari et al., 1984) that the mag-netic flow of these components reflects thefunctional modifications in the secondarysomatosensory cortex, SII.

ReferencesAine, C. J., Bodis-Wollner, I., and George, J. S. (1990).

Spatial frequency segregation and evidence formultiple sources. In “Advances in Neuro-logy. Magnetoencephalography” (S. Sato, ed.), pp. 141–155. Raven Press, NewYork.

Aine, C. J., Supek, S., and George, J. S. (1995).Temporal dynamics of visual evoked neuro-magnetic sources: Effects of stimulus parametersand selective attention. Int. J. Neurosci. 80, 79–104.

Allison, T. (1984). Recording and interpreting event-related potentials. In “Cognitive Psycho-physiology. Event-related Potentials and the Studyof Cognition” (E. Donchin, ed.), pp. 1–36.Lawrence Erlbaum Assoc., Hillsdale, New Jersey.

Allison, T., McCarthy, G., Nobre, A., Puce, A., andBelger, A. (1994). Human extrastriate visual cortexand the perception of faces, words, numbers, andcolors. Cerebral Cortex 4, 544–554.

Bentin, S., Mouchetant-Rostaing, Y., Giard, M. H., et al. (1999). ERP manifestations of processingprinted words at different psycholinguistic levels:Time course and scalp distribution. J. Cogn.Neurosci. 11, 235–260.

Bodis-Wollner, I., Brinnan, J. R., Nicoll, J., Frkovic, S.,and Mylin, L. (1992). A short latency cortical com-ponent of the foveal VEP is revealed by hemifieldstimulation. Electroencephalogr. Clin. Neurophysiol.84, 201–205.

Brunia, C. H. M., and Van Boxtel, G. J. M. (2000).Motor preparation. In “Handbook of Psycho-physiology” (J. T. Cacioppo, L. G. Tassinary, andG. G. Berntson, eds.), 2nd Ed., pp. 507–532.Cambridge University Press, Cambridge.

Buzsaki, G. (1991). The thalamic clock: Emergentnetwork properties. Neuroscience 41, 351–364.

Celesia, G. G., and Daly, R. F. (1977).Visual electroen-cephalographic computer analysis (VECA). A newelectrophysiologic test for the diagnosis of opticnerve lesions. Neurology 27(7), 637–641



Clark, V. P., and Hillyard S. A. (1995). Identification ofearly visual evoked potential generators by retino-topic and topographic analyses. Human BrainMapping 2, 170–187.

Connolly, J. F., and Phillips, N. A. (1994). Event-related potential components reflect phonologicaland semantic processing of the terminal word ofspoken sentences. J. Cogn. Neurosci. 3,pp. 256–266.

Deecke, L., Bashore, T., Brunia, C. H., Grunewald-Zuberbier, E., and Grunewald, G. (1984).Movement-associated potentials and motorcontrol. Ann. N.Y. Acad. Sci. 425, 398–428.

Del Gratta, C., and Romani, G. L. (1999). MEG:Principles, methods, and applications, Biomed.Technik 44, Suppl. 2, pp. 11–23.

De Munck, J. C. (1990). The estimation of time varyingdipoles on the basis of VEP. Electroencephalogr. Clin.Neurophysiol. 77, 156–160.

De Munck, J. C., Van Dijk, B. W., and Spekreijse, H.(1988). Mathematical dipoles are adequate todescribe realistic generators of human brain activ-ity. IEEE Trans. Biomed. Eng. 11, 960–966.

Desmedt, J. E., and Robertson, D. (1977). Differentialenhancement of early and late components of thecerebral somatosensory evoked potentials duringforced-paced cognitive tasks in man. J. Physiol. 271,761–782.

Desmedt, T. J., and Tomberg, C. (1989). Mapping earlysomatosensory evoked potentials in selectiveattention: Critical evaluations of control conditionsused for titrating by difference the cognitive P30,P40, P100 and N140. Electroencephalogr. Clin.Neurophysiol. 74, 321–346.

Donchin, E. (1979). Event-related brain potentials inthe study of human information processing. In“Evoked Brain Potentials and Behaviour” (H.Begleiter, ed.), pp. 13–88, Plenum Press, New York.

Donchin, E. (1981). Surprise!…Surprise! Psycho-physiology 9, 493–513.

Donchin, E., Kubovy, M., Kutas, M., Johnson, R., Jr.,and Hernin, G. F. (1973). Graded changes inevoked response (P300) amplitude as a function ofcognitive activity. Perception Psychophys. 14,319–324.

Drasdo, N. (1980). Cortical potentials evoked bypattern presentation in the foveal region. In“Evoked Potentials” (C. Barber, ed.), pp. 167–174.MTP Press, Falcon House, Lancaster, England.

Duncan-Johnson, C. C., and Donchin, E. (1982). TheP300 component of the event-related brain poten-tial as an index of information processing. Biol.Psychol. 14, 1–52.

Elberling, C., Bak, C., Kofoed, B., Lebech, J., andSaermark, K. (1982). Auditory magnetic fields.Scand. Audiol. 11, 61–65.

Fabiani, M., Gratton, G., and Coles, M.G. H. (2000).Event-related brain potentials. Methods, theoryand applications. In “Handbook of Psycho-

physiology” (J. T. Cacioppo, L. G. Tassinary, andG. G. Berntson, eds.), 2nd Ed., pp. 27–52. Cam-bridge University Press, Cambridge.

Friederici, A.D. (1997). Neurophysiological aspects oflanguage processing. Clin. Neurosci. 4, 64–72.

Friederici, A.D., Steinhauer, K., and Frisch. S. (1999).Lexical integration: Sequential effects of syntacticand semantic information. Memory Cogn. 27,438–453.

Friedman, D., Vaughan, H. G., Erlen-Meyer-Kimling,L. (1978). Stimulus and response related compo-nents of the late positive complex in visualdiscrimination tasks. Elecroencephalogr. Clin. Neuro-physiol. 45, 319–330.

Galbraith, G. C., Threadgill, M. R., Hemsley, J., Salour,K., Songdej, N., Ton, J., and Cheung, L. (2000).Putative measure of peripheral and brainstem fre-quency-following in humans. Neurosci. Lett. 6,123–127.

Garcia-Larrea, L., Bastuji, H., and Mauguier, E. F.(1991). Mapping study of somatosensory evokedpotentials during selective spatial attention.Electroencephalogr. Clin. Neurophysiol. 80, 201–214.

Gibson, W. P. R. (1980). The auditory evoked potential(AEP). In “Evoked Potentials” (C. Barber, ed.), pp. 43–54. MIP Press, Falcon House, Lancaster,England.

Giraud, A. L., Lorenzi, C, Ashburner, J., Wable, J.,Johnsrude, I., Frackowiak, R., and Kleinschmidt,A. (2000). Representation of the temporal enve-lope of sounds in the human brain. J. Neurophysiol.84, 588–598.

Gunter, T.C., and Friederici, A.D. (1999). Concerningthe automaticity of syntactic processing.Psychophysiology 36, 126–137.

Hagoort, P., Brown, C., and Groothusen, J. (1993). Thesyntactic positive shift (SPS) as an ERP measure ofsyntactic processing. Lang. Cogn. Processes 8,439–483.

Hahne, A., and Friederici, A. D. (1999). Electro-physiological evidence for two steps in syntacticanalysis. Early automatic and late controlledprocesses. J. Cogn. Neurosci. 11, 194–205.

Hansen, J. C., and Hillyard, S. A. (1980). Endogenousbrain potentials associated with selective auditoryattention. Elecroencephalogr. Clin. Neurophysiol. 49,277–290.

Hari, R. (1994). Magnetoencephalography in thestudy of human brain functions. In “CognitiveElectrophysiology” (H. J. Heinze, T. F. Münte, andG. R. Mangun, eds.), pp. 368–378. Birkhäuser,Boston.

Hari, R., Reinikainen, K., Kaukoranta, E.,Hamalainen, M., Ilmoniemi, R., Penttinen, A.,Salminen, J., and Teszner, D. (1984).Somatosensory evoked cerebral magnetic fieldsfrom SI and SII in man. Elecroencephalogr. Clin.Neurophysiol. 57, 254–263.

REFERENCES 37


Hari, R., Pelizzone, M., Mäkelä, J. P., Hällström, J.,Leinonen, L., and Lounasamaa, O. V. (1987). Neuro-magnetic responses of the human auditory cortex toon- and off-sets of noise burst. Audiology 25, 31–43.

Harter, M. R., and Aine, C. J. (1984), Brain mecha-nisms of visual selective attention. In “Varieties ofAttention” (R. Parasuraman, and R. Davies, eds.),pp. 293–321. Academic Press, New York.

Heinze, H. J., Mangun, G. R., Burchert, W., Hinrichs,H., Scholze, M., Münte, T. F., Gös, A., Johannes, S.,Scherg, M., Hundeshagen, H., Gazzaniga, M. S.,and Hillyard, S. A. (1994). Combined spatial andtemporal imaging of spatial selective attention inhumans. Nature 372, 543–546 .

Hellige, J. B. (1993). “Hemispheric Asymmetries.What’s Right and What’s Left.” HarvardUniversity Press, Cambridge.

Hillyard, S. A., and Kutas, M. (1983). Electro-physiology of cognitive processing. Annu. Rev.Psychol. 34, 33–61.

Hillyard, S. A., Picton, T. W., and Regan, D. M. (1978).Sensation, perception, and attention: Analysisusing ERPs. In “Event-related Brain Potentials inMan” (E. Callaway, P. Tueting, and S. Koslow,eds.), pp. 223–322. Academic Press, New York.

Hoffman, J. E. (1990). ERP and automatic and controlledprocessing. In “Event-related Brain Potentials. BasicIssues and Applications” ( J. W. Rohrbaugh, R.Parasuraman, and R. Johnson, Jr., eds), pp. 145–156.Oxford University Press, Oxford.

Hudnell, H. K., Boyes, W. K., and Otto D. A. (1990).Stationary pattern adaptation and the early com-ponents in human visual evoked potentials. Electro-encephalogr. Clin. Neurophysiol. 77, 190–193.

Hugdhal, K. (1993). Psychophysiology. “The Mind–Body Perspective”. Harvard University Press,Cambridge, Massachusetts.

Hughes, H. C., Nozawa, G., and Kitterle, F. (1996).Global precedence, spatial frequency channels andthe statistics of natural images. J. Cogn. Neurosci. 8,197–230.

Isreal, J. B, Chesney, G. L., Wickens, C. D., andDonchin, E. (1980). P300 and tracking difficulty:Evidence for multiple resources in dual-task per-formance. Psychophysiology 17, 259–273.

Jeffreys, D. A. (1970). Striate and extra-striate originsof pattern-related visual evoked potential (VEP)components. J. Physiol. 211 (Suppl.), 29.

Jeffreys, D. A. (1971). Cortical source localizations ofpattern related visual evoked potentials recordedfrom the human scalp. Nature 229, 502–504.

Jeffreys, D. A. (1977). The physiological significance ofpattern visual evoked potentials. In “VisualEvoked Potentials in Man” (J. E. Demedt, ed.), pp. 134–167. Clarendon, Oxford.

Kaan, E., Harris, A., Gibson, E., and Holcomb, P.(2000). The P600 as an index of syntactic inte-gration difficulty. Lang. Cogn. Processes 15, 159–201.

Karhu, J., Hari, R., Mäkelä, J. P., Hottunen, J., andKnuutila J. (1982). Somatosensory evoked mag-netic fields in multiple sclerosis. Electroencephalogr.Clin. Neurophysiol. 83, 192–200.

Karlin, L., Mariz, M. J., Brauth, S. E, and Mardkoff, A.M. (1971). Auditory evoked potentials, motorpotentials and reaction times. Electroencephalogr.Clin. Neurophysiol. 31, 129–136.

Katznelson, R. D. (1981). EEG recording, electrodeplacement, and aspects of generator localization.In “Electric Fields of the Brain. The Neurophysicsof EEG” (P. L. Nunez, ed.), pp. 176–213. OxfordUniversity Press, New York and Oxford.

Kleins, M., Coles, M. G. H., and Donchin, E. (1984).People with absolute pitch process tones withoutproducing a P300. Science 224, 1306–1309.

Kristeva, R., Cheyne, D., Lang, W., Lindinger, G., andDeecke L. (1990). Movement-related potentialsaccompanying unilateral and bilateral fingermovements with different inertial loads. EEG Clin.Neurophysiol. 75, 410–418.

Kulikowski, J. J., Murray, I. J., and Perry, N. R. A.(1989). Electrophysiological correlates of chro-matic-opponent and achromatic stimulation inman. Color Vision Defici. 9, 145–153.

Kuriki, S., Takeuchi, F., and Hirata, Y. (1988). Neuralprocessing of words in the human extrastriatevisual cortex. Cogn. Brain Res. 6, 193–203.

Kutas, M., and Dale, A. (1997). Electrical and magneticreading of mental functions. In “CognitiveNeuroscience” (M. D. Rugg, ed.), pp. 197–242.Psychology Press, Taylor & Francis Group, Hove,East Sussex, UK.

Kutas, M., and Hillyard, S. A. (1980). Reading sense-less sentences: Brain potentials reflect semanticincongruity. Science 207, 203–205.

Kutas, M., and Kluender, R. (1994). What is who vio-lating? A reconsideration of linguistic violations inlight of event-related brain potentials. In“Cognitive Electrophysiology” (H. J. Heinze, T. F.Münte, and G. R. Mangun, eds.), pp. 183–210.Birkhäuser, Boston.

Kutas, M., and Van Petten, C. (1988). Event–relatedbrain potentials studies of language. In “Advancesin Psychophysiology” (P. K. Ackles, J. R. Jennings,and M. G. H. Coles, Eds.), Vol. 3, pp. 139–188. JAIPress Inc., Greenwich, Connecticut.

Kutas, M., and Van Petten, C. K. (1994).Psycholinguistics electrified: Event–related brainpotential investigations. In “Handbook ofPsycholinguistics” (M. A. Gernsbacher, ed.), pp. 83–143. Academic Press, San Diego.

Kutas, M., McCarthy, G., and Donchin, E. (1977).Augmenting mental chronometry: The P300 as ameasure for stimulus evaluation time. Science 197,792–795.

Kutas, M., Federmeier, K. D., and Sereno, M. I. (1999).Current approaches to mapping language inelectromagnetic space. In “The Neurocognition of



Language” (C. M. Brown and P. Hagoort, eds.),pp. 359–392. Oxford University Press, Oxford.

Lesèvre, N. (1982). Chronotopographical analysis ofthe human evoked potential in relation to visualfield data from normal individuals and hemian-optic patients. Ann. N.Y. Acad. Sci.”, 388, 635–641.

Lesevre, N., and Joseph, J. P. (1979). Modifications ofthe pattern-evoked potential (PEP) in relation tothe stimulated part of the visual field (clues for themost probable origin of each component).Electroencephalogr. Clin. Neurophysiol. 47, 183–203.

Livingstone, M. S., and Hubel D. H. (1988). Segregationof color, form, movement and depth: Anatomy,physiology and perception. Science 240, 740–749.

Magliero, A., Bashore, T. R., Coles, M. G. H., andDonchin, E. (1984). On the dependence of P300latency on stimulus evaluation processes. Psycho-physiology 21, 171–186.

Mangun, G.R., Hinrichs, H., Scholz, M., Mueller-Gaertner, H.W., Herzog, H., Krause, B.J., Tellman,L., Kemna, L., and Heinze H. J. (2001). Integratingelectrophysiology and neuroimaging of spatialselective attention to simple isolated visualstimuli. Vision Res. 41, 1423–1435.

McCandles, G. A. (1987). Electrophysiological mea-surements in the assessment of the young child. In“Childhood Deafness: Causation, Assessment andManagement” (F. H. Bess, ed.), pp. 161–169. Grune& Stratton, New York, San Francisco, London.

McCarthy, G., and Donchin. F. (1981). A metric forthought. A comparison of P300 latency and reac-tion time. Science 211, 77–80.

Mecklinger, A., Schriefers, H., Steinhauer, K., andFriederici A. D. (1995). Processing relative clausesvarying on synctactic and semantic dimensions:An analysis with event-related potentials. Memoryand Cogn. 23, 477–494.

Münte, T. F., Heinze, H. J., Matzke, M., Wieringa, B. M.,and Johannes, S. (1998). Brain potentials andsyntactic violations revisited: No evidence forspecificity of the syntactic positive shift.Neuropsychologia 36, 217–226.

Näätänen, R. (1982). Processing negativity: Anevoked-potential reflection of selective attention.Psychol. Bull. 92, 605–640.

Näätänen, R. (1992). “Attention and Brain Function.”Erlbaum, London.

Näätänen, R. (1995). The mismatch negativity—Apowerful tool for cognitive neuroscience. EarHearing 16, 6–18.

Näätänen, R., and Michie, P. T. (1979). Early selectiveattention effects on the evoked potentials. A criticalreview and reinterpretation. Biol. Psychol. 8, 81–136.

Näätänen, R., Gaillard, A. W. K., and Mantisalo, S.(1978). Early selective-attention effect reinter-preted. Acta Psychol. 42, 313–329.

Näätänen, R., Alho, K., and Sams, M. (1985). Selectiveinformation processing and event-related brainpotentials. In “Psychophysiological Approaches

to Human Information Processing” (F. Klix, R.Näätänen, and R. Zimmer, eds.), pp. 49–83. Elsevier,Amsterdam.

Neville, H., Snyder, E., Woods, D., and Galambos, R.(1982). Recognition and surprise alter the humanvisual evoked response. Proc. Natl. Acad. Sci.U.S.A. 79, 2121–2123.

Nielsen-Bohlman, L., and Knight, R. (1994). Electro-physiological dissociation of rapid memory mecha-nisms in humans. Neuroreport 1221, 1517–1521.

Nunez, P. L., Brett, M. W., and Silberstein, R. B. (2001).Spatial-temporal structures of human alpharhythms: Theory, microcurrent sources, multiscalemeasurements, and global binding of local net-works. Human Brain Mapping 13, 125–164.

Okada, Y. C., Kaufman, C., Brenner, D., andWilliamson, S. S. (1982). Modulation transfer func-tions of the human visual system revealed by mag-netic field measurements, Vision Res. 22, 319–333.

Ossenblock, P., and Spekreijse, H. (1991). The extra-striate generators of the EP to checkerboard onset.A source localization approach. Electroencephalogr.Clin. Neurophysiol. 80, 181– 193.

Osterhout, L. (1994). Event-related brain potentials astools for comprehending language comprehension.In “Perspectives on Sentence Processing” (J. CharlesClifton, L. Frazier, and K. Rayner, eds.), pp. 15–44.Lawrence Erlbaum Assoc., Hillsdale, New Jersey.

Osterhout, L., and Holcomb, P. J. (1992). Event-relatedbrain potentials elicited by syntactic anomaly. J. Memory Lang. 31(6), 785–806.

Osterhout, L., and Holcomb, P. J. (1993). Event-relatedpotentials and syntactic anomaly: Evidence ofanomaly detection during the perception of continuous speech. Lang. Cogn. Processes 8, 413–437.

Osterhout, L., and Holcomb, P. J. (1995). Event relatedpotentials and language comprehension. In “Elec-trophysiology of Mind: Event-Related Brain Poten-tials and Cognition” (M. D. Rugg, and M. G. H.Coles, eds.), Vol. 25, pp. 171–215. Oxford UniversityPress, Oxford.

Papanicolau, A. C., and Johnston, J. (1984). Probeevoked potentials: Theory, method and applica-tions. Int. J. Neurosci. 24, 107–131.

Paulus, W. M., and Plendl, H. (1988). Spatial dissocia-tion of early and late colour evoked components.Electroencephalogr. Clin. Neurophysiol. 71, 81–88.

Peters, M. J., and De Munck, J. C. (1990). On theforward and the inverse problem for EEG andMEG. In “Auditory Evoked Magnetic Fields andElectric Potentials, Advances in Audiology” (F. Grandori, M. Hoke, and G. L. Romani, eds.),Vol. 6, pp. 70–102. Karger, Basel.

Petersen, S. E., Fox, P. T., Posner, M. I., Mintun, M., andRaichle M. E. (1988). Positron emission tomographystudies of the processing of single words. J. Cogn.Neurosci. 1, 153–170.

Picton, T. W., and Hillyard, S. A. (1988). Endogenousevent-related potentials. In “Handbook of Electro-

REFERENCES 39


encephalography and Clinical Neurophysiology” (T. W. Picton, ed.), Vol. 3, pp. 361–426. Elsevier,Amsterdam.

Picton, T. W., Hillyard, S. A., Krautsz, H. I., andGalambos, R. (1974). Human auditory evokedpotentials. I. Evaluation of components. Electro-encephalogr. Clin. Neurophysiol. 36, 179–190.

Pilgreen, K. L. (1995). Physiological, medical, and cog-nitive correlates of electroencephalography. In“Neocortical Dynamics and Human EEGRhythms” (P. L. Nunez, ed.), pp. 195–248. OxfordUniversity Press, New York and Oxford.

Polk, T. A., Stallcup, M., Aguirre, G. K., Alsop, D. C.,D’Esposito, M., Detre, A. J., and Farah M. J. (2002).Neural specialization for letter recognition. J. Cogn.Neurosci. 14,145–159.

Previc, F. H. (1988). The neurophysiological significanceof the N1 and P1 components of the visual evokedpotential. Clin. Visual Sci. 3, 195–202.

Proverbio, A. M. (1993). The role of left and righthemisphere in sustained and selective attention.Ph.D. Doctoral Thesis. University of Padua.

Proverbio, A. M., Zani, A., and Avella, C. (1996).Differential activation of multiple current sourcesof foveal VEPs as a function of spatial frequency.BrainTopogr. 9, 59–69.

Proverbio, A. M., Zani, A., and Avella, C. (1997a).Hemispheric asymmetries for spatial frequencydiscrimination in a selective attention task. BrainCogn. 34, 311–320.

Proverbio, A. M., Lilli, S., Zani, A., and Semenza, C.(1997b). Neural basis of common vs. proper nameretrieval: An electrophysiological investigation,Brain Lang. 60, 31–33.

Proverbio, A. M., Minniti, A., and Zani, A. (1998).Electrophysiological evidence of a perceptualprecedence of global vs. local visual information.Cogn. Brain Res. 6, 321–334.

Proverbio, A. M., Lilli, S., and Zani, A. (1999). ERPmapping of brain activation during phono-logical processing. Biomed. Technik 44, Suppl. 2,178–180.

Proverbio, A. M., Cok B., and Zani A. (2002a).Electrophysiological measures of language processing in bilinguals. J. Cogn. Neurosci. (inpress).

Proverbio, A. M., Esposito, P., and Zani, A. (2002b).Early involvement of temporal area in attentionalselection of grating orientation: An ERP study.Cogn. Brain Res. 13, 139–151.

Puce, A., Allison, T., Asgari, M., Gore, J. C., andMcCarthy, G. (1996). Differential sensitivity ofhuman visual cortex to faces, letter strings, andtextures: A functional magnetic resonance imagingstudy. J. Neurosci. 16, 5205–5215.

Rif, J., Hari, R., Hämäläinen, M., and Sams, S. A.(1991). Auditory attention affects two diffrentareas in the human auditory cortex. Electro-encephalogr. Clin. Neurophysiol. 79, 464–472.

Romani, G. L., Williamson, S. S., and Kaufman, L.(1982). Tonotopic organization of the human audi-tory cortex. Science 216, 1339–1340.

Rösler, F., and Manzey, D. (1986). Automatization ofcognitive operations as reflected in event-relatedbrain potentials: Methodological considerations anddata. In “Human Memory and Cognitive Capa-bilities. Mechanisms and Performances” (F. Klix,and H. Hagendorf , eds.), pp. 659–673. Elseviers,Amsterdam.

Roth, W., Ford, J. M., and Kopell, B. S. (1978). Long-latency evoked potentials and reaction times.Psychophysiology 15, 17–23.

Ruchkin, D. S., Sutton, S., and Mahaffey, D. (1987).Functional differences between members of theP300 complex: P3a and P3b. Psychophysiology 24,87–103.

Ruchkin, D. S., Johnson, R., Jr., Mahaffey, D., andSutton, S. (1988). Toward a functional categoriza-tion of slow waves. Psychophysiology 25, 339–353.

Scherg, M. (1992). Functional imaging and localizationof electromagnetic brain activity, Brain Topogr. 5,103–111.

Shulman-Galambos, C., and Galambos, R. (1978).Cortical responses from adults and infants incomplex visual stimuli. Electroencephalogr. Clin.Neurophysiol. 45, 425–35.

Silberstein, R. B. (1995a). Neuromodulation ofneocortical dynamics. In “Neocortical Dynamicsand Human EEG rhythms” (P. Nunez, ed.),pp. 591–627. Oxford University Press, New York.

Silberstein, R. B. (1995b). Steady-state visually evokedpotentials, brain resonances, and cognitiveprocesses. In “Neocortical Dynamics and HumanEEG Rhythms” (P. Nunez, ed.), pp. 272–303.Oxford University Press, New York.

Skrandies, W. (1984). Scalp potential fields evoked bygrating stimuli: Effects of spatial frequency andorientation. Electroencephalogr. Clin. Neurophysiol.58, 325–332.

Van Petten, C. (1995). Words and sentences: Event-related brain potential measures. Psychophysiology32, 511–525.

Woldorff, M. G., Gallen, C. C., Hampson, S. A.,Hillyard, S. A., Pantev, C., Sobel, D., and Bloom, C.G. (1993). Modulation of early processing inhuman auditory cortex during auditory selectiveattention. Proc. Natl. Acad. Sci. 90, 8722–8726.

Zani, A. (1989). Brain evoked responses reflect infor-mation processing changes with the menstrualcycle in young female athletes. J. Sport Med.Physical Fitness 29,113–121.

Zani, A., and Proverbio, A. M. (1995). ERP signs ofearly selective attention effects to check size.Electroencephalogr. Clin. Neurophysiol. 95, 277–292.

Zani, A., and Proverbio, A. M. (1997). Attention mod-ulation of short latency ERPs by selective attentionto conjunction of spatial frequency and location. J. Psychophysiol. 11, 21–32.




C H A P T E R

3

Seeing the Forest through the Trees:

The Cross-Function Approachto Imaging Cognition

Roberto Cabeza and Lars Nyberg

INTRODUCTION

During the past decade, the field offunctional neuroimaging of cognition hasgrown exponentially. From a handful ofstudies in the early 1990s, this researchdomain expanded to more than 800 studiesby the early 2000s. Today, positron emis-sion tomography (PET) and functionalMRI (fMRI) studies cover almost everyaspect of human cognition, from motionperception to moral reasoning. If eachstudy is seen as a tree, the field has grownfrom minimal vegetation to a luxurianttropical forest in less than 10 years. Yet,functional neuroimaging researchers some-times focus exclusively on their own cog-nitive domain and do not see the forestthrough the trees. The goal of the presentchapter is to call attention to the forest—that is, to what many functional neuro-imaging studies of cognition have incommon.

When we say that most researchers arefocused on the trees, we refer to the factthat the vast majority of functional neuro-imaging studies investigate a single cog-nitive function, such as attention, workingmemory, or episodic memory. Yet, with the

accumulation of functional neuroimagingdata, it has become obvious that the brainis not organized like a cognitive psychol-ogy textbook, with dedicated systems forperception, attention, working memory,episodic memory, and so forth. Instead, theneural correlates of cognitive functionsoverlap considerably, with each brainregion being involved in a variety of cogni-tive functions. What cognitive processes dothese common regions mediate? By com-paring patterns of brain activity across dif-ferent cognitive functions, answers to thisquestion can be generated.

The matrix in Fig. 1 illustrates the dif-ference between the traditional within-function approach and the cross-functionapproach we are advocating in this chap-ter. Let us assume that in functional neuro-imaging studies Cognitive Function Atypically is associated with activations inBrain Regions 1 and 3, Cognitive FunctionB with activations in Brain Regions 2 and 3,and Cognitive Function C with activationsin Brain Regions 1 and 2. In the standardwithin-function approach, functional neuro-imaging researchers are primarily con-cerned with one cognitive function andinterpret activations in relation to this par-

ticular function. Thus, in a situation suchas the one depicted in Fig. 1, researchers ofFunction A would attribute the activationof Region 1 to a certain aspect of FunctionA, whereas researchers of Function Cwould attribute the activation of the sameregion to a certain aspect of Function C.For instance, left ventrolateral prefrontalcortex activations have been attributed to language processes by languageresearchers, to working memory processesby working memory researchers, to seman-tic memory processes by semantic memoryresearchers, and so forth (Cabeza andNyberg, 2000). In contrast with the within-function approach, the cross-functionapproach focuses on the columns of thematrix rather than on the rows, and asksquestions about the functional role of abrain region (e.g., Region 1) that isrecruited by different cognitive functions(e.g., Functions A and C).

Thus, the basic question the cross-function approach asks concerns why thesame brain region is recruited by differentcognitive functions. There are at least threepossible answers to this question. First,according to a sharing view, the commonregion is involved in cognitive operationsthat are recruited by different cognitivefunctions. In the case of Fig. 1, the sharingview would argue that Region 1 mediatesprocesses that are engaged both byFunction A and by Function C. A reduction-

istic interpretation of the sharing viewwould say that shared operations “belong”to one of the two functions, and are “bor-rowed” by the other function. For example,prefrontal cortex (PFC) regions activatedboth by episodic memory and by workingmemory could mediate working memoryprocesses that are also tapped by episodicmemory (e.g., Wagner, et al., 1998a). In con-trast, an abstractive interpretation of thesharing view would argue that the sharedprocesses should be described in moreabstract terms than either of the two func-tions. For instance, dorsolateral PFCregions common to episodic and workingmemory could reflect general monitoringoperations that were tapped by both func-tions (e.g., Cabeza et al., 2002a).

Second, according to a subdivision view,when different functions activate the sameregion, the region actually consists ofseveral subregions that are differentiallyinvolved in each of the functions. In thecase of Fig. 1, the subdivision view wouldargue that Function A and Function C acti-vate different subregions of Region 1 (e.g.,Subregion 1a and Subregion 1c). From thispoint of view, the goal of the cross-functionapproach would not be to identify acommon process shared by different func-tions, but to dissociate the functions ofeach subregion by increasing the spatialresolution of functional neuroimagingtechniques and/or the specificity of experi-mental manipulations. Much of the func-tional neuroimaging work on visualrecognition favors the subdivision view,and researchers in this area have identifiedsubregions on the ventral surface of thetemporal lobes that are specialized in rec-ognizing faces, places, animals, and tools(for reviews, see Kanwisher et al., 2001;Martin, 2001). It should be noted, however,that the subdivision view does not neces-sarily imply modularity, because one mayargue that a region can be divided intosubregions without assuming that thesesubregions have the properties typicallyattributed to neurocognitive modules, such

42 3. NEUROIMAGING ACROSS FUNCTIONS


FIGURE 1 Illustration of within-function andcross-function approaches to functional neuro-imaging.

as domain specificity, cognitive impenetra-bility, shallow output, etc. (Fodor, 1983).

Third, according to a network view, thefinding that the same region is activated byseveral functions does not imply that thecommon region performs the same cogni-tive operations in all the functions. On thecontrary, this view assumes that the cogni-tive operations performed by a brainregion depend on the interactions betweenthe region and the rest of the brain, andbecause these interactions change acrossfunctions, so do the operations performedby the region (McIntosh, 1999; Nyberg andMcIntosh, 2000). In the case of Fig. 1, thenetwork view would argue that Region 1performs different operations in FunctionsA and C, because during Function A itinteracts with Region 3, whereas duringFunction C it interacts with Region 2. Anextreme version of the network viewwould say that—with the exception ofprimary sensory and motor cortices—brainregions are not specialized in particularcognitive process, and their operations arecompletely determined by network inter-actions. A moderate version of the networkview would state that there is a broad spe-cialization, but the specific operations per-formed by a region are determined bynetwork interactions. For example, Region1 may have a broad specialization in fastonline computations, but these computa-tions may be applied to speech processingduring a language task or to rotating anobject during an imagery task. It should benoted that the idea that brain regions havebroad specializations is close to theabstractive interpretation of the sharingview.

Regardless of what view one endorses—the three views are not incompatible andmay be combined in different ways—itseems clear that in order to understandwhy the same brain regions are activatedby a variety of cognitive functions onemust go beyond the standard within-function approach and adopt a cross-function approach. The cross-function

approach has two basic methods, and bothare described in the present chapter. Onemethod is to conduct a metaanalysis com-bining the results of studies that originallyinvestigated a single cognitive function(Cabeza and Nyberg, 2000; Christoff andGabrieli, 2000; Duncan and Owen, 2000;Fletcher and Henson, 2001). Anothermethod is to conduct functional neu-roimaging studies that compare differentcognitive functions directly within-subjects(Braver et al., 2001; Cabeza et al., 2002a,b;LaBar et al., 1999; Nyberg et al., 2002a,b;Ranganath and D’Esposito, 2001). Thepresent chapter describes both methods:first, we report a cross-study metaanalysis,and then we review the results of the firstcrop of studies that compare differentfunctions within-subject.

COMPARING DIFFERENTCOGNITIVE FUNCTIONS

ACROSS STUDIES

This section reports a metaanalysis offunctional neuroimaging data. First, wedescribe the methods of the metaanalysis,including the characteristics of the data set,the rationale for the classificationemployed, and the calculation of activa-tion frequency. Then, we describe anddiscuss the results of the metaanalysis forprefrontal, midline, parietal, temporal, andmedial temporal regions.

Methods

From the data set of a previous large-scale metaanalysis (Cabeza and Nyberg,2000), we selected 136 studies in five cog-nitive domains: (1) attention, (2) percep-tion, (3) working memory, (4) semanticmemory retrieval and episodic memoryencoding, and (5) episodic memoryretrieval. The rationale for consideringsemantic memory retrieval and episodicmemory encoding within the same cate-gory is that these two processes tend to

COMPARING DIFFERENT COGNITIVE FUNCTIONS ACROSS STUDIES 43


co-occur and are very difficult to differenti-ate: semantic retrieval involves incidentalepisodic encoding, and intentional episodicencoding involves incidental semanticretrieval (for a discussion, see Nyberg et al.,1996a; Tulving et al., 1994). Consistent withthis idea, semantic retrieval and episodicencoding tend to show very similar activa-tion patterns, except in medial temporallobe (MTL) regions (see below).

As Table 1 shows, for each cognitivedomain, we classified activations accordingto whether the study involved verbal,object, or spatial stimuli. There were slightdifferences in the types of stimuli used ineach domain, however. For verbal stimuli,the Working Memory domain includedboth words and numerical stimuli. In theAttention domain, studies in the verbal cat-egory used the Stroop task, and hence arecharacterized not only by their verbalnature but by the conflict-monitoring oper-ations tapped by this task as well. In thePerception domain, the verbal categorycorresponded to word-reading studies inwhich overt verbal responses were absentor subtracted out by the control task.Differences in the spatial category existedas well. The memory conditions alsoincluded imagery operations and thesemantic retrieval category included letterrotation studies.

As in our previous metaanalysis(Cabeza and Nyberg, 2000), we used

Brodmann areas (BAs) as the unit of analy-sis (see Fig. 2). Some BAs (e.g., 3/1/2, 4, 5,33, 43, 38) were excluded because theyshowed very few activations, and someareas were collapsed together (BAs 41 and42; BAs 30 and 31) to simplify analyses.Medial temporal lobe regions were treatedas a unit because the overall area is smalland the localization of activations in differ-ent regions is not always clear. Due to



FIGURE 2 Brodmann areas. From Elliott (1964).“Textbook of Neuroanatomy,” with permission ofLippincott Williams & Wilkins.

TABLE 1 Number of Activations (Reported Peaks) in 136 PET/fMRI Studies According to CognitiveFunction and Stimulus Involved

Stimulus

Function Verbal Object Spatial Total

Attention 37 60 83 180

Perception 106 109 27 242

Working memory 238 85 177 500

Semantic retrieval/episodic encoding 171 124 60 355

Episodic retrieval 216 49 45 310

Total 768 427 392 1587

space limitations, we do not report ordiscuss the results for occipital regions,basal ganglia, thalamic, and cerebellarregions.

In our previous metaanalysis (Cabezaand Nyberg, 2000), we identified typicalactivation patterns for each cognitive func-tion using a qualitative evaluation of thefrequency of activations across studies.Although we emphasized representativeactivations, we also noted exceptions to thegeneral pattern and discussed specificfindings and studies. By contrast, in thepresent metaanalysis, we used a quantita-tive measure of activation frequency andfocused on aggregate results without dis-cussion of individual studies. The quanti-tative measure employed is the number ofactivations reported in each BA or brain

region in the studies reviewed. The num-ber of activations is defined by the numberof different coordinates reported, withsome studies reporting more than onecoordinate in each BA. The rationale forusing this measure is that the numbers ofcoordinates reported tended to be posi-tively correlated with the size of the acti-vations, and hence, this measure providesan indirect index of both frequency andrelative size of the activations. Thenumbers of activations in each brainregion were separately analyzed for eachcognitive function and stimulus type (see Figs. 2–7). Because the total number of activations varied across cognitivedomains and stimulus type, we express thenumber of activations as a percentage ofthe number of activations in each cell in



FIGURE 3 Percentage of activations in anterior and dorsolateral prefrontal regions in five cognitive domains.BA, Brodmann area; V, verbal; O, object; S, spatial.

Table 1. Although using percentages pro-vides a straightforward method to com-pare across category/stimuli cells, thesepercentages should be considered withcaution in the case of cells with few activa-tions (e.g., spatial perception), because theymay reflect the peculiarities of the fewstudies in the cell rather than a generalpattern.

Results

Prefrontal Regions

The distribution of activations in theprefrontal cortex for the five cognitivefunctions is shown in Figs. 3 and 4. The lat-

eralization of activation for semanticretrieval/episodic encoding and forepisodic retrieval was consistent with thehemispheric encoding/retrieval asymme-try (HERA) model (Nyberg et al., 1996a,1998; Tulving et al., 1994), which postulatedthat the left PFC is differentially moreinvolved in retrieving information fromsemantic memory and in simultaneouslyencoding novel aspects of this informationinto episodic memory, whereas the rightPFC is differentially more involved inretrieving information from episodicmemory. At the same time, the lateraliza-tion of PFC activity was also affected bystimulus type: right-lateralized activations



FIGURE 4 Percentage of activations in ventrolateral and posterior prefrontal regions in five cognitivedomains. BA, Brodmann area; V, verbal; O, object; S, spatial.

during semantic retrieval/episodic encod-ing usually occurred for nonverbal materi-als and left-lateralized activations duringepisodic retrieval usually occurred forverbal materials. Thus, the lateralization ofPFC activity depends both on processes(Nyberg et al., 1996a, 1998) and stimuli(e.g., Kelley et al., 1998; McDermot et al.,1999; Wagner et al., 1998b).

Turning to the activation pattern for dif-ferent PFC subregions, Fig. 3 shows the fre-quency of activation for anterior (BA 10)and dorsolateral (BAs 9 and 46) PFCregions. Anterior PFC (BA 10) activationswere frequent for episodic memoryretrieval, particularly in the right hemi-sphere. Frontopolar activity during epi-sodic retrieval has been attributed to thegeneration and maintenance of the mentalset of episodic retrieval, or episodic retrievalmode (Cabeza et al., 1997; Düzel et al., 1999;Lepage et al., 2000; Nyberg et al., 1995).This idea is supported by evidence thatactivity in BA 10 remains constant acrossdifferent levels of episodic retrievalperformance (Nyberg et al., 1995) and dif-ferent types of episodic retrieval tasks(Cabeza et al., 1997) and is sustainedthroughout the retrieval task (Düzel et al.,1999). Although not shown in Fig. 3, acti-vations in BA 10 are also frequent duringproblem solving (Cabeza and Nyberg,2000), suggesting that this region has amore general role of monitoring internallygenerated information (Christoff andGabrieli, 2000). Dorsolateral PFC (BAs 9and 46) activations were frequent forworking memory, semantic retrieval/episodic encoding, and episodic retrieval(see Fig. 3). A sharing account of thisoverlap would be that semanticretrieval/episodic encoding and episodicretrieval depend on working memory(reductionistic interpretation) or that thethree functions depend on monitoringoperations (abstractive interpretation).However, sharing accounts cannot easilyexplain why the lateralization of dorsolat-eral PFC changes across functions, tending

to be bilateral for working memory, left lat-eralized for semantic retrieval/episodicencoding, and right lateralized for episodicretrieval (see Fig. 3). The subdivision viewcould argue that this different lateral-ization pattern reflects the existence of dif-ferent subregions within dorsolateral PFC,whereas the network view could arguethat the function of dorsolateral PFC, aswell as its lateralization, is determined byits interactions with other brain regions(e.g., parietal cortex in for workingmemory, left temporal cortex for semanticretrieval/episodic encoding).

Figure 4 shows the frequency of activa-tions in ventrolateral (BAs 47 and 45),ventral/posterior (BA 44), and posterior(BA 6) PFC regions. BA 47 activations werelateralized similarly to dorsolateral PFC,but unlike dorsolateral PFC activations,they were also frequent not only formemory domains but also for the attentiondomain. This pattern suggests that BA 47is involved in processes that are shared byworking, semantic, and episodic memoryas well as by attention tasks. In contrast,BA 45 seems to be more specific to seman-tic retrieval/episodic encoding, particu-larly in the left hemisphere and for verbalstimuli, consistent with the idea that thisregion is involved in semantic processing(Gabrieli et al., 1998; Poldrack et al., 1999).Like BAs 47 and 45, BA 44 also shows left-lateralized activation during semanticretrieval/episodic encoding, but unlikeBAs 47 and 45, left BA 44 is often activatedduring verbal working memory tasks. Thispattern is in keeping with the notion thatleft BA 44, which overlaps with Broca’sarea, is involved in phonological mainte-nance and rehearsal (for a review, seeSmith and Jonides, 1999). Thus, the presentresults are consistent with the notion thatthe anterior part of the left inferior frontalgyrus (BAs 47 and 45) is involved insemantic processing, whereas the posteriorpart (BA 44) is involved in phonologicalworking memory (Kapur et al., 1996;Poldrack et al., 1999). Yet, left BA 44 activa-



tions have also been found in workingmemory studies that employed faces andmeaningless shapes (Cabeza and Nyberg,2000), suggesting that the function of thisregion is not strictly verbal. The role ofright BA 44—the homolog of Broca’s areain the right hemisphere—is also unclear.Finally, the distribution of activations inposterior PFC (BA 6) looks quite dif-ferent than the ones previously described.Activation overlaps in BA 6 are difficult tointerpret because this is a very largeBrodmann area that probably comprisestwo or more different functional subre-gions. The inferior part of BA 6 is close toBroca’s area, and some of the left BA 6activations during verbal working memoryand semantic retrieval may reflect phono-logical rehearsal. In contrast, more dorsalparts of BA 6 may be more related to atten-tional and working memory processes.

Midline Regions

Figure 5 shows the frequency of activa-tions in midline regions, including the ante-rior cingulate cortex (BAs 32 and 24) andthe precuneus (BA 31). Central cingulateactivations are not depicted because theywere scarce, but they show a pattern similarto those in the anterior cingulate. Anteriorcingulate activations were frequent duringattention (e.g., Stroop tasks), workingmemory, and episodic retrieval. The role ofthe anterior cingulate in cognition has beenattributed to initiation of action (Posner andPetersen, 1990) and to conflict monitoring(for a review, see Botvinick et al., 2001),among other processes (for a review, seeDevinsky et al., 1995). The initiation-of-action hypothesis accounts well for activa-tions during demanding cognitive tasks,such as working memory and episodicretrieval, whereas the conflict-monitoring



FIGURE 5 Percentage of activations in midline regions in five cognitive domains. BA, Brodmann area; V,verbal; O, object; S, spatial.

hypothesis provides a better account foranterior cingulate activations during Strooptasks (see verbal attention in Fig. 5).Obviously, these two views are not incom-patible: the anterior cingulate cortex couldboth initiate appropriate responses and sup-press inappropriate ones (Paus et al., 1993).Given the heterogeneous structure andcomplex connectivity of the anterior cingu-late (Devinsky et al., 1995), it is quite possi-ble that different processes are tappeddepending on the particular subregionengaged [subdivision view, e.g., Bush et al.,(2002)] and its interactions with the rest ofthe brain (network view).

Precuneus activations in BA 31 showeda very different functional pattern com-pared to those in the anterior cingulatecortex. They were not frequent duringStroop or object working memory tasks,

and were more specifically associated withthe processing of spatial information inworking memory, semantic retrieval/episodic encoding, and episodic retrievaltasks. The association between the pre-cuneus region and memory for spatialstimuli fits well with the idea that thisregion is involved in imagery (Fletcher et al., 1995; Shallice et al., 1994), althoughevidence against this hypothesis has beenreported (Buckner et al., 1996; Krause et al.,1999). The present results also link the pre-cuneus to the processing of spatial infor-mation, which is a link that has beendiscussed relatively little in the functionalneuroimaging literature.

Parietal Regions

Figure 6 shows the frequency of activa-tions in parietal regions (BAs 7, 40, and 39).



FIGURE 6 Percentage of activations in parietal regions in five cognitive domains. BA, Brodmann area; V, verbal; O, object; S, spatial.

BA 7 activations were frequent dur-ing attention, perception, and workingmemory, and also common during se-mantic retrieval/episodic encoding tasksinvolving spatial stimuli. This pattern fitswell with the idea that this region is part ofa dorsal pathway involved in spatial per-ception (Ungerleider and Mishkin, 1982).However, BA 7 is also activated by verbalattention, working memory, and episodicretrieval studies, and it is unclear how thespatial-processing hypothesis can accountfor these activations. Also, the spatial-processing hypothesis cannot easilyaccount for activations in BA 40, which arefrequent for attention, perception, andworking memory, regardless of stimuli.Working memory activations in left BA 40have been attributed to the storage ofverbal information in working memory(Awh et al., 1996; for a review, seeD’Esposito, 2001; Jonides et al., 1998;Paulesu et al., 1993). The involvement ofthis region in spatial semantic retrievalcould be related to a verbal component insome of these studies [e.g., letter rotation(Alivisatos, 1997); encoding the location ofnameable objects (Owen, 1996)]. Finally, BA39 activations seem to reflect both a spatialand verbal processing component. Left BA39 activations in verbal perception studiesmay reflect the link between the leftangular gyrus and graphemic/phonologi-cal processing (for a review, see Binder andPrice, 2001).

Temporal Regions

Figure 7 shows the frequency of activa-tions in lateral temporal regions: BAs 22,21, 20, and 37. Activations in left BA 22were much more frequent for verbal thanfor object and spatial stimuli, suggestingthey are primarily associated with lan-guage processing. Left BA 22 overlaps withWernicke’s area, which has been stronglylinked to language comprehension inresearch with aphasic patients (Benson,1988). In contrast, left BA 21 activationswere frequent not only for verbal but

also for object stimuli. From a sharing–-abstractive point of view, this pattern sug-gests that BA 21 is involved in a processcommon to processing verbal and objectstimuli, such as meaning-based analyses.Finally, consistent with its location alongthe ventral pathway for object processing(Ungerleider and Mishkin, 1982), activa-tions in BAs 20 and 37 were more frequentwhen objects were used as stimuli. Thus,the distribution of activations over thelateral surface of the left temporal lobes,from the superior to the inferior temporalgyri, can be described as a gradient fromverbal to object processing, with moreabstract (stimulus-independent) semanticprocessing in the middle.

Medial Temporal Lobes

Finally, Fig. 8 shows the frequency ofactivations in the medial temporal lobes.These activations were frequent duringepisodic memory encoding and retrieval,consistent with the evidence that lesions inthis area impair episodic memory func-tions (for a review, see Squire, 1992). Therole of the MTL in episodic memory hasbeen attributed to binding (Henke et al.,1997; Lepage et al., 2000) and noveltydetection (Tulving et al., 1996) duringencoding, and to trace recovery (Cabezaet al., 2001; Nyberg, et al., 1996b) and recol-lection (Eldridge et al., 2000; Schacter et al.,1996a) during retrieval (for a review, seeCohen et al., 1999). The MTL has been alsolinked to spatial processing (Maguire et al.,1998; O’Keefe and Nadel, 1978), but thisidea cannot be easily explain the relativelack of MTL activations during spatialattention, spatial perception, and spatialworking memory. In contrast, MTL activa-tions during attention and perception taskswere frequent when object stimuli wereemployed (see Fig. 8). This pattern fits wellwith the notion that the MTL is an anteriorpart of the ventral pathway for objectprocessing. It is unclear if the episodicmemory and object-processing roles ofMTL can be subsumed under a more





FIGURE 7 Percentage of activations in temporal regions in five cognitive domains. BA, Brodmann area; V,verbal; O, object; S, spatial.

FIGURE 8 Percentage of activations in medial temporal lobe regions in five cognitive domains. BA, Brodmannarea; V, verbal; O, object; S, spatial.

general function (sharing view), whetherthey involve different subregions of MTL(subdivision view), or whether they reflectdifferent interactions between MTL and therest of the brain (network view).

Summary

Table 2 summarizes the results previ-ously displayed in Figs. 3–8. For each brainregion (e.g., BA 10), the presence of asymbol indicates that the frequency of acti-vations for a stimuli/function combination(e.g., verbal episodic retrieval) was abovethe average for the region, and the size ofthe symbol indicates whether the frequencywas slightly above average, high, or veryhigh. The symbols indicate how activationsare lateralized (black symbol, right lateral-ized; white symbol, left lateralized; rosette,bilateral; diamonds, midline) when the fre-quency in one hemisphere was at least twicethe frequency in the other hemisphere.

As illustrated by Table 2, anterior PFCactivations in BA 10 were particularly fre-quent during episodic memory retrievaltasks. These activations have been attributedto episodic retrieval mode (Cabeza et al., 1997; Düzel et al., 1999; Lepage et al.,2000; Nyberg et al., 1995), but the involve-ment of anterior PFC in problem solving(Cabeza and Nyberg, 2000) suggests thisregion has a general role in monitoring inter-nally generated information (Christoff andGabrieli, 2000). Dorsolateral PFC activationsin BAs 9 and 46 were very frequent duringworking memory, semantic retrieval/episodic encoding, and episodic retrievaltasks, possibly reflecting working memory(Wagner, 1999) or monitoring (Cabeza et al.,2002a) operations. Ventrolateral PFC activa-tions in BA 47 were also common duringattention tasks. In the left hemisphere, BA 45activations were particularly frequent dur-ing verbal semantic retrieval/episodic en-coding tasks, and BA 44 activations werefrequent during verbal working memorytasks. This pattern is consistent with the ideathat the anterior part of the left inferior

frontal gyrus is involved in semantic pro-cessing and the posterior part (i.e., Broca’sarea) is involved in phonological rehearsal(Kapur et al., 1996; Poldrack et al., 1999).Finally, posterior PFC activations in BA 6were most frequent during attention andworking memory tasks.

Midline activations include those inanterior cingulate (BAs 32 and 24) and pre-cuneus (BA 31) regions. Anterior cingulateactivations were common for all functions,but—consistent with the conflict-monitor-ing hypothesis (for a review, see Botvinicket al., 2001)—they were particularly fre-quent for Stroop tasks (verbal attentioncategory). In contrast, precuneus activa-tions were especially frequent for spatialmemory tasks. Although the precuneus hasbeen associated with imagery (Fletcheret al., 1995; Shallice et al., 1994), a specificlink with spatial processing has been dis-cussed very little in the literature.

Table 2 also shows the frequency of acti-vation in parietal, temporal, and MTLregions. Parietal activations in BA 7 werecommon for all functions, and—consistentwith the ventral/dorsal pathway distinction(Ungerleider and Mishkin, 1982)—they wereoften found during spatial tasks. In contrast,parietal activations in BA 40 were particu-larly frequent during attention and workingmemory tasks regardless of the stimuli.BA 39 activations were often found in theleft hemisphere during verbal perception(reading) and verbal semantic retrieval/episodic encoding tasks, possibly reflectingthe role of the left angular gyrus in languageprocessing (for a review, see Binder andPrice, 2001). Temporal activations in the lefthemisphere can be described as a gradientfrom verbal processing in the superior tem-poral gyrus (BA 21) to object processing inthe inferior temporal gyrus (BAs 20 and 37),with more abstract semantic processing inthe middle (BA 22). Finally, MTL activationswere common during long-term memoryand episodic and semantic tasks, consistentwith the involvement of the MTL in declara-tive memory (Squire, 1992). They were also



associated with object perception and atten-tion, suggesting that the MTL is the anteriorpart of the ventral pathway for object pro-cessing (Ungerleider and Mishkin, 1982)

COMPARING DIFFERENTCOGNITIVE FUNCTIONS

WITHIN SUBJECTS

Although cross-function comparisons inlarge-scale metaanalyses of imaging data

like the one previously described (see also,Cabeza and Nyberg, 2000; Christoff andGabrieli, 2000; Duncan and Owen, 2000)can help identify regions that show activ-ation overlap across functions, their resultsare usually confounded with differences in stimuli, tasks, and imaging methods.Although these differences could beconsidered an advantage from a sharingpoint of view (because activation overlapsreflect similarities in processes rather than similarities in methods), they are a

COMPARING DIFFERENT COGNITIVE FUNCTIONS WITHIN SUBJECTS 53


Table 2 Typical Activation Patterns in PET and fMRI Studies of Five Cognitive Functionsa

Activation pattern

Prefrontal Midline Parietal Temporal

Function 10 9 46 47 45 44 6 32 24 31 7 40 39 22 21 20 37 MTL

Attention

Verbal

Object Spatial

Perception

Verbal Object Spatial

Working memory

Verbal Object Spatial

Semant retriev/episod encod

Verbal

Object Spatial

Episodic retrieval

Verbal

Object Spatial

a Notes: , Left lateral; , right lateral; , bilateral lateral; , midline. For each brain region (e.g., BA 10), a symbol is shown ifthe frequency of activations for a particular stimulus cognitive function cell (e.g., verbal episodic retrieval) was higher than theaverage frequency for the region. The size of the symbol approximately corresponds to the relative proportion of activations foreach function compared to the rest of the functions. Activations are shown as lateralized when the frequency for one hemispherewas at least double that in the other hemisphere. MTL, Medial temporal lobes.

disadvantage from a subdivision point ofview (location differences may reflectmethodological differences rather thanfunctional differences). More generally, theproblem of cross-study comparisons is thatthey only allow qualitative statementsabout the involvement of a certain regionin various functions but not quantitativestatistical measures of the strength of theseactivations across functions. Thus, activa-tions that appear to differ across functionsmay actually be similar (e.g., a thresholdeffect), and activations that appear similaracross functions may actually be different(e.g., activation intensity). Thus, in order todetermine accurately similarities and dif-ferences in activation across different func-tions it is critical to compare thesefunctions directly, within-subjects, andunder similar experimental conditions.

Only a few functional neuroimagingstudies have tried such direct within-subject comparisons. One reason for thisscarcity is historical: because most cog-nitive researchers specialize in a single cognitive function, it is only natural thatthey maintained this specialization whenthey started conducting functional neuro-imaging studies. In addition, functionalneuroimaging researchers inherited a longlist of research questions about each partic-ular function from cognitive psychology,and this list of questions kept them focusedfor many years on their favorite function.Another reason for the dearth of cross-function studies is that these studies areparticularly difficult to design. First, theparadigms used to investigate differentcognitive functions tend to be dissimilar(e.g., the cuing paradigm used to studyattention vs. the old/new recognition para-digm used to study episodic retrieval), andit is challenging to design tasks for two dif-ferent functions that have a similar struc-ture in terms of stimuli, responses, andtiming. Thus, trying to compare differentfunctions may appear sometimes as try-ing to “compare apples and oranges.” Itshould be noted, however, that if one does

not compare apples and oranges, one maymiss the fact that they are both round andsweet fruits. Second, some functions areinherently more difficult than others. Forexample, in the case of working memoryand episodic memory, if the memory loadis kept constant (e.g., one word), thenretrieval from working memory is alwayseasier than retrieval from long-term mem-ory. In these situations, one is faced withthe dilemma of matching experimentalconditions at the expense of having differ-ences in task difficulty or matching taskdifficulty at the expense of introducingdifferences in experimental conditions.Despite all these problems, successfuldirect cross-function studies can bedesigned, and they offer unique insightsinto the role of different brain regionsacross various functions.

The next two sections review compar-isons of different functions within-subjects(see Table 3). The first section reviewsstudies that used blocked fMRI and PETdesigns, and the second section reviewsstudies that employed event-related fMRIdesigns. An advantage of blocked designsfor cross-function comparisons is that theymeasure both item- and state-related activ-ity. Item-related activity refers to transientchanges associated with cognitive opera-tions specific to particular items within thetask (e.g., old vs. new stimuli in a memorytest), whereas state-related activity refers tosustained changes associated with mentalstates that are characteristic of the task(Donaldson et al., 2001; Düzel et al., 1999).Because different functions can resembleeach other or differ both in terms of item-related or state-related activity, it is con-venient that blocked designs measure both of them at the same time. Event-relatedfMRI designs have several advantages for cross-function research. Importantly, thesedesigns provide separate measures of different task components, such as theencoding, maintenance, and retrieval com-ponents of working memory tasks. Also,they allow separate analyses of trials asso-



ciated with successful vs. unsuccessfulbehavioral responses, thereby providing abetter control for differences in taskdifficulty across functions.

Blocked Studies

LaBar et al. (1999)

One of the first functional neuroimagingstudies that directly compared differentcognitive functions within-subjects is prob-ably the study by LaBar et al. (1999), whichcontrasted verbal working memory andvisuospatial attention using fMRI. Thisstudy compared a working memory taskwithout a spatial attention component (a letter two-back task) to a spatial atten-tion task without a working memorycomponent (a Posner cuing paradigm inwhich the cue remained on until the targetwas presented). Each task was comparedto its own control task, and commonregions were identified with a conjunctionanalysis.

As shown in Table 3, the conjunctionanalysis identified a common network ofbrain regions for verbal working memoryand spatial attention, including poste-rior/dorsomedial PFC (BA 6), parietal(BAs 7/40), and left temporal regions. Thisoverlap in posterior PFC and parietalregions is consistent with the results of the foregoing metaanalysis (see spatialattention and verbal working memory inTable 2). In the LaBar et al. study, the verbalworking memory tasks involved shiftsbetween external (letters on display) andinternal (letters in working memory)frames of reference, whereas the spatialattention task involved shifts to differentlocations in space. Thus, the authors pro-posed that the common network reflectsshifts in attentional focus, irrespective ofwhether the shifts occur over space, time,or other cognitive domains.

In addition to conjunction analyses,LaBar et al. compared verbal workingmemory and spatial attention directly—after subtracting out their respective

control conditions (see Table 3). Workingmemory was associated with activations insupplementary motor area (SMA), leftopercular PFC (BA 44), precuneus, andinferior parietal regions (right BA 40),whereas spatial attention was associatedwith occipitotemporal and extrastriatecortex. The finding that SMA was moreactivated for verbal than for spatial stimuliis intriguing because this region was previ-ously associated with spatial processing(e.g., Courtney et al., 1998). The findingthat Broca’s area (left BA 44) was moreactivated for working memory than forspatial attention is consistent with themetaanalysis reported in this chapter (seeTable 2)

Braver et al. (2001)

The blocked fMRI study by Braver et al.(2001) compared PFC activations duringworking memory (two-back task), episodicencoding (intentional learning), andepisodic retrieval (old/new recognition).Each task was investigated using wordsand unfamiliar faces. The authors madethree predictions. (1) Dorsolateral PFCshould be selectively activated by theworking memory task. According toBraver et al., dorsolateral PFC “is criticallyimportant for tasks requiring active main-tenance over intervening items and/or themonitoring and manipulation of main-tained information,” and these processesare not engaged during episodic memoryencoding or retrieval. (2) Regardless of thetask, ventrolateral PFC activity should beleft lateralized for verbal materials andright lateralized for spatial materials (e.g.,Kelley et al., 1998; McDermott et al., 1999).(3) Frontopolar PFC should be selectivelyactivated by episodic memory retrieval.The authors based this prediction on theaforementioned idea that anterior PFCregions are involved in episodic retrievalmode (e.g., Lepage et al., 2000; Nyberg et al., 1995).

The results confirmed the first two pre-dictions but not the third. Dorsolateral



56

TABLE 3 Results of PET/fMRI Studies That Compared Different Cognitive Functions Within-Subjectsa

Activation pattern

Prefrontal Midline Parietal Temporal

Study/comparison 10 9 46 47 45 44 6 32 24 31 7 40 39 22 21 20 37 MTL

Blocked paradigms

LaBar et al. (1999)

Both WM and spatial attention WM > spatial attention Spatial attention > WM

Braver et al. (2001): PFC

WM > other tasks Episodic enc. > other tasksb Episodic ret. > other tasksb

Nyberg et al. (2002): task PLS

Exp. 1

Both WM and episodic ret. Episodic ret. > other tasks Semantic ret. > other tasks WM > other tasks

Exp. 2

Both autobio. memory and sem. ret. Nyberg et al. (2002): seed PLS

WM, episod. ret., and sem. ret.

57

Event-related paradigms

Ranganath et al. (2002): PFC; Ranganathand and D’Esposito (2001): MTL

Both WM enc. and episodic enc. Both WM ret. and episodic ret. WM > episodic m. Episodic m. > WM

Cabeza et al. (2002)

Both WM and episodic ret. Episodic ret. > WM WM > episodic ret.

Cabeza et al. (2002)

Both episodic ret. and visual attention Episodic ret. > visual attention Visual attention > episodic ret.

a Notes: , Left lateral; , right lateral; l, bilateral lateral;l, midline; Exp, experiment; enc., encoding; ret., retrieval; sem., semantic; WM, working memory; MTL, medial temporal lobes.b The activation did not meet the significance criteria used for other contrasts.

PFC regions were activated duringworking memory but not during episodicencoding or retrieval. Although thisfinding confirms the first prediction ofBraver et al., it is surprising because dozensof PET and fMRI studies of episodicmemory encoding and retrieval havereported significant activations in dorsolat-eral PFC regions (Cabeza and Nyberg,2000). The second prediction that ventrolat-eral PFC regions should be lateralizedaccording to materials (left for words, rightfor faces) for both working and episodicmemory was confirmed, suggesting thatthis lateralization pattern is a function-independent phenomenon. Finally, becauseanterior PFC was not differentially acti-vated by episodic memory retrieval, thethird prediction was not confirmed. On thecontrary, a left anterior PFC (BA 10) wasmore activated for working memory thanfor episodic encoding and retrieval. Thismay have happened because the episodicretrieval task involved only retrievalprocesses whereas the working memorytask (two-back) involved encoding, main-tenance processes, and retrieval processes.Thus, it is possible that the workingmemory task was more complex andinvolved greater time-on-task activity thandid the episodic encoding and retrievaltasks. Because blocked designs cannot dis-tinguish encoding, maintenance, andretrieval phases of working memory, theydo not allow an appropriate comparisonbetween episodic retrieval and the retrievalphase of working memory.

Nyberg et al. (2002a,b)

In two PET experiments, the neural cor-relates of working memory, semanticmemory, and episodic memory were com-pared (Nyberg et al., 2002a). Across the twoexperiments, three measures were used foreach of the examined memory systems.Results were analyzed using multivariatestatistical technique (partial least squares;PLS) (McIntosh et al., 1996) that can iden-tify the combinations of experimental con-

ditions that account for most variance inthe brain images. In Experiment 1, bothworking memory and episodic retrievalwere associated with activations in rightanterior PFC, precuneus/cuneus, and bi-lateral parietal regions (see Table 3). Inkeeping with the results of Braver et al.(2001), right BA 47 was more activated forepisodic retrieval than for workingmemory. Consistent with the metaanalysispreviously reported, left PFC (BAs 9, 45,and 44) was differentially activated bysemantic retrieval tasks, and posterior PFC(BA 6), by working memory tasks (seeTable 3). Experiment 2 yielded an un-expected finding: an autobiographicalmemory task, which can be classified asepisodic, activated a set of regions similarto that activated by semantic retrievaltasks, including left ventrolateral PFC. Wesuggested that the cued word retrievalused in the task elicited general semanticretrieval, and therefore had a sharedpattern of brain activity with tests ofsemantic retrieval.

In a follow-up study (Nyberg et al.,2002b), the frontal regions common toworking memory, episodic retrieval, andsemantic retrieval were identified and thefunctional connectivity of these regionsand the rest of the brain was investigatedacross tasks (e.g., McIntosh et al., 1997).This undertaking is closely related to thenetwork view discussed in the Intro-duction, and to our knowledge this is thefirst study to consider connectivity analy-ses in cross-function comparisons. Threefrontal regions were identified: left anteriorPFC (BA 10), left ventrolateral PFC (BA 45),and dorsal anterior cingulate cortex (BA32) (see Table 3). Of the three commonregions, only the ventrolateral PFC regionshowed a shared pattern of functional con-nectivity. Thus, despite the fact that theanterior cingulate and anterior PFC regionswere consistently activated across workingmemory, episodic retrieval, and semanticretrieval, they apparently played differentroles in each of these functions. In contrast,



the left ventrolateral region appeared toplay the same role across the three memoryfunctions, consistent with the idea that thisregion is involved in semantic generation(Fletcher and Henson, 2001) and activeretrieval (Owen et al., 2000) processes.

Event-Related fMRI Studies

Ranganath and D’Esposito (2001) andRanganath et al. (2002)

Ranganath and collaborators (Ranga-nath and D’Esposito, 2001; Ranganath et al., 2002), compared brain activity duringthe encoding and retrieval phases ofworking memory and episodic memory. InPFC (Ranganath et al., 2002), bilateral pos-terior (BA 6) and ventrolateral (BAs 44, 45,and 47) regions were activated during bothencoding and retrieval phases of bothworking memory and episodic memory.Additionally, bilateral dorsolateral PFCregions (BAs 9, and 46) and left anteriorPFC region (BA 10/46) were activatedduring the retrieval phase but not duringthe encoding phase of both functions (seeTable 3). A left ventrolateral PFC region(BA 47) was more activated duringepisodic retrieval than during workingmemory retrieval, but this region did notshow a significant difference with respectto base line in either condition. Thus,working memory and episodic memoryrecruited similar PFC regions, including aleft anterior PFC region that was associatedwith the retrieval phase of both functions.According to the authors, the left anteriorPFC activation reflected the “online” moni-toring and evaluation of specific memorycharacteristics during retrieval.

In MTL regions, the study yielded avery interesting finding: whereas the hip-pocampus was activated during the main-tenance phase of the working memorytask, the parahippocampal gyrus was acti-vated during the encoding and retrievalphases of both working memory andepisodic memory (Ranganath andD’Esposito, 2001). A second experiment

replicated these results, and addition-ally showed that both hippocampal andparahippocampal activations were greaterfor novel than for familiar faces (Ranga-nath and D’Esposito, 2001). The authorsargued that both regions play a role inepisodic memory (e.g., Aggleton andBrown, 1999). To explain why the hip-pocampus was not significantly activatedduring episodic encoding and retrievaltasks, the authors suggested that the hip-pocampus may use sparse representations(e.g., Fried et al., 1997), and as a result, itstransient activity during episodic encodingand retrieval may be difficult to detect. Incontrast, prolonged hippocampal activitymay be easier to detect during workingmemory tasks, or during episodic tasksinvolving sustained recollective processing(e.g., Eldridge et al., 2000). More generally,the authors argued that their results castdoubts on the idea that working memoryand episodic memory depend on distinctneural correlates, and endorsed the notionthat working memory maintenance is the outcome of controlled activation ofepisodic memory networks (e.g., Fuster,1995).

Cabeza et al. (2002a)

In this study by Cabeza et al., (2002a),the neural correlates of episodic retrievaland working memory for verbal materialswere compared. The trials of both tasksconsisted of two phases (Phase 1 andPhase 2). In the episodic retrieval trials,Phase 1 consisted of an instruction to thinkback to a previous study episode, andPhase 2 consisted of a retrieval cue, towhich subjects made a Remember–Know–New recognition response. Phase 1 wasexpected to elicit retrieval mode activity,and Phase 2 was expected to elicit cue-specific retrieval activity. In workingmemory trials, Phase 1 consisted of amemory set of four words in two columns,and Phase 2 consisted of a word probe, towhich subjects made a Left–Right–Newresponse. Thus, Phase 1 measured



working memory encoding and mainte-nance activity, and Phase 2 measuredworking memory retrieval activity.

The fMRI data yielded two main find-ings (see Table 3). First, there were simi-larities and differences in PFC activityacross tasks: (1) a left dorsolateral region(BA 9) was similarly activated for work-ing memory and episodic retrieval tasks,(2) anterior (BA 10) and ventrolateral (BAs47 and 45) regions were more activated forepisodic retrieval, and (3) Broca’s (leftBA 44) and posterior/dorsal (BAs 44 and6) regions were more activated for workingmemory (see Table 3). The first finding ofoverlapping dorsolateral PFC activity forepisodic retrieval and working memory isconsistent with the aforementioned resultsby Ranganath et al. (2002). Dorsolateralprefrontal activations have been attributedto monitoring in both episodic retrievaland working memory studies, and theresults were consistent with this idea. Thesecond finding of anterior PFC activationduring episodic retrieval was consistentwith the hypothesis that this region isinvolved in retrieval mode (Cabeza andNyberg, 2000; Düzel et al., 1999; Lepageet al., 2000; Nyberg et al., 1995). In keepingwith this idea, the anterior PFC activationstarted during Phase 1, before the presenta-tion of the retrieval cue, and was sustainedthroughout the trial (see Fig. 9). This time-course could explain why Braver et al.(2001) and Ranganath et al. (2002) failed todetect differential anterior PFC activity

during episodic retrieval because thosestudies did not differentiate betweenretrieval mode and cue-specific aspects ofepisodic retrieval. Greater ventrolateralPFC activity for episodic retrieval than forworking memory is consistent with theresults of Braver et al. (2001) and Nyberg et al. (2002a). The fact that this activationoccurred during Phase 2 following therecognition cue (see Fig. 9) fits well withthe notion that this region is involved inthe specification of episodic retrieval cues(Henson et al., 1999). Finally, the thirdfinding, that Broca’s area was differentiallyengaged during working memory, is con-sistent with the results of LaBar et al.(1999), and the fact that this activation wasmaximal during the maintenance phase ofworking memory harmonizes with theidea that this region mediates phonologicalrehearsal (for a review, see Smith andJonides, 1999).

The second main result of the study wasthe unexpected finding that anterior MTL was activated during both episodicretrieval and working memory. Althoughconsistent with the aforementioned studyby Ranganath and D’Esposito (2001), thisfinding is surprising because MTL hasbeen strongly associated with episodicretrieval but not with working memory. Wespeculated that the MTL overlap mightreflect the indexing functions of thisregion, which could play a role not onlyduring the access of stored long-termmemory traces but also during the mainte-



FIGURE 9 Time courses of brain activity during episodic retrieval and working memory in three prefrontalregions (after Cabeza et al., 2002a).

nance of short-term memory representa-tions. As discussed below, however, morerecent findings show that anterior MTL isalso activated during an attention taskwithout a mnemonic component, suggest-ing that the representations indexed byMTL are not necessarily mnemonic.

Cabeza et al. (2002b)

In an event-related fMRI study, theneural correlates of episodic memoryretrieval and visual attention were com-pared. The motivation for comparing thesetwo functions was that many of the brainregions typically activated during episodicretrieval tasks (for reviews, see Cabeza,1999; Rugg and Henson, 2002), such as pre-frontal, parietal, anterior cingulate, andthalamic areas, are also frequently acti-vated during visual attention tasks (forreviews, see Handy et al., 2001; Kanwisherand Wojciulik, 2000). Thus, although theinvolvement of these regions duringepisodic retrieval has been attributed toepisodic retrieval processes (e.g., postre-trieval monitoring), it may actually reflectattentional operations. To investigate thisidea, the previously described episodicretrieval task with retrieval mode and cue-specific retrieval phases (Cabeza et al.,2002a) was compared to a visual attentiontask with an important sustained attentioncomponent. In the visual attention task,participants stared at a letter in the centerof the screen to determine whether itblipped once, twice, or never during a 12-sec interval.

The study yielded three main findings.First, consistent with previous functionalneuroimaging evidence, the study ident-ified a common frontoparietal–cingulate–thalamic network for episodic retrieval andvisual attention. This finding suggests thatmany of the PFC and parietal activationsfrequently found during episodic retrievalreflect basic attentional processes ratherthan complex mnemonic operations. Act-ually, some of the memory-related inter-pretations proposed in episodic retrieval

studies can be easily rephrased in terms ofsimpler attentional processes. For example,right dorsolateral PFC activations duringepisodic retrieval have been attributed topostretrieval monitoring (Henson et al.,1999), but because monitoring involvessustained attention and sustained attentionis associated with right PFC activations(for reviews, see Coull, 1998; Sarter et al.,2001), then right PFC activations duringepisodic retrieval may be described assustained attention to the retrieval output.

Second, several subregions were differ-entially involved in episodic retrieval vs.visual attention. For example, left PFC wasmore activated for episodic retrieval thanfor visual attention, possibly reflectingsemantically guided information produc-tion, whereas right PFC was more activatedfor visual attention than for episodicretrieval, possibly reflecting monitoringprocesses (Cabeza et al., 2002c). Consistentwith Cabeza et al. (2002a), anterior PFC (BA10) was differentially involved in episodicretrieval, possibly reflecting retrieval mode.The precuneus and neighboring regionswere more activated for episodic retrievalthan for visual attention, suggesting thatthese areas are involved in processinginternally generated information.

Finally, the study yielded an unexpectedfinding: anterior MTL regions were simi-larly activated during episodic retrievaland during visual attention (see Fig. 10). Ifone assumes that MTL has an indexingfunction (e.g., MClelland et al., 1995), thenthis finding suggests that MTL indexes notonly episodic memory and working mem-ory representations (Cabeza et al., 2002a),but also perceptual representations. Inother words, anterior MTL may index rep-resentations in the focus of consciousness,regardless of whether the representationsoriginate in episodic memory, workingmemory, or the senses. This idea is consis-tent with Moscovitch’s proposal that MTL is a module specialized in automa-tically registering the conscious experience



(Moscovitch, 1992). Also, the idea is coher-ent with evidence that MTL activity duringepisodic retrieval differs for old items asso-ciated with different forms of conscious-ness (remembering vs. knowing) (Eldridgeet al., 2000) but is similar for old and newitems associated with similar forms of con-sciousness (veridical vs. illusory recogni-tion) (Cabeza, et al., 2001; Schacter et al.,1996b). Because MTL lesions do notusually impair working memory andattention tasks, but sometimes disruptimplicit tasks (e.g., Chun and Phelps,1999), one has to conclude that MTL activ-ity may reflect processing the contents ofconsciousness but is neither necessary norsufficient for this processing.

CONCLUSIONS

To summarize the main findings of com-paring different cognitive functions acrossstudies (Figs. 3–8, Table 2) and within-sub-jects (Table 3), we performed the drasticsimplification shown in Table 4. First, wecollapsed over verbal, object, and spatialstimuli because the modulatory effects ofstimulus type can be summarized in twosimple statements: differences betweenobject and spatial stimuli generally follow

the ventral/dorsal pathway distinction(Ungerleider and Mishkin, 1982), and differ-ences between verbal and nonverbal stimulifollow hypothetical hemisphere specializa-tions (e.g., Milner, 1971). Second, we consid-ered only activation overlaps supported byboth cross-study (Table 2) and within-sub-jects (Table 3) analyses. Thus, Table 4 doesnot include the perception domain, whichhas not been investigated by direct cross-function studies. Finally, we focused on themost consistent activation patterns and didnot emphasize exceptions to these patternsor theoretical controversies.

As illustrated by Table 4, anterior PFC(BA 10) plays a prominent role in episodicretrieval and, to a lesser degree, in workingmemory. The involvement of this region inepisodic retrieval is consistent with theretrieval mode hypothesis (Cabeza et al.,2002a) and the overlap between episodicretrieval and working memory (Nyberg et al., 2002a; Ranganath et al., 2002) isconsistent with a more general role in moni-toring internally generated information(Christoff and Gabrieli, 2000). DorsolateralPFC (BA 9/46) is most strongly associatedwith working memory and episodicretrieval, possibly reflecting a role in moni-toring (Cabeza et al., 2002a). Anterior ven-trolateral PFC (BA 47/45) is also involved in



FIGURE 10 Time courses of brain activity during episodic retrieval (ER) and visual attention (VA) in a leftanterior medial temporal lobe region. Reprinted from Neuropsychologia; R. Cabeza, F. Dolcos, S. Prince, H. Rice, D.Weissman, and L. Nyberg; Attention-related activity during episodic memory retrieval: A cross-function fMRIstudy. Copyright 2002, with permission from Elsevier Science.

63

TABLE 4 Common Regions for Four Cognitive Functions and Their Hypothetical Roles in Cognitiona

Region/Brodmann area

PFC ACC: Precun.: Pariet: Temp.:

Function Ant.: 10 DL: 9, 46 aVL: 47, 45 pVLp: 44 postDors: 6 32, 24 31 40, 7 21 MTL

Attention 1, 7 1,2,7 1,7 1,2,7 1,7

Working memory 1, 3, 4, 5 1, 3, 4, 5, 6 1, 4, 5, 6 1, 2, 5, 6 1, 3, 4, 5, 6 1, 4, 5, 6 1, 4 1, 4, 5, 6 5, 6

Epi. enc./sem. ret. 1, 4 1, 4, 5 1, 4, 5 1, 4 1, 4 1

Episodic retrieval 1, 4, 5, 6, 7 1, 4, 5, 6, 7 1, 4, 5, 6, 7 1, 4, 5, 6, 7 1, 4, 6, 7 1, 4, 5, 7 1, 4, 7 1, 5, 6

Hypothetical Retrieval Monitoring Semantic In left Top-down Initiation of Orienting Shifts of Semantic Indexing ofprocesses mode and processing hemisphere: selection action and/ attention to attention processing representa-

/or moni- and inhib- phonological or conflict internally among tions within toring of itory control rehearsal monitoring generated external or the focus of internally information internal consciousnessgenerated eventsinformation

a Abbreviations: epi., episodic; enc., encoding; ret., retrieval; sem., semantic; PFC, prefrontal cortex; ACC, anterior cingulate; precun., precuneus; pariet., parietal cortex; temp., temporalcortex; MTL, medial temporal lobes; ant., anterior; DL, dorsolateral; aVL, anterior ventrolateral; pVLp, posterior ventrolateral-posterior; postDors, posterior dorsal. Symbols have the samemeaning as in Tables 2 and 3; the symbol size approximately corresponds to the relative proportion of activations for each function compared to the rest of the functions. The numbers adja-cent to the symbols refer to the following studies: 1, present metanalysis (see Table 2); 2, LaBar et al. (1999); 3, Braver et al. (2001); 4, Nyberg et al. (2002a, b); 5, Ranganath and D’Esposito (2001)and Ranganath et al. (2002); 6, Cabeza et al. (2002a); 7, Cabeza et al. (2002b).

working memory and episodic retrieval, butin the left hemisphere it plays a major rolein episodic encoding/semantic retrieval,possibly reflecting semantic processing (e.g.,Gabrieli et al., 1998). Ventrolateral PFC isalso known to play an important role ininhibitory control (D’Esposito et al., 1999;Jonides et al., 1998), and this function couldaccount for its involvement in Stroop tasks.In the left hemisphere, posterior ventrolat-eral PFC (BA 44) is strongly associated withverbal working memory, consistent with ahypothetical role in phonological rehearsal(Cabeza et al., 2002a; Kapur et al., 1996;Poldrack et al., 1999). Finally, posterior–dorsal PFC (dorsal BA 6) seem primarilyassociated with attention and workingmemory, consistent with a hypothetical rolein top-down selection (Corbetta andShulman, 2002)

Table 4 shows that whereas parietalregions (BAs 40 and 7) were primarilyassociated with attention and workingmemory, left temporal regions (particularlyBA 21) were primarily associated withsemantic and episodic memory. Theinvolvement of parietal regions in attentionand working memory, as well as inepisodic retrieval, can be explained if oneassumes that these regions are involved inshifting attention not only among externalevents (spatial and nonspatial attention),but also among internal events (workingmemory and episodic retrieval). Theinvolvement of left temporal regions inboth semantic and episodic memory tasks(Nyberg et al., 2002a) can be explained by ageneral role in semantic processing.

Finally, the MTL seems to be involved inall four cognitive functions. Although thecross-study metaanalysis did not link MTLto working memory, two cross-functionfMRI studies found the MTL to be acti-vated for both episodic retrieval andworking memory (Cabeza et al., 2002a;Ranganath and D’Esposito, 2001).Overlapping MTL activations for episodicretrieval and visual attention have beenfound and suggest that this region indexes

representation within the focus of con-sciousness (Cabeza et al., 2002b). Thishypothesis fits well a popular cognitiveneuroscience model (Moscovitch, 1992)and can account for the involvement ofthese regions in several different cognitivefunctions.

As noted above, Table 4 shows anextremely simplified (almost simplistic)description of typical activation patterns,which does not acknowledge exceptions tothe patterns or theoretical controversiesabout the functions of different brainregions. For example, although anteriorPFC tends to play a more important role inepisodic retrieval than in working memory,Ranganath et al. (2002) found similar ante-rior PFC activity across these functions andBraver et al. (2001) found greater activityfor working memory. Even if the findingsof Ranganath et al. reflect a lack of powerand the results of Braver et al. reflect thelack of differentiation between workingmemory encoding and retrieval, furtherresearch is clearly warranted. Also, thereare inconsistencies about the lateralizationof overlapping PFC activations. Forinstance, overlapping anterior PFC activa-tions for episodic retrieval and workingmemory have been found in the righthemisphere (Nyberg et al., 2002a), in theleft hemisphere (Ranganath et al., 2002),and bilaterally (Cabeza et al., 2002a).Moreover, Table 4 collapsed over regions islikely to have different roles in cognition,such as inferior (BA 40) and posterior(BA 7) parietal regions. For example, wehave found that the posterior parietalcortex was similarly involved in workingmemory and episodic retrieval, whereasthe anterior parietal cortex in the left hemi-sphere was differentially involved inworking memory (Cabeza et al., 2002a; seealso Ranganath et al., 2002). If there is dis-agreement about the activation patterns inTable 4, there is of course much more dis-agreement about the theoretical interpreta-tions of these activations. For instance, afMRI study (Bush et al., 2002) pointed out



that if one considers only research withhuman subjects, the list of cognitive pro-cesses attributed to the anterior cingulatecortex include the following: attention-for-action/target selection, motivationalvalence assignment, motor response selec-tion, error detection/performance monitor-ing, competition monitoring, anticipation,working memory, novelty detection, andreward assessment. Cross-function com-parisons could help decide among thesedifferent functional interpretations

Cross-function comparisons provideimportant constraints to functional inter-pretations, particularly if one assumes thesharing view. For example, although wehave attributed MTL activity to the recov-ery of episodic memory traces (e.g., Cabezaet al., 2001; Nyberg et al., 1996b), theinvolvement of these regions in attention,working memory, and semantic memorysuggests a much more general function incognition. Cross-function comparisonswork against the natural tendency to inter-pret activations in terms of our favoritecognitive function (episodic memory inour case). They help us to overcome func-tion chauvinism and to see the “bigpicture.” In other words, cross-functioncomparisons allow us to see the forestthrough the trees.

ReferencesAggleton, J. P., and Brown, M. W. (1999). Episodic

memory, amnesia, and the hippocampal-anteriorthalamic axis. Behav. Brain Sci. 22(425–289).

Awh, E., Jonides, J., Smith, E. E., Schumacher, E. H., et al. (1996). Dissociation of storage and rehearsalin verbal working memory: Evidence frompositron emission tomography. Psychol. Sci. 7(1),25–31.

Benson, D. F. (1988). Classical syndromes of aphasia.In “Handbook of Neuropsychology” (F. Foller and J. Grafman, eds.), Vol. 1, pp. 267–280. Elsevier,Amsterdam.

Binder, J. R., and Price, C. J. (2001). Functionalneuroimaging of language. In “Handbook ofFunctional Neuroimaging of Cognition”(R. Cabeza and A. Kingstone, eds.), pp. 187–251.MIT Press, Cambridge, Massachussetts.

Botvinick, M. M., Braver, T. S., Barch, D. M., Carter,C. S., and Cohen, J. D. (2001). Conflict monitoringand cognitive control. Psychol. Rev. 108(3),624–652.

Braver, T. S., Barch, D. M., Kelley, W. M., Buckner, R. L., Cohen, N. J., Miezin, F. M., Snyder, A. Z.,Ollinger, J. M., Akbudak, E., Conturo, T. E., andPetersen, S. E. (2001). Direct comparison ofprefrontal cortex regions engaged by working andlong-term memory tasks. Neuroimage 14, 48–59.

Buckner, R. L., Raichle, M. E., Miezin, F. M., andPetersen, S. E. (1996). Functional anatomic studiesof memory retrieval for auditory words and visualpictures. J. Neurosci. 16(19), 6219–6235.

Bush, G., Vogt, B. A., Holmes, J., Dale, A. M., Grave,D., Jenike, M. A., and Rosen, B. R. (2002). Dorsalanterior cingulate cortex: A role in reward-baseddecision making. Proc. Natl. Acad. Sci. U.SA. 99, 523–528.

Cabeza, R. (1999). Functional neuroimaging ofepisodic memory retrieval. In “Memory, Con-sciousness, and the Brain: The Tallinn Conference” (E. Tulving, ed.), pp. 76–90. The Psychology Press,Philadelphia.

Cabeza, R., and Nyberg, L. (2000). Imaging CognitionII: An empirical review of 275 PET and fMRIstudies. J. Cogn. Neurosci. 12, 1–47.

Cabeza, R., Kapur, S., Craik, F. I. M., McIntosh, A. R.,Houle, S., and Tulving, E. (1997). Functionalneuroanatomy of recall and recognition: A PETstudy of episodic memory. J. Cogn. Neurosci. 9(2),254–265.

Cabeza, R., Rao, S. M., Wagner, A. D., Mayer, A. R.,and Schacter, D. L. (2001). Can medial temporallobe regions distinguish true from false? An event-related fMRI study of veridical and illusory recog-nition memory. Proc. Natl. Acad. Sci. U.S.A. 98,4805–4810.

Cabeza, R., Dolcos, F., Graham, R., and Nyberg, L. (2002a). Similarities and differences in theneural correlates of episodic memory retrieval andworking memory. Neuroimage 16, 317–330.

Cabeza, R., Dolcos, F., Prince, S., Rice, H., Weissman,D., and Nyberg, L. (2002b). Attention-related activ-ity during episodic memory retrieval: A cross-function fMRI study. Neuropsychologia. In press.

Cabeza, R., Kester, J., and Anderson, N. D. (2002c).Lateralization of prefrontal cortex activity duringepisodic memory retrieval: Evidence for the gen-erate-recognize hypothesis. Submitted.

Christoff, K., and Gabrieli, J. D. E. (2000). The fron-topolar cortex and human cognition: Evidence fora rostrocaudal hierarchical organization withinthe human prefrontal cortex. Psychobiology 28,168–186.

Chun, M. M., and Phelps, E. A. (1999). Memorydeficits for implicit contextual information inamnesic subjects with hippocampal damage. Nat.Neurosci. 2, 844–847.

REFERENCES 65


Cohen, N. J., Ryan, J., Hunt, C., Romine, L., Wszalek,T., and Nash, C. (1999). Hippocampal system anddeclarative (relational) memory: Summarizing thedata from functional neuroimaging studies.Hippocampus 9, 83–98.

Corbetta, M., and Shulman, G. L. (2002). Control ofgoal-directed and stimulus-driven attention in thebrain. Nat. Rev. 3, 201–215.

Coull, J. T. (1998). Neural correlates of attention andarousal: Insights from electrophysiology, func-tional neuroimaging and psychopharmacology.Prog. Neurobiol. 55, 343–361.

Courtney, S. M., Petit, L., Maisog, J. M., Ungerleider,L. G., and Haxby, J. V. (1998). An area specializedfor spatial working memory in human frontalcortex. Science 279, 1347–1351.

D’Esposito, M. (2001). Functional neuroimaging ofworking memory. In “Handbook of functionalneuroimaging of cognition” (R. Cabeza and A. Kingstone, eds.), pp. 293–327. MIT Press,Cambridge, Massachusetts.

D’Esposito, M., Postle, B. R., Jonides, J., and Smith, E. E. (1999). The neural substrate and temporaldynamics of interference effects in workingmemory as revealed by event-related functionalMRI. Proc. Natl. Acad. Sci. U.S.A. 96, 7541–7519.

Devinsky, O., Morrell, M. J., and Vogt, B. A. (1995).Contributions of anterior cingulate cortex tobehaviour. Brain 118, 279–306.

Donaldson, D. I., Petersen, S. E., Ollinger, J. M., andBuckner, R. L. (2001). Dissociating state and itemcomponents of recognition memory using fMRI.Neuroimage 13, 129–142.

Duncan, J., and Owen, A. M. (2000). Common regionsof the human frontal lobe recruited by diverse cognitive demands. Trends Cogn. Sci. 23, 473–483.

Düzel, E., Cabeza, R., Picton, T. W., Yonelinas, A. P.,Scheich, H., Heinze, H.-J., and Tulving, E. (1999).Task- and item-related processes in memoryretrieval: A combined PET and ERP study. Proc.Natl. Acad. Sci. U.S.A. 96, 1794–1799.

Eldridge, L. L., Knowlton, B. J., Furmanski, C. S.,Bookheimer, S. Y., and Engle, S. A. (2000).Remembering episodes: A selective role for thehippocampus during retrieval. Nat. Neurosci. 3(11),1149–1152.

Elliott, H. (1964). “Textbook of Neuroanatomy.”Lippincott, Philadelphia.

Fletcher, P. C., and Henson, R. N. A. (2001). Frontallobes and human memory: Insights from func-tional neuroimaging. Brain 124, 849–881.

Fletcher, P. C., Frith, C. D., Grasby, P. M., Shallice, T.,Frackowiak, R. S. J., and Dolan, R. J. (1995). Brain systems for encoding and retrieval ofauditory–verbal memory: An in vivo study inhumans. Brain 118, 401–416.

Fodor, J. (1983). “The Modularity of Mind” MIT/Bradford Press, Cambridge, Massachusetts.

Fried, I., MacDonald, K. A., and Wilson, C. L. (1997).Single neuron activity in human hippocampus andamygdala during recognition of faces and objects.Neuron 18, 753–765.

Fuster, J. M. (1995). “Memory in the Cerebral Cortex.”MIT Press, Cambridge, Massachusetts.

Gabrieli, J. D., Poldrack, R. A., and Desmond, J. E.(1998). The role of left prefrontal cortex in lan-guage and memory. Proc. Natl. Acad. Sci. U.S.A.95(3), 906–913.

Handy, T. C., Hopfinger, J. B., and Mangun, G. R.(2001). Functional neuroimaging of attention. In“Handbook of Functional Neuroimaging ofCognition” (R. Cabeza and A. Kingstone, eds.),pp. 75–108. MIT Press, Cambridge, Massachusetts.

Henke, K., Buck, A., Weber, B., and Wieser, H. G.(1997). Human hippocampus establishes associa-tions in memory. Hippocampus 7(3), 249–256.

Henson, R. N. A., Shallice, T., and Dolan, R. J. (1999).Right prefrontal cortex and episodic memoryretrieval: A functional MRI test of the monitoringhypothesis. Brain 122, 1367–1381.

Jonides, J., Smith, E. E., Marshuetz, C., Koeppe, R. A.,and Reuter-Lorenz, P. A. (1998). Inhibition inverbal working memory revealed by brain activa-tion. Proc. Natl. Acad. Sci. U.S.A. 95(14), 8410–8413.

Kanwisher, N., and Wojciulik, E. (2000). Visual atten-tion: Insights from brain imaging. Nat. Rev. 1,91–100.

Kanwisher, N., Downing, P., Epstein, R., and Kourtzi,Z. (2001). Functional neuroimaging of visualrecognition. In “Handbook of FunctionalNeuroimaging of Cognition.” (R. Cabeza and A. Kingstone, eds.), pp. 109–151. MIT Press,Cambridge, Massachusetts.

Kapur, S., Tulving, E., Cabeza, R., McIntosh, A. R.,Houle, S., and Craik, F. I. (1996). The neural corre-lates of intentional learning of verbal materials: A PET study in humans. Cogn. Brain Res. 4(4),243–249.

Kelley, W. M., Miezin, F. M., McDermott, K. B.,Buckner, R. L., Raichle, M. E., Cohen, N. J.,Ollinger, J. M., Akbudak, E., Conturo, T. E.,Snyder, A. Z., and Petersen, S. E. (1998).Hemispheric specialization in human dorsalfrontal cortex and medial temporal lobe for verbaland nonverbal memory encoding. Neuron 20(5),927–936.

Krause, B. J., Schmidt, D., Mottaghy, F. M., Taylor, J., Halsband, U., Herzog, H., Tellmann, L., andMüller-Gärtner, H.-W. (1999). Episodic retrievalactivates the precuneus irrespective of the imagerycontent of word pair associates: A PET study. Brain122, 225–263.

LaBar, K. S., Gitelman, D. R., Parrish, T. B., andMesulam, M. (1999). Neuroanatomic overlap ofworking memory and spatial attention networks:A functional MRI comparison within subjects.Neuroimage 10(6), 695–704.



Lepage, M., Ghaffar, O., Nyberg, L., and Tulving, E.(2000). Prefrontal cortex and episodic memoryretrieval mode. Proc. Natl. Acad. Sci. U.S.A. 97(1),506–511.

Maguire, E. A., Frith, C. D., Burgess, N., Donnett, J. G., and O’Keefe, J. A. (1998). Knowing wherethings are: Parahippocampal involvement inencoding object locations in virtual large-scalespace. J. Cogn. Neurosci. 10(1), 61–76.

Martin, A. (2001). Functional neuroimaging of seman-tic memory. In “Handbook of FunctionalNeuroimaging of Cognition.” (R. Cabeza and A.Kingstone, eds.), pp. 153–186. MIT Press,Cambridge, Massachusetts.

McDermot, K. B., Buckner, R. L., Petersen, S. E.,Kelley, W. M., and Sanders, A. L. (1999). Set and code-specific activation in the frontal cortex: An fMRI study of encoding and retrieval of faces and words. J. Cogn. Neurosci. 11, 631–640.

McDermott, K. B., Buckner, R. L., Petersen, S. E.,Kelley, W. M., and Sanders, A. L. (1999). Set- andcode-specific activation in frontal cortex: an fMRIstudy of encoding and retrieval of faces andwords. J. Cogn. Neurosci. 11(6), 631–640.

McClelland, J. L., McNaughton, B. L., and O’Reilly, R. C. (1995). Why there are complementary learn-ing systems in the hippocampus and neocortex:Insights from the successes and failures of connec-tionist models of learning and memory. Psychol.Rev. 102, 419–457.

McIntosh, A. R. (1999). Mapping cognition to thebrain through neural interactions. Memory 7,523–548.

McIntosh, A. R., Nyberg, L., Bookstein, F. L., andTulving, E. (1997). Differential functional connec-tivity of prefrontal and medial temporal corticesduring episodic memory retrieval. Hum. BrainMapping 5, 323–327.

Milner, B. (1971). Interhemispheric differences in thelocalization of psychological processes in man. Br. Med. Bull. 27, 272–277.

Moscovitch, M. (1992). Memory and working-with-memory: A component process model basedon modules and central systems. J. Cogn. Neurosci.4(3), 257–267.

Nyberg, L., and McIntosh, A. R. (2000). Functionalneuroimaging: Network analyses. In “Handbookof Functional Neuroimaging of Cognition”R. Cabeza and A. Kingstone, eds.), pp. 49–72. MITPress, Cambridge, Massachusetts.

Nyberg, L., Tulving, E., Habib, R., Nilsson, L.-G.,Kapur, S., Houle, S., Cabeza, R., and McIntosh, A. R. (1995). Functional brain maps of retrievalmode and recovery of episodic information.NeuroReport 7, 249–252.

Nyberg, L., Cabeza, R., and Tulving, E. (1996a). PETstudies of encoding and retrieval: The HERAmodel. Psychonom. Bull. Rev. 3(2), 135–148.

Nyberg, L., McIntosh, A. R., Houle, S., Nilson, L.-G.,and Tulving, E. (1996b). Activation of medial tem-poral structures during episodic memoryretrieval. Nature 380(6576), 715–717.

Nyberg, L., Cabeza, R., and Tulving, E. (1998).Asymmetric frontal activation during episodicmemory: What kind of specificity? Trends Cogn.Sci. 2, 419–420.

Nyberg, L., Forkstam, C., Petersson, K. M., Cabeza,R., and Ingvar, M. (2002a). Brain imaging ofhuman memory systems: Between-systems simi-larities and within-system differences. Cogn. BrainRes. 13, 281–292.

Nyberg, L., Marklund, P., Persson, J., Cabeza, R.,Forkstam, C., Petersson, K. M., and Ingvar, M.(2002b). Common prefrontal activations duringworking memory, episodic memory, and semanticmemory. Submitted.

O’Keefe, J. A., and Nadel, L. (1978). “The Hippocampusas a Cognitive Map.” Oxford, London.

Owen, A. M., Lee, A. C. H., and Williams, E. J. (2000).Dissociating aspects of verbal working emmorywithin the human frontal lobe: Further evidencefor a “process-specific” model of lateral frontalorganization. Psychobiology 28, 146–155.

Paulesu, E., Frith, C. D., and Frackowiak, R. S. J.(1993). The neural correlates of the verbal compo-nent of working memory. Nature 362, 342–345.

Paus, T., Petrides, M., Evans, A. C., and Meyer, E.(1993). Role of the human anterior cingulate cortexin control of oculomotor, manual, and speechresponses: A positron emission tomography study.J. Neurophysiol. 70, 453–469.

Poldrack, R. A., Wagner, A. D., Prull, M. W.,Desmond, J. E., Glover, G. H., and Gabrieli, J. D. E.(1999). Functional specialization for semantic andphonological processing in the left inferior pre-frontal cortex. Neuroimage 10(1), 15–35.

Posner, M. I., and Petersen, S. E. (1990). The attentionsystem of the human brain. Annu. Rev. Neurosci.13, 25–42.

Ranganath, C., and D’Esposito, M. (2001). Medialtemporal lobe activity associated with activemaintenance of novel information. Neuron 31,865–873.

Ranganath, C., Johnson, M. K., and D’Esposito, M.(2002). Temporal dynamics of prefrontal activityduring working and long-term memory.Submitted.

Rugg, M. D., and Henson, R. N. A. (2002). Episodicmemory retrieval: An (event-related) functionalneuroimaging perspective. In “The CognitiveNeuroscience of Memory Encoding and Retrieval”(A. E. Parker, E. L. Wilding, and T. Bussey, eds.),Psychology Press, Hove, UK. In press.

Sarter, M., Givens, B., and Bruno, J. P. (2001). Thecognitive neuroscience of sustained attention:Where top-down meets bottom-up. Brain Res. Rev.35, 146–160.

REFERENCES 67


encoding/retrieval asymmetry in episodic mem-ory: Positron emission tomography findings. Proc.Natl. Acad. Sci. U.S.A. 91, 2016–2020.

Tulving, E., Markowitsch, H. J., Craik, F. I. M., Habib,R., and Houle, S. (1996). Novelty and familiarityactivations in PET studies of memory encodingand retrieval. Cereb. Cortex 6(1), 71–79.

Ungerleider, L. G., and Mishkin, M. (1982). Twocortical visual systems. In “Analysis of VisualBehavior” (D. J. Ingle, M. A. Goodale, and R. J. W. Mansfield, eds.), pp. 549–589. MIT Press,Cambridge, Massachusetts.

Wagner, A. D. (1999). Working memory contributionsto human learning and remembering. Neuron 22,19–22.

Wagner, A. D., Desmond, J. E., Glover, G., andGabrieli, J. D. E. (1998a). Prefrontal cortex andrecognition memory: Functional-MRI evidence forcontext-dependent retrieval processes. Brain 121,1985–2002.

Wagner, A. D., Poldrack, R. A., Eldridge, L. L.,Desmond, J. E., Glover, G. H., and Gabrieli, J. D. E.(1998b). Material-specific lateralization of pre-frontal activation during episodic encoding andretrieval. NeuroReport 9, 3711–3717.



Schacter, D. L., Alpert, N. M., Savage, C. R., Rauch, S. L., and Albert, M. S. (1996a). Conscious recollec-tion and the human hippocampal formation:Evidence from positron emission tomography.Proc. Natl. Acad. Sci. U.S.A. 93, 321–325.

Schacter, D. L., Reiman, E., Curran, T., Yun, L. S.,Bandy, D., McDermott, K. B., and Roediger, H. L.,3rd. (1996b). Neuroanatomical correlates ofveridical and illusory recognition memory:Evidence from positron emission tomography.Neuron 17(2), 267–274.

Shallice, T., Fletcher, P., Frith, C. D., Grasby, P.,Frackowiak, R. S. J., and Dolan, R. J. (1994). Brainregions associated with acquisition and retrieval ofverbal episodic memory. Nature 368(6472),633–635.

Smith, E. E., and Jonides, J. (1999). Storage and execu-tive processes in the frontal lobes. Science283(1657–1661).

Squire, L. R. (1992). Memory and the hippocampus: Asynthesis from findings with rats, monkeys, andhumans. Psychol. Rev. 99, 195–231.

Tulving, E. (1983). “Elements of Episodic Memory.”Oxford University Press, Oxford.

Tulving, E., Kapur, S., Craik, F. I. M., Mosco-vitch, M., and Houle, S. (1994). Hemispheric

CHAPTER 3, FIGURE 10 Time courses of brain activity during episodic retrieval (ER) and visual attention (VA) in a left anterior medial temporal lobe region. Reprinted from Neuropsychologia; R. Cabeza, F. Dolcos, S. Prince, H. Rice, D. Weissman, and L. Nybert; Attention-related activity during episodic memory retrieval: A cross-function fMRI study. Copyright 2002, with permission from Elsevier Science.

+3.0[

-3.0

CHAPTER 8, FIGURE 2 Voltage maps showing the oscillatory nature of the ERN and sensorimotor potentials. Time is relative to a button press, which is at 0 msec.

15.

\ \

0 -5

-10

CHAPTER 8, FIGURE 4 Left: Location of the generators of the ERN. Right: Source wave forms illustrating the relative contribution of each source to the scalp-recorded ERN. The box indicates the window of the ERN.

- - Roar~A(~ /

, /


C H A P T E R

4

Evoked Potentials Studies ofVisual Information Processing

Wolfgang Skrandies

INTRODUCTION

Sensory and perceptual processes of thehuman mind have been extensively stu-died in the past by psychophysicalmethods. Modern neurosciences offer sev-eral anatomical and physiological methodsto complement such studies by analyzingthe neurophysiological correlates of sensoryinformation processing (Kandel et al., 2000).In animal experiments, neuronal mecha-nisms are investigated using invasivebiochemical, anatomical, and neurophysio-logical methods that tag activity andprocesses within various structures of thenervous system. For the examination ofhuman subjects, in general only noninva-sive procedures can be used. Becausehuman information processing takes placein fractions of a second, one of the mostfeasible methods constitutes the recordingof brain electrical activity. The strengths ofelectrophysiological methods lie in theirvery high time resolution (in the order ofmilliseconds) and their sensitivity to detectfunctional changes of global brain statesand of nervous activity. The high temporalresolution as well as their noninvasivenature constitute a significant advantage ofthese methods over brain imaging tech-niques [computer topography (CT),positron emission tomography (PET), or

functional magnetic resonance imaging(fMRI)], and evoked brain activity datareveal steps in sensory information pro-cessing that occur very rapidly. Com-parison of imaging and neurophysiologicalmethods shows that there is a reasonablebut imperfect correlation between electro-physiological data and hemodynamicresponses measured by fMRI (George et al.,1995).

For the study of perception and cogni-tion, measures with very high temporalresolution are needed. This is reflected bythe fact that brain mechanisms related toperception are fast (note that motor reac-tion to visual stimuli occurs in a fraction ofa second), and that individual steps ininformation processing are associated withrapid changes of the spatiotemporal char-acteristics of spontaneous and evoked elec-trical brain activity. In addition, thebinding of stimulus features by the cooper-ation of distinct neural assemblies has beenproposed to be mediated through high-frequency oscillation and its coherence ofneuronal activation in different parts of thebrain (Singer, 1999).

The present chapter illustrates how therecording of brain electrical activity incombination with knowledge of the humanvisual system may be employed to studyinformation processing in healthy volun-

teers as well as in patients with selectivevisual deficiency. Data are presented ondifferent experimental questions related tohuman visual perception, including con-trast and stereoscopic vision as well as per-ceptual learning.

Psychophysical procedures are inher-ently subjective because the dependentvariable consists of the verbal response ormotor reaction to a physical stimulus of thesubject under study. Thus, the subject’swillingness and ability to cooperate in the examination determines the result ofthe experiment. We also draw attention tothe fact that psychophysical results alwaysreflect the final outcome of the completechain of information processing involvingsensory transduction in specialized recep-tor organs (Corey and Roper, 1992), subcor-tical and cortical processes, as well ashigher, cognitive strategies. This mostlyprevents the identification and direct inter-pretation of behavioral data in terms of iso-lated steps of sensory processing.

Electrophysiological recordings consti-tute a supplement to such psychophysicalmethods, whereby the electrical activity oflarge neuronal populations is obtainednoninvasively via electroencephalograms(EEGs) or evoked and event-related poten-tials; these may be used to quantify neu-ronal correlates of perceptual and cognitiveprocesses and relate them to anatomicalstructures and functional systems of thehuman central nervous system.

NEUROPHYSIOLOGICAL BASESOF EVOKED ELECTRICAL

BRAIN ACTIVITY

Neurons communicate by transmittingto their neighbors membrane potentialchanges in their synaptic endings, and theyare able to connect different structures bysending over long distance in the braininformation in the form of frequency-modulated action potentials of constantamplitude. In animal experiments, it is

possible to record such activity directlyinside of neurons or in their vicinity,whereas human studies have to rely onrecordings from the intact skull. Due to thefact that scalp-recorded activity has ampli-tudes in the order of only microvolts (µV),for technical reasons it was not until about70 years ago that EEG recordings becamepossible (Berger, 1929). For the interpreta-tion of such data it is important to keep inmind that we are dealing with mass activ-ity originating from large neuronal popu-lations, spreading by volume conduction tothe scalp. The electrodes commonly usedare very large (about 10 mm in diameter)as compared to the dimension of individ-ual neurons (about 20 µm in diameter),which implies that the area of a single elec-trode corresponds to that of about 250,000neurons. The spatial integration of electri-cal nervous activity is even larger consider-ing the fact that activity simultaneouslyoriginating in distant neuronal structures is always picked up by the recordingelectrodes (Skrandies et al., 1978).

In contrast to the spontaneous EEG ofrelatively large amplitude, stimulus-relatedbrain activity is much weaker, and may berevealed only by some processing of thedata. More than 100 years ago sensoryevoked potentials had been recorded bythe English physiologist Caton, (1875)directly from the cortical surface of rabbitand monkey brains, whereas the recordingof human evoked potentials became feas-ible only after the advent of electronic and computerized signal-processing capa-bilities in the 1950s and 1960s.

Evoked potentials are systematic changesof the EEG induced by incoming informa-tion to the brain. Every sensory stimuluselicits electrical activity that is projected byselective and specialized afferent fiber sys-tems to the corresponding cortical sensoryareas, where it induces changes of theongoing electrical activity. These changesdepend on (1) the function state of thebrain (information processing is differentduring various sleep stages and in differ-

72 4. EVOKED POTENTIALS STUDIES

II. COGNITIVE AND AFFECTIVE PROCESSING OF THE BRAIN

ent states of wakefulness), (2) the specificmeaning and importance of a given stimu-lus (attention and cognitive set of thesubject determine the way stimuli areprocessed), and (3) the physical stimulusparameters and sensory thresholds of theorganism (which also influence thesubject’s sensory capability to perceivestimuli).

A single sensory stimulus evokes brainactivity of only very small amplitude;although the ongoing EEG has amplitudesin the order of up to 150 µV, evoked brainactivity reaches amplitudes between only0.1 and 20 µV, and such small changescannot be detected in the spontaneousEEG. Signal averaging enables ident-fication of evoked activity: the same stimu-lus is presented repeatedly, and typicallyEEG segments of 10–1000 msec in length,following each stimulus presentation, areaveraged. Stimulus-related brain activitylooks similar in repeated trials, whereas thespontaneous EEG shows a randomizedamplitude distribution over time.

Figure 1 illustrates how visual evokedpotentials (VEPs) are obtained by averag-ing. From animal experiments it is knownthat neurons in the mammalian visualcortex can be activated optimally by con-trast changes. In order to obtain the dataillustrated in Fig. 1, at time 0 msec the con-trast of a checkerboard pattern is reversed,and the EEG segment following stimula-tion is recorded for an epoch of 1000 msec.This procedure is repeated at a rate of2 Hz, thus two stimuli occur during therecording epoch. The contrast reversalelicits an occipitally positive componentwith a latency of about 100 msec that,however, cannot be seen in the raw EEG.With the averaging of only a few trials it isobvious that large, independent potentialfluctuations occur with positive and nega-tive polarity at random (see upper twocurves in Fig. 1); with further averagingtheir mean value tends toward 0 µV. Onthe other hand, all stimulus-related, time-locked VEP activity shows consistent

polarities over the recording epoch, and astable potential configuration emergesafter summation of several single poten-tials. This can be seen for both contrastreversals that occur every 500 msec (stimu-lus onset is indicated in Fig. 1 by the solidvertical lines at 0 and 500 msec). The aver-aging of 32 single potentials yields a VEPwave form with a consistent positive com-ponent with a latency of about 100 msec(indicated by dashed lines in Fig. 1), andadditional stimulus presentations result in

NEUROPHYSIOLOGICAL BASES OF EVOKED ELECTRICAL BRAIN ACTIVITY 73


FIGURE 1 Averaging of evoked EEG potentials:every 500 msec the contrast of a checkerboard patternpresented electronically on a monitor is reversed(denoted by vertical lines), and the EEG is recordedover the occipital brain areas for an epoch of 1000msec. The numbers on the right indicate the numberof trials averaged, and it is obvious that in the sponta-neous EEG (upper two curves) no consistent patternof electrical brain activity can be detected. The averag-ing of the EEG following a different number of stimu-lus presentations (n = 2, 4, 8, 16, 32, 64, 128) showshow stimulus-dependent activity emerges, whereasindependent potential fluctuations with positive andnegative polarity occurring at random cancel eachother. A positive component with a latency of about110 msec marked by the dashed vertical line, can be seen after 32 individual sweeps. Note that the VEP amplitude reaches about 15 V, whereas thestimulus-independent EEG is in the order of 20 to 30 V.

a further decrease in variability of therecorded signal. Evoked components are commonly denoted by their relativepolarity (positive, P; negative, N) followedby the number of milliseconds of theirapproximate latency. Thus, Fig. 1 displaysthe so-called P100 component of the VEP.

The latencies and amplitudes of evokedpotential components as well as thenumber of stimulus presentations neededto obtain a stable evoked brain responseare dependent on the physical stimulusparameters as well as on the sensorymodality: the amplitudes of the so-calledbrain stem auditory evoked potentials arein the order of 0.2–1.0 µV, thus 1500–2000single evoked responses have to be used(Jewett et al., 1970). The small amplitudesare due to the fact that the neuronal ele-ments activated are located in the brainstem structures of the auditory pathway,and thus are far away from the recordingelectrodes on the scalp. Along a similarline, fewer cortical neurons are selectivelysensitive to visual horizontally disparatestimuli than they are to contrast changes,and 300–600 trials are needed to obtain a VEP elicited by stereoscopic stimuli (see later discussion on stereoscopicperception).

Evoked brain activity consists of asequence of components that are inter-preted to reflect steps in information pro-cessing; these must be determined byquantitative methods (Lehmann andSkrandies, 1980; Skrandies, 1987). Suchcomponents occur at times of high neuralactivity accompanied by strong electricalfields, and large potential differences areseen in the recorded wave forms. The mainparameters extracted from evoked brainactivity are component latencies (timebetween stimulus presentation and theoccurrence of a given component indicat-ing neural processing times), amplitude(strength of the evoked electrical field,indicating the degree of synchronous neu-ronal activation), and amplitude topogra-

phy, which may give some indication ofthe neuronal populations involved in theprocessing of a given stimulus.

It is important to note that mapping ofelectrical activity does not make it possibledirectly to draw conclusions on the exactneuroanatomical locations of the intracra-nial sources. Neuronal mass activity pro-duces electrical fields that spread viavolume conduction throughout the brain,and these can be recorded at locationsdistant from the generating source. Thishas been shown in a study on variousstages of the cat visual system, wherebysingle unit activity and field potentialswere compared and the spread of electricalactivity was evident throughout the brain (Skrandies et al., 1978). Thus, modelsource computations on scalp-recordeddata always have to rest on certain explicit(and sometimes implicit) assumptions con-cerning the number, location, and spatialextent of dipoles as well as the homogene-ity and geometry of the intracranial mediain order to arrive at physiologically mean-ingful solutions (Koles, 1998). It is alsoobvious that the “inverse problem” of howto determine the sources of potentials in aconductive medium, when the scalp poten-tial field is given, has no unique solution(Von Helmholtz, 1853). The computation ofequivalent dipoles thus must be regardedas a further step of data reduction. Themultidimensional scalp data space con-sisting of potential measurements frommany electrode locations may be explainedor modeled by an equivalent dipole dataspace with typically fewer dimensions. It is evident that such a data reduction has to be performed for each poststimulustime point separately, whereby the solu-tion reflects the instantaneous sourceconfiguration.

As will be shown in the next section,direct interpretations of scalp potentialfields in terms of intracranial neuronal gen-erator locations may be misleading. This isdue to the fact that the inverse problem,and source localizations, should mainly be



viewed as models that explain the scalp-recorded activity.

Visually elicited activity may berecorded noninvasively, both at the level ofthe retina as electroretinogram (ERG) andfrom the visual cortex as visual evokedpotential. Here the focus is concentrated onVEP activity, and the reader is referredelsewhere for a more detailed descrip-tion of electroretinographical methods in basic research and in clinical settings(Armington, 1974; Heckenlively andArden, 1991). Skrandies and Leipert (1988)give some instructive examples on how thecombination of cortical and retinal electro-physiological recordings allows the topo-logical identification and diagnosis of thecauses of visual field defects in neuro-ophthalmologic patients.

MULTICHANNEL RECORDINGAND TOPOGRAPHIC MAPPING

EEG activity derives from an electricalfield, the characteristics of which vary withtime and space. Thus, the position of record-ing electrodes on the scalp determines thepattern of the recorded activity; multichan-nel recordings of EEGs and evoked poten-tials enable topographical analysis of theelectrical fields of the brain as they are

reconstructed from many spatial samplingpoints. [For details of topographical analy-sis of EEG data, see also Skrandies (2002)and Appendix E, this volume.]

Figure 2 illustrates the scalp distributionof evoked potential fields between 60 and190 msec obtained from 30 electrodes, withcontrast reversing stimuli presented to dif-ferent retinal areas. The upper row in Fig. 2shows maps of activity evoked by stimulipresented to the left hemiretina; thebottom map series shows activity elicitedby visual stimuli presented to the rightretinal half. The maps in the middle rowillustrate the activity evoked when thesame stimulus was foveated by the subject,and one can see that the major positivecomponent that occurs at a latency of 110 msec shows a symmetrical distributionover the occipital areas. When lateralizedstimuli are presented to the subject, it isobvious that the evoked potential fieldsshow a strong lateralization of activitydepending on retinal stimulus location. In humans, the retinal projections to thevisual cortex are very orderly, resulting ina retinotopic cortical representation of thevisual field. Due to the decussation of the ganglion cell fibers originating fromthe nasal retina in the optic chiasm, eachvisual half-field projects to the contra-lateral visual cortex. Thus, with lateralized

MULTICHANNEL RECORDING AND TOPOGRAPHIC MAPPING 75


FIGURE 2 Topographical distribution of potential fields between 60 and 190 msec at 10-msec intervals evokedby contrast reversal stimuli presented to different retinal areas. Recordings were obtained simultaneously from 30electrodes distributed over the head (note head scheme in inset). Checkerboard reversal stimuli were presented tothe left or right hemiretina or with central fixation of the subject. In all map series a major positive componentoccurs with a latency of 110 msec with a symmetrical occipital distribution for central stimuli, and with a lateral-ized distribution for lateral stimuli. Equipotential lines in steps of 2 V; hatched areas are negative with respect tothe average reference.

stimulation, neurons are activated in thevisual cortex ipsilateral to the retinal halfthat receives the stimulus. Inspection of thepotential distributions in Fig. 2 reveals,however, that the evoked potential dataappear to show a different picture: withlateral half-field stimulation we see anoccipital positive component occurringbetween 100 and 120 msec after stimula-tion that is largest over the contralateralhemisphere (maps series in Fig. 2, top andbottom). This effect is called “paradoxicallateralization” of the VEP. From earlierwork it is known that visual stimuli pre-sented in the lateral half-fields yield acomplex pattern of asymmetric distribu-tions of evoked potential components onthe scalp: ipsilateral and contralateral com-ponent locations have been described, andthe direction and amount of scalp potentiallateralization appears to depend criticallyon the physical stimulus parameters(Skrandies and Lehman, 1982). Of course,the lateralization of evoked potential com-

ponents is different from hemisphericspecialization effects.

One aim of electrophysiological record-ings of human brain activity is theidentification of the underlying sources inthe brain. Information is processed in cir-cumscribed areas of the central nervoussystem, and spontaneous activity also orig-inates from specific brain structures. Thus,it appears of consequence to try to explainthe topography of scalp distribution pat-terns in terms of anatomical localization ofneuronal generators. To arrive at validinterpretations of scalp-recorded data is notrivial task: a fundamental and severe com-plication constitutes the so-called “inverse”problem that cannot be uniquely solved.Any given surface distribution of electricalactivity can be explained by an endlessvariety of intracranial neural source distri-butions that produce an identical surfacemap. Thus, there is no unique numericalsolution when model sources are deter-mined, but knowledge of the anatomy and



FIGURE 3 Original and reconstructed potential fields evoked by stimuli in the left or right visual field atcomponent latency (110 msec for left, 105 msec for right visual field stimuli). Original maps represent the originalpotential distribution; the model maps are the surface distribution computed from the model dipoles. The greatsimilarity between the recomputed and original potential fields indicates that the model dipoles are able toexplain most of the variance in the data (more than 95% for each data set). The head schemes with the results ofmodel dipole computations are shown from above or from the left side. Dots mark the locations of the dipole, thelines indicate dipole orientation and strength. Recordings are in 30 channels, with electrodes evenly distributedbetween the inion and 25% of the nasion–inion distance (see scheme in Fig. 2).

physiology of brain systems allows deduc-tion of a meaningful source localization(see below and Fig. 3). We note that thisstill implies that model sources are deter-mined and this aspect needs to be consid-ered when interpreting evoked potential orEEG data (Skrandies, 2002).

Data such as those illustrated in Fig. 2indicate that the scalp locations ofstrongest electrical activity do not necessar-ily coincide with the intracranial localiza-tion of the neuronal generators, and mostattention should be drawn to the areaswhere steep potential gradients occur. Thiswill now be illustrated with results fromdipole location computations. With mathe-matical approaches it is possible tocompute the electric potential distributionon the surface of a homogeneous conduct-ing sphere surrounded by air, which is dueto a point current dipole inside the sphere(cf. Pascual et al., 1990). In addition, itappears reasonable to assume the head tobe represented best by a concentric three-shell model (Ary et al., 1981). Othermethods for localization of sources of elec-trical brain activity consider more complexanatomical (realistic head model) andphysiological (distributed sources) infor-mation, as is illustrated by the merging ofdata from imaging methods and electro-physiological data (Fuchs et al., 1998;George et al., 1995; Koles, 1998; Pascual-Marqui et al., 1994; Skrandies, 2002).

The results of such a source localizationcomputation for lateralized visual evokedpotential fields are given in Fig. 3. In orderto control for positional errors, prior tocomputation all electrode locations havebeen quantified by digitizing their posi-tions in three dimensions on the subject’shead. This information, along with thepositional information on the majoranatomical landmarks of the head, wasused for computation of best-fit dipolelocalizations. Figure 3 illustrates the dipolesource location of a component occurringbetween 105 and 110 msec latency after thepresentation of a visual stimulus in the left

or right visual half-field. Componentlatency was determined by the computa-tion of maximal field strength as the meanstandard deviation within the field at eachtime point [i.e., global field power (GFP)](Lehmann and Skrandies, 1980; Skrandies,1987) (see also Appendix E, this volume).

The evoked potential fields at com-ponent latency [110 msec for stimuli pre-sented in the left visual field (i.e., on the right hemiretina; upper part of Fig. 3),105 msec for stimulation of the right visualfield (i.e., on the left hemiretina; lower partof Fig. 3)] are illustrated in the maps on theleft side of Fig. 3. As seen before, lateralvisual stimuli result in a “paradoxical”lateralization, with potential maximaoccurring contralateral to the retinal halfstimulated. The results of the dipole com-putation are given as source localizationsin the schematic head as seen from aboveor from the left side. Despite the lateraliza-tion of high peaks of activity over the con-tralateral hemisphere, the model sourcesare located in the hemisphere ipsilateral tothe hemiretina stimulated. This is in linewith the anatomy of the visual cortexwhereby the retinal projections arrive inthe calcarine fissure in the medial part ofthe ipsilateral occipital cortex. Electrodesover the contralateral hemisphere, how-ever, appear to be located optimally torecord most of the activity originatingfrom the calcarine cortex. This observationhas been confirmed by intracranial record-ings in human patients (Lehmann et al.,1982), and it also explains why the retinalextension of the stimulus determines theamount of lateralization of the evokedbrain activity (Skrandies and Lehmann,1982).

We must stress the point that suchdipole computations result in a model thatbest explains the electrical field recordedon the scalp, but due to the inverseproblem discussed above, there is nounique solution for such computations.Anatomical locations of the neuronalsources determined may be quite different

MULTICHANNEL RECORDING AND TOPOGRAPHIC MAPPING 77


if the same data have to be fit by more thanone dipole or if different assumptionsabout conductivity and head geometry aremade. On the other hand, the “model”maps computed from the dipole sourcesolution (the so-called forward solution)are very similar to the measured data ofour example: for both data sets illustratedin Fig. 3, more than 95% of the variance isexplained. This indicates that the datareduction achieved by the model dipolecomputation yields reasonable results. It isimportant to keep in mind that theabsolute locations of the potential maximaor minima in the field do not necessarilyreflect the location of the underlying gener-ators (this fact has led to confusion in the

EEG literature, and for visual evoked activ-ity this phenomenon became known as“paradoxical lateralization”). Rather, thelocation of steepest potential gradients inthe scalp fields is a more adequate parame-ter, indicating the intracranial source loca-tions. This is evident when the potentialdistribution maps and the location of themodel dipoles are inspected in Fig. 3.

STEADY-STATE VEPs: INFLUENCEOF STIMULATION FREQUENCY

The data illustrated in Fig. 1 and 2 wereelicited by a checkerboard pattern revers-ing in contrast at a rate of 2 reversals/sec,



FIGURE 4 Visual potential wave forms evoked by a checkerboard pattern reversing with 14, 12, 10, 8, and 6 reversals/sec. Note how response frequencies follow stimulation frequency. On the right the normalized ampli-tude spectra are shown stemming from a fast Fourier transform computation. For each condition the amplitudecorresponding to the stimulation frequency is plotted. Data recorded from Oz versus a reference at Fz.

and two responses occur in 1 sec. Withincreasing stimulation frequency the brainresponses follow the reversal rates becauseeach single stimulus elicits time-lockedbrain activity, and subsequent componentsoverlap. This is shown in Fig. 4, where VEPactivity was recorded while a checkerboardstimulus was reversed in contrast at 6, 8,10, 12, or 14 reversals/sec.

All VEP wave forms are highly corre-lated with the stimulation frequency: e.g.,for 6 reversals/sec three components occurwithin 500 msec, and 8 reversals/sec elicitfour components, etc. This means thatevery contrast reversal is followed byevoked activity in the visual cortex. Suchpotentials are also called “steady-state”VEPs because it is assumed that repetitivestimulation results in a continuous streamof steady responses. It is also obvious thatdifferent frequencies yield brain responsesof different strengths. Note that amplitudesare largest with the stimulus changing at6 reversals/sec, and there is some ampli-tude tuning for the higher frequencies,with a relative maximum occurring for astimulation frequency of 10 reversals/sec.Frequency analysis may be used to quan-tify amplitudes in given frequency bandsand also to detect brain responses in noisysignals. Conventionally, this is done bycomputing a fast Fourier transform (FFT)on the VEP amplitudes, resulting in adescription of the data in the frequencydomain. As is evident from the wave formsin Fig. 4, low stimulation rates evoke time-locked activity as can be determined byvisual inspection of the VEPs for frequen-cies up to a rate of 12 reversals/sec. On theother hand, it looks like there is no stimu-lus-related activity when the subjectobserves a checkerboard pattern changingat 14 reversals/sec.

The results of a frequency analysis,however, reveal that also with 14 rever-sals/sec significant stimulus-related VEPactivity may be detected (see spectra inFig. 4). Although its amplitude is rathersmall and the VEP wave form is largely

dominated by lower frequencies, thereoccurs a clear peak at 14 Hz when thepower spectra resulting in a frequencyanalysis are consulted (see Fig. 4, rightside). Thus, prior knowledge of the tempo-ral course of stimulation allows one todetect brain activity related to the process-ing of visual input.

Stimulation with high temporal fre-quencies may also be employed in order tostudy the time resolution and refractoryperiods of the human visual system. Withdouble-flash stimulation (two flashesoccurring at intervals in the order of mil-liseconds) Skrandies and Raile (1989)demonstrated that both retinal and corticalactivity may be recorded with suchstimuli. Most interestingly, even when thesubject was not able to perceive twoflashes, the evoked neuronal activityshowed two responses. In addition, therewere significant differences of retinal andcortical structures in temporal resolutioncapabilities: with intervals below 40 mseconly very few subjects displayed a VEPresponse to the second flash, whereas theelectroretinogram yielded two separateresponses to a double flash even with aninterval of only 10 msec. Differencesbetween retinal and cortical processing arefurther supported by the fact that thereappears to exist no direct relationshipbetween the amplitudes and latencies ofretinal and cortical potentials (Skrandiesand Raile, 1989) indicating that knowledgeof electrophysiological parameters in theretina does not allow prediction of howcortical potentials will be influenced by thevariation of stimulus parameters.

PERCEPTUAL LEARNING,NEURAL PLASTICITY, AND

HIGHER COGNITIVE PROCESSES

Cognitive effects also affect evokedbrain activity, which depends on theinformation processing during a task.Factors such as attention, motivation, or

PERPETUAL LEARNING, NEURAL PLASTICITY, AND HIGHER COGNITIVE PROCESSES 79


expectancy as well as the occurrence prob-ability of stimuli determine the pattern ofelectrical brain activity. The presentation oftask-relevant information yields so-calledendogenous components, commonly occur-ring with large latencies of more than 300 msec. In general, such components areelicited only when the stimulus is relevantfor a task and the subject attends to thestimuli. An identical stimulus presentedwithout task relevancy is not followed bythis kind of activity. These endogenouscomponents are interpreted to reflecthigher, cognitive information-processingsteps (see also Gazzianiga et al., 1998;Picton, 1988; Skrandies, 1995).

One may find that attentional processesalso influence earlier components. In astudy on visual information processing therandomized presentation of relevant andirrelevant alphanumeric and geometricstimuli yielded significant differences inevoked components at a latency of onlyabout 100 msec, in addition to the expectedeffects occurring at much longer latencies(Skrandies et al., 1984). This indicates thatrather early information-processing steps,probably in primary visual cortical areas,are influenced when the subject is involvedin a pattern discrimination task [seeSkrandies (1983) for a more detaileddescription]. Such data are in line with thereport by Zani and Proverbio (1995), whoillustrated the effect of selective attentionto the size of checks (squares) on early VEPcomponents.

In addition, there are systematic changesin the scalp topography of event-relatedbrain activity during processing of lan-guage stimuli (Skrandies, 1998). Accordingto the “semantic differential technique” theaffective meaning of words can bequantified in statistically defined, indepen-dent dimensions, in which every word isuniquely located in three dimensions—evaluation (good/bad), potency (strong/weak), and activity (active–passive). Thesedimensions are very stable and cultureindependent (Osgood et al., 1975). It is

important to note that visual processing ofwords yields a scalp topography of theP100 component which is globally similar tocheckerboard evoked brain activity. Withmore detailed topographical analysis,however, small but significant differencesappeared when the locations of the posi-tive and negative centers of gravity (so-called centroids; for details see AppendixE) were compared (Skrandies, 1998).

When brain activity evoked by differentsemantic word classes was analyzed,significant effects were not restricted to late“cognitive” components, but brain activityat early latencies (corresponding to theP100 component) was affected by semanticmeaning of the stimuli. These data showhow visually evoked brain activity is mod-ulated by the meaning of the stimuli atearly processing stages (see Skrandies,1998). In a similar way, it has been illus-trated that attention affects early steps ofvisual processing (Skrandies et al., 1984).Thus, the influence of attention and cogni-tive parameters on activation of the visualcortex can be studied electrophysiologi-cally by recordings of VEP activity.

Similarly, there are effects on brain elec-trical activity that accompany learningprocesses. From psychophysical experi-ments it is known that performance of anumber of perceptual tasks improves as aresult of training, not only during the onto-genetic development in early childhoodbut also in adults, reflected by perceptualimprovements in sensory discrimina-tion ability (Fiorentini and Berardi, 1981;Gibson, 1953). Similarly, neurophysiologi-cal studies on the cortical plasticity in adultanimals have established that the repre-sentation of sensory functions in corticalareas is not hard wired, and it can changeas a function of repeated stimulus pro-cessing. Selective deafferentiation is themost drastic alteration of adequate sens-ory input: the interruption of peripheralsomatosensory afferences is followed by an extensive rewiring of cortical projec-tions in the somatosensory cerebral cortex



(Merzenich et al., 1984). Such modificationswere also found in animals trained for per-ceptual discrimination, whereby functionalchanges of cortical mechanisms have beendocumented with invasive neurophysio-logical methods; there is a correlationbetween functional changes observed insingle-unit responses and sensory dis-crimination performance in adult monkeysobserved with somatosensory stimuli(Recanzone et al., 1992) as well as inauditory frequency-discrimination tasks(Recanzone et al., 1993). For the mam-malian visual system, functional changesin receptive field organization of corticalneuronal assemblies were demonstrated(Gilbert and Wiesel, 1992).

Electrophysiological correlates of im-proved discrimination performance of spe-cial visual stimuli have been reported bySkrandies and Fahle (1994) and bySkrandies et al. (2001). The human visualsystem is able to resolve stimuli with anaccuracy that is much better than thespacing of individual photoreceptors in theretina: the small offset of line stimuli (so-called vernier targets) in the order of only a few seconds of arc may be detected bynormal human subjects, yielding muchhigher than usual visual acuity coined(“hyperacuity”) (see overview in West-heimer, 1982). The repeated presentation ofsuch visual vernier targets during anexperimental session is accompanied by asignificant improvement in sensory thresh-olds within less than half an hour of train-ing. This learning is stimulus specific,based on the demonstration that there is notransfer of improved performance whenthe orientation of the stimuli is changed by90° (Poggio et al., 1992). Thus, there are noattention effects, and the subject does notlearn simply to adjust to the experimentalprocedure, but rather very specific information-processing strategies improveby training. In different populations ofhealthy adults, Skrandies and Fahle (1994)and Skrandies et al. (1996) consistentlyfound that the mean performance

improved significantly within about halfan hour of passive training. Such improve-ments in performance derived from psy-chophysical testing are paralleled bysignificant alterations of electrical brainactivity. Analysis of potential distributionsrecorded over the occipital areas of thesubjects during the learning phaserevealed highly significant changes. Theseeffects were reflected by differences in thetopography of the scalp potential distribu-tions, with short latencies below 100 msecas well as at latencies extending to over500 msec.

Figure 5 illustrates the mean potentialfields of 10 subjects at 250 msec latencyevoked by the first or second block of 600presentations of vertical or horizontalvernier targets. Note the very small ampli-tude of the brain electrical response tosuch weak stimuli. This is commonlyobserved also with three-dimensionalvisual stimuli that selectively activate onlya small population of neurons in thehuman visual cortex (see below). From Fig. 5 it is obvious that for both stimulusorientations the scalp potential fields thatare elicited after learning are very differentfrom those evoked by the same stimulusbefore learning. Before the training thepotential fields are shallow, and displayonly little activity. After training a strongnegative component over the occipitalareas can be seen. The direct statisticalcomparisons computed as paired t-testsbetween maps turn out to be highlysignificant, indicating the occurrence ofdifferent electrical brain activity afterlearning (see significance probability mapsillustrated in Fig. 5). These differencescannot be explained by a different timecourse of evoked activity in the two condi-tions, as has been shown in a number ofdifferent studies (Skrandies and Fahle,1994; Skrandies et al., 1996, 2001). Learningof a vertical vernier target and learning ofa horizontal target are both followed bysimilar electrophysiological changes,although differences in electrical activity,

PERPETUAL LEARNING, NEURAL PLASTICITY, AND HIGHER COGNITIVE PROCESSES 81


depending on the orientation of gratingstimuli, have been reported (Skrandies,1984). Similarly, in our psychophysicalexperiments we could verify that improve-ments in performance over time are inde-pendent of the orientation of the stimulus(Skrandies and Fahle, 1994; Skrandies et al.,1996), although with vertical vernier stimuli,in general, sensory thresholds are smallerthan with horizontal stimuli (Skrandies et al.,2001).

We note that at certain latencies similarfield configurations occurred, and mainlyfield strength increased after training (e.g., at50, 350, or 550 msec), whereas at other timesthe evoked potential fields were completelyand significantly different, as illustrated inFig. 5. This suggests that the activation ofnew neuronal populations by an identical

stimulus is induced after perceptual learn-ing. In Fig. 5, the data from a componentelicited by vernier stimuli at 250 mseclatency are illustrated, showing large differ-ences in the pattern of electrical brain activ-ity. We note that such significant differencesin electrical brain activity occurred not onlyat this time point but also over extendedperiods of the recording epoch (Skrandiesand Fahle, 1994; Skrandies et al., 1996, 2001).These results suggest that after trainingintracranial neuronal generator populationswere activated in a different way by an iden-tical visual stimulus. It is important that inboth psychophysical and electrophysiologi-cal experiments significant effects of percep-tual learning were obtained with a similartime course. The high correlation betweenthe change in perceptual performance andthe change of stimulus-related brain poten-tial topography as a function of practice, aswell as the spatiotemporal pattern of neu-ronal activation with steep gradients overthe primary visual cortex, corroborates thenotion that perceptual learning occurs at an“early” level of visual processing. This inter-pretation is in line with the fact that neuronsin the striate cortex respond to the presenta-tion of vernier offset stimuli (Swindale andCynader, 1986).

In summary, significant effects of per-ceptual learning can be obtained in bothpsychophysical and electrophysiologicalexperiments with a similar time course—within about half an hour. The covariation ofthe subjective sensory and neurophysiologi-cal results illustrates that more efficient per-ceptual processing is paralleled by activationchanges, mainly over the primary visualcortex, suggesting that implicit learning mayoccur at a relatively early level of perceptualprocessing. The neurophysiological datarecorded noninvasively from large neuronalassemblies of human subjects simultane-ously during perceptual processing reflectthe dynamic changes going on at the corticallevel (Darian-Smith and Gilbert, 1995).

Similar learning takes place with three-dimensional perception. Random-dot stere-



FIGURE 5 Mean evoked potential fields of 10 sub-jects at 250 msec latency evoked by the first (beforetraining) or second block of 600 presentations (aftertraining) of visual hyperacuity targets. The results ofcorresponding t-tests are illustrated on the bottom assignificance probability maps (SPM). The left-handmaps show the data obtained with horizontal stimuli;the right-hand maps display the fields evoked by ver-tical stimuli. Note pronounced and highly significantchanges in topography after learning, which cor-relates with significant improvements of sensorythresholds. Recordings are from 16 electrodes over theoccipital areas (note head scheme in inset); shadedareas are negative with respect to the average refer-ence; steps are in 0.1 V for the potential fields; stepsare in 1.0 t-values units for the t-maps. Data fromSkrandies and Fahle (1994).

ograms (RDSs) have been used in studies formany years (Julesz, 1971; Bergua andSkrandies, 2000), and these stimuli containonly binocular horizontal disparity as depthcue to be extracted by the visual system.Three-dimensional information containedand hidden in RDSs is not easy to see.Similar stimuli are the so-called autostere-ograms, printed in the well-known “Magic-Eye” books; in these it is obvious that naiveobservers must learn to perceive a three-dimensional structure. With RDS stimulimost normal subjects also need severalseconds or even longer to identify stereo-scopic targets, and this response time dropssignificantly with practice (O’Toole andKersten, 1992). Skrandies and Jedynak (1999)have shown in a combined psychophysicaland electrophysiological study that morethan half of the 16 subjects tested learned tosee stereoscopic targets after 8 min of train-ing. It is important to note that significantimprovements in sensory discriminationoccur even after viewing only stimuli belowperceptual threshold. These observations arein line with previous studies on subliminalperception and implicit learning in normalsubjects (Berry and Dienes, 1993) and “blind sight” in brain-damaged patients(Weiskrantz et al., 1974; Zihl, 1980), whichalso suggests that perceptual processing andlearning are possible without consciousawareness. On a cortical level, this learningwas accompanied by topographic changesin the electrophysiological pattern of activa-tion of neural assemblies in the visual cortex,where the center of activity shifted towardthe right hemisphere. Subjects who did notimprove in perception displayed no sucheffects (Skrandies and Jedynak, 1999).

STEREOSCOPIC PERCEPTIONAND EVOKED POTENTIALS:PHYSIOLOGICAL BASIS AND

CLINICAL APPLICATION

Binocular and depth perception havebeen examined for many years and are

being studied today by employing com-puter-generated random-dot stereograms.A historical account of the invention anddevelopment of random-dot stimuli hasbeen given by Bergua and Skrandies(2000).

Three-dimensional depth perception isbased in part on the fact that our two eyessee the environment from slightly differentviewpoints, and depth information isextracted from the horizontal disparity ofvisual stimuli on the two retinas. Slightdifferences in the image viewpoints—hori-zontal disparity—are the crucial cue fordepth perception. Because the input intothe two eyes remains separated up to thelevel of the visual cortex (Bishop, 1973;Gonzalez and Peres, 1998), evoked brainactivity generated exclusively by corticalstructures may be investigated whenrandom-dot stereograms (Julesz, 1971) arepresented binocularly. Such RDS stimulido not contain any contrast information,and can be used to investigate three-dimensional vision in isolation. Com-monly, such stimuli are presenteddynamically at high frequency by moderncomputer graphics (Skrandies, 2001). Theperception of these stereograms dependson the fusion of two nonidentical, horizon-tally disparate inputs in the visual cortex,and under monocular viewing conditionsa dynamic RDS (dRDS) stimulus cannot beperceived stereoscopically. The neuronalcorrelates of these processes are corticalneurons selectively responsive to disparatebinocular stimuli; these as have beenfound in the monkey visual cortex (Hubeland Wiesel, 1970; Von der Heydt et al.,1981).

For experimental studies of humanvision, the use of dRDSs offers the possi-bility to investigate selectively cortical pro-cessing of visual information. Withrandom-dot displays any desired three-dimensional form can be created: e.g., athree-dimensional checkerboard pattern isproduced when each eye sees a differentarray of randomly arranged dots, and one

STEREOSCOPIC PERCEPTION AND EVOKED POTENTIALS 83


of the patterns is horizontally displaced insuch a way that the area of each individualcheck square contains crossed binoculardisparity. After binocular fusion, theseareas will stand out in front of the plane offixation, and a study participant will per-ceive a checkerboard pattern hovering infront of the monitor. The time-locked aver-aging of electrical brain activity is trig-gered by the sudden change in horizontaldisparity, but independent stimulation ofthe two eyes may be achieved by employ-ing polarizing foils or anaglyphs (red andgreen dots in combination with red/greengoggles), or by liquid crystal diode shutterglasses. With such glasses, dRDSs are pre-sented as monocular half-images alternat-ing at the refresh rate of the monitor, andthe glasses are electronically synchronizedwith the monitor frequency so that everysecond image is seen by one eye only. Forpractical application, the different methodsyield very similar perceptual effects, theyare employed only in order to route inde-pendent information to the left and theright eye.

Figure 6 illustrates evoked potentialsrecorded from a healthy young adult.Checkerboard reversal stimuli with con-trast borders were presented either monoc-ularly to the left and right eyes (lowest twocurves) or binocularly (second curve). It isobvious that differences in componentlatencies can be seen when VEPs evokedby monocular and binocular stimulationare compared: stimulation of both the leftand right eyes yields latencies of 113 msec,whereas binocularly evoked brain activitydisplays a component latency of only 103msec. Such binocular summation effectshave been described before, illustratingthat latencies or amplitudes of the evokedpotential are significantly different whenbinocular and monocular conditions arecompared (e.g., Nakayama et al., 1982;Skrandies, 1993). The physiological cor-relate for such findings is the fact thatmany neurons of the mammalian visualcortex are influenced by input from both

eyes (Gonzalea and Perez, 1998), and visionwith two eyes and monocular vision resultin different perceptions. Binocular fusion ofdisparate retinal images yields stereopsis(see below), and vision with two eyesenlarges the visual field. In psychophysicalexperiments it was shown that with binocu-lar stimuli higher contrast sensitivity isobtained, as compared with monoculartargets (Campbell and Green, 1965).

Figure 6 also shows stereoscopic VEPsthat were elicited by the presentation of athree-dimensional checkerboard pattern:red/green anaglyph dynamic RDS patternsthat were generated every 20 msec byspecial hardware were presented on a colormonitor. By using red/green spectacles,each eye saw only a portion of the dotsdisplayed, and the green pattern was hori-zontally displaced in such a way that thesubject could perceive a checkerboard



FIGURE 6 Visual evoked potentials recorded fromthe occipital areas of a healthy subject; obtained withthe reversal in depth of a stereoscopic checkerboardpattern with horizontal disparities changing by 13.8 min of visual angle (top), or with binocular(second curve) or monocular contrast reversal stimuli(lower two curves). Major components are markedby arrows. Note shorter latencies for binocular (103 msec) as compared to monocular stimulation(113 msec). With three-dimensional stimuli, ampli-tudes are small, thus the stereo VEP is scaled differ-ently (see vertical calibration bar). Reference electrodeat Fz; occipital positive is up.

pattern standing in front of the plane of themonitor. Disparities were changed locallyevery 256 msec by 13.8 min of visual angle,resulting in the percept of a three-dimen-sional checkerboard pattern that stood infront of the monitor and reversed in depth.Stereoscopic evoked activity yields evokedcomponents with latencies very similar tothose due to contrast evoked brain activity,whereas stereoscopically elicited am-plitudes were significantly smaller. This isevident in Fig. 6, and has been describedrepeatedly (Skrandies, 1986, 1991, 2001;Skrandies and Vomberg, 1985), suggestingthat the afferent binocular informationflow to the human visual cortex is ofsimilar velocity for processing of bothdynamic RDS patterns and contrastborders. On the other hand, three-dimen-sional stimuli are significantly less effectivein exciting many visual neurons synchro-nously, which is in line with the assump-tion that there are many more neuronssensitive to contrast changes than there areneurons selectively sensitive to binoculardisparate stimuli. In addition, topographiccomparisons of stereoscopic and contrastevoked potential fields suggest that dis-parate retinal stimuli are processed prefer-entially by neuronal populations outsidearea 17 (Skrandies, 1986, 1991, 1997;Skrandies and Vomberg, 1985), which isconsistent with results obtained withsingle-unit recordings in cats and monkeys(Hubel and Wiesel, 1970; Von der Heydt,1981).

Within certain physiological limits,increasing disparities lead to the percep-tion of increasing depth, and one wouldexpect some optimal disparity range whendepth perception is strongest. Thus, stereo-scopic evoked brain activity also dependson the horizontal disparity of the RDSstimulus, and significant effects of horizon-tal disparities on brain electrical activityare observed (Skrandies, 1997).

Figure 7A illustrates the topographicaldata recorded in 30 channels over theoccipital areas with dynamic RDS stimuli

of different disparities, ranging from 7 to24.5 min of arc. Stimulation frequency was6 Hz, and stimulus-locked brain activitywas obtained with all disparity values. Themaps in Fig. 7A are the responses occur-ring at stimulation frequency. With large orsmall disparities, potential field strengthwas rather small, whereas the largestresponses were obtained with intermediatedisparities, as is evident from the fewerfield lines in Fig. 7A. This observationillustrates that there is a functional dispar-ity tuning of cortical activation: low andhigh disparities yield less synchronizedneural activity as compared to intermedi-ate disparities. This tuning of responsestrength is in line with studies on singleneurons in the monkey visual cortex(Gonzalez and Perez, 1998; Hubel andWiesel, 1970). In addition, significant dif-ferences were observed in dRDS evokedbrain activity when central and lateralstimulus locations were compared. Withlateral stimuli (extending from the fovea to17.1° eccentricity), maximal amplitudeswere obtained at larger disparities thanwith central stimuli (Fig. 7B): with RDSstimuli presented to foveal areas, thelargest amplitudes occurred with meandisparities of 10.5 min of arc, whereas withlateral stimuli, sensitivity was largest withstimuli of 14 min of arc (Skrandies, 1997).These observations indicate that moreperipheral and lateral areas are less sensi-tive to disparity information, supportingdata on disparity thresholds for fusionwhere for patent and qualitative stereopsisas well as stereo, thresholds increase withretinal eccentricity (Ogle, 1962).

Further analysis of the data also revealsthat there are not only differences betweencentral and eccentric stimulation, but thatthere are also pronounced differencesbetween brain activity evoked with stereostimuli presented in the left or right visualfield (Fig. 7B). Stimuli located in the rightvisual field show a tuning function with aclear response peak at 14 min of arc dis-parity. Neuronal response strength is



significantly reduced for higher and lowerdisparities, as is reflected by smaller am-plitudes of brain electrical activity. Sur-prisingly, with stimuli presented in the leftvisual field the brain responses lack thistuning function: here all horizontal dispari-ties appear to be processed in a similarway, and there is no peak in the tuningcurve. In a population of 22 test subjectsthere was no preference for a certain dis-parity of the RDS stimuli, and an analysisof variance confirmed a significant interac-tion between visual field location and dis-parity. These results are independent of thelocation of the recording site, becauserecording electrodes over the left and righthemispheres yield very similar results(Skrandies, 1997). The lack of disparitytuning of VEP responses with stimuli inthe left visual field is explained by highintersubject variability of amplitudes inthis stimulus condition. In the case of

large-amplitude variation between sub-jects, the tuning effect is expected to disap-pear. Thus, the basic difference betweenthe processing of disparity information inthe left and right visual fields may beexplained by the smaller variation andhigher consistency of brain activity elicitedby three-dimensional stimuli presented tofoveal areas or parafoveal areas extendingtoward the right visual field, indicating dif-ferences in global processing of three-dimensional information.

Knowledge of the influence of the hori-zontal disparity of VEP activity may beuseful for the clinical application of record-ings of stereoscopically evoked brain activity. In psychophysical and electro-physiological experiments, we comparedpatients with selective disturbance ofstereoscopic vision and healthy youngadults (Vomberg and Skrandies, 1985;Skrandies, 1995, 2001). It is important to



FIGURE 7 (A) Scalp distributions of power of the 6-Hz band recorded in 30 channels (electrode locations asindicated by head scheme). Disparity varied between 7 and 24.5 min of arc, stimulation frequency was 6 depthreversals/sec. Individual maps are shown for stimuli of different disparity; numbers in the figure refer to dispar-ity values, lines are in steps of 0.5 V. Note different response strength with different disparities. (B) Mean fieldstrength computed on 30 channels of the responses in the 6-Hz band as a function of horizontal disparity of thestimulus between 7 and 24.5 min of arc. Note differences in absolute strength between central (solid line) andlateralized stimuli. As ANOVA revealed, disparity tuning is significantly different for dynamic RDS stimulioccurring in the right (dashed line) or left visual field (dotted line). Mean values computed from data on 22 sub-jects. Data from Skrandies (1997).

note that meaningful comparisons are pos-sible only with patients who have normalvisual acuity in both eyes and who possessother normal visual functions (color vision,visual fields, contrast sensitivity). It is com-prehensible that these are rare casesbecause the patients have no subjectivesymptoms and experience only very littledisturbance.

The data displayed in Fig. 8 illustrateevoked potentials recorded from a subjectwith microstrabism. The conventionalVEPs elicited by contrast reversal stimula-tion of the left and right eyes or binocu-larly, show amplitudes and latencies in thenormal range. As in the healthy subject,with binocular stimuli, significantly shortercomponent latencies (108 msec) areobserved as compared with monocularstimulation (115 msec). However, withdRDS stimuli, quite a different pictureemerges: when a stereogram with a hori-

zontal disparity of 13.8 min of visual angleis presented, the major VEP componentdisplays a reduced amplitude as well as alatency prolongation by more than 20 msec, resulting in a component latencyof about 140 msec. These data suggest thatthere is some deficiency that affects onlystereovision in this patient, leaving theprocessing of contrast information intact.This is in agreement with the fact that thispatient had increased psychophysicalthresholds for three-dimensional stimuli.With an increase of horizontal disparity(“more depth”), the pathological VEPactivity may become normal: a dRDSpattern with a horizontal disparity of 27.6 min of visual angle elicits brain activ-ity with normal component latencies (113 msec) and amplitudes that are similarto those of the healthy subject (comparethe upper curves of Figs. 6 and 8).

Related electrophysiological results havebeen described by Vomberg and Skrandies(1985) and Skrandies (1995, 2001) ingroups of patients with various degrees ofstereovision deficiency but normal binocu-lar visual acuity: there is a high correlationbetween disparity thresholds that weredetermined psychophysically and electro-physiologically in a group of patients withvarious degrees of stereovision deficiencybut normal binocular visual acuity.

Such results indicate that the recordingof brain activity evoked by three-dimen-sional stimuli may be employed to objec-tively determine stereovision capability.Another application is the monitoring ofvisual development in infants: similar tomany other sensory functions, depth per-ception is no innate capacity but has to belearned in the first months of life. This hasalso been demonstrated in evoked poten-tial studies in which, with contrast stimuli,evoked brain activity is recordable inneonates; using RDS stimuli, stimulus-related brain activity is elicited only afterabout 4 months of age. The critical periodof stereovision development as deter-mined electrophysiologically is in the



FIGURE 8 Visual evoked potentials recorded fromthe occipital areas of a patient with microstrabism.The upper two curves illustrate stereoscopic VEPselicited by the reversal in depth of a stereoscopiccheckerboard pattern with horizontal disparities of27.6 or 13.8 min of visual angle. The third curve wasobtained with binocular, and the lowest two curveswith monocular, contrast reversal stimuli. Major com-ponents are marked by an arrow. Note latency pro-longation with small disparities, and normal latencieswith large horizontal disparities. With three-dimensional stimuli, amplitudes are small, thus thestereo VEP is scaled differently (see calibration bar).Reference electrode at Fz; occipital positive is up.

order of 10 and 19 weeks of age, which issomewhat earlier than when it is deter-mined by behavioral methods (Petrig et al.,1981).

CLINICAL APPLICATIONS IN NEUROLOGY

AND OPHTHALMOLOGY

The classical application of VEP mea-surements for clinical purposes is its con-tribution to the diagnosis of multiplesclerosis (MS). The main characteristic ofthis disease is the patchy demyelination ofafferent and efferent nerve fibers distrib-uted all over the nervous system, and thepatients have neurological symptoms thatcannot be explained by a single lesion.Myelin is an insulating sheath, found onmost axons, that increases conductionvelocity (cf. Kandel et al., 2000), thus,electrophysiologically, latency prolonga-tions are expected when demyelinationoccurs. In many patients the visual systemis affected at an early state of the disease,and one finds patients with pathologicalVEP results who, however, do not displayany subjective visual symptoms (i.e., theyhave normal visual acuity and visualfields). An optic neuritis occurs frequentlyat a very early stage of the disease, whichin general is followed by a recovery afterseveral weeks, whereas other symptoms—hemiparesis, ataxia, and sensory distur-bances—may be seen only after someyears. The diagnosis of MS is warrantedonly if several independent lesions can bequantified, or if several repeated attacks ofsimilar neurological symptoms occur overtime. Thus, with pathological VEP mea-surements clinically silent lesions can bedetected in patients with no visual symp-toms, and this may contribute to a finaldiagnosis of MS. For more information onpathophysiological and clinical details ofMS the reader is referred to Bauer et al.,(1980) and McKhann (1982).

Other applications of VEP measure-ments in ophthalmology and neurologycomprise the documentation of visualdevelopment in infants (as discussed in thesection on stereoscopic vision) as well asthe topologic localization of disturbancesin the visual system (Heckenlively andArden, 1991). The combined recordings ofVEP and ERG activity allow localization oflesions in the afferent visual pathway inpatients with visual field defects. Skrandiesand Leipert (1988) could demonstrate asignificant relationship between pathologi-cal electrical acivity and the site of thelesion in a group of neuro-ophthalmologi-cal patients. After lesions of the optic nerveor optic tract, ERG changes appear in par-allel to a retrograde degeneration of theaxons of the retinal ganglion cells. On theother hand, in adult patients with corticallesions, no electrophysiological sign of sub-sequent retinal alterations can be found.This has also been demonstrated in con-trolled lesion experiments performed onadult cats (see Skrandies and Leipert, 1988).In summary, such data illustrate the topo-diagnostic possibilities of the combinationof various electrophysiological recordingsin patients with defective vision. Due totheir noninvasive nature and their sensitiv-ity to functional (and not only structuraland anatomical) changes, these methodsare commonly applied for a wide variety ofdiagnostic questions.

CONCLUSION

Classical psychophysics allows study ofintegrative, subjective aspects of sensoryinformation processing, but electrophysio-logical experiments give us tools for theassessment of neuronal mechanisms atvarious levels of the central nervoussystem. In addition to the visual processesdescribed in this chapter, various sensorymodalities may be investigated with onlyminimal active cooperation of the subject,and primary sensory evoked brain activity



is largely independent of cognitive pro-cessing strategies.

High temporal resolution inherent inelectrophysiological recordings helps toreveal steps in information processingoccurring in fractions of a second. Forfunctional analyses of sensory processesthis is a significant advantage over brainimaging techniques such as computertopography, positron emission tomogra-phy, or magnetic resonance imaging, thestrengths of which are the exact anatomicalthree-dimensional identification of struc-tures of the central nervous system. ForMRI, substantial improvements in time res-olution are in reach, as reports on “func-tional MRI” suggest (cf., Belliveau et al.,1991; Frackowiak et al., 1997). However, thedirect relationship between neuronal acti-vation and local hemodynamic changesthat occur on a much cruder time scale still remains unclear (George et al., 1995).Although it has been demonstrated thatfunctions of the human visual cortex maybe successfully studied by fMRI (Wandell,1999), due to the huge cost of the equip-ment and the technical operating expense,however, one may predict that in mostcases the access to such techniques will berestricted to medical centers specialized forclinical diagnosis, and in general it will notbe available on a routine basis for workersin the fields of sensory physiology orexperimental psychology. On the otherhand, electrophysiological recording ofbrain electrical activity is relatively easy toperform, and it has widespread applica-tions in the fields of human basic and clini-cal neurophysiology as well as in cognitiveneuroscience (Gazzanigga et al., 1998).

The noninvasive recording of evokedpotentials constitutes a powerful supple-ment to psychophysical testing, and it mayreveal steps of information processing witha high resolution in the time domain. Ashas been illustrated in this chapter, electro-physiological measures may also beemployed for functionally localizingcertain effects in the central nervous

system, as is also evident from the clinicalapplications of auditory brain stem poten-tials, electroretinography, and visual evokedbrain activity. On the other hand, it isimportant to keep in mind that directinterpretations of electrical brain activity interms of absolute anatomical localizationof neuronal generator populations are notwarranted. Careful experimental planningas well as profound knowledge on theanatomical and neurophysiological basesof perceptual processes are mandatory inorder to arrive at a meaningful inter-pretation of the recorded data. Thus, acombination of psychophysical and elec-trophysiological methods in controlledexperiments holds the promise for furtherinsight into the mechanisms of sensoryinformation processing in the human brainin the future.

Acknowledgment

Supported in part by DeutscheForschungsgemeinschaft, DFG SK 26/8-3.

ReferencesArmington, J. C. (1974). “The Electroretinogram.”

Academic Press, New York.Ary, J. P., Klein, S. A. and Fender, D. H. (1981),

Location of sources of evoked scalp potentials:Corrections for skull and scalp thickness. IEEETrans. Biomed. Eng. BME-28, 447–452.

Bauer, H. J. , Poser, S., and Ritter, G. (eds.) (1980).“Progress in Multiple Sclerosis Research.”Springer, Berlin.

Belliveau, J. W., Kennedy, D. D., Kinstry, R. C.,Buchbinder, B. R., Weisskoff, R. M., Cohen, M. S.,Venea, J. M., Brady, T. J., and Rosen, B. R. (1991).Functional mapping of the human visual cortexby magnetic resonance imaging. Science 245,716–719.

Berger, H. (1929). Über das Elektrenkephalogrammdes Menschen. I. Mitteilung. Arch. Psychiat.Nervenkr. 87, 527–570.

Bergua, A., and Skrandies, W. (2000). An earlyantecedent to modern random dot stereograms—“The Secret Stereoscopic Writing” of Ramón yCajal. Int. J. Psychophysiol. 36, 69–72.

Berry, D., and Dienes, Z. (1993). “Implicit Learning.Theoretical and Empirical Issues.” LawrenceErlbaum Assoc., Hillsdale, New Jersey.

REFERENCES 89


Bishop, P. O. (1973). Neurophysiology of binocularsingle vision and stereopsis. In “Handbook ofSensory Physiology” (R. Jung, ed.), (VII/3A), pp. 255–305. Springer, Berlin.

Campbell, F. W., and Green, D. G. (1965). Monocularversus binocular visual acuity. Nature 208, 191–192.

Caton, R. (1875). The electric currents of the brain. Br. Med. J. 2, 278.

Corey, D. P., and Roper, S. D. (eds.) (1992). “SensoryTransduction.” The Rockefeller University Press,New York.

Cormack, L. K., Stevenson, S. B., and Schor, C. M.(1993). Disparity-tuned channels of the humanvisual system. Vis. Neurosci. 10, 585–596.

Darian-Smith, C., and Gilbert, C. D. (1995).Topographic reorganization in the striate cortex ofthe adult cat and monkey is cortically mediated. J. Neurosci. 15, 1631–1647

Fiorentini, A., and Berardi, N. (1981). Perceptuallearning specific for orientation and spatialfrequency. Nature 287, 43–44.

Frackowiak, R. S. J., Friston, K. J., Frith, C. D., Dolan,R. J., and Mazziotta, J. C. (1997). “Human BrainFunction.” Academic Press, San Diego.

Fuchs, M., Drenckhahn, R., Wischmann, H. A., andWagner, M. (1998). An improved boundaryelement method for realistic volume-conductormodeling. IEEE Trans. Biomed. Eng. 45, 980–997.

Gazzaniga, M., Ivry, R., and Mangun, G. R. (1998).“Cognitive Neuroscience: The Biology of theMind.” Norton, New York.

George, J. S., Aine, C. J., Mosher, J. C., Schmidt, D. M.,Ranken, D. M., Schlitt, H. A., Wood, C. C., Lewine,J. D., Sanders, J. A., and Belliveau, J. W. (1995).Mapping function in the human brain with mag-netoencephalography, anatomical magnetic reso-nance imaging, and functional magnetic resonanceimaging. J. Clin. Neurophysiol. 12, 406–431.

Gibson, E. J. (1953). Improvement in perceptual judge-ments as a function of controlled practice or train-ing. Psychol. Bull. 50, 401–430.

GIlbert, C. D., and Wiesel, T. N. (1992). Receptive fielddynamics in adult primary visual cortex. Nature356, 150–152.

Gonzalez, F., and Perez, R. (1998). Neural mechanismsunderlying stereoscopic vision. Prog. Neurobiol. 55,191–224.

Heckenlively, J. R., and Arden, G. B. (eds.) (1991).“Principles and Practice of Clinical Electro-physiology of Vision.” Mosby, Year Book Inc., St.Louis.

Hubel, D. H., and Wiesel, T. N. (1970). Cells sensitiveto binocular depth in area 18 of the macaquemonkey cortex. Nature 225, 41–42.

Jewett, D. L., Romano, M. N., and Williston, J. S.(1970). Human auditory evoked potentials:Possible brainstem components detected on thescalp. Science 167, 1517–1518.

Julesz, B. (1971). “Foundations of CyclopeanPerception.” The University of Chicago Press,Chicago.

Kandel, E. R., Schwartz, J. H., and Jessel, T. M. (2000).“Principles of Neural Science,” 4th Ed. McGraw-Hill, New York.

Koles, Z. J. (1998). Trends in EEG source localization.Electroencephalogr. Clin. Neurophysiol. 106, 127–137.

Lehmann, D., and Skrandies, W. (1980). Reference-freeidentification of components of checkerboard-evoked multichannel potential fields. Electro-encephalogr. Clin. Neurophysiol. 48, 609–621.

Lehmann, D., Darcey, T. M., and Skrandies, W. (1982).Intracerebral and scalp fields evoked by hemireti-nal checkerboard reversal, and modelling of theirdipole generators. In “Clinical Applications ofEvoked Potentials in Neurology.” (J. Courjon, F. Mauguière, and M. Révol, eds.), pp. 41–48.Raven Press, New York.

McKhann, G. M. (1982). Multiple sclerosis. Annu. Rev.Neurosci. 5, 219–239.

Merzenich, M. M., Nelson, R. J., Stryker, M. P.,Cynader, M. S., Schoppmann, A., and Zook, J. M.(1984). Somatosensory cortical map changesfollowing digit amputation in adult monkeys. J. Comp. Neurol. 224, 591–605.

Nakayama, K., Apkarian, P., and Tyler, C. W. (1982).Spatial frequency limitations in binocular neurons:Visual evoked potential evidence. Ann. N.Y. Acad.Sci. 388, 610–614.

Ogle, N. K. (1962). The optical space sense. In “TheEye” (H. Davson, ed.), Vol.4, pp. 211–432.Academic Press, New York.

Osgood, C. E., May, W. H., and Miron, M. S. (1975).“Cross-cultural Universals of Affective Meaning.”University of Illinois Press, Urbana, Illinois.

O’Toole, A. J., and Kersten, D. J. (1992). Learning to seerandom-dot stereograms. Perception 21, 227–243.

Pascual, R., Biscay, R., and Valdes, P. (1990). The phys-ical basis of electrophysiological brain imaging:Exploratory techniques for source localization andwaveshape analysis of functional components ofelectrical brain activity. In “Machinery of theMind—Data, Theory, and Speculations aboutHigher Brain Function,” (E. R. John, ed.), pp. 435–459. Birkhäuser, Boston.

Pascual-Marqui, R. D., Michel, C. M., and Lehmann,D. (1994). Low resolution electromagnetic tomo-graphy: A new method for localizing electricalactivity in the brain. Int. J. Psychophysiol. 18, 49–65.

Petrig, B., Julesz, B., Kropfl, W., Baumgartner, G., andAnliker, M. (1981). Development of stereopsis andcortical binocularity in human infants. Science 213,1402–1405.

Picton, T. W. (ed.) (1988). “Handbook of Electro-encephalography and Clinical Neurophysiology,Revised Series, Vol. 3, Human Related Potentials.”Elsevier, Amsterdam.



Poggio, T., Fahle, M., and Edelman, S. (1992). Fast per-ceptual learning in visual hyperacuity. Science 256,1018–1021.

Recanzone, G. H., Merzenich, M. M., and Schreiner, C.E. (1992). Changes in the distributed temporalresponse properties of SI cortical neurons reflectimprovements in performance on a temporallybased tactile discrimination task. J. Neurophysiol.67, 1071–1091.

Recanzone, G. H., Schreiner, C. E., and Merzenich, M.M. (1993). Plasticity in the frequency represen-tation of primary auditory cortex followingdiscrimination training in adult owl monkeys. J. Neurosci. 13, 87–103.

Singer, W. (1999). Neuronal synchrony: A versatilecode for the definition of relations? Neuron 24,49–65.

Skrandies, W. (1983). Information processing andevoked potentials: Topography of early and latecomponents. Adv. Biol. Psychiatr, 13, 1–12.

Skrandies, W. (1984). Scalp potential fields evoked bygrating stimuli: Effects of spatial frequency andorientation. Electroencephalogr. Clin. Neurophysiol.58, 325–332.

Skrandies, W. (1986). Visual evoked potential topogra-phy: Methods and results. In “TopographicMapping of Brain Electrical Activity,” (F. H. Duffy,ed.), pp. 7–28 Butterworths, Boston.

Skrandies, W. (1987). The upper and lower visual fieldof man: Electrophysiological and functional differ-ences. Prog. Sens. Physiol. 8, 1–93.

Skrandies, W. (1991). Contrast and stereoscopic visualstimuli yield lateralized scalp potential fields asso-ciated with different neural generators.Electroencephalogr. Clin. Neurophysiol. 78, 274–283.

Skrandies, W. (1993). Monocular and binocularneuronal activity in human visual cortex revealedby electrical brain activity mapping. Exp. BrainRes. 93, 516–520.

Skrandies, W. (1995). Visual information processing:Topography of brain electrical activity. Biol.Psychol. 40, 1–15.

Skrandies, W. (1997). Depth perception and evokedbrain activity: The influence of horizontal dispar-ity. Vis. Neurosci. 14, 527–532.

Skrandies, W. (1998). Evoked potential correlates ofsemantic meaning: A brain mapping study. Cogn.Brain Res. 6, 173–183.

Skrandies, W. (2001). The processing of stereoscopicinformation in human visual cortex: Psycho-physical and electrophysiological evidence. Clin.Electroencephalogr. 32, 152–159.

Skrandies, W. (2002) EEG topography. In “TheEncyclopedia of Imaging Science and Technology,”(J. P. Hornak, ed.). John Wiley & Sons, New York(in press).

Skrandies, W., and Fahle, M. (1994). Neuro-physiological correlates of perceptual learning inthe human brain. Brain Topogr. 7, 163–168.

Skrandies, W., and Jedynak, A. (1999). Learning to see3-D: Psychophysical thresholds and electricalbrain topography. NeuroReport 10, 249–253.

Skrandies, W., and Lehmann, D. (1982). Spatial prin-cipal components of multichannel maps evokedby lateral visual half-field stimuli. Electro-encephalogr. Clin. Neurophysiol. 54, 662–667.

Skrandies, W., and Leipert, K. P. (1988). Visual fielddefects are not accompanied by electro-physiological evidence of transsynaptic retrogradedegeneration. Clin. Vis. Sci. 3, 45–57.

Skrandies, W., and Raile, A. (1989). Cortical andretinal refractory periods in human visual system.Int. J. Neurosci. 44, 185–195.

Skrandies, W., and Vomberg, H. E. (1985). Stereo-scopic stimuli activate different cortical neurons in man: Electrophysiological evidence. Int. J.Psychophysiol. 2, 293–296.

Skrandies, W., Wässle, H., and Peichl, L. (1978). Arefield potentials an appropriate method for demon-strating connections in the brain? Exp. Neurol. 60,509–521.

Skrandies, W., Chapman, R. M., McCrary, J. W., andChapman J. A. (1984). Distribution of latent com-ponents related to information processing. Ann.N.Y. Acad. Sci. 425, 271–277.

Skrandies, W., Lang, G., and Jedynak, A. (1996). Sens-ory thresholds and neurophysiological correlates ofhuman perceptual learning. Spatial Vis. 9, 475–489.

Skrandies, W., Jedynak, A., and Fahle, M. (2001)Perceptual learning: Psychophysical thresholdsand electrical brain topography. Int. J. Psycho-physiol. 41, 119–129

Swindale, N. V., and Cynader, M. S. (1986). Vernieracuity of neurons in cat visual cortex. Nature 319,591–593.

Vomberg, H. E., and Skrandies, W. (1985).Untersuchung des Stereosehens im Zufallspunkt-muster-VECP: Normbefunde und klinischeAnwendung. Klin. Monatsbl. Augenheilkd. 187,205–208.

Von der Heydt, R., Hänni, P., Dürsteler, M., andPeterhans, E. (1981). Neuronal responses to stereo-scopic stimuli in the alert monkey—A comparisonbetween striate and prestriate cortex. PflügersArch. 391, R34.

Von Helmholtz, H. (1853). Über einige Gesetze derVertheilung elektrischer Ströme in körperlichenLeitern, mit Anwendung auf die thierelektrischenVersuche. Ann. Phys. Chem. 29, 211–233, 353–377.

Wandell, B. (1999). Computational neuroimaging ofhuman visual cortex. Annu. Rev. Neurosci. 22,145–173

Weiskrantz, L., Warrington, E. K., Sanders, M. D., andMarshall, J. (1974). Visual capacity in the hemiano-pic field following a restricted occipital ablation.Brain 97, 709–728.

Westheimer, G. (1982). Visual hyperacuity. Prog. Sens.Physiol. 1, 1–30.

REFERENCES 91


Zani, A., and Proverbio, A. M. (1995). ERP signs of early selective attention effects to check size. Electroencephalogr. Clin. Neurophysiol. 95,277–292.

Zihl, J. (1980).”Blindsight”: Improvement of visuallyguided eye movements by systematic practice inpatients with cerebral blindness. Neuropsychologia18, 71–77.




C H A P T E R

5

MEG Studies ofVisual Processing

C. J. Aine and J. M. Stephen

INTRODUCTION

Numerous magnetoencephalographic(MEG) investigations of the functionalorganization of the human visual systemhave been conducted since 1968, whenCohen from MIT made the first recordingsof neuromagnetic fields associated withalpha rhythms (Cohen, 1968). In 1972,Cohen recorded spontaneous brain activityusing a single-channel superconductingquantum interference device (SQUID) in ashielded room (Cohen, 1972), and by 1975,visual evoked responses to a strobe flashwere acquired (Teyler et al., 1975).Williamson and Kaufman at NYU beganconducting a number of visual and audi-tory evoked response studies using aseven-channel system (two channels wereused for noise rejection) in the followingyears (e.g., Brenner et al., 1975; Williamsonet al., 1978; Kaufman and Williamson,1980). Currently, MEG systems contain64–306 sensors, providing whole-headcoverage.

MEG can provide sensitive temporalinformation about sensory and cognitivefunctions (on the order of milliseconds)and can provide good spatial resolution aswell. Hemodynamic measures (e.g., func-tional magnetic resonance imaging andpositron emission tomography), although

providing good spatial resolution, have alower temporal resolution (seconds andtens of seconds, respectively) that is inade-quate for documenting changes in conduc-tion velocities or neural processing times.MEG methods provide some benefits fordata analysis compared to event-relatedpotential (ERP) methods in terms of visual-izing the data and ease in localizing thesources. In the first case, because MEGsignals are generated from intracellularcurrents (Okada et al., 1997; Tesche et al.,1988; Wu and Okada, 1998), the skull is vir-tually transparent to magnetic fields (Barthet al., 1986; Okada et al., 1999a,b), andbecause MEG measurements are referencefree, the field distributions at the surface ofthe head are more focal, making it easier tovisualize general source locations from thesurface patterns. In the second case,because the skull is virtually transparent tomagnetic fields, simple head models (e.g.,homogeneous spherical conductive medium)may be used for source localization ofMEG data (Hämäläinen and Sarvas, 1987,1989; Cuffin, 1990). Therefore, the field ofMEG is in a unique position to draw fromboth the single- and multiunit studies inmonkeys and the hemodynamic studies of functional neuroimaging. The mostserious limitation of MEG and electro-encephalographic (EEG) methods is the

nonuniqueness of the solutions (i.e., anumber of different spatial configurationsof sources within the brain can create thesame field pattern at the surface of thehead).

Early MEG investigations of the visualsystem attempted to corroborate findingsbetween noninvasive MEG measures andinvasive studies in monkeys, becausemuch of the knowledge of the visualsystem had been gained from theseanatomical, lesional, and electrophysiologi-cal studies. Invasive studies reveal anumber of different areas in monkey brainthat contain different representations of thevisual field, which process information inslightly different ways (Zeki, 1978). Forexample, visual area 4 (V4) in monkeyscontains a heavy representation of thecentral visual field and a large proportionof color selective cells, whereas the medialtemporal area (MT) emphasizes peripheralvision and is quite sensitive to motion(Zeki, 1973, 1978, 1980; Maunsell and VanEssen, 1983; Albright, 1984). Felleman andVan Essen (1991) have identified 32 differ-ent visual areas in monkey brains,although some of these areas are still underdebate. Most early MEG studies of basicvision focused on examining properties(e.g., spatial frequency/temporal fre-quency tuning functions and retinotopicorganization) of single visual areas (e.g.,V1, V4, or MT). A review of these studiesindicates that by carefully selecting stimu-lus parameters (e.g., field position, color,size, and motion) it is possible to identifyand characterize several different visualareas in the human brain.

More recently, MEG studies have exam-ined cognitive issues such as the represen-tation of language processes in the brain,as well as memory and imagery (e.g.,Salmelin et al., 1994; Michel et al., 1994;Kuriki et al., 1996; Eulitz et al., 1996;Koyama et al., 1998; Zouridakis et al., 1998;Walla et al., 2001; Iwaki et al., 1999). Studiesof language and imagery rely less on inva-sive results in monkeys for indirect valida-

tion but depend more on results from otherfunctional neuroimaging methods, such aspositron emission tomography (PET) andfunctional magnetic resonance imaging(fMRI) for corroboration. But even con-temporary views of higher cognitivefunctions (i.e., memory) draw on knowl-edge gleaned from invasive studies ofprimate sensory/perceptual systems tosome extent, which emphasize that anumber of different neural systems par-ticipate in the representation of an object orevent (Squire, 1986; Kosslyn, 1988).Although there is overwhelming evidencethat feature integration relies on conver-gent hierarchical processing, i.e., the visualsystem can be viewed as a series of pro-cessing stages that represent a progressiveincrease in complexity of neu-ronal repre-sentations (e.g., Zeki, 1978; Van Essen,1985; Van Essen and Maunsell, 1983; De Yoe et al., 1994), there is also over-whelming evidence for the existence ofat least two functionally specialized pro-cessing streams in the visual system(“dorsal” and “ventral”) operating inparallel (e.g., Ungerleider and Mishkin,1982; Ungerleider, 1995; Van Essen andMaunsell, 1983; Merigan and Maunsell,1993; De Yoe and Van Essen, 1988). Manyinvestigators currently believe that infor-mation concerning the attributes of stimuliis not stored as a unified percept in a singlecortical location, but rather, appears to bestored in a distributed cortical system inwhich information about specific featuresis stored close to the regions of cortex thatmediate the perception of those features(e.g., Ungerleider, 1995; Mesulam, 1998;Goldman-Rakic, 1988). The issue of howfeatures and attributes of stimuli becomeintegrated across widespread corticalregions has been of intense interest anddebate. By capitalizing on the temporalresolution of neuromagnetic measures inconjunction with anatomical MRI, insightsinto the connectivity patterns of the differ-ent cortical regions (i.e., functional systemsor networks) can be obtained.

94 5. VISUAL MEG STUDIES


The MEG studies reviewed in thischapter represent steps toward the generalgoal of a noninvasive delineation of visualinformation-processing pathways in thehuman brain. The following general topicsare examined: (1) retinotopic organization,(2) basic visual functions within or acrossvisual areas, (3) synchronization or oscilla-tory behavior, (4) higher order processesthat may alter the flow of informationthrough the visual system (e.g., selectiveattention), and (5) general issues. Thisreview is not intended to be comprehen-sive in the sense of referencing all visualMEG studies conducted during the past 30years. Instead, we provide a conceptualphysiological/anatomical framework foreach of the subject headings providedabove and discuss representative MEGstudies published in journals, relative toeach of these subject headings.

IDENTIFICATION OF VISUALAREAS AND RETINOTOPY

Visual information flows from retina tocortex through two primary pathways. Thetectopulvinar system in monkeys projectsfrom the retina to the superior colliculus,pulvinar, or lateral posterior nucleus of thethalamus, to predominantly extrastriateareas such as parietal cortex (Schneider,1969; Rodieck, 1979; Van Essen, 1979;Wilson, 1978). In contrast, the phylogeneti-cally newer geniculostriate system, thefocus of this review, is the one most fre-quently studied using neuroimagingmethods. In monkeys, the geniculostriatesystem projects from the retina to thelateral geniculate nucleus (LGN) of thethalamus, and from the LGN to layer IV ofvisual cortical area 1 (V1), the primaryvisual cortex. V1 contains a point-to-pointrepresentation of the entire contralateralvisual hemifield (Rodieck, 1979; Van Essen,1979; Felleman and Van Essen, 1991).Visually responsive cortex projects from V1to additional areas within the occipital lobe

(e.g., V2, V3, V3A, VP) and to large por-tions of the temporal (e.g., V4, TEO, TE)and parietal lobes (MT, MST, STP, PO, VIP,LIP). Many of the visual areas containedwithin these regions have a topographicmapping of the contralateral visual hemi-field; however, some areas have either avery crude retinotopic representation (i.e.,point-to-point projection of the visual fieldonto striate and extrastriate cortex) or noneat all. Visual areas in monkeys have beenidentified using any of the following crite-ria: (1) retinotopic organization, (2) ana-tomical connections, (3) neuronal responseproperties, (4) architectonics, and (5) be-havioral deficits resulting from ablation(Maunsell and Newsome, 1987; Zeki,1978). The general strategy employed fornoninvasive studies attempting to identifyvisual areas in humans has relied heavilyon two of the criteria mentioned above: (1) evidence of distinct retinotopic organ-ization and (2) functional specialization(e.g., motion selective, color selective).

Several event-related potential studiesattempted to examine the field representa-tion of V1 in humans. Jeffreys and Axford(1972a,b), for example, have suggested thatthe field representation of V1 is organizedinto four basic quadrants (cruciformmodel); this division results from the inter-section of the longitudinal and calcarinefissures (i.e., they form the vertical andhorizontal axes of the cruciform). The rightfield projects to the left hemisphere andthe left field projects to the right hemi-sphere. Similarly, the lower visual fieldprojects to the upper bank of the calcarinefissure and the upper field projects to thelower bank of the calcarine fissure.Unfortunately, although many ERP studiesin humans have routinely noted robusteffects of retinal location on amplitude,wave shape, and polarity on the 100-msecresponse to stimuli, there has been consid-erable disagreement concerning the neuralgenerators of the evoked responses(Michael and Halliday, 1971; Drasdo, 1980;Lesevre and Joseph, 1979; Butler et al.,

IDENTIFICATION OF VISUAL AREAS AND RETINOTOPY 95


1987; Maier et al., 1987; Darcey et al., 1980).The temporal overlap of activities frommultiple cortical areas presents a clearchallenge for ERP methods. Activity frommore than one visual area can summateand contribute to a single peak in the waveform, making it difficult to infer discretecortical generators from the scalp distribu-tions alone (Jeffreys and Axford, 1972a,b).Although mathematical models for sourcelocalization have been available for ERPmeasures, only limited attempts have beenmade to localize sources using ERPs.Inconsistencies between ERP studies alsoarose from differences in (1) stimulus para-meters, (2) reference electrode locations,and (3) the number of electrode locationsutilized in the study (Regan, 1989).

Early MEG studies examining retino-topy used only a few sensors for recordingthe magnetic fields and did not identify thestructures generating the fields measuredat the surface of the head (i.e., they did not superimpose the calculated sourcelocations onto MRIs). Maclin et al., (1983)presented a contrast reversed (13-Hz) sinu-soidal grating in the right visual field andmasked off portions of this stimulus tocreate the appearance of either a semicircu-lar area of adjustable outer radius, or asemiannular area of adjustable inner andouter radii. They found that the depth ofthe source (inferred to be primary visualcortex) increased with greater eccentricityof the stimulus, consistent with the cruci-form model.

Ahlfors and colleagues (1992) usedcheckerboard octants placed in eightcentral locations (octant sectors within a1.8° radius circle) or eight parafoveal loca-tions in octant sectors of an annulus withan inner radius of 1.8° and an outer radiusof 3.8°. Larger stimuli were presented tothe parafoveal locations due to the corticalmagnification factor, i.e., larger stimuli areneeded in the periphery to activate thesame amount of tissue in primary visualcortex that is activated by smaller stimuliin the central field (e.g., Rovamo and Virsu,

1979; Perry and Cowey, 1985). These inves-tigators also used smaller stimuli (<2°)than were used in previous visual studiesbecause of the assumption of point currentdipoles used in dipole modeling; the use oflarge stimuli would violate the basicassumption that the modeled source is apoint source, because large stimuli activateextended regions of tissue in cortex. Asingle-dipole model and a minimum normmethod were applied to the 80-msecdeflection, which revealed activity con-tralateral to the field of stimulation.However, the results did not reveal clearorder for upper versus lower field stimula-tion (i.e., activity evoked by the upper fieldstimuli were not located below activeregions associated with lower field stimuli,respectively). In addition, they reportedthat most responses to parafoveal stimuliwere superior to the corresponding fovealstimuli, and the depths of the sources wereapproximately equal. In general, theseresults did not correspond well with theclassical cruciform model, except for thecontralateral projections. These investiga-tors also noted considerable differences inthe pattern of the minimum norm results(i.e., orientation of net current flow) acrosssubjects and studies. Again, source loca-tions were not superimposed on MRIs tovisualize the generators of these signals,and due to the infancy of multidipolemodels at that time, they used a single-dipole model to account for activity at80 msec even though they acknowledgedthat there was evidence suggesting that atleast two sources must have been active at that time.

Aine and colleagues (1996) attempted tomap the borders of different human visualareas using methods similar to thoseapplied in monkey studies. The boundariesof different visual areas within a singlehemisphere may be outlined by focusingon the representations of the vertical andhorizontal meridia in the visual field,because these locations typically project tothe edges of cortical areas (Cragg, 1969;



Zeki, 1969, 1978; Van Essen et al., 1982; VanEssen, 1985). Callosal fibers terminatepreferentially in regions representing thevertical midline of the visual field, and anexamination of their distribution has led tothe identification of different visual areasin nonhuman primates (Cragg, 1969; Zeki,1969, 1978; Van Essen et al., 1982) and inhuman autopsy specimens (Clarke andMiklossy, 1990). Aine et al. (1996) placedsmall circular sinusoids (ranging in sizefrom 0.4 to 1° for central versus moreeccentric field placements) in seven dif-ferent positions along the lower verticalmeridian and right horizontal meridian.Stimuli were located off the meridia by 0.5° and were scaled according to thecortical magnification factor. A seven-channel system was positioned over 10–15different head surface locations coveringoccipital, occipitoparietal, and occipito-temporal regions. Multidipole spatiotem-poral models were applied to various timeintervals ranging from 80 to 165 msec post-stimulus.

The results of Aine and colleaguesconfirm the general features of the classicalretinotopic model for V1: (1) lower fieldstimuli generally activated regions in theupper bank of the calcarine fissure and(2) V1 sources were more anterior foreccentric placements along the verticalmeridian. However, there was a discrep-ancy between their results and the classicalmodel for that portion of the V1 retinotopicmap corresponding to eccentric placementsbelow the horizontal meridian, followingits extent. With the use of small stimuli, itwas clear that the most eccentric place-ments of the stimuli along the horizontalmeridian did not project onto the upperbank, as predicted by the classical model.It is generally assumed that the representa-tion of the horizontal meridian lies pre-cisely at the base (i.e., lateral extent) of thecalcarine fissure, but the data of Aine et al.showed that the representation of the hori-zontal meridian in some cases deviatesdownward from a posterior to anterior

direction or lies primarily in the lowerbank for some individuals. It turns out thatextensive anatomical studies of human V1show that the anterior boundary of V1 isordinarily found in the lower lip of thefissure (Polyak, 1957), unlike the cruciformmodel. These results are a reminder thatthe classical model represents an ideal caseand that most human anatomical dataindicate extreme variability across subjectsand that the calcarine fissure per se is notsynonymous with V1 (Polyak, 1957;Stensaas et al., 1974).

Although Aine and colleagues (1996)identified multiple visual areas, includingoccipitoparietal, occipitotemporal, andipsilateral occipital activity, the retinotopicorganization of these areas was not deter-mined. Supek and colleagues (1999),however, did examine retinotopic organi-zation of the occipitoparietal and occipi-totemporal regions, as one of their goals,by placing small difference of gaussiansstimuli (DOGs) in the lower right visualfield, just to the right of the vertical merid-ian. Again, a seven-channel system wasmoved over several locations of the headsurface and multidipole spatiotemporalmodels were applied to the data spanningan 80- to 170-msec interval of time. Eachsource in both occipitoparietal and occipito-temporal areas evidenced a systematicshift in location associated with changes in stimulus placement parallel to the verti-cal meridian. These results point to thesensitivity of neuromagnetic techniques.Additional studies using fMRI have alsoshown general retinotopic organizationwithin visual areas. These methods areable to define the boundaries of differentvisual areas quickly and effectively (Serenoet al., 1995; Tootell et al., 1995b; 1998;Hadjikhani et al., 1998).

Hari and colleagues examined theoccipitoparietal region to determine if it isthe homolog of monkey area V6, situatedon the anterior bank of the medial pari-etooccipital sulcus (POS), or if it is an areaunique to humans (Portin et al., 1998;

IDENTIFICATION OF VISUAL AREAS AND RETINOTOPY 97


Portin and Hari, 1999; Vanni et al., 2001).Portin and Hari (1999) noted that as they varied the eccentricity of 5.5° semi-circles presented in the left and right visualfields (0°–16° eccentricity), the V1 sourcerevealed retinotopic organization whereasthe medial parietal source did not. Theysuggest that both the lack of retinotopicorganization and the lack of enhancedfoveal representation suggest that the POS area in humans is the homolog of V6 in monkeys (Shipp et al., 1998; Gallettiet al, 1999). Vanni and colleagues (2001)attempted to examine the temporaldynamics of these two cortical areas, V1and the anteromedial cuneus (V6 com-plex). They presented pattern reversalcheckerboard stimuli to four quadrants,extending 4–12° from the fixation point.Their results suggested that V1 and theanteromedial cuneus had similar onsettimes (~56 msec poststimulus) comparedto other posterior sources such as temporo-occipital and superior temporal regions.

Previously, Aine and colleagues (1995)noted the simultaneous activation of V1and occipitoparietal sources [the sourcelocations of POS found by Vanni et al.(2001) were quite similar to those found byAine and colleagues] for peripheral stimu-lation. Central field stimulation, in con-trast, resulted in the sequential activationof V1, occipitoparietal, and occipitotempo-ral sources. This interesting timing differ-ence noted between central and peripheralfield stimulation (centered 0 and 7° in theright visual field, respectively) was investi-gated further to assess the generality ofthis effect using stimuli that were notscaled by cortical magnification factorversus those that were scaled. In bothcases, V1 and occipitoparietal sources(POS) appeared simultaneous for periph-eral stimulation only. Stephen and col-leagues (2002) tested this issue further byexamining the speed of transmissionthrough dorsal structures such as the POS,versus ventral structures (e.g., area V4),and found equally short onset latencies

throughout the dorsal stream structurescompared to the progressive lengtheningof onset latencies throughout the ventralstream. Vanni and colleagues (2001) didnot compare central versus peripheral fieldconditions to demonstrate this differentialeffect, but the fact that several studiesfound simultaneous activation for V1 andPOS for peripheral stimulation speaks tothe generality of the effect, becausedifferent stimuli, analysis procedures, andrecording instruments were used. How-ever, whether POS is or is not retinotopi-cally organized remains to be determined.Supek and colleagues (1999) did revealretinotopy in this region whereas Portinand Hari (1999) did not. Based on recentresults, three to four of the regions iden-tified in the time interval of 60–250 msecappear to reside in parietal cortex, suggest-ing the possibility that the data may havebeen undermodeled in the studies of POSmentioned above. [We note that Vanni andUutela (2000) also resolved three distinctparietal sources and that Supek et al. (1999)explicitly noted the possibility of under-modeling their data.] In addition, it isdifficult to see small systematic shifts insource location as a function of systematicshifts in field location when large stimuliare utilized.

BASIC VISUAL FUNCTIONS

Spatial Frequency, Temporal Frequency,and Contrast Threshold

Early single-unit studies of feline retinalganglion cells suggested that the visualsystem contains two or more classes of neurons that differ in their receptive field (RF) and signal transmission charac-teristics (Enroth-Cugell and Robson, 1966;Breitmeyer and Ganz, 1976). The RF of aneuron is that area of the retina in whichstimulation by light leads to a response ofthe cell. Y-Type neurons have larger RFs,faster conduction velocities, and respond



best to lower spatial frequencies and highertemporal frequencies than do X-Typeneurons. X-Type neurons have smaller RFs,slower conduction velocities, and respondbest to higher spatial frequencies and lowertemporal frequencies. Spatial frequencyrelates to the amount of detail cells canprocess and is inversely related to cell size(e.g., Enroth-Cugell and Robson, 1966).Within the feline retina, RF size tends toincrease from foveal to peripheral retina,and there is a corresponding shift in the ratioof X to Y cells (Wright and Ikeda, 1974). Inother words, the central retina has a higherproportion of X cells and the peripheralretina has more Y cells. As noted above,large-diameter cells generally have fasterconduction velocities and prefer lowerspatial frequencies. The X and Y cell dis-tinctions in felines are similar to the parvo-cellular, or P-like, and magnocellular, orM-like, neurons in monkey retina and cortex(Stone and Johnston, 1981). The M and Pcells will be discussed in more detail below.

Breitmeyer (1975) examined reactiontimes (RTs) in humans to gratings of dif-ferent spatial frequencies and found a correlation; lower spatial frequency grat-ings yielded shorter reaction times com-pared to higher spatial frequency gratings.Breitmeyer hypothesized that Y-type cellsmediated the responses to the lower spatialfrequencies, because their conduction timeswere fast. Kaufman and Williamson (1980)used MEG to examine this relationship.They measured the phase lag of the evokedneuromagnetic response (i.e., the time fromstimulus presentation to response detection)to contrast-reversal gratings when thespatial frequency and reversal rate of thesegratings were varied. These data showedthat for a grating of a particular spatial fre-quency, the phase of the response is pro-portional to the temporal frequency ofpresentation. Similar to Breitmeyer’s RTdata, there was an increase in the latency of the neuromagnetic response with anincrease in spatial frequency content.

Okada and colleagues (1982) systemati-cally examined peak-to-peak amplitudes(e.g., the amplitude from the maximumpositive to the maximum negative peak ofthe initial response) of the visually evokedmagnetic fields (VEFs) to varying spatialfrequencies, temporal frequencies, andcontrast levels using vertical, contrast-reversal gratings. EEG responses wererecorded as well from two active electrodelocations and threshold contrasts weredetermined psychophysically. The peak-to-peak means showed greater amplitude forhigher spatial frequencies at low temporalfrequencies (see top row of Fig. 1) and,conversely, higher temporal frequenciesrevealed greater amplitudes to low spatialfrequencies (see bottom row). The charac-teristics of these transfer functions agreedqualitatively with transfer functionsobtained with visual evoked potentials(VEPs) (Campbell and Maffei, 1970; Camp-bell and Kulikowski, 1972; Regan, 1978)and the psychophysical contrast sensitivityfunction (Kelly, 1966; Robson, 1966). Thecontrast sensitivity function of the VEFwas quite similar in shape to the psy-chophysical contrast sensitivity functiondetermined in the same experiment andthe phase lag increased linearly with tem-poral frequency of the stimulus, similar toresults found by Williamson and col-leagues (Williamson et al., 1978; Kaufmanand Williamson, 1980). Okada and col-leagues also showed that the latency of theVEF increased with higher spatial fre-quency and decreased when contrast wasincreased. Although many of these resultshad been shown previously, this was thefirst study to document quantitatively thelinear relation between the steady-statemagnetic field and electrical potential forboth phase and amplitude. These data sug-gested that the VEP and VEF were pro-duced by a common source.

Several studies set out to examine therelationship between MEG responsestrength and latency as a function of checksize and/or contrast of a checkerboard

BASIC VISUAL FUNCTIONS 99


stimulus. In addition, the neural origin ofthe P100 visual response, which was soelusive in the ERP studies, became a focusof MEG studies because MEG was capableof better spatial resolution compared toERPs (Leahy et al., 1998). Early ERP studieslabeled peaks in the evoked responseseither as components 1, 2, and 3 (CI, CII,CIII) or as peaks denoted by polarity (neg-ative versus positive) and latency (N70,P100, N200, P200, P300), depending on thetype of stimulation (e.g., pattern reversal,pattern onset, flash stimulation) and thecountry in which the studies were con-ducted (e.g., United Kingdom versus UnitedStates). Considerable effort was expendedon attempts at localizing the underlyinggenerators of these peaks either qualita-tively (Jeffreys and Axford, 1972a,b; Michaeland Halliday, 1971) or quantitatively via

source localization procedures (e.g., Butleret al., 1987; Darcey et al., 1980; Maier et al.,1987; Ossenblok and Spekreijse, 1991), butit eventually became clear that singlepeaks/components in the wave forms (e.g.,P100) do not necessarily reflect activityfrom a single cortical area (i.e., a peak canreflect activity from a number of differentsources). A MEG study by Seki and col-leagues (1996) reported very few differ-ences in source locations associated withthe MEG correlate of P100 to pattern rever-sals of full-field (subtending 17° visualangle), half-field, and quadrant-field stim-ulation. They used a single-dipole model toaccount for activity occurring in the 90- to135-msec time window and found that allof the sources localized to the bottom ofthe calcarine fissure. Other MEG studiesfocused on localizing the sources of the



FIGURE 1 Mean peak-to-peak amplitudes of the visually evoked magnetic fields of four subjects are shown asa function of spatial frequency and contrast level. Each subplot shows this relation for four different temporalfrequencies (3.5, 6.5, 8.5, and 11.5 Hz). Mean luminance = 10 cpd/m2. Reprinted from Vision Research, 22; Y. C.Okada, L. Kaufman, D. Brenner, and S. J. Williamson; Modulation transfer functions of the human visual systemrevealed by magnetic field measurements; pp. 319–333. Copyright 1982, with permission from Elsevier Science.

different components of the VEFs andshowed various localizations around thecalcarine fissure for the first through thirdcomponents (Harding et al. 1991, 1994;Hashimoto et al. 1999; Seki et al., 1996,Shigeto et al., 1998). Unfortunately, theMEG results were quite variable, similar tothe ERP results. This was probably becausemultiple generators were contributing tothe peaks in the MEG wave forms but thepeaks were analyzed using primarily asingle-dipole model.

Armstrong and colleagues (1991)attempted to establish norms for MEGresponses to visual stimuli, similar to whathad been done with EEG, using a second-order gradiometer in an unshielded envi-ronment. They studied 100 subjects aged18–87 years and found that pattern reversalstimuli evoked a major positive componentbetween 90 and 120 msec, whereas flashstimulation produced a major positivecomponent between 90 and 140 msec. Theynoted that the latencies were considerablymore variable in MEG than in EEG. This ismost likely because MEG primarily mea-sures intracellular current flow rather thanthe return currents, which can causesignificant differences from subject tosubject in the field patterns at each sensorlocation, due to the variable orientations ofthe sources that contribute to these compo-nents. Armstrong did find, however, thatthere was a steep increase in latency afterage 55.

Hashimoto and colleagues (1999) usedlarge checkerboard patterns to identifythree peaks in the MEG data similar tothose found in the ERPs (N75, P100, N145)and attempted to localize the cortical gen-erators of these peaks by applying a single-dipole model around the peak latencies.These investigators noted that the sourcesfor the first and third peaks were adjacentto one another with similar orientationswithin calcarine fissure, but the generatorof the second peak had a different sourcelocation in the medial occipital lobe.Furthermore, when different contrast

levels were applied, the first componentwas quite sensitive to the contrast modu-lations whereas the third component wasnot. They concluded that the first and thirdcomponents had different physiologicalproperties and recalled data from Aineet al. (1995) indicating a recurrence ofactivation in both striate and extrastriatecortices, suggesting that the third compo-nent might be a reactivation of primaryvisual cortex. More recently, Aine andStephen (2002) have carefully character-ized temporal response profiles fromseveral cortical areas and have concludedthat response profiles from primary visualand primary auditory areas have initial“spikelike” activity followed by “slowwave” activity. The “spikelike” activitydoes appear to have different physio-logical properties, compared to the “slowwave” activity, even though both of theseactivities are generated from the samecortical region; the former appears toreflect primarily feedforward activity and the latter primarily reflects efferentactivity. These data are discussed morefully under the section “Higher OrderProcesses.”

Nakamura and colleagues (2000), awareof the animal studies indicating that theretinal ganglion cells most sensitive tohigher spatial frequency stimuli have apredominantly foveal location (Novak et al., 1988), tested whether checkerboardpatterns consisting of different check sizeswould evoke neuromagnetic activity inslightly different regions of V1, as a func-tion of spatial frequency. These investiga-tors used half-field checkerboard patternspresented to the right field (subtending12.5° visual angle) with pattern reversalrates at 1 Hz. Similar to many ERP studies,these investigators found that check sizesignificantly affected the latency andamplitude of the 100-msec peak (i.e.,longer latencies and reduced amplitude formany of the higher spatial frequencies),but check size did not produce a shift insource location within V1.



Dorsal and Ventral Processing Streamsor Object and Spatial Vision

De Monasterio and Gouras (1975) founda division of retinal ganglion cells inmonkey retina, similar to those reportedpreviously in the cat retina. Two types pre-vailed, color-opponent and broadbandganglion cells; color-opponent cells weremostly found in the fovea and broadbandcells were more prominent in the periphery(Wiesel and Hubel, 1966; De Monasterioand Gouras, 1975; Leventhal et al., 1981;Perry et al., 1984). The color-opponent cellsproject to the four dorsal (parvocellular)layers of LGN and the broadband cellsproject to the two ventral (magnocellular)layers (Wiesel and Hubel, 1966; Schillerand Malpeli, 1978; Leventhal et al., 1981;Perry et al., 1984). These parallel streams ofprocessing continue through primary andhigher order visual areas in primate cortex.The large cell types (magnocellular, Mcells) predominantly project to posteriorparietal cortex, or dorsal stream areas,while the small cell types (parvocellular, Pcells) predominantly project to inferiortemporal cortex or ventral stream areas(Livingstone and Hubel, 1987; DeYoe andVan Essen, 1988; Shipp and Zeki, 1985;Tootell et al., 1988; Maunsell et al., 1990). Ingeneral, “dorsal” stream structures (relatedto motion processing and spatial vision)are sensitive to luminance differences, rela-tively insensitive to color, and more sensi-tive to lower spatial frequencies and highertemporal frequencies. “Ventral” streamstructures (related to color processing andobject vision), in contrast, are sensitive tochromatic contrast, higher spatial frequen-cies, and lower temporal frequencies(DeYoe and Van Essen, 1988; Van Essenand Maunsell, 1983; Livingstone andHubel, 1988; Maunsell and Newsome,1987; Ungerleider and Mishkin, 1982). Themajor components of the color/form orobject pathway include V1, V2, V4, TEO,and TE (TEO and TE are located within theinferior temporal lobe), and the major com-

ponents of the motion or spatial pathwayinclude V1, V2, V3, middle temporal area,medial superior temporal area (MST), andarea 7A of parietal cortex (Maunsell andNewsome, 1987).

It was Ungerleider and Mishkin (1982)who originally proposed the existence oftwo cortical streams mediating object andspatial vision, based on lesion studies inmonkeys. Livingstone and Hubel (1987,1988) later suggested that the M and Ppathways may be the neural substrates ofthese processing streams. The goal ofseveral early functional neuroimagingstudies in humans was to dissociate thedorsal and ventral processing streams byfocusing on color and motion tasks (e.g.,Corbetta et al., 1991; Zeki et al., 1991;Kleinschmidt et al., 1996; Shah et al., 1998);they attempted to dissociate M and P path-ways in ways similar to those used byLivingstone and Hubel (1987, 1988). It hasbecome very clear, however, that the twostreams of processing are not completelysegregated, even as early as V1 (e.g.,Lachica et al., 1992; Nealy and Maunsell,1994). Area MT, associated with theparietal/dorsal stream, receives predomi-nantly M cell input from the lateral geni-culate nucleus, but area V4, associatedwith the temporal/ventral stream, receivesstrong input from both M and P subdivi-sions of the LGN (Maunsell et al., 1990;Ferrera et al., 1994; Merigan and Maunsell,1993). Given the controversy that ensuedover the separation or lack of separationbetween the M and P streams, the focus ofthe neuroimaging studies appeared to turntoward tasks emphasizing object versusspatial vision.

A dissociation between object andspatial pathways was attempted by Haxbyand colleagues (1991) and others (Haxby et al., 1994; Kohler et al., 1995), using PET methods. These investigators com-pared regional cerebral blood flow (rCBF)responses when subjects were engaged ina face-matching task (object process-ing) versus a dot-location matching task



(spatial processing). They concluded thatlateral occipital extrastriate cortex isinvolved in both the face- and dot-match-ing tasks whereas the face-matching taskalso activated an occipital temporal regionthat was anterior and inferior to the lateraloccipital region. In addition to activatingthe lateral occipital region, the dot-match-ing task also engaged a lateral superiorparietal region.

Sato and colleagues (1999), using MEG,attempted to separate out the two process-ing streams by having subjects discrimi-nate familiar from unfamiliar scenes andfaces. They hypothesized that discrimina-tion of familiar versus unfamiliar faceswould activate lingual and fusiform gyri,whereas discrimination of familiar versusunfamiliar scenes would invoke activity inthe right parahippocampal gyrus and theright POS. In the latter case, they rational-ized that both parahippocampal gyrus andparietal cortex are involved in orientationand navigation in space (topographic pro-cessing). Using single-dipole modelingmethods, they found prominent signals inthe MEG response to scenes occurring200–300 msec after stimulus onset in rightparahippocampal and parietooccipitalregions. In contrast, prominent MEG sig-nals in response to face processingappeared between 150 and 200 msec in thelingual or bilateral fusiform gyri.

Portin and colleagues (1998) used MEGto study the properties of POS, belongingto the dorsal pathway, by presentinghemifield luminance versus patternedstimuli to the left and right visual fields.Hemifield luminance stimuli were hypoth-esized to activate dorsal stream structureswhereas patterned stimuli were hypothe-sized to activate ventral stream structurespreferentially. The hemifield stimuli, semi-circular in shape, subtended either a com-plete semicircle covering the fovea (acircular radius of 8°; foveal stimuli) or asegment of a circle starting at an eccentric-ity of 1.5° (nonfoveal stimuli). Source loca-tions were determined by first applying a

single-dipole model to various groups ofsensor locations (28–32 sensors) over themaximum response area, at various timeintervals. The source locations determinedfrom three different cortical regions (bilat-eral occipital and midline POS sources)were then introduced into a time-varyingthree-dipole model (fixed locations) todetermine the time courses of the threecortical locations. Dipole strengths wereallowed to vary as a function of time inorder to explain the measured signals in all122 sensor locations. Patterned stimulievoked strong contralateral activity in V1(65–75 msec, maximum peak), followed bysustained activation during the presenta-tion of the stimulus. Weaker activation waslocalized in the POS. In contrast, the lumi-nance stimulus (also of semicircular shape)evoked bilateral activity in occipital cortex,which occurred 10 msec earlier on averagethan for patterned stimuli, and causedstrong activation of POS. These investiga-tors concluded that they had preferentiallyactivated the parvocellular pathway viathe patterned stimulus, and the magnocel-lular pathway via the luminance stimulus.They also speculated that the strongerbilateral activation of occipital cortexnoted for the luminance stimuli was due tothe preferential interhemispheric transferof low spatial frequencies.

As mentioned earlier, Stephen and col-leagues (2002) tested the hypothesis thatperipheral field stimulation should lead tofaster onset latencies in the dorsal streamstructures relative to central field stimula-tion, because (1) peripheral representationsof V1 and V2 have been associated withfaster conduction velocities [associatedwith larger cell sizes (Nowak and Bullier,1997)] and (2) peripheral field representa-tions of V1/V2 have direct projections toMT and parietal cortex (Nowak andBullier, 1997; Rockland, 1995; Movshonand Newsome, 1996; Ungerleider andDesimone, 1986; Felleman and Van Essen,1991). Small circular sinusoids with lumi-nance-matched backgrounds were contrast



reversed at four alternation rates (0–14 Hz)along the horizontal meridian in centraland peripheral visual fields (2.3 and 24°,respectively). A whole-head MEG system(Neuromag 122) was used to acquire thedata and new automated analysis methodswere applied to the MEG data (Huang etal., 1998; Aine et al., 2000). Multiple startingparameters (e.g., 3000) were randomlychosen from within a head volume for themultidipole, spatiotemporal modelsapplied to the entire sensor array [seeStephen et al., (2002) for details on how thebest fits to the data were determined].Onset latencies for each cortical response(i.e., the time course for a specific corticalarea) were calculated for the differentvisual areas for each subject. The onsetlatencies in the two representative dorsalstream structures, superior lateral occipitalgyrus (S. LOG, putative MT) and intrapari-etal sulcus (IPS), were significantly shorterfor peripheral versus central field stimula-tion (see tabular data in Fig. 2A), whereasthe onset latencies in V1, V2, and inferiorlateral occipital gyrus (I. LOG, putative V4)were statistically identical across the fieldof stimulation. The results indicate thatalthough central and peripheral field stim-ulation activates similar cortical regions,information from central and peripheralfields arrives in some higher visual areasvia different routes. These results are con-sistent with results from several nonhumanstudies (e.g., Breitmeyer and Ganz, 1976;Nowak and Bullier, 1997). Unlike Portinand colleagues (1998), the current resultsdid not find a difference in onset times inV1 when stimulating foveal versus periph-eral representations of V1 with the sametype of patterned stimulus (circular sinu-soids), scaled by the cortical magnification.It is possible that the earlier onset latencynoted in V1 by Portin and colleagues, asso-ciated with the luminance stimulus, wasdue to the greater mean luminance of theirluminance stimuli compared to their pat-terned stimuli, as acknowledged by theseauthors. As noted earlier, higher contrast

stimuli (clearly exhibited by the luminancestimuli) shorten peak latencies and reac-tion times (Campbell and Kulikowski,1972; Robson, 1966; Okada et al., 1982).

Color/Form or Object Vision (VentralProcessing Stream)

A brief overview of structures compris-ing the ventral stream in monkeys is con-sidered next, in order to discuss the wealthof neuroimaging studies that haveattempted to separate face-selective areasfrom object areas within the ventral pro-cessing stream of humans. Dorsal streamstructures are discussed later under theheading “Motion or Spatial Vision (DorsalProcessing Stream).” Zeki (1973, 1978) orig-inally proposed that area V4 is specificallyand selectively involved in color analysis.However, it appears that there are at leastfour extrastriate visual areas that play arole in color analysis: V2, V3, VP, and V4(Van Essen and Maunsell, 1983). Recentstudies indicate that V4 has a crude retino-topic organization (e.g., Schein andDesimone, 1990) and the RFs in this areacontain the central 20–30° of the retina(Zeki, 1980). The major outputs of V4 are toareas TEO and TE in the inferior temporalcortex (ITC) (e.g., Desimone et al., 1980).The RFs of TEO in monkeys are intermedi-ate in size compared to V4 and TE. Theneuronal properties in TEO are also inter-mediate in complexity between V4 and TE,suggesting that the neural coding of visualobjects in TEO is based on object featuresthat are more global than those in V4 butnot as global as those in TE (Boussaoud etal., 1991). TEO is probably more importantfor making fine visual discriminations asopposed to memory functions. Area TE islocated at the end of the “ventral” stream,in the ITC. The RFs in this area are largeand always include the fovea and extendinto the ipsilateral visual field (Richmondet al., 1983; Saleem and Tanaka, 1996;Boussaoud et al., 1991). TE RFs havecomplex selectivities, responding best in





FIGURE 2 (A) The mean onset times across individual time courses (along with the standard error and thenumber of observations shown in parentheses) for five different cortical regions evoked by stimulating the centralversus peripheral visual fields. Onset latencies were computed by an algorithm, where onset time was defined asthe time when the activity exceeded two standard deviations of the noise level. The group-averaged corticalresponse profiles shown for two cortical areas [V1 and intraparietal sulcus (IPS)] display curves associated withcentral field stimulation (solid lines) and peripheral field stimulation (dashed lines). The IPS is a dorsal streamstructure. The MRIs shown at the right reveal the locations of these visual areas for two subjects. The small triangles and circles shown on the MRIs reflect source locations associated with peripheral field and central field stimulation, respectively. (B) Another parietal location (dorsal stream structure), the superior lateral occi-pital gyrus (S. LOG), which revealed earlier onset times for peripheral field stimulation, as well.

the region of the fovea for real objects orfaces (Desimone and Gross, 1979, Gross etal., 1974; Perrett et al., 1982; Baylis et al.,1987). Responses to faces are 2–10 timesgreater than responses to grating stimuli,simple geometrical stimuli, or complexthree-dimensional objects (Perrett et al., 1982;Baylis et al., 1987). These cells have someproperties of perceptual invariance becausethey respond well even when the faces areinverted or rotated. TE projects heavily intothe perirhinal region, a region important forvisual recognition memory of objects,because this region projects to the hippo-campus via entorhinal cortex. Ablation of TE produces long lasting deficits in themonkey’s ability to learn visual discrimina-tions but leaves intact the ability to learn discriminations in other modalities (e.g.,Mishkin, 1982; Cowey and Weiskrantz, 1967;Gross, 1973; Sahgal and Iversen, 1978;Bagshaw et al., 1972; Butter, 1969).

The use of tasks involving face percep-tion or face recognition in functional neu-roimaging studies became popular becauseof the converging lines of evidence cross-ing nonhuman animal studies, humanlesion studies, and noninvasive functionalimaging studies in humans, suggestingthat a region of the temporal lobe, belong-ing to the object-processing stream, isselective for face processing. Face-selectiveneurons in monkeys have been identifiedin (1) area TE in monkeys (Gross et al.,1972; Rolls et al., 1977), (2) superior tempo-ral sulcus (STS), which receives projectionsfrom inferotemporal cortex (Gross et al.,1972; Perrett et al., 1982), and (3) amygdala,parietal cortex, and frontal cortex, all ofwhich receive projections from the fundusof the STS (Sanghera et al., 1979; Aggletonet al., 1980; Leinonen and Nyman, 1979;Seltzer and Pandya, 1978; Pigarev et al.,1979; Jacobsen and Trojanowski, 1977;Rolls, 1984).

Clinical evidence for face-selectiveregions in humans comes from cases ofprosopagnosia, a difficulty in recognizingfaces of familiar persons that is associated

with damage to the inferior occipitotempo-ral region (Meadows, 1974; Whiteley andWarrington, 1977; Damasio, 1985; Sergentand Poncet, 1990). Object recognition, incontrast, is not typically impaired inprosopagnosic patients (De Renzi, 1986).As Damasio and colleagues (1982) andLogothetis and Sheinberg (1996) point out,prosopagnosics tend to have difficultiesmaking within-category discriminations.For example, these patients have dif-ficulties differentiating between variouscars, or fruits.

Dissociation between face- and object-selective areas within ITC was attemptedby Sergent and colleagues (1992) by com-paring rCBF measures when subjects wereengaged in several tasks (e.g., discriminat-ing the orientation of sine wave gratings,face gender, face identity, object identity).They concluded that face identity causedactivation of right extrastriate cortex,fusiform gyrus, and the anterior temporalcortex of both hemispheres. In contrast,object recognition primarily activated leftoccipitotemporal cortex. Other PET studies(Kim et al., 1999; Kapur et al., 1995;Campanella et al., 2001) and fMRI studies(e.g., Puce et al., 1995; Courtney et al., 1997;Kanwisher et al., 1997; Haxby et al., 1999;Halgren et al., 1999; Jiang et al., 2000;Maguire et al., 2001) attempted to demon-strate the selectivity and locus of face-selective areas. Results from all of thesestudies generally agree in showingfusiform gyrus involvement (either medial,lateral, or posterior), but some studiesrevealed additional areas as well, such asinferior temporal cortex, right middleoccipital gyrus, right lingual gyrus, supe-rior temporal, inferior occipital sulcus,lateral occipital sulcus, and middle tempo-ral gyrus/STS, in addition to prefrontalregions.

In addition, the fMRI/PET studies gen-erally suggest that object perception acti-vated occipitotemporal regions (fusiformgyrus), an area similar to the area activatedby face perception, but more medial



(Malach et al., 1995; Martin et al., 1995; Puceet al., 1995; Halgren et al., 1999; Haxby et al.,1999; Martin and Chao, 2001; Maguire etal., 2001). However, there are clear differ-ences of opinion concerning the exact loca-tions of the object areas. Malach andcolleagues (1995), for example, proposedthat lateral occipital cortex (LO) near thetemporal border was the homolog ofmonkey TE, because pictures of objectsactivated this region much more stronglythan did scrambled images. Kanwisherand colleagues (1997) also found a regionselective to images of objects compared toscrambled images, but this region was justanterior and ventral to Malach’s LO. Inboth cases, the putative object regionevoked strong responses to objects andfaces, as well as familiar and novel objects.

MEG examinations of face processingprimarily focused on deflections seen inMEG-averaged wave forms to face stimuliversus other objects or degraded facestimuli. Lu and colleagues (1991) foundthat faces evoked greater amplitudedeflections than did birds at 150, 260, and500 msec, which were localized to bilateraloccipitotemporal junction, inferior parietalcortex, and middle temporal lobe, respec-tively. Sams and colleagues (1997) found aface-selective area in four of seven subjectsin occipitotemporal cortex peaking at150–170 msec. They noted, however, thatthe source locations of the face-specificresponse varied across subjects and thateven nonface stimuli can activate the facearea, although with less magnitude.Swithenby and colleagues (1998) alsofound a significant difference for facesversus other images (i.e., greater normal-ized regional power) at 140 msec in thesensors over the right occipitotemporalregion, and source modeling suggested theventral occipitotemporal region (see Fig. 3).Each stimulus image subtended a visualangle of 8 × 6° and was luminance matchedto the other images. In contrast, Liu andcolleagues (1999), using magnetic fieldtomography (MFT) analysis on single-trial

data, found no evidence of a face-specificarea in the sense that all complex objectsappear to activate the fusiform gyrus (at125–175 msec and 240–265 msec in the lefthemisphere and 150–180 msec for the rightfusiform), along with many other areas. Aprevious report from the same laboratory(Streit et al., 1999), using MFT analysis onaveraged responses from single subjects,found early activity (~160 msec) related tothe emotional content of faces in the poste-rior sector of superior right temporal



FIGURE 3 Averaged evoked field patterns recordedover right occipitotemporal cortex for four subjects.Responses to faces are depicted by solid tracings;responses to other images are shown as dotted lines.The largest peaks, occurring between 130 and 150 msec for each subject, were evoked by facestimuli. Reprinted with permission from ExperimentalBrain Research; Neural processing of human faces: Amagnetoencephalographic study; S. J. Swithenby, A. J. Bailey, S. Bräutigam, O. E. Josephs, V. Jousmäki,and C. D. Tesche; Vol. 118, p. 505, Fig. 3, 1998.Copyright 1998 Springer-Verlag.

cortex and inferior occipito temporalcortex, followed by bilateral activity (righthemisphere leading the left hemisphere) in the middle sector of temporal cortex(~200 msec) and in the amygdala (~220 msec).

In contrast with the Liu et al. study (1999),for each condition (faces, pointillized faces,and inverted faces), Linkenkaer-Hansen andcolleagues (1998) also localized activityaround 170 msec to the fusiform gyrus, butthey interpreted these results as evidence forface-selectivity [Halgren et al. (2000) foundbilateral fusiform activity]. They also sug-gested that face selectivity may occur asearly as 120 msec (although differences inlatencies were not statistically significant),because latency differences were evident in the MEG wave forms between faces and pointillized faces (i.e., degraded faces).However, even though these investigatorscontrolled for many of the low-level featuressuch as luminance, contrast, and intensitydistribution of the gray-scale levels, suchlatency differences can occur due to differ-ences in spatial frequency between thesestimuli (see Okada et al., 1982). Face stimuliare usually regarded as containing contentof low spatial frequency, but pointillizedfaces have additional high-frequency con-tent. In the face and inverted-face condi-tions, however, where spatial frequency washeld constant, inverted faces resulted ingreater amplitude and longer peak latencywhen compared to upright faces. Liu andcolleagues (2000) did not attempt sourcelocalization but they found that face stimulievoked larger responses as compared tononface stimuli (animal and human hand) at160 msec after stimulus onset in sensorsover both left and right occipitotemporalcortex. Inverted-face stimuli and face stimuliproduced similar response magnitudes, butthe former occurred 13 msec later than thelatter. The latency differences were similar to results noted above in the study byLinkenkaer-Hansen and colleagues (1998)

In conclusion, the MEG studies, alongwith PET and fMRI studies of face selectiv-

ity, generally suggest that the rightfusiform gyrus produces larger responseamplitudes to face stimuli as comparedwith other nonface stimuli. Other regionswere also identified (e.g., extrastriate) asbeing more strongly activated in responseto face stimuli. However, most of thesestudies did not apply appropriate controlsto eliminate an alternative interpretationthat response strengths or peak latenciesand moments may vary between faces andother objects because (1) the area of retinalstimulation was not the same, (2) contrastor luminance was not equated, and (3)spatial frequency content was not equated.The MEG studies presented here indicatethat each of these parameters can have alarge impact on source amplitude, latency,and location. In this sense, these studieshave not demonstrated that a region of thefusiform gyrus is selective for faces until allof these lower level features are controlled.

Other functional neuroimaging studies(e.g., Shen et al., 1999; Haxby et al., 1999)also question the selectivity of the facearea, because face perception does notappear to be associated with a region orsets of regions that are dedicated solely toface processing. These “face-selective”regions also respond significantly toobjects such as houses. Gauthier and col-leagues (1999) suggest that results showinga specialized face area in the fusiformgyrus merely reflect the expertise we haveat perceiving and remembering faces,rather than being specific to faces. Theseinvestigators showed that expertise withnonface objects can also activate the rightfusiform gyrus. Recent single-unit studieshave also shown that the selectivity ofinferotemporal (IT) neurons found forfaces could be generated for any novelobject (e.g., computer-generated wire andspheroidal objects) as the result of exten-sive training (reviewed in Logethetis andSheinberg, 1996; Kobatake et al., 1998).According to Logothetis and Sheinberg(1996), the anterior region of IT is moreconcerned with object class, whereas the



posterior end of IT is concerned with thespecifics of an object item (also see reviewby Tanaka, 1997). Consistent with the sug-gestion by Gauthier and colleagues, faceselectivity is viewed as another form ofobject recognition. The site of activationwithin the inferior temporal lobe (i.e., ante-rior versus posterior) depends on whetherobject class or object identity is manipu-lated by the experimental task. Lesionstudies also suggest two functional subdi-visions of inferotemporal cortex: (1) lesionsin posterior TEO lead to simple patterndeficits and (2) lesions in anterior TE leadto associative and visual memory deficits(see review by Logothetis and Sheinberg,1996).

Cue Invariance

A few studies have tackled the issue ofcue invariance of the object-related pro-cessing areas. As touched on briefly above,monkey studies indicate both segregationof visual cues into different processingstreams such as the M and P streams (e.g.,Livingstone and Hubel, 1988) and conver-gence of several primary cues within singlevisual areas or even within single neurons.In the latter case, the visual system can beviewed as a series of processing stages thatrepresent a progressive increase in com-plexity of neuronal representations that aredependent on the output of precedingstages (e.g., Zeki, 1978; Van Essen, 1985;Van Essen and Maunsell, 1983; De Yoe etal., 1994; Sary et al., 1993). As Grill-Spectorand colleagues (1998) note, the organiza-tional principles underlying the specializa-tion of visual areas remain a matter ofdebate (e.g., Ungerleider and Haxby, 1994;Goodale et al., 1994). Grill-Spector and col-leagues used fMRI to determine the cueinvariance of object-related areas; i.e., doesthe same preferential activation exist forobjects defined solely by luminance,motion, and texture cues? These authorsfound a region in LO that was preferen-tially activated by objects defined by all

cues tested. They also found an earlierretinotopic area, V3a, which exhibited con-vergence of object-related cues as well.

Okusa and colleagues (2000), usingMEG, examined cue-invariant shapeperception in humans. These investi-gators presented three different dynamicrandom-dot patterns (flicker, texture, andluminance), subtending 5 × 5° in the central field. Three different stimulusfigures (diamond, noise, and a cross) werepresented in the foreground against thebackground random dots. In the luminancecondition, for example, dots comprising thestimulus figure (e.g., diamond) wereabruptly reduced in luminance. MEGrecordings were made from occipito-temporal cortex using a 37-channel system.By measuring the peak amplitude andpeak latency of the first component in the wave forms, they found that the peak latency was different for the cues(250 msec for luminance, 270 msec forflicker, and 360 msec for texture) but not forthe figures (diamond, cross, noise). In con-trast, the peak amplitude was different forthe figures (96–114 fT), but not for the cues.Source locations were determined using asingle-dipole model for a time interval of0–500 msec poststimulus. Source locationsevoked by the figures were similar acrossconditions within subjects but were differ-ent across subjects (fusiform gyrus, lateraloccipital gyrus, etc.). Reaction times corre-lated with the peak latency differencesnoted above for the different cues. Theyconcluded that the shape defined by differ-ent visual cues activated the same region inlateral extrastriate cortex regardless of dif-ferences between the visual cues. The cor-respondence between the RTs and peaklatencies for the cues suggest that LO isresponsible for the perception of shape.

Motion or Spatial Vision (DorsalProcessing Stream)

The magnocellular layers of monkeyLGN, associated with motion pathways,



project to V1, V2, V3, MT, MST, ventralintraparietal area (VIP), and area 7a in pari-etal cortex (Van Essen and Maunsell, 1983).In general, the retinotopic specificitydecreases progressively in successive levelsof the motion pathway and average RFarea increases (Maunsell and Newsome,1987). MT, found on the posterior bank andfloor of the superior temporal sulcus, is thelowest order area in which a selectiveemphasis on motion analysis is apparent(Van Essen and Maunsell, 1983). V1, V2,and MT all have direction-selective cells,but the preferred range of speeds for theoverall population of cells is nearly anorder of magnitude greater in MT than inV1. In addition, there are pronounced sur-round interactions in MT permitting theinhibition of responses within the excita-tory RF due to motion in other parts of thevisual field (signaling relative motion).MST receives strong input from MT, isposition invariant, and is sensitive to trans-lational motion, expansion, contraction,and rotation (Geesaman and Andersen,1996). In general, there is convergence ofinputs in the STS from the far peripheralrepresentations of V1 and V2. Inputs toparietal cortex tend to arise either fromareas that have been implicated in spatialor motion analysis (e.g., areas within theSTS) or from peripheral field representa-tions in the prestriate cortex (Baizer et al.,1991).

Many types of motion have beenstudied by the functional neuroimagingtechniques that have been available overthe past 20 years (Watson et al., 1993;Tootell et al., 1995a,b; Cheng et al., 1995;Cornette et al., 1998). Most of this researchhas focused on trying to identify and char-acterize the human homolog of monkeyarea MT. MT in monkeys has been studiedextensively by a number of differentgroups to identify the characteristics of thismotion-sensitive area, including identify-ing cells that are direction selective, speedselective, and orientation selective (Zeki,1980; Maunsell and Van Essen, 1983;

Albright, 1984; Lagae et al., 1993), as wellas defining the receptive field properties(Felleman and Kaas, 1984) and M and Pcontributions to this area (Maunsell et al.,1990). The overall conclusion of thesestudies is that the monkey area MT is sen-sitive to a diverse range of motion stimuliwhile being fairly insensitive to othervisual characteristics such as color. There-fore, functional neuroimaging studies havestudied a variety of motion stimuli, includ-ing continuous motion using random-dotdisplays, changes in direction of motion,onset and offset of motion, as well asapparent motion. The functional imagingtools that rely on measuring metabolic orblood flow changes have had reasonablesuccess at locating a region in humans nearthe parieto-temporo-occipital border that ishomologous to monkey area MT/MST(Watson et al., 1993; Tootell et al., 1995a,b;Cheng et al., 1995; Cornette et al., 1998).MEG studies have shown reasonableagreement in the location of the motion-specific activity while adding temporalinformation about this motion-sensitivearea.

The visual MEG studies of motion haveattempted to characterize fully the differ-ent types of motion, as many VEP (Clarke,1973a,b), PET, and fMRI studies have donein the past. For example, ffytche et al.(1995a) found that area V5 (or MT) wasalso activated by the perception of motiondefined by hue, rather than luminancecues. In this study, area V4 (typically asso-ciated with color processing) was notactive, suggesting that although V5 has tra-ditionally been seen as insensitive to colorit can apparently use this information toextract the motion information fromstimuli. ffytche and colleagues interpretedthese results as signifying a parvocellularcontribution to area V5, as seen inmonkeys. Fylan and colleagues (1997)similarly used isoluminant red/green sinu-soidal gratings to demonstrate the pres-ence of chromatic-sensitive units in V1,which play a role in processing motion



information. Patzwahl et al. (1996) foundthat the initial response to motion wasdipolar and localized to area V5. However,the later activity was much more complex,suggesting multiple areas are necessary tointerpret motion stimuli. Holliday et al.(1997) found evidence of an additionalpathway to V5 based on a case study inwhich the patient’s left hemisphere repre-sentation of V1 was absent. The normalhemisphere showed a bimodal response inarea V5, but the response was delayed by25–40 msec compared to controls. Theaffected side showed a unimodal responsethat was consistent in time with the secondpeak from the normal side. This pattern ofresults suggests that the input for the firstpeak originated from V1 whereas thesecond peak had a nongeniculostriateorigin.

Anderson and colleagues (1996) per-formed an extensive set of experiments onarea V5 to characterize the spatial and tem-poral frequency preferences of this area.They found that similar to the monkey lit-erature, area V5 is selective for low spatialfrequencies [in cycles per degree (cpd),≤ 4 cpd] and a large range of temporal driftfrequencies ( ≤ 35 Hz). In addition, therewas clear response saturation at 10% con-trast. Based on psychophysical findingsthat show that the visual system is sensi-tive to motion for spatial frequencies ≤ 35cpd, and contrary to the ffytche et al.(1995a) findings, Anderson et al. suggestthat the P pathway conveys this informa-tion and that this motion information is notprocessed in area V5. They also suggestthat although the M pathway conveyssome motion information it is primarilyconveying information to the parietalcortex with a more specific goal of identify-ing motion in the peripheral field.

Although many of the MEG studies onthe perception of motion have attempted tolocalize the motion-specific cortical areassimilar to the fMRI and PET studies, theyhave often extended the analysis toprovide additional information about the

temporal characteristics as well. In anotherstudy, ffytche et al. (1995b) looked at thetiming of the V5 response relative to differ-ent drift speeds. They found that forspeeds greater than 22°/sec, the signals inV5 arrived before the signals in V1,whereas for speeds less than 6°/sec, thesignals in V1 arrived before the signals inV5. This timing information is also inagreement with the case study by Hollidayet al. (1997) that suggested the existence ofa separate pathway to area V5.

Lam et al. (2000) studied the differ-ences between coherent versus incoherentmotion using random-dot stimuli.Although onset latency did not changewith speed of the stimuli, there was aninverse relationship between offset latencyand speed of the stimuli. They also foundthat the sources evoked by slower motionstimuli localized more laterally than forfaster motion stimuli. Similar results wereseen for coherent as well as incoherentmotion, suggesting that the same areaprocessed both types of motion. In a studyexamining changes in the direction ofmotion (Ahlfors et al., 1999), fMRI loca-tions were used to help guide multidipoleanalysis of the MEG data (although theMEG locations were not constrained by thefMRI activation areas). These investigatorsfound five different areas responsive to themotion stimuli: MT+, frontal eye field(FEF), posterior STS (pSTS), V3A, andV1/V2. They used the notation MT+ torefer to the collection of motion-sensitiveareas, including MT and MST, locatedalong the occipitotemporal border. Theonset of the response was first seen in areaMT+ around 130 msec, followed by activa-tion of FEF 0–20 msec later. The remainingareas were active later. In general, twotypes of responses were seen in the fiveareas, transient (MT, V1/V2) and sustained(pSTS, frontal). They suggest that pSTS isresponsible for processing informationfrom a number of different areas due to thelong-duration response, consistent withresults from monkey studies.



Uusitalo et al. (1996) looked at thememory lifetimes in the visual system; thatis, how long does it take visual areas torecover from a previous stimulus? Theyfound that the primary and early visualareas with short-onset latencies also hadshort memory traces (0.2–0.6 sec), whereasthe higher order areas (prefrontal,supratemporal gyrus, parieto-occipital-temporal junction), which had later onsettimes, showed significantly longer memorytraces (7–30 sec). A later study examinedthe memory lifetime of area V5, specifically(Uusitalo et al., 1997). This study showedthat area V5 had an activation lifetime ofbetween 0.4 and 1.4 sec across subjects, butthe difference between hemispheres,within subjects, was less than 0.1 sec. Theactivation lifetime of area V5 implies that itis a later stage in the processing hierarchy,as suggested by Felleman and Van Essen(1991).

In addition to testing the different char-acteristics of continuous motion, therehave been a number of studies examiningthe phenomenon of apparent motion.Apparent motion stimuli are spatially dis-tinct stimuli that are presented sequentiallyat a sufficiently fast rate for one to perceivethe offset of the first stimulus and onset ofthe second stimulus as a moving stimulus(phi phenomenon). Technically, all motionstimuli created by a computer reflectapparent motion, but the apparent motionstimuli discussed here refer to stimuli thatthe eye can physically distinguish in astatic state but can also be perceived asmotion with appropriate timing. Kaneokeand colleagues (1997) have performed themost extensive studies on this type ofapparent motion. They initially comparedapparent motion with continuous orsmooth motion (using random-dot stimuli)to determine if they were perceived in thesame way. Although the same area wasactive in response to these two types ofstimuli, the peak latency was significantlyshorter for apparent motion (162 and 171msec) as compared to smooth motion

stimuli (294–320 msec). They suggest that adifferent pathway is available to the appar-ent motion stimuli. Based on two addi-tional studies, Kaneoke and colleaguesconcluded that apparent motion was not asimple summation of on/off responses(Kaneoke et al., 1998; Kawakami et al.,2000). Naito et al. (2000) found an interest-ing field effect in apparent motion. In thelower visual field there were no directionpreferences; however in the upper visualfield, downward motion produced asignificantly stronger response comparedto upward motion despite the same loca-tion and orientation of the modeledresponse.

Two additional types of visual motionstimuli that have been studied briefly withsignificant results are visuomotor integra-tion and action observation. In the firststudy, Nishitani et al. (1999) had the sub-jects perform three tasks: visual fixation,eye pursuit, and finger–eye pursuit. Theyfound four main areas of activation: V1,anterior intraparietal lobule (aIPL), dorso-lateral frontal area (DLF), and superiorparietal lobule (SPL). They suggest aIPLis critical for visuomotor integration,whereas SPL played a role in visuospatialattention. They suggested that the lack ofV5 activation was due to a weakerresponse to change in direction, rather thanonset/offset of motion in this area. Astudy by Vanni et al. (1999) foundenhanced 8- to 15-Hz mu rhythm activityin the postcentral gyrus after a change inperception to a motion stimulus in a binocular rivalry task. This task had ahigh-contrast stationary horizontal gratingstimulus presented to one eye, whereas theother eye was presented with a low-contrast vertical grating. Movement of the weak vertical grating caused per-ceptual shifts of attention from the high-contrast grating to the low-contrast grating(noted by a verbal response from thesubject). The mu enhancement effects were primarily seen during the binocularrivalry task and only very weakly during a



separate visual motion task, suggestingthat somatosensory cortex plays a role inperception even with no associated move-ment. They suggest this is possibly relatedto eye fixation or micro-saccades duringthe task.

In the remaining studies, the primary taskwas arm/hand movement with a visualcomponent. In monkeys, it was found thatmotor cortex was active in response toaction observation as well as in response toactually performing the action. There wasadditional frontal area activation (area F5 inmonkeys) that is considered part of themirror system, i.e., the network of corticalareas employed to mirror someone else’smovements or actions (e.g., Gallese et al.,1996; Rizzolatti et al., 1996). Similar MEGstudies were performed in humans; thesubject was asked to perform hand move-ments as well as observe another per-son performing similar hand movements.Salenius (1999) looked initially at the sup-pression of spontaneous 20-Hz activity inresponse to hand movement and observa-tion of hand movements. This study showedthat viewing the movements significantlydiminished the rebound of 20-Hz activity,similar to actual movements. Salenius sug-gested that the motor cortex is a critical com-ponent of the mirror system not only forlanguage but also for inferring someoneelse’s thoughts and actions. Nishitani andHari (2000) performed a similar study withmore extensive localization analysis andfound activation of area V5, as well as acti-vation of Brodmann’s areas BA44 and BA4in both the action and the action observationtasks. In addition, BA44 was active muchearlier than BA4, suggesting that it plays acritical role in the observation/executionloop. They suggest that BA44, traditionallyseen as a language area, is part of the mirror system due to the role hand gesturesplayed in prelingual communication. Inaddition, these studies show the close link between the different sensory areas increating a unified percept of the outsideworld.

OSCILLATORY BEHAVIOR IN THE VISUAL SYSTEM

There are a number of different types ofoscillatory behaviors associated with thevisual system. First, there is spontaneousoscillatory activity, generally referred to asvisual alpha rhythms, which demonstrateoscillatory activity in the 8- to 13-Hz fre-quency range. Second, one can create oscil-latory behavior by driving the system withperiodic stimuli. Third, gamma-band oscil-latory activity (in the 30- to 45-Hz fre-quency range) can be induced by variousstimuli that do not have any oscillatorycharacteristics, but evoke an oscillatoryresponse in a particular frequency range.

Alpha rhythms have been studiedextensively using EEGs. These studieshave ranged from characterizing thenormal frequency range of alpha (8–13 Hz)to determining how alpha activity inter-acts with periodic visual stimulation (e.g.,Regan, 1966; Klemm et al., 1982; Mast andVictor, 1991; Pigeau and Frame, 1992).However, due to the lower spatial resolu-tion of EEGs, most of these studies did notfocus on the source of the alpha rhythms.Many early MEG studies used the higherspatial resolution of this technique to local-ize this activity (Ciulla et al., 1999;Chapman et al., 1984; Salmelin and Hari,1994). Results from these studies primarilyagree that the source of the spontaneousactivity was located in and around thePOS as well as the calcarine fissure.Although results in monkeys suggest thatthe strongest source of alpha rhythms isfound in striate cortex, Salmelin and Haripointed out that the POS source was prob-ably as strong, if not stronger, in MEGstudies due to cancellation of the MEGsignal in the calcarine fissure.

A number of different studies haveinvestigated the role of alpha rhythmsfurther by establishing the levels of inter-action different stimuli have on the sponta-neous alpha activity. Hari et al. (1997) used

OSCILLATORY BEHAVIOR IN THE VISUAL SYSTEM 113


a simple visual stimulus to show that theposterior alpha activity could be damp-ened within 200 msec of the onset of avisual stimulus. They also saw a reboundof alpha activity beyond baseline levelswith the offset of the stimulus. A numberof different studies showed differentialalpha suppression during different cogni-tive tasks (Salenius et al., 1995; Tesche et al.,1995; Williamson et al., 1996; Vanni et al.,1997). For example, Salenius and col-leagues (1995) and Williamson and col-leagues (1996) both found that a visuallypresented cognitive task and a visualcontrol task caused alpha suppression.However, there was a difference in thedegree of alpha suppression between these two studies; Salenius and colleagues found that the alpha suppression wasweaker during mental imagery, whereasWilliamson and colleagues found equiva-lent alpha suppression between visualimagery and the visual control task (visualimagery is discussed further under“Higher Order Processes”). Vanni et al.(1997) found that alpha suppression wasgreater for objects than for nonobjectswhen the objects were the target stimuli,but that there was no difference when thetask did not differentiate between thesetwo classes of stimuli. This suggests thatthe apparent discrepancy in the previoustwo studies is most likely representing thesensitivity of task relevance, or attention,on the interaction of alpha activity withparticular stimuli. In addition to seeingalpha modulation during a mental imagerytask, Tesche and colleagues also saw 20-Hzmodulation. They suggested that there wasprobably higher frequency activity in theprevious studies that averaged out withthe spectral analysis techniques used. Asexpected, different types of analysesprovide different information from a givendata set. Given the temporal resolution ofMEG, there are a number of different time-and frequency-domain analyses that areappropriate, depending on the tasks andhypotheses of any particular study.

One obvious goal in these studies,besides determining the source of thealpha activity, is to try to determine thepurpose of the oscillations. Hari andSalmelin (1997) suggested three possibleroles for the spontaneous oscillatory activ-ity seen in many of the sensory modalitieswhile at rest. First, oscillations may reflecta type of idling of the system that allowsthe system to react more quickly to unex-pected incoming stimuli. Second, it maysignify the ongoing transfer of informationbetween the peripheral and centralnervous system. Third, it might signifyperiodic stimulation of neuronal groupsused to reinforce synaptic connections.

A couple of investigators have sug-gested clinical applications for the moni-toring of alpha activity using MEGs. Parraet al. (2000) found that alpha activity and afixation-off sensitivity-related abnormality(FOS-RA) were separable by applying acluster analysis to the data. With applica-tion of ethosuximide for 1 year, this abnor-mal activity was reduced. Tesche andKajola (1993), using a novel time-frequencyanalysis of the data, found that a spike andslow-wave event found in the time-domainanalysis from an epileptic subject localizedto a location similar to that of abnormal 2-to 6- Hz activity found in the frequency-domain analysis. They suggested that atime frequency-domain analysis can helpwith MEG analysis by separating signaland noise, if they are in different frequencybands, as well as by identifying neuralactivity based on the characteristic that it isspatially correlated, whereas noise often isnot.

Driving Oscillatory Activity withPeriodic Stimulation

An additional way to interrogate thevisual system is not only to observe thespontaneous oscillatory activity but also todrive the system at different frequencies todetermine its sensitivity to different tempo-ral characteristics. Narici and colleagues



have performed a number of studieslooking at the frequency sensitivity of thedifferent sensory systems to stimuli drivenat different frequencies (Narici et al., 1990,1998; Narici and Peresson, 1995). Theyfound that although the different sensorysystems can be driven at a number of dif-ferent frequencies, the preferred frequencyis the dominant frequency that is found inthe spontaneous activity. That is, when thevisual system was driven at the alpha fre-quency, the driven oscillations lasted con-siderably longer compared to oscillationsat the surrounding frequencies. This isuseful in enhancing the signal and improv-ing the stationarity of the oscillations.Although the discussion in the previoussection has shown that it is possible tolocalize and study the alpha activity, thetransitory nature of alpha activity seen inshort bursts makes the analysis moredifficult to perform. In a second study,Narici found that there appeared to be twodifferent components of the occipital alpha,one at 10.5 Hz and one around 12 Hz. The10.5-Hz component appeared to continuein the background regardless of stimula-tion, whereas the 12-Hz activity was verystable with periodic stimulation, but it didnot continue spontaneously. They sug-gested the 10.5-Hz activity had an atten-tion component whereas the 12-Hz activitywas related to cognitive processing.

In a set of experiments, Tononi et al.(1998) and Srinivasan et al. (1999) investi-gated the cortical response to alternatingvisual stimuli in a binocular rivalry task.That is, different stimuli were presented tothe two eyes. These stimuli competed forperceptual dominance similar to a Neckercube that spontaneously changes orienta-tion as the subject maintains fixation. Inthis set of studies, one eye was presentedwith a red vertical grating alternating atfrequency f1, while the other eye was pre-sented with a blue horizontal grating alter-nating at frequency f2. The alternationfrequencies presented to the two eyes werechosen from the following set: 7.41, 8.33,

9.5, and 11.12 Hz. The subject activatedone of two switches with a right fingermovement when the red or blue stimuluswas dominant (none if neither was domi-nant). The stimuli changed dominanceevery 2–3 sec on average. By collecting along time interval while the subject per-formed this task, they were able to obtain afrequency resolution of 0.0032 Hz. Theyfound control values for the power by pre-senting the two stimuli separately (alter-nating randomly) for the same length oftime as the binocular rivalry task. Tononiand colleagues first found that a signalfrom both frequency bands was seenduring the perceptual dominance of eachstimulus. This implies that although onlyone stimulus was consciously perceived,the visual information from both visualstimuli was registered in cortex at somelevel. However, there was a significantchange in power relative to the consciouslyperceived stimulus (decreased by 50–85%during the unconscious versus consciousperception). This difference in power wasseen in sensors over occipital, temporal,and frontal cortex. Srinivasan et al. (1999)extended the work on this task by lookingat the coherence at the different frequen-cies between different cortical areas (com-pared sensors >12 cm apart). Power at agiven channel suggests the level of locallysynchronized activity, whereas coherencebetween channels suggests the level ofsynchronization in distant brain areas.They found that in addition to an increasein power, the coherence between thesewidely separated areas was also increasedat the stimulus frequency (both intra- andinterhemispheric). The most robust changein coherence occurred between interhemi-spheric sensor pairs between occipital andparietal channels. Other significant coher-ence pairings included temporal or frontalof one hemisphere with temporal, parietal,and occipital channels of the other hemi-sphere, and to a lesser extent frontal/tem-poral pairings with occipital/parietalchannels within hemisphere. This study



also showed that the coherence informa-tion was independent of the change inpower, implying that additional informa-tion can be gained by looking at coherenceresults. If one were to extend thesemethods by performing source analysis ofthe coherence data, these results suggestone could provide a macroscopic view ofnetwork activity and cortical synchronywithin the human brain.

Induced Oscillatory Activity

The third type of oscillatory activity thatis seen in response to visual stimuli isinduced oscillatory activity. That is, thestimulus does not have any oscillatorycharacteristics, but oscillatory behavior isobserved by using a variety of techniquesto look at the spectral content of theresponse. This type of behavior is mostcommonly seen during cognitive tasks,suggesting that the induced oscillations arepart of different cell assemblies that areactivated to bind temporally different stim-ulus characteristics or the activation of aparticular system of distributed areas thatare necessary to attend or respond to thetask. Although most of the original gammaband (30–45 Hz) studies in MEG focusedon auditory stimulation, a number ofadditional studies using visual stimuli incognitive tasks have shown gamma bandactivity.

Despite reliable reports of gamma bandactivation in cats and monkeys, it has beenmore difficult to see this activity in humansusing MEG, as compared to EEG. Eulitz etal. (1996) notes that one possible reasonwhy it is difficult to see gamma band activ-ity using MEG relates to the number ofneurons in one region that must be activesimultaneously (e.g., tens to hundreds ofthousands) in order to produce a measur-able MEG signal at the surface of the head.If gamma band activity originates from cellassemblies, and at most tens of thousandsof neurons make up a particular cellassembly with only a fraction of those cells

active at any one time (Aertsen et al., 1994),then there is an insufficient number ofactive cells to produce a measurable signalat the surface of the head. An additionaldifficulty with observing high-frequencyoscillations is that if the oscillations are notwell phase-locked to the onset of the stim-ulus, these responses easily average out inthe presentation of the 100 stimuli nor-mally needed to achieve an adequatesignal-to-noise ratio to obtain a reasonableMEG response. Despite these difficulties,Eulitz and colleagues did see activity in thetraditional gamma band using MEG thatwas reduced during presentation of lan-guage stimuli relative to nonlanguagestimuli at 291 and 506 msec. They alsoobserved 60-Hz activity that was specific tolanguage stimuli over language-specificcortex in response to visually presentedwords and nonword control stimuli. In arelated study, Eulitz et al. (2000) found thatevoked activity and induced activityshowed different topographies of activityacross the surface of the head. In this case,evoked activity was defined as the normalvisual evoked response with its multi-spectral characteristics and induced activ-ity was defined as specific oscillatoryactivity that was generated by the non-oscillatory stimulus. Although they con-firmed their previous results, only post-hoc analysis showed that the induced60-Hz activity near the frontotemporalregion was specific to automatic wordprocessing.

As suggested by the paucity of studiesexamining visually induced gamma bandactivity, novel data analysis techniques arerequired to provide more informationabout this type of oscillatory behavior. Alarge amount of work in this area has beenperformed by Tallon-Baudry and col-leagues (Tallon-Baudry et al., 1997; Tallon-Baudry and Bertrand, 1999). They applieda time–frequency analysis technique that looks for wave packets of gammaband activity using wavelet analysis onunaveraged MEG data. The assumption



was that different stimuli would inducegamma band activity at different times.With the wavelet analysis, one can obtain apower value at different time points thatcan be added together across trials withouthaving to worry about the signal being outof phase and canceling out with averaging.This analysis has provided a significantstep forward in using gamma band infor-mation to further the understanding of thevisual system and cognitive functioning.They have seen both phase-locked gammaband responses (generally early) and non-phase-locked gamma band activity inresponse to delayed match to sample tasks.Based on a simulation study, they alsosuggest that the origins of the gamma bandactivity are the horizontally oriented den-dritic fields, which may further explainwhy MEG has a difficult time detectingthis induced activity (Tallon-Baudry andBertrand, 1999).

Regardless of these difficulties, anumber of other studies have reportedgamma band activity in a variety of visu-ally presented tasks. Sokolov et al. (1999)found that there was an enhancement ingamma band activity in response to coher-ent motion. In this experiment they pre-sented coherent and incoherent visual andauditory stimuli. The coherent visualstimuli were horizontally oriented barsmoving downward (3°/sec) in either theupper or lower half of the screen whereasthe coherent auditory stimuli consisted oftwo alternating tones (300 and 1000 Hz, 60-msec duration). The incoherent visualstimuli consisted of an irregular displace-ment of the horizontal bars, and the inco-herent auditory stimulus was white noise.The auditory and visual coherent stimuliwere randomly presented first, always fol-lowed by the opposite modality stimulus(unattended condition). The subject had torespond to the offset of the first stimulusand decide if it was auditory or visual(with a choice of finger movements). Theyfound modality-specific attention effects tothe coherent stimuli (increased gamma

over visual cortex during the visual stimu-lus and vice versa), as well as gamma bandenhancement for the attended coherentvisual stimuli over occipital cortex in 50- to250-msec time interval (locked to stimulusonset). However, this was only the casewhen the coherent motion stimulus wasattended. If it was not attended, thegamma band activity was the same forcoherent and incoherent motion stimuli,suggesting a strong attention rather than afeature-binding component to the gammaband activity.

Braeutigam et al. (2001) found a verycomplex spatio temporal pattern ofgamma band responses to visually pre-sented congruent and incongruent wordsin a sentence completion task. Both con-gruent and incongruent words showed atypical visual response to the visual wordstimuli. However, differences betweenthese two types of stimuli were evident as early as 200 msec poststimulus. The incongruent words showed two differ-ent periods of phase-locked responses(230–375 and 570–940 msec). In addition,the incongruent words evoked the well-studied N400 response. The congruentwords, on the other hand, showed gammaactivity clusters only around the time thatthe N400 component in the incongruentwords was decreasing. This activity wasnear the traditional Broca’s and Wernicke’sareas; the authors suggested that this activ-ity was related to a type of syntacticclosure. In an interesting, albeit somewhattangential, study, Amidzic et al. (2001)found different patterns of gamma burstsin amateur chess players and in grandmasters in the 5 sec following a “move” inchess. The amateur chess players primarilyshowed gamma activity in the medial tem-poral lobe, suggesting they were looking atnew patterns. In contrast, the grandmasters showed no gamma activity in themedial temporal lobe area but revealedactivity in the parietal and frontal lobes,suggesting they were searching the database for known patterns.



Although most of the induced oscillatorystudies have focused on gamma band activ-ity, other regions of cortex reveal inducedoscillatory behavior in other frequencybands. For example, the hippocampus hasbeen shown in rats to exhibit induced oscil-latory activity in the theta frequency band(4–12 Hz) (Vanderwolf, 1969). In anotherstudy, Tesche and Karhu (2000) used MEG todetermine if this technique could detectstimulus-induced theta activity in thehuman hippocampus. By engaging theirsubjects in a working memory task (using avariant of the Sternberg paradigm), theywere able to demonstrate stimulus-lockedtheta activity. Unlike most MEG studies thatrandomize the timing of the presentation ofthe stimuli to reduce habituation effects, thestimuli in this task were presented with aconstant temporal spacing. They found stim-ulus-locked theta activity in both left andright hippocampus; however, the left hip-pocampus theta activity was evident at ~120msec poststimulus, whereas the right hip-pocampus theta activity was evident at ~80msec prestimulus. The duration of the stim-ulus-locked theta activity was dependent onmemory load, suggesting that the durationof theta activity was related to the cognitiveprocess of monitoring incoming sensorystimuli. The memory-load theta was alsofound to last at most 600 msec, despite a sys-tematic increase in memory load. Tesche andKarhu suggest that the timing of this stimu-lus-locked theta might be a physiologicalcorrelate of the generally accepted memory-load limit of 7 ± 2 items. The prestimulusphase-locking is most likely associated withinternal timing mechanisms of the cortex,which are linked to both right hemisphereand cerebellar activity.

HIGHER ORDER PROCESSES

Selective Attention

For the past three decades, behavioraland neurophysiological investigations of

selective attention have attempted tospecify the functional level at which infor-mation is selected for or rejected fromfurther processing (Johnston and Dark,1986; Kahneman and Treisman, 1984).“Early selection” theories (e.g., Broadbent,1958; Treisman and Geffen, 1967) have pro-posed that irrelevant information can berejected before the semantic analysis of thestimulus, i.e., attention operates at thesensory or perceptual level. “Late selec-tion” theories, in contrast, propose thatselection occurs after both the physical andsemantic analysis of all stimuli impingingon an organism (e.g., Deutsch and Deutsch,1963). According to this view, stimuli auto-matically activate nodes in long-termmemory; attention in this case operates atthe level of decision or response processes(Deutsch and Deutsch, 1963; Shiffrin andSchneider, 1984; Hoffman, 1978; Posner et al., 1980).

Early event-related potential studies ofvisual selective attention in humansattempted indirectly to address the ques-tion of when and where stimulus informa-tion is selected for further processing.These studies examined peak latencies inthe wave forms, peak amplitudes, andscalp distributions, evoked by selective-attention tasks (e.g., Eason et al., 1969; VanVoorhis and Hillyard, 1977; Harter et al.,1982; Hillyard, 1985; Rugg et al., 1987;Neville and Lawson, 1987). The question of“early” versus “late” selection in cognitivestudies of selective attention translated intothe question of how early in the ERP waveforms the effects of selective attentionoccur, and more specifically, if selectiveattention can influence very early levels ofthe visual system such as primary visualcortex (V1) (e.g., Eason, 1981; Harter andAine, 1984; Hillyard and Munte, 1984;Hillyard, 1985). Although it is generallyassumed that ERP effects at increasinglatencies reflect the differential activationof populations of neurons at successivelevels in the nervous system, little informa-tion is available about the neural structures



responsible for attention-related ERPeffects in humans.

Early invasive studies of attention, usingsingle-unit recordings in monkeys, wereinitially unsuccessful in finding any modu-lation of primary visual cortex as a func-tion of selective attention, at any point intime (e.g., Wurtz and Mohler, 1976; Moranand Desimone, 1985; Haenny and Schiller,1988; Motter, 1993). However, recentinvasive studies in monkeys (e.g., Ito andGilbert, 1999; Roelfsema et al., 1998;Vidyasagar, 1998; Mehta et al., 2000a,b) andnoninvasive fMRI studies in humans (e.g.,Tootell et al., 1998; Worden et al., 1996;Brefczynski and De Yoe, 1999; Ghandi etal., 1999; Huk and Heeger, 2000; Shulmanet al., 1997, Somers et al., 1999; Watanabe etal., 1998) routinely demonstrate such mod-ulation of activity. Investigators using inva-sive methods generally agree that attentionacts to enhance the mean firing rate of indi-vidual neurons (Bushnell et al., 1981;Haenny et al., 1988; Roelfsema et al., 1998;McAdams and Maunsell, 1999; Spitzer et al., 1988; Connor et al., 1997; Motter, 1994; Seidemann and Newsome, 1999).Although effects of attention have beendemonstrated at many levels within thevisual system of monkeys (Desimone andDuncan, 1995; Ito and Gilbert, 1999;Reynolds and Desimone, 1999; Mehta et al.,2000a,b; Schroeder et al., 2001; Maunsell,1995; Treue and Martinez-Trujillo, 1999;Moran and Desimone, 1985; Motter, 1993;Treue and Maunsell, 1996), there is stillsome debate concerning how early effectsof attention may be evidenced. Somestudies indicate that attention can influ-ence the initial visual response (e.g., Lucket al., 1997; Ito and Gilbert, 1999), butothers suggest that attention-related modu-lations clearly lag the earliest response(~250–300 msec) (Lamme et al., 1998;Haenny and Schiller, 1988; Mehta et al.,2000a; Seidemann and Newsome, 1999;Roelfsema et al., 1998; Motter, 1994).

Spatial location appears to be an espe-cially effective visual cue for guiding selec-

tive attention. A number of studies haveshown that ERPs recorded over the poste-rior regions of the head are significantlymodulated when subjects are required tofocus attention on a sequence of visualstimuli at a particular spatial locationwhile ignoring a concurrent sequence offlashes in the opposite visual field. Stimulipresented at the attended location, com-pared to stimuli presented at the samelocation while attention was directedtoward stimuli in the opposite visual field,elicit an enhanced sequence of ERP com-ponents over the occipital scalp that typi-cally include P1 and N1 components (peaklatencies of 80–140 and 140–170 msec,respectively), as well as longer latencydeflections (Eason et al., 1969; Van Voorhisand Hillyard, 1977; Harter et al., 1982;Hillyard and Munte, 1984; Mangun andHillyard, 1988; Neville and Lawson, 1987;Rugg et al., 1987). These early attention-sensitive ERP effects are thought to arise inone or more regions of visual cortex,although direct evidence for this conclu-sion is lacking. Modulation of even earlierstages of the afferent pathway has beenreported by Eason and colleagues (Easonet al., 1983). Taken together, these datasuggest that selective attention to spatiallocation may involve modulation of theearliest stages of the cortical visual systemand possibly subcortical structures.

Luber et al. (1989) examined spatialattention using MEG by presenting 1°square gratings 5° to the left or right ofcentral fixation (above the horizontalmeridian) and instructing subjects toattend to either the right or the left field fora block of trials, while maintaining centralfixation. They noted an enhancement ofsource strengths occurring around 200–360msec, “well after at least the first compo-nent of the cortical response has occurred.”This is unlike the earlier ERP studies (e.g.,Van Voorhis and Hillyard, 1977; Harter etal., 1982), wherein attention-related effectswere noted around 100 msec poststimulus.Effects of attention were not accompanied

HIGHER ORDER PROCESSES 119


by changes in the location or orientation ofthe equivalent current dipoles, suggestingthat attention modulates sensory areas thatnormally process visual stimuli.

Aine and colleagues (1995) examinedeffects of attending a conjunction of loca-tion and spatial frequency. Gratings of 1and 5 cpd were randomly presented to twolocations along the horizontal meridian:the central visual field and 7° in the rightvisual field. Subjects were instructed toattend to one of the four combinations offield location and spatial frequency (e.g.,attend to the 1-cpd stimulus presented tothe right visual field). During the nonat-tend condition in this specific example,striate and occipito temporal regions didnot remain active for long; by 130 msec, anoccipitoparietal source dominated. A dif-ference field was constructed by subtract-ing the non-attend wave form the attendwave form and a single-dipole model wasapplied to this difference wave. Thedifference wave revealed a strong effect ofattention around 150 msec, which localizedto striate cortex. When striate sources were compared for the nonattend condi-tion (90 msec) and the attend condition(150 msec), the source locations were adja-cent to one another with opposing orienta-tions. The apparent differences inorientation and in the temporal dynamicsof the response were interpreted asreflecting feedback into striate cortex,causing a reactivation of this region as afunction of attention.

Vanni and Uutela (2000) examined thefrontoparietal attentional network usingMEG [e.g., frontal eye fields and lateralintraparietal area; Corbetta et al., (1991)],which has been reported to be active whenshifts of attention occur between objects orlocations and shifts of gaze. Precentral(putative FEF) and parietal responses toperipheral stimuli were compared whensubjects were instructed to fixate centrallyon a small square while attending versusnot attending to peripheral stimuli, versuswhen subjects were engaged in detecting

auditory tones (i.e., they were not attend-ing to the central fixation point). Regions ofinterest (ROIs) were selected based on theaverage of three conditions representingsteadily active areas within 400 msec fromstimulus onset. Minimum current esti-mates were calculated for these ROIs. Themost systematic difference between experi-mental conditions was noted in the rightprecentral region (frontal eye fields). Theseauthors concluded that the focusing ofattention on a fixation point enhances rightprecentral cortical responses (putative FEF)to stimuli in all parts of the visual field,whether attended or not. They suggest thatvisually evoked right FEF responses,evident around 100 msec in their study,depend on visual attention (e.g., attentionto the fixation point), instead of beingdependent on making overt responses totarget stimuli, as suggested by Corbettaand colleagues (1993). We also note FEFactivity (e.g., Stephen et al., 2002) whensubjects are required to maintain centralfixation but are not required toattend/respond to the visual stimuli. Thisissue is discussed in more detail under“Final Comments.”

Visual Mental Imagery

As mentioned previously, many investi-gators currently believe that informationconcerning the attributes of stimuli is notstored as a unified percept in a single corti-cal location; but rather, it appears to bestored in a distributed cortical system inwhich information about specific featuresis stored close to the regions of cortex thatmediate the perception of those features(e.g., Ungerleider, 1995; Mesulam, 1998;Goldman-Rakic, 1988). Perception occurswhen information is registered directlyfrom the senses. Mental imagery occurswhen perceptual information is accessedfrom memory; perhaps previously per-ceived objects or events are recalled ormental images may result from a new com-bination or modification of stored percep-



tual information. Mental imagery draws onmuch of the same neural machinery as per-ception in the same modality, and canengage mechanisms in memory, emotion,and motor control (see Kosslyn et al., 2001).However, although visual mental imageryand visual perception share many mecha-nisms, they do not draw on identicalprocesses. For example, imagery, unlikeperception, does not require low-levelorganizational processing. On the otherhand, perception, unlike imagery, does notrequire the activation of information inmemory when the stimulus is not present.Kosslyn and colleagues report results froma functional neuroimaging study suggest-ing that, of the brain areas active duringvisual perception and visual imagery,approximately two-thirds were activated inboth cases (Kosslyn et al., 1997). An inva-sive study conducted in humans examinedthe neuronal substrates of visual percep-tion and visual imagery by recording fromsingle neurons in the human medial tem-poral lobe of patients with pharmacologi-cally intractable epilepsy (Kreiman et al.,2000). Neurons in the hippocampus, amyg-dala, entorhinal cortex, and parahippocam-pal gyrus altered their firing rates duringmental imagery. In addition, of the neuronsthat fired selectively during both visualstimulation and imagery, 88% had identicalselectivity. In general, careful selection oftask instruction and stimuli for an imagerytask can produce preferential activation ofdifferent processing streams. For example,the fusiform face area may be activated byhaving subjects image faces while parahip-pocampal regions may become active byinstructing subjects to visualize indoor oroutdoor scenes (O’Craven and Kanwisher,2000).

As was noted in the section on oscilla-tory behavior, many functional neuroimag-ing studies investigated whether visualimagery could alter activity of the primaryvisual area (V1). Two studies with resultsthat replicate each other are noted here,indicating that visual mental imagery can

alter activity in primary visual cortex justas visual selective attention can alter activ-ity in primary visual cortex. Both PET andfMRI results examined responses evokedby having subjects image different sizes ofobjects. When the object was imaged as alarger size, activation in the calcarinefissure shifted to more anterior locations,whereas activation shifted to more poste-rior locations when the object was imagedas a smaller size (i.e., retinotopic organiza-tion) (Kosslyn et al., 1995; Tootell et al.,1998).

Several studies examining visual mentalimagery utilized a visual mental rotationtask similar to one described by Cooperand Shepard (1973). Michel and colleagues(1994) used MEG to test whether the tem-poral duration of alpha suppression overvisual cortex correlates with behavioralreaction time measures, as the timerequired for completing the mental rota-tion task increases. In a previous study,Kaufman et al. (1990) showed that recogni-tion of previously presented polygonshapes was accompanied by suppressionof alpha activity, which correlated withreaction time. In addition, the source of thealpha suppression appeared to reside invisual cortex, similar to earlier blood flowstudies (reviewed by Roland and Gulyas,1994). In the study by Michel and col-leagues, subjects were asked to dis-criminate normal from mirror-reflectedversions of alphanumeric characters whenthe characters were tilted at differentangles relative to the upright position. Thereaction times paralleled results fromCooper and Shepard (1973), showingincreased reaction time with increasedrotation of the target figure from thenormal orientation. Responses to mirror-reflected letters appeared to be 50 mseclonger than responses to letters with thecorrect orientation. In addition, there was aclear relationship between the increase induration of suppression of parietooccipitalalpha band neuronal activity and reactiontime; i.e., both increased by about 200 msec



for the largest (most difficult) angles ofrotation. This result indicates that the POSregion was involved in the mental rotationtask. Furthermore, small changes in thespatial pattern during alpha suppressionwere observed to be dependent on taskdemands. These authors concluded thatalpha suppression directly reflected thecognitive functions that were involved inmental rotation.

Iwaki and colleagues (1999) andKawamichi and colleagues (1998) alsoexamined cortical processes related tomental rotation using source localizationmethods. In the study by Iwaki andcolleagues, line-drawn objects were pre-sented simultaneously and subjects wereinstructed to discriminate rotationally sym-metrical pairs from mirror-reversed pairsvia a finger lift response. A control taskconsisted of having subjects view the samevisual stimuli without making judgmentsand to respond alternately with their rightor left index finger. These investigatorsreport initial activity in occipital cortex(120–180 msec) for both target and controltasks, followed by activation in the poste-rior part of the inferotemporal or temporalcortex for the target condition around170–230 msec. Finally, activity was evidentin parietal or inferior parietal regions forthe target condition around 200–270 msec.Kawamichi and colleagues (1998) foundsimilar regions of activity and temporalsequence when subjects were engaged inmaking judgments of hand orientation in amental rotation task. These studies provideinformation concerning the dynamic prop-erties of distributed cortical activity relatedto mental rotation processes.

Raij (1999) examined imagery-relatedactivity in visual cortex produced by audi-tory phonemes. Subjects were instructed toimagine visually the letter (Roman alphabet)corresponding to the auditory phoneme andto examine its visuospatial properties. Thisactivity was compared to responses to thesame stimuli when subjects detected tonepips. Imagery-related responses were de-

tected over multiple cortical areas, includingleft lateral occipital cortex in 2 of 10 subjectsand in the calcarine fissure of 1 subject.Posterior parietal cortex, close to the midline(precuneus), was active around 390 msecafter voice onset. Aine and colleagues (2002)similarly demonstrated activation of visualcortex (around 200 msec after word onset)during a size categorization task using audi-tory word stimuli that represent commonobjects. Mellet and colleagues (2000), usingfMRI methods, have also demonstrated thatauditory verbal descriptions can produceblood oxygenation changes in visual cortex.

An additional MEG and EEG study thatfalls somewhere between the categories ofselective attention and mental imagery,reported by Gaetz and colleagues (1998),examined the spontaneous reversals of theNecker cube. The subjects were instructedto close their eyes after reversals to marktrigger points in the data. A neuralnetwork classification analysis on the MEGdata revealed that the individual epochsrelated to the control stimulus (a square)and the Necker cube conditions wereclassified appropriately (10/10), showingsignificant differences between the condi-tions. The EEG data showed somewhat lessspecific results, with correct classificationsin 6/8 conditions. Both the MEG and EEGdata showed that there was a significantincrease in correlation across the channelsduring the Necker cube reversals relativeto the control condition. Gaetz and col-leagues conclude that there is increasedsynchrony in a distributed network of cor-tical areas (including occipital, parietal,and temporal areas) during the perceptionof Necker cube reversals.

Working Memory

Wang and colleagues (2001) showedMEG results from a Wisconsin CardSorting Task (WCST). This task has beenroutinely used as a neuropsychological testof frontal lobe dysfunction (Milner, 1963).Subjects are asked to match a “target” card



to a “reference” card, each of which con-tain colored symbols varying in terms ofcolor, shape, and number of symbols. Thecorrect sorting principle (i.e., sort based oncolor, sort based on number of symbols,sort based on shape of symbols) is neverexplicitly described; subjects must deter-mine for themselves, based on the feed-back signals received, whether or not theyare sorting according to the correct cate-gory. Patients suffering from frontal lobedysfunction have difficulties in switchingto a new, correct category. Wang and col-leagues were interested in comparing MEGresponses to (1) card sorting when subjectswere sorting to the appropriate card cate-gory versus after they were first notifiedthat they needed to switch their attentionto another card category and (2) feedbackfor correct versus incorrect responses. In

general, a frontal–parietal network wasidentified across subjects (BA9, BA44/45,and BA40) common to both continuoussorting and feedback responses (see Fig. 4).After subjects were first notified that they needed to switch categories, strong activity in middle and inferior prefrontalcortex was evident at 200 msec (primarilyBA44/45); at 370 msec, strong activity wasevident again in prefrontal cortex (primar-ily BA9). Differences in response to thewrong versus correct feedback signalswere evident around 460–640 msec in thedorsolateral prefrontal and middle frontalgyrus (primarily BA9). These authors con-cluded that the same general brain areas(frontal–parietal network) were involvedin (1) shifting attention after receivingfeedback indicating the wrong sortingcategory was being used and (2) visual



FIGURE 4 Group-averaged RMS from frontal, temporoparietal, and occipital regions are shown across sub-jects for the card presentation and feedback signal conditions. For card presentation, differences in the frontal andtemporoparietal regions can be seen when subjects needed to shift to another category (solid lines) versus whensubjects were sorting based on the correct category (thin lines). Note that occipital regions did not reveal suchdifferences. Differences associated with the feedback signal can be seen in frontal regions when the first incorrectfeedback signal was delivered (solid lines) versus when the correct feedback signal was delivered (thin line). Shaded regions represent periods when the differences between the two conditions were statisticallysignificant as determined by paired t-tests. Frontal regions revealed different patterns of effects—same regionsbut at different points in time. Reprinted from Cognitive Brain Research 12; L. Wang, R. Kakigi, and M. Hoshiyama; Neural activities during Wisconsin Card Sorting Test-MEG observation; pp. 19–31. Copyright2001, with permission from Elsevier Science.

working memory associated with correctperformance. The timing between thesestructures differed according to task condi-tion. Numerous studies ranging fromsingle-unit studies in monkeys to noninva-sive fMRI studies in humans have shownenhanced activity in prefrontal regions that is maintained throughout workingmemory tasks (e.g., Jiang et al., 2000). Thecurrent MEG study (Wang et al., 2001),however, emphasizes the importance of thetemporal dynamics of brain activity.Different functions are not only repre-sented by networks of different brain struc-

tures, but different functions may also berepresented as different timing patternswithin the same network of brain struc-tures. This latter aspect of brain organiza-tion has hardly been explored.

Aine and Stephen (2002) explored themechanisms underlying the enhancementof brain areas (or saliency), as a function ofselective-attention/working-memorytasks. Several MEG articles alluded eitherto the recurrence of activity within a visualarea or to the variability in the duration ofactivity (i.e., lifetimes of activation traces)across visual areas. In the former case, Hari



FIGURE 5 “Spikes” and “slow waves” evoked by visual and auditory stimuli. (A) Response profiles forprimary visual cortex in two subjects (left and right columns, respectively) evoked by visual stimuli whileengaged in a working-memory task (Active Condition). These responses were acquired to the sample stimulus ofthe delayed nonmatch to sample task. Each tracing represents an average of ~250 individual neuromagneticresponses. MRIs for each subject are shown below, with primary visual cortex highlighted by the symbols repre-senting the best-fitting solutions. (B) Cortical response profiles evoked by the auditory classification task for thesame two subjects. The top row shows response profiles from primary auditory cortex and the bottom row showsresponse profiles from visual cortex when subjects were classifying whether the common nouns they heard rep-resented items larger than a television set. The MRIs below reveal that bilateral lower level visual areas wereengaged in this auditory task.

and colleagues (1980) noted that both theN100 and P180 components of the auditoryMEG localized to primary auditory cor-tex. Aine and colleagues (1995) had noted a similar finding in the visual MEG, i.e., there appeared to be a recurrence ofactivation within primary visual cortex.Williamson and colleagues (Lu et al., 1992;Uusitalo et al., 1996), on the other hand,studied the decay times of neural activityin primary and secondary auditory andvisual cortex. They noted that the decaytime of the sensory traces in primary areascorresponded to psychophysical memoryexperiments examining iconic memory(initial representation of visual stimuli)and echoic memory (initial representationof auditory stimuli). As mentioned earlier,Uusitalo and colleagues (1996) suggestedthat there are two forms of memory repre-sentation in each sensory modality. In the visual system, occipital lobes revealedshort lifetimes (e.g., 200–300 msec), whereashigher order visual structures revealed lifetimes lasting from 7 to 30 sec.

Our recent memory studies in bothauditory and visual modalities, using ournewer automated analysis procedures(Huang et al., 1998; Aine et al., 2000), haveshown that activity in the primary visualand auditory cortices can be characterizedas having an early “spikelike” componentfollowed by later “slow-wave” activity (seerows, A and B, Fig. 5). The early “spike-like” activity in the visual response wasexperimentally separated from the later“slow-wave” activity by conducting anauditory size classification task whichevoked activity in visual cortex, presum-ably due to imagery (see bottom row ofFig. 5B). Visual cortical activity was evokedwhen subjects were asked if the auditorywords, representing common objects, werelarger than a television set. These resultswere interpreted as reflecting early feedfor-ward activity (similar to Uusitalo’s brieflifetime traces from occipital activity)(Uusitalo et al., 1996) and later feedbackactivity, which was of longer duration and

was particularly sensitive to attention(similar to Uusitalo’s long lifetime tracesevident in higher order visual areas). Thedifference between these studies is thatprimary and secondary areas reflect bothtypes of activity, which can account for theresults by Hari et al. (1980) and Aine et al.(1995) that suggest a recurrence of activa-tion, even in primary areas. Aine andStephen (2002) link these results to recentstudies examining feedforward and feed-back components in monkeys when theywere engaged in selective attention tasksand studies examining temporal binding.

In the visual working-memory task (amodified version of a delayed nonmatch tosample task), conducted by Aine andStephen (2002), stimuli were constructed ofblack and white squares (Walsh functions)with the characteristics that (1) eachmember had an equal amount of black andwhite, allowing for equal luminance acrossthe stimulus sets, and (2) they weredifficult to label verbally. Subjects werepresented with a “Sample” stimulus for 1sec duration in the central visual field(visual angle 2°) followed by two distrac-tor stimuli in the lower left and right quad-rants. A second Walsh stimulus (“Target”of 1 sec duration) was presented approxi-mately 2.5 sec after the offset of the“Sample” stimulus and subjects wereinstructed to respond with a button pressto the “Target” stimulus if it did not matchthe “Sample” stimulus. Results from the“Sample” stimuli (no behavioral responseswere made to these stimuli) showed thatselective attention or working memory cansustain activity over quite a number of dif-ferent cortical areas (thereby increasing thelifetimes of traces in these regions), com-pared to a passive control task (see Fig.6A), and acts to increase the synchronicityof the activity (see Fig. 6B) across wide-spread cortical areas (temporal binding?).Both of these effects were most pro-nounced for the later “slow-wave” activity.

The topic of temporal binding (i.e., howfeatures and attributes of stimuli become





FIGURE 6 (A) MEG cortical response profiles to “Sample” stimuli when the subject was not engaged (first row—Passive Task) versus engaged (second row—Active Task) in the working-memory task. Note that activity localized toleft occipital–temporal and prefrontal regions reveal elevated and sustained activity in the later portion of theresponses for the “Active” versus “Passive” profiles (bottom vs. top rows, respectively). The clusters of triangles andsquares on the MRIs to the right indicate the cortical regions of activity localized for the “Passive” versus “Active”tasks, respectively. The right hemisphere is displayed on the left side of the MRIs, according to radiological conven-tion. (B) Cross-covariance plots of different cortical areas relative to V1. The autocorrelation of V1 with itself isincluded as a reference. The left and right columns in A show cross-covariance plots of the time-courses for the bestsolution relative to V1 during the passive and active portions, respectively. Both early (90–200 msec) and late(200–450 msec) time intervals for each “Passive” and “Active” task are shown. Notice that during the early timeinterval for the “Passive” task there is a large dispersion of the lag at the maximum correlation (peak), whereasduring the late time interval of the “Active” task, the maximum correlation of most of the sources is almost synchro-nous with the V1 slow-wave response. It is also interesting to note that the time-courses appear more synchronouseven in the early time interval during the “Active” task, although there is no evidence of an enhancement of the earlyresponse (compare initial time courses for “Passive” and “Active” tasks in A).

integrated across widespread corticalregions) has been one of intense interestand debate lately. Singer (1999), along withmany others, suggest that synchronousneuronal firing provides one mechanismfor binding the different features/attributesof stimuli across these widespread corticalareas. Recent results from nonhumanstudies support the idea that corticalneurons form networks by synchronizingtheir activity with other neurons thatprocess similar information (Singer, 1995;1999; Engel et al., 1991, 1992; Gray, 1999;Roelfsema et al., 1997; Towle et al., 1999).Temporal binding could be postulated ifin-phase oscillatory behavior was observedacross different cortical areas. One type ofactivity that has been suggested to mediatetemporal binding in this case is the gammaband activity discussed in the previoussection. A second situation in which temporal binding could be postulated is if different cortical areas exhibited low-frequency synchronous activity. Our MEGresults are consistent with the secondnotion of temporal binding in that we findthat selective attention helps synchronizethe late activity across brain regions,similar to a study conducted in monkeysby Fries and colleagues (2001). Our resultsand results of others reviewed in this papersuggest that by capitalizing on the tempo-ral resolution of neuromagnetic measures,insights into the connectivity patterns ofthe different cortical regions (i.e., func-tional systems or networks) can beobtained.

FINAL COMMENTS

A review of MEG visual studies con-ducted over the past 30 years leads one tonote the variety of methods applied to theanalysis of MEG data as well as differencesin the designs of the studies. A few issuesmentioned below capture differences in investigator opinions, both implicit and explicit, which deserve some attention.

Source Modeling Issues

The most frequently used source modelin MEG assumes that the observed fieldpattern at any instant in time can beapproximated by using a single equivalentcurrent dipole (ECD). For multiple-dipolemodeling, the surface distribution can beapproximated by a set of ECDs with fixedlocations, the moments of which vary withtime. In the ECD approach, the fitting pro-cedure is generally overdeterminedbecause the number of dipole parametersis less than the number of independentMEG measurements. Several investigatorshave tested the accuracy of source localiza-tion using this approach on computer-sim-ulated data and by using physical models,for which the source and head models areknown (e.g., Weinberg et al., 1986; Barth etal., 1989; Achim et al., 1991; Supek andAine, 1993).

Another type of source model dividesthe entire brain or just the cortex into alarge number of grid sites (e.g., distributedsource model). Because the number ofunknown parameters for the distributedsource model is greater than the number ofMEG measurements (10,000–1,000,000unknowns versus 100–200 measurements),the problem is highly underdetermined,thus requiring additional mathematicalconstraints. The current distribution isobtained at these sites by fitting the datausing mathematical constraints such asminimum norm (e.g., Ioannides et al., 1994;Wang et al., 1992; Dale and Sereno, 1993;Hämäläinen and Ilmoniemi, 1994; Pascual-Marqui et al., 1994; Grave de Peralta-Menendez and Gonzalez-Andino, 1998;Uutela et al., 1999).

Analysis methods in the early days ofMEG were clearly dominated by use of asingle-dipole model applied to instants intime. Currently, single-dipole modelingtechniques are still commonly used, aswell as multidipole, spatiotemporalmodels, but variants of minimum normmethods are beginning to gain ground

FINAL COMMENTS 127


[e.g., minimum current estimates (Uutela etal., 1999)]. Regardless of the method ofchoice for the investigator, it is importantfor investigators to apply modeling proce-dures that are objective and easilydescribed. Current methods that do notrequire investigators to provide startingparameters and are relatively easy todescribe are as follows: (1) variants of mul-tidipole, spatiotemporal modeling (e.g.,Scherg, 1990) using R- or RAP-MUSICalgorithms (Mosher and Leahy, 1998;Ermer et al., 2000); (2) variants of multidi-pole, spatiotemporal modeling using mul-tistart methods (Huang et al., 1998; Aine etal., 2000); (3) variants of minimum normmethods (e.g., Hämäläinen and Ilmoniemi,1994) using magnetic field tomography(Ioannides et al., 1990, 1993); (4) variants ofminimum norm methods using minimumcurrent estimates (Uutela et al., 1999); and(5) bayesian inference (Schmidt et al., 1999).

One criticism of the spatiotemporal,multidipole methods focuses on theassumption that cognitive processes arerepresented as extended regions of tissue(rather than focal). Indeed, older hierarchi-cal models of cortical organization, whichview sensory signals as being elaboratedon at successive stages in sensory associa-tion cortices with information flow in onebasic direction, implied that cognitionsmight be represented in brain as extendedin nature. However, current anatomicaland physiological data depict cortical orga-nization of cognitive functions as distrib-uted module-mosaics that are reciprocallyinterconnected by a finite number of dedi-cated networks (e.g., Goldman-Rakic,1988). Ojemann and colleagues (1989), forexample, show that even language ishighly localized in most patients, usingelectrical stimulation mapping techniques;it is composed of several mosaics of 1 to 2cm2, usually one in the frontal region andone or more in the temporoparietal lobe.Numerous invasive studies in monkeyspoint to similar conclusions, even forhigher order functions such as memory.

Fahy et al. (1993), for example, found evi-dence for clustering of neurons with recog-nition memory-related activity as if suchneurons may occur in cortical columns.

Central Fixation: Attention or Eye Movement?

Many visual functional neuroimagingstudies are reporting bilateral activity inthe lateral frontal gyri, medial frontalgyrus, and bilateral parietal cortical activ-ity (e.g., intraparietal sulcus) across anumber of different tasks (e.g., Kim et al.,1999; Nobre et al., 1997; Kosslyn et al., 1997;Courtney et al., 1997; Campanella et al.,2001; Aine and Stephen, 2002). These corti-cal regions correspond well with thefrontal eye fields, supplementary eyefields, and the parietal eye fields, as dis-cussed by several authors (e.g., Andersonet al., 1994; Berman et al, 1999; Petit andHaxby, 1999; Petit et al., 1999; Paus, 1996;Paus et al., 1991; Ploner et al., 1999; Morrowand Sharpe, 1995). Are these regionsunique to the specific task instructions athand (e.g., mental figure rotation or spatialattention), are they related to eye move-ments, or are they a result of attending tothe fixation point? Corbetta (1998) suggeststhat the attention and eye movementsystems share the same neural network,i.e., attention is covert eye movements. Tomake things even more confusing, eyemovements can be as subtle as maintainingfixation, as noted in results reported byPetit and colleagues (1999) and Leinonenand Nyman (1979). This pattern of resultshas become a hallmark of visual studies,including those that do not require cogni-tive processing at all, other than maintain-ing central fixation (e.g., Stephen et al.,2002). Vanni and Uutela (2000) attemptedto examine this complex issue and notedlarge differences in the putative right FEFactivity between conditions in which sub-jects were required to fixate centrallyversus a control condition in which sub-jects were detecting tones. In the latter



case, the FEF is not likely to be as activebecause subjects were not as concernedabout maintaining central fixation whileengaged in an auditory task. Unfor-tunately, the details of this control experi-ment, which was conducted on only 2 ofthe 11 subjects, is lacking (e.g., how do weknow these subjects were fixating centrallyduring the auditory task?). It is clear thatadditional studies will be required in orderto understand the complex relationshipbetween eye movements, maintainingcentral fixation, and attention on thisfrontal-parietal system.

Multimodality Imaging

Most investigators would agree thatmultimodality imaging is a reasonabledirection to pursue because different tech-niques provide complementary informa-tion (e.g., George et al., 1995; Heinze et al.,1994; Ahlfors et al., 1999; Liu et al., 1998;Dale et al., 2000; Dale and Halgren, 2001).However, initial simplistic notions suggest-ing the use of fMRI locations to constrainmultidipole, spatiotemporal models inorder to obtain the time courses for fMRIlocations can be misleading. Because we donot understand the exact relationshipbetween changes in membrane potentialsand changes in blood oxygenation levels, itseems prudent to compare and contrastMEG and fMRI measures initially and touse fMRI location information to “guide,”as opposed to “constrain,” MEG analyses(e.g., Ahlfors et al., 1999). MEG computersimulations indicate that undermodelingthe data, or even having the correct modelorder (i.e., number of dipoles) but a poorset of starting parameters for an errorsurface containing many local minima (i.e.,many sources and/or low signal-to-noiseratio), can provide erroneous time courses(Supek and Aine, 1997).

There are several reasons why onewould not expect to find a one-to-one cor-respondence between active source regionslocalized from MEG versus fMRI. First, the

optical imaging experiments find that manyof the largest signals corresponding withblood volume changes originated from thelarger elements of the microvasculature,which were outside of the activated area(Frostig et al., 1990); i.e., the later phase ofthe vascular response is less localized(Malonek and Grinvald, 1996). Second,fMRI results represent activity averagedacross seconds of time whereas most MEGresults focus on millisecond windows intime. It may be the case that later slow-waveactivity noted in many cognitive MEGstudies correlates best with fMRI, because itis sustained for >1 second. This issue needsfurther investigation. We have also notedthat when long time intervals are modeledin the MEG studies (e.g., 500–1000 msec),some of the early activity is not modeled, asif the later and longer duration activitydominates longer time intervals. Thisproblem often prompts one to examinemore than one window of time. Third, MEGcan detect subtle changes in stimulus para-meters (e.g., spatial frequency content, con-trast, temporal frequency, and area of retinalstimulation), whereas fMRI requires strongstimulus parameters (larger and brighterstimuli) in order to evoke a noticeablechange in blood oxygenation.

In conclusion, the past 30 years hasproduced very exciting changes in MEGhardware, providing dense, whole-headcoverage. Although a significant amountof work has also been devoted towarddeveloping more automated multisourcemodeling techniques and global minimiz-ers, many of these analysis tools have notbeen sufficiently evaluated nor widely dis-tributed throughout the MEG community,as evidenced by the number of studiesusing a single-dipole model for analysis ofsensory and cognitive data. However, weare likely to witness significant improve-ments in this regard during the nextdecade, because at this point in time, non-invasive MEG and EEG methods provideour best hope for understanding the tem-poral dynamics of the different functional

FINAL COMMENTS 129


networks mediating sensory and cognitiveprocesses. A complete understanding ofthe functional organization of humanvisual system must incorporate when andhow long events are occurring, in additionto where they are occurring.

Okada’s group recently demonstrated inswine that MEG is capable of monitoring thesynchronized population spikes of the thala-mocortical axonal terminals and corticalneurons from outside the skull (Ikeda et al., 2002), thereby providing the most com-pelling evidence yet that MEG is capable ofdirectly examining population spike activity.Our own personal experiences indicate thatMEG is capable of showing much moredetail about brain functions than originallyimagined, and MEG has much better spatialresolution than originally postulated, giventhat appropriate models are applied to thedata. Once appropriate head models becomeavailable to EEG studies, many of the MEGanalysis tools can be applied to EEG data aswell, which will enable us to examine thecomplementary nature of these twomethods. We look forward to a meaningfulintegration of MEG, EEG, fMRI, and PETresults to help elucidate the functional orga-nization of the human brain.

Acknowledgments

This work was supported by the NIH (NEI, 5R01-EY08610-10; and NCRR, 1-P20-RR-15636; the MIND Institute (#2006);the VA Merit Review (“Functional Neuro-imaging of Normal Aging”); the ResearchService, Department of Veterans Affairs; andthe Department of Radiology, UNM Schoolof Medicine. We wish to thank Selma Supek,David Hudson, Sanja Kovacevic, and ChadWoodruff for their helpful comments on themanuscript.

ReferencesAchim, A., Richer, F., and Saint-Hilaire, J. M. (1991).

Methodological considerations for the evalu-ation of spatiotemporal source models. Electro-encephalogr. Clin. Neurophysiol. 79, 227–240.

Aersten, A., Erb, M., Palm, G., and Schütz, A. (1994).Coherent assembly dynamics in the cortex:Multineuron recordings, network simulations andanatomical considerations. In “Oscillatory EventRelated Brain Dynamics” (C. Pantev et al., eds.),pp. 59–83. Plenum, New York.

Aggleton, J. P., Burton, M. J., and Passingham, R. E.(1980). Cortical and subcortical afferents to theamygdala in the rhesus monkey (Macaca mulatta).Brain Res. 190, 347–368.

Ahlfors, S. P., Ilmoniemi, R. J., Hämäläinen, M. S.(1992). Estimates of visually evoked cortical cur-rents. Electroencephalogr. Clin. Neurophysiol. 82,225–236.

Ahlfors, S. P., Simpson, G. V., Dale, A. M., Belliveau, J.W., Liu, A. K., Korvenoja, A., Virtanen, J.,Huotilainen, M., Tootell, R. B., Aronen, H. J., andIlmoniemi, R. J. (1999). Spatiotemporal activity of acortical network for processing visual motionrevealed by MEG and fMRI. J. Neurophysiol. 82,2545–2555.

Aine, C. J., and Stephen, J. M. (2002). Task relevanceenhances and synchronizes late slow-wave activ-ity of distributed cortical sources (submitted).

Aine, C. J., Supek, S., and George, J.S. (1995). Temporaldynamics of visual-evoked neuromagnetic sources:Effects of stimulus parameters and selectiveattention. Int. J Neurosci. 80, 79–104.

Aine, C. J., Supek, S., George J. S., Ranken, D., Lewine,J., Sanders, J., Best, E., Tiee, W., Flynn, and R.R.,Wood, C. C. (1996). Retinotopic organization ofhuman visual cortex: Departures from the classicalmodel. Cereb. Cortex 6, 354–361.

Aine, C. J., Huang, M., Stephen, J., and Christner, R.(2000). Multistart algorithms for MEG empiricaldata analysis reliably characterize locations andtime courses of multiple sources. NeuroImage 12,159–172.

Albright, T. D. (1984). Direction and orientation selec-tivity of neurons in visual area MT of themacaque. J. Neurophysiol. 52, 1106–1130.

Amidzic, O., Riehle, H. J., Fehr, T., Wienbruch, C., andElbert, T. (2001). Pattern of focal gamma-bursts inchess players. Nature 412, 603.

Anderson, S. J., Holliday, I. E., Singh, K. D., andHarding, G. F. (1996). Localization and functionalanalysis of human cortical area V5 using magneto-encephalography. Proc. R. Soc. Lond. B. Biol. Sci.263, 423–431.

Anderson, L. J., Jenkins, I. H., Brooks, D. J., Hawken,M. B., Frackowiak, R. S. J., and Kennard, C. (1994).Cortical control of saccades and fixation in man: APET study. Brain 117, 1073–1084.

Armstrong, R. A., Slaven, A., and Harding, G. F.(1991). Visual evoked magnetic fields to flash andpattern in 100 normal subjects. Vis. Res. 31,1859–1864.

Bagshaw, M.H., Mackworth, N. H., and Pribram, K.H. (1972). The effect of resections of the infero-



temporal cortex or the amygdala on visual orienting and habituation. Neuropsychologia. 10,153–162.

Baizer, J. S., Ungerleider, L. G., and Desimone, R.(1991). Organization of visual inputs to the inferiortemporal and posterior parietal cortex inmacaques. J. Neurosci. 11, 168–190.

Barth, D. S., Baumgartner, C., and Sutherling, W. W.(1989). Neuromagnetic field modeling of multiplebrain regions producing interictal spikes in humanepilepsy. Electroencephalogr. Clin. Neurophysiol. 73,389–402.

Barth, D. S., Sutherling, W., Broffman, J., and Beatty, J.(1986). Magnetic localization of a dipolar currentsource implanted in a sphere and a humancranium. Electroencephalogr. Clin. Neurophysiol. 63,260–273.

Baylis, G. C., Rolls, E. T., and Leonard, C. M. (1987).Functional subdivisions of the temporal lobe neo-cortex. J. Neurosci. 7, 330–342.

Berman, R. A., Colby, C. L., Genovese, C. R.,Voyvodic, J. T., Luna, B., Thulborn, K. R., andSweeney, J. A. (1999). Cortical networks subserv-ing pursuit and saccadic eye movements inhumans: an fMRI study. Human Brain Mapping 8,209–225.

Boussaoud, D., Desimone, R., and Ungerleider, L. G.(1991). Visual topography of area TEO in themacaque. J. Comp. Neurol. 306, 554–575.

Braeutigam, S., Bailey, A. J., and Swithenby, S. J.(2001). Phase–locked gamma band responses tosemantic violation stimuli. Brain. Res. Cogn. Res.10, 365–377.

Brefczynski, J. A., and De Yoe, E. A. (1999). A physio-logical correlate of the ‘spotlight’ of visual atten-tion. Nature Neurosci. 2, 370–374.

Breitmeyer, B. (1975). Simple reaction time as ameasure of the temporal response properties oftransient and sustained channels. Vis. Res. 15,1411–1412.

Breitmeyer, B. G., and Ganz, L. (1976). Implications ofsustained and transient channels for theories of visual pattern masking saccadic suppression,and information processing. Psychol. Rev. 83, 1–36.

Brenner, D., Williamson, S. J., and Kaufman, L. (1975).Visually evoked magnetic field of the humanbrain. Science 190, 480–482.

Broadbent, D. E. (1958). “Perception and Com-munication.” Pergamon, London.

Bushnell, M. C., Goldberg, M. E., and Robinson, D. L.(1981). Behavioral enhancement of visualresponses in monkey cerebral cortex. I.Modulation in posterior parietal cortex related toselective visual attention. J. Neurophysiol. 46,755–772.

Butler, S. R., Georgiou, G. A., Glass, A., Hancox, R. J.,Hopper, J. M., and Smith, K. R. H. (1987). Corticalgenerators of the CI component of the pattern-

onset visual evoked potential. Electroencephalogr.Clin. Neurophysiol. 68, 256–267.

Butter, C. M. (1969). Impairments in selective atten-tion to visual stimuli in monkeys with inferotem-poral and lateral striate lesions. Brain Res. 12,374–383.

Campanella, S., Joassin, F., Rossion, B., De Volder, A.,Bruyer, R., and Crommelinck, M. (2001).Association of the distinct visual representationsof faces and names: A PET activation study.NeuroImage 14, 873–882.

Campbell, F. W., and Kulikowski, J. J. (1972). Thevisual evoked potential as a function of contrastof a grating pattern. J. Physiol. 222, 345–356.

Campbell, F. W., and Maffei, L. (1970).Electrophysiological evidence for the existence oforientation and size detectors in the human visualsystem. J. Physiol. 207, 534–652.

Chapman, R. M., Ilmoniemi, R. J., Barbanera, S., andRomani, G. L. (1984). Selective localization ofalpha brain activity with neuromagnetic measure-ments. Electroencephalogr. Clin. Neurophysiol. 58,569–572.

Cheng, K., Fujita, H., Kanno, I., Miura, S., andTanaka, K. (1995). Human cortical regions acti-vated by wide-field visual motion: An H2 (15)OPET study. J. Neurophysiol. 74, 413–427.

Ciulla, C., Takeda, T., and Endo, H. (1999). MEG char-acterization of spontaneous alpha rhythm in thehuman brain. Brain Topogr. 11, 211–222.

Clarke, P. G. (1973a). Visual evoked potentials tochanges in the motion of patterned field. Exp.Brain Res. 18, 145–155.

Clarke, P. G. (1973b). Comparison of visual evokedpotentials to stationary and to moving patterns.Exp. Brain Res. 18, 156–164.

Clarke, S., and Miklossy, J. (1990). Occipital cortex inman: Organization of callosal connections, relatedmyelo- and cytoarchitecture, and putative bound-aries of functional visual areas. J. Comp. Neurol.298, 188–214.

Cohen, D. (1968). Magnetoencephalography: evi-dence of magnetic fields produced by alpha-rhythm currents. Science 161, 784–786.

Cohen, D. (1972). Magnetoencephalography: Detec-tion of the brain’s electrical activity with a superconducting magnetometer. Science 175,664–666.

Connor, C. E., Preddie, D. C., Gallant, J. L., and VanEssen, D. C. (1997). Spatial attention effects inMacaque Area V4. J. Neurosci. 17, 3201–3214.

Cooper, L. A., and Shepard, R. N. (1973). The timerequired to prepare for a rotated stimulus. MemoryCogn. 1, 246–250.

Corbetta, M. (1998). Frontoparietal cortical networksfor directing attention and the eye to visual loca-tions: identical, independent, or overlappingneural systems? Proc. Natl. Acad. Sci. U.S.A. 95,831–838.

REFERENCES 131


Corbetta, M., Miezin, F. M., Dobmeyer, S., Shulman,G. L., and Petersen, S. E. (1991). Selective anddivided attention during visual discriminations ofshape, color, and speed: functional anatomy bypositron emission tomography. J. Neurosci. 11,2383–2402.

Corbetta, M., Miezin, F. M., Shulman, G. L., andPetersen, S. E. (1993). A PET study of visuospatialattention. J . Neurosci. 13, 1202–1226.

Cornette, L., Dupont, P., Spileers, W., Sunaert, S.,Michiels, J., Van Hecke, P., Mortelmans, L., andOrban, G. A. (1998). Human cerebral activityevoked by motion reversal and motion onset. APET study. Brain 121, 143–157.

Courtney, S. M., Ungerleider, L. G., Kell, K., andHaxby, J. V. (1997). Transient and sustained activ-ity in a distributed neural system for humanworking memory. Nature 386, 608–611.

Cowey, A., and Weiskrantz, L. (1967). A comparisonof the effects of inferotemporal and striate corticallesions on visual behavior of rhesus monkeys. Q. J. Exp. Psychol. 19, 246–253.

Cragg, B. G. (1969). The topography of the afferentprojections in the circumstriate visual cortex of themonkey studied by the Nauta method. Vis. Res. 9,733–747.

Cuffin, B. N. (1990). Effects of head shape on EEG’sand MEG’s. IEEE BME 32, 905–910.

Dale, A. M., and Halgren, E. (2001). Spatiotemporalmapping of brain activity by integration of multi-ple imaging modalities. Curr. Opin. Neurobiol. 11,202–208.

Dale, A. M., and Sereno M. I. (1993). Improved local-ization of cortical activity by combining EEG andMEG with MRI cortical surface reconstruction: alinear approach. J. Cogn. Neurosci. 5, 162–176.

Dale, A. M., Liu, A. K., Fischl, B. R., Buckner, R. L.,Belliveau, J. W., Lewine, J. D., and Halgren, E.(2000). Dynamic statistical paramagnetic map-ping: Combining fMRI and MEG for high-resolution imaging of cortical activity. Neuron 26,55–67.

Damasio, A. R. (1985). Prosopagnosia. Trends Neurosci.8, 132–135.

Damasio, A. R. Damasio, H., and Van Hoesen, G. W.(1982). Prosopagnosia: Anatomic basis and behav-ioral mechanisms. Neurology 32, 331–334.

Darcey, T. M., Ary, J. P., and Fender, D. H. (1980).Spatiotemporal visually evoked scalp potentials in response to partial-field patterned stimu-lation. Electroencephalogr. Clin. Neurophysiol. 50,348–355.

De Monasterio, F. M., and Gouras, P. (1975).Functional properties of ganglion cells of theRhesus monkey retina. J. Physiol. 251, 167–195.

De Renzi, E. (1986). Current issues in prosopagnosia.In “Aspects of Face Processing” (H. D. Ellis, M. A.Jeeves, F. Newcombe, and A. Young, eds),pp. 153–252. Marinus Nijhoff, Dordrecht.

Desimone, R., and Duncan, J. (1995). Neural mecha-nisms of selective visual attention. Annu. Rev.Neurosci. 18, 193–222.

Desimone, R., Fleming, J., and Gross, C. G. (1980).Prestriate afferents to inferior temporal cortex: anHRP study. Brain Res. 184, 41–55.

Desimone, R., and Gross, C. G. (1979). Visual areas inthe temporal cortex of the macaque. Brain Res. 178,363–380.

Deutsch, J. A., and Deutsch, D. (1963). Attention:Some theoretical considerations. Psychol. Rev. 70,80–90.

De Yoe, E. A., and Van Essen, D. C. (1988). Concurrentprocessing streams in monkey visual cortex. TrendsNeurosci. 11, 219–226.

De Yoe, E. A., Felleman, D. J., Van Essen, D. V., andMcClendon, E. (1994). Multiple processing streamsin occipitotemporal cortex. Nature 371, 151–154.

Drasdo, N. (1980). Cortical potentials evoked bypattern presentation in the foveal region. In“Evoked Potentials” (C. Barber ed.), pp. 167–174MTP Press, Lancaster.

Eason, R. G. (1981). Visual evoked potential correlatesof early neural filtering during selective attention.Bull. Psychonom. Soc. 18, 203–206.

Eason, R. G., Harter, M. R., and White, C. T. (1969).Effects of attention and arousal on visually evokedcortical potentials and reaction time in man.Physiol. Behavior. 4, 283–289.

Eason, R. G., Oakley, M., and Flowers, L. (1983).Central neural influences on the human retinaduring selective attention. Physiol. Psychol. 11,18–28.

Engel, A. K., König, P., Kreiter, A. K., Schillen, T. B.,and Singer, W. (1992). Temporal coding in thevisual cortex: new vistas on integration in thenervous system. Trends Neurosci. 15, 218–226.

Engel, A. K., König, P., Kreiter, A. K., and Singer, W.(1991). Interhemispheric synchronization of oscil-latory neuronal responses in cat visual cortex.Science 252, 1177–1179.

Enroth-Cugell, C., and Robson, J. G. (1966). The con-trast sensitivity of retinal ganglion cells of the cat.J. Physiol. 187, 517–551.

Ermer, J. J., Mosher, J. C., Huang, M., and Leahy, R. M.(2000). Paired MEG data set source localizationusing recursively applied and projected (RAP)MUSIC. IEEE Trans. Biomed. Eng. 47, 1248–1260.

Eulitz, C., Maess, B., Pantev, C., Friederici, A. D.,Feige, B., and Elbert, T. (1996). Oscillatory neuro-magnetic activity induced by language and non-language stimuli. Cogn. Brain Res. 4, 121–132.

Eulitz, C., Eulitz, H., Maess, B., Cohen, R., Pantev, C.,and Elbert, T. (2000). Magnetic brain activityevoked and induced by visually presented wordsand nonverbal stimuli. Psychophysiology. 37,447–455.

Fahy, F. L., Riches, I. P., and Brown, M. W. (1993).Neuronal activity related to visual recognition



memory: Long-term memory and the encoding ofrecency and familiarity information in the primateanterior and medial inferior temporal and rhinalcortex. Exp. Brain Res. 96, 457–472.

Felleman, D. J., and Kaas, J. H. (1984). Receptive-fieldproperties of neurons in middle temporal visualarea (MT) of owl monkeys. J. Neurophysiol 52,488–513.

Felleman, D. J., and Van Essen, D. C. (1991).Distributed hierarchical processing in the primatecerebral cortex. Cereb. Cortex. 1, 1–47.

Ferrera, V. P., Nealey, T. A., and Maunsell, J. H. R.(1994). Responses in Macaque visual area V4 fol-lowing inactivation of the parvocellular and mag-nocellular LGN pathways. J. Neurosci. 14,2080–2088.

ffytche, D. H., Skidmore, B. D., and Zeki, S. (1995a).Motion-from-hue activates area V5 of humanvisual cortex. Proc. R. Soc. Lond. B Biol. Sci. 260,353–358.

ffytche, D. H., Guy, C. N., and Zeki, S. (1995b). Theparallel visual motion inputs into areas V1 and V5of human cerebral cortex. Brain 118, 1375–1394.

Fries, P., Reynolds, J. H., Rorie, A. E., and Desimone,R. (2001). Modulation of oscillatory neuronal syn-chronization by selective visual attention. Science291, 1560–1563.

Frostig, R., Lieke, E., Ts’o, D., and Grinvald, A. (1990).Cortical functional architecture and local couplingbetween neuronal activity and the microcircula-tion revealed by in vivo high-resolution opticalimaging of intrinsic signals. Proc. Natl. Acad. Sci.U.S.A. 87, 6082.

Fylan, F., Holliday, I. E., Singh, K. D., Anderson, S. J.,and Harding, G. F. A. (1997). Magnetoencephalo-graphic investigation of human cortical area V1using color stimuli. NeuroImage 6, 47–57.

Gaetz, M., Weinberg, H., Rzempoluck, E., and Jantzen,K. J. (1998). Neural network classifications andcorrelation analysis of EEG and MEG activityaccompanying spontaneous reversals of theNecker cube. Cogn. Brain Res. 6, 335–346.

Gallese, V., Fadiga, L., Fogassi, L., and Rizzolatti, G.(1996). Action recognition in the premotor cortex.Brain 119, 593–609.

Galletti, C., Fattori, P., Kutz, D. F., and Gamberini, M.,(1999). Brain location and visual topography ofcortical area V6A in the Macaque monkey. Euro. J.Neurosci. 11, 575–582.

Gauthier, I., Tarr, M. J., Anderson, A. W., Skudlarski,P., and Gore, J. C. (1999). Activation of the middlefusiform ‘face area’ increases with expertise in rec-ognizing novel objects. Nat. Neurosci. 2, 568–573.

Geesaman, B. J., and Andersen, R. A. (1996). The analysis of complex motion patterns byform/cue invariant MSTd neurons. J. Neurosci. 16,4716–4732.

George, J. S., Aine, C. J., Mosher, J. C., Schmidt, D. M.,Ranken, D. M., Schlitt, H. A., Wood, C. C., Lewine,

J. D., Sanders, J. A., and Belliveau, J. W. (1995).Mapping function in the human brain withmagnetoencephalography, anatomical magnetic res-onance imaging and functional magnetic resonanceimaging. J. Clin. Neurophysiol. 12, 406–431.

Ghandi, S. P., Heeger, D. J., and Boynton, G. M.(1999). Spatial attention affects cortical activity inhuman primary visual cortex. Proc. Natl. Acad. Sci.U.S.A. 96, 3314–3319.

Goldman-Rakic, P. S. (1988). Topography of cognition:Parallel distributed networks in primate associa-tion cortex. Annu. Rev. Neurosci. 11, 137–156.

Goodale, M. A., Meenan, J. P., Bulthoff, H. H., Nicolle,D. A., Murphy, K. J., and Racicot, C. I. (1994).Separate neural pathways for the visual analysisof object shape in perception and prehension.Curr. Biol. 4, 604–610.

Grave de Paralta-Menendez, R., and Gonzalez-Andino, S. L. (1998). A critical analysis of linearinverse solutions to the neuroelectromagneticinverse problem. IEEE Trans. Biomed. Eng. 45,440–8.

Gray, C. M. (1999). The temporal correlation hypo-thesis of visual feature integration: Still alive andwell. Neuron 24, 31–47.

Grill-Spector, K., Kushnir, T., Edelman, S., Itzchak, Y.,and Malach, R. (1998). Cue-invariant activation inobject-related areas of the human occipital lobe.Neuron 21, 191–202.

Gross, C. G. (1973). Visual functions of inferotemporalcortex. In “Handbook of Sensory Physiology” (R. Jung, ed), Vol. 7, pp. 451–482. Springer-Verlag,Berlin.

Gross, C. G., Rocha-Miranda, C. E., and Bender, D. B.(1972). Visual properties of neurons in infero-temporal cortex of the macaque. J. Neurophysiol.35, 96–111.

Gross, C. G., Bender, D. B., and Rocha-Miranda, C. E.(1974). Inferotemporal cortex: a single-unitanalysis. In “The Neurosciences (Third StudyProgram)” (F. O. Schmitt and F. G. Worden, eds.),pp. 229–238, MIT Press, Cambridge.

Hadjikhani, N., Liu, A. K., Dale, A. M., Cavanagh, P.,and Tootell, R. B. H. (1998). Retinotopy and colorsensitivity in human visual cortical area V8. Nat.Neurosci. 1, 235–241.

Haenny, P. E., and Schiller, P. H. (1988). State depen-dent activity in monkey visual cortex. I. Single cellactivity in V1 and V4 on visual tasks. Exp. BrainRes. 69, 225–244.

Haenny, P. E., Maunsell, J. H. R., and Schiller, P. H.(1988). State dependent activity in monkey visualcortex. II. Extraretinal factors in V4. Exp. Brain Res.69, 245–259.

Halgren, E., Dale, A. M., Sereno, M. I., Tootell, R. B.H., Marinkovic, K., and Rosen, B. R. (1999).Location of human face-selective cortex withrespect to retinotopic areas. Human Brain Mapping.7, 29–37.

REFERENCES 133


Halgren, E., Raij, T., Marinkovic, K., Jousmäki, V., andHari, R. (2000). Cognitive response profile of thehuman fusiform face area as determined by MEG.Cereb. Cortex. 10, 69–81.

Hämäläinen, M. S., and Ilmoniemi, R. J. (1994).Interpreting magnetic fields of the brain: Mini-mum norm estimates. Med. Biol. Eng. Comput. 32,35–42.

Hämäläinen, M. S., and Sarvas, J. (1987). Feasibility ofthe homogeneous head model in the interpretationof neuromagnetic fields. Phys. Med. Biol. 32, 91–97.

Hämäläinen, M. S., and Sarvas, J. (1989). Realistic con-ductivity geometry model of the human head forinterpretation of neuromagnetic data. IEEE BME36, 165–171.

Harding, G. F. A., Janday, B., and Armstrong, R. A.(1991). Topographic mapping and source localiza-tion of the pattern reversal visual evoked magneticresponse. Brain Topogr. 4, 47–55.

Harding, G. F. A., Degg, C., Anderson, S. J. , Holliday,I., Fylan, F., Barnes, G., and Bedford, J. (1994).Topographic mapping of the pattern onset evokedmagnetic response to stimulation of different por-tions of the visual field. Int. J. Psychophysiol. 16,175–183.

Hari, R., and Salmelin, R. (1997). Human corticaloscillations: a neuromagnetic view through theskull. Trends Neurosci. 20, 44–49.

Hari, R., Salmelin, R., Makela, J. P., Salenius, S., andHelle, M. (1997). Magnetoencephalographic corti-cal rhythms. Int. J. Psychophysiol. 26, 51–62.

Hari, R., Aittoniemi, L., Järvinen, M. L., Katila, T., andVarpula, T. (1980). Auditory evoked transient andsustained magnetic fields of the human brain. Exp.Brain Res. 40, 237–240.

Harrison, R. R., Aine, C. J., Chen, H.-W., and Flynn, E. R. (1996). Comparison of minimization methodsfor spatio-temporal electromagnetic source local-ization using temporal constraints. NeuroImage 3,S64.

Harter, M. R., and Aine, C. J. (1984). Brainmechanisms of visual selective attention. In(R. Parasuraman and D. R. Davies, (eds.),“Varieties of Attention” pp. 293–321. AcademicPress, New York.

Harter, M. R., Aine, C. J., and Schroeder, C. (1982).Hemispheric differences in the neural processingof stimulus location and type: Effects of selectiveattention on visual evoked potentials. Neuro-psychologia 20, 421–438.

Hashimoto, I., Kashii, S., Kikuchi, M., Honda, Y.,Nagamine, T., and Shibasaki, H. (1999). Temporalprofile of visual evoked responses to pattern-reversal stimulation analyzed with a whole-headmagnetometer. Exp. Brain. Res. 125, 375–382.

Haxby, J. V., Grady, C. L., Horwitz, B., Ungerleider, L. G.,Mishkin, M., Carson, R. E., Herscovitch, P.,Schapiro, M. B., and Rapoport, S. I. (1991).Dissociation of object and spatial visual processing

pathways in human extrastriate cortex. Proc. Natl.Acad. Sci. U.S.A. 88, 1621–1625.

Haxby, J. V., Horwitz, B., Ungerleider, L. G., Maisog, J.M., Pietrini, P., and Grady, C. L. (1994). The func-tional organization of human extrastriate cortex: APET-rCBF study of selective attention to faces andlocations. J. Neurosci. 14, 6336–6353.

Haxby, J. V., Ungerleider, L. G., Clark, V. P., Schouten,J. L., Hoffman, E. A., and Martin, A. (1999). Theeffect of face inversion on activity in human neuralsystems for face and object perception. Neuron 22,189-199.

Heinze, H. J., Mangun, G. R., Burchert, W., Hinrichs,H., Scholz, M., Munte, T. F., Gos, A., Scherg, M.,Johannes, S., and Hundeshagen, H. (1994).Combined spatial and temporal imaging of thebrain activity during visual selective attention inhumans. Nature 372, 543-546.

Hillyard, S. A. (1985). Electrophysiology of humanselective attention. Trends Neurosci. 8, 400–405.

Hillyard, S. A., and Munte, T. F. (1984). Selective atten-tion to color and locational cues: An analysis withevent-related brain potentials. Percept. Psychophys.36, 185–198.

Hoffman, J. E. (1978). Search through a sequentiallypresented visual display. Percept. Psychophys. 23,1–11.

Holliday, I. E., Anderson, S. J., and Harding, G. F.(1997). Magnetoencephalographic evidence fornon-geniculostriate visual input to human corticalarea V5. Neuropsychologia 35, 1139–1146.

Huang, M., Aine, C. J., Supek, S., Best, E., Ranken, andFlynn, E. R. (1998). Multi-start downhill simplexmethod for spatiotemporal source localization inmagnetoencephalography. Electroencephalogr. Clin.Neurophysiol. 108, 32–44.

Huk, A. C., and Heeger, D. J. (2000). Task-relatedmodulation of visual cortex. J. Neurophysiol. 83,3525–3536.

Ikeda, H., Leyba, L., Bartolo, A., Wang, Y., and Okada, Y. C. (2002). Synchronized spikes of thalamo-cortical axonal terminals and cortical neurons aredetectable outside the brain of piglet with high-resolution of MEG. J. Neurophysiol. 87, 626–630.

Ioannides, A. A., Bolton, J. P., and Clarke, C. J. (1990).Continuous probabilistic solutions to the bio-magnetic inverse problem. Inverse Problem 6,523–542.

Ioannides, A. A., Muratore, R., Balish, M., and Sato, S.(1993). In vivo validation of distributed sourcesolutions for the biomagnetic inverse problem.Brain Topogr. 5, 263–273.

Ioannides, A. A., Fenwick, P. B., Lumsden, J., Liu, M. J., Bamidis, P. D., Squires, K. C., Lawson, D., and Fenton, G. W. (1994). Activationsequence of discrete brain areas during cog-nitive processes: results from magnetic fieldtomography. Electroencephalogr. Clin. Neurophysiol.91, 399–402.



Ito, M., and Gilbert, C. D. (1999). Attention modulatescontextual influences in the primary visual cortexof alert monkeys. Neuron 22, 593–604.

Iwaki, S., Ueno, S., Imada, T., and Tonoike, M. (1999).Dynamic cortical activation in mental image pro-cessing revealed by biomagnetic measurement.NeuroReport 10, 1793–1797.

Jacobsen, S., and Trojanowski, J. Q. (1977). Pre-frontal granular cortex of the rhesus monkey. I. Intrahemispheric cortical afferents. Brain Res.132, 209–233.

Jeffreys, D. A., and Axford, J. G. (1972a). Source loca-tions of pattern-specific components of humanvisual evoked potentials. I. Component of striatecortical origin. Exp. Brain. Res. 16, 1–21.

Jeffreys, D. A., and Axford, J. G. (1972b). Source loca-tions of pattern-specific components of humanvisual evoked potentials. II. Component of extras-triate cortical origin. Exp. Brain. Res. 16, 22–40.

Jiang, Y., Haxby, J. V., Martin, A., Ungerleider, L. G.,and Parasuraman, R. (2000). Complementaryneural mechanisms for tracking items in humanworking memory. Science 287, 643–646.

Johnston, W. A., and Dark, V. J. (1986). Selective atten-tion. Annu. Rev. Psychol. 37, 43–75.

Kahneman, D., and Treisman, A. (1984). Changingviews of attention and automaticity. In “Varietiesof Attention” (R. Parasuraman and D. R. Davies,eds.), pp. 29–61. Academic Press, London.

Kaneoke, Y., Bundou, M., Koyama, S., Suzuki, H., andKakigi, R. (1997). Human cortical area respondingto stimuli in apparent motion. NeuroReport 8,677–682.

Kaneoke, Y., Bundou, M., and Kakigi, R. (1998).Timing of motion representation in the humanvisual system. Brain Res. 790, 195–201.

Kanwisher, N., McDermott, J., and Chun, M. M.(1997). The fusiform face area: A module in humanextrastriate cortex specialized for face perception.J. Neurosci. 17, 4302–4311.

Kapur, N., Friston, K. J., Young, A., Frith, C. D., andFrackowiak, R. S. J. (1995). Activation of humanhippocampal formation during memory for faces:A PET study. Cortex 31, 99–108.

Kaufman, L., and Williamson, S. J. (1980). The evokedmagnetic field of the human brain. Ann. N.Y. Acad.Sci. 340, 45–65.

Kaufman, L., Schwartz, B., Salustri, C., andWilliamson, S. J. (1990). Modulation of sponta-neous brain activity during mental imagery. J. Cogn. Neurosci. 2, 124–132.

Kawakami, O., Kaneoke, Y., and Kakigi, R. (2000).Perception of apparent motion is related to theneural activity in the human extrastriate cortex asmeasured by magnetoencephalography. Neurosci.Lett. 285, 135–138.

Kawamichi, H., Kikuchi, Y., Endo, H., Takeda, T., andYoshizawa, S. (1998). Temporal structure ofimplicit motor imagery in visual hand-shape dis-

crimination as revealed by MEG. NeuroReport9,1127–1132.

Kelly, D. H. (1966). Frequency doubling in visualresponses. J. Opt. Soc. Am. 56, 1628–1633.

Kim, J. J., Andreasen, N. C., O’Leary, D. S., Wiser, A.K., Boles Ponto, L. L., Watkins, G. L., and Hichwa,R. D. (1999). Direct comparison of the neural sub-strates of recognition memory for words andfaces. Brain 122, 1069–1083.

Kleinschmidt, A., Lee, B. B., Requardt, M., andFrahm, J. (1996). Functional mapping of color pro-cessing by magnetic resonance imaging ofresponses to selective P- and M-pathway stimula-tion. Exp. Brain. Res. 110, 279–288.

Klemm, W. R., Gibbons, W. D., Allen, R. G., andHarrison, J. M. (1982). Differences among humansin steady-state evoked potentials: evaluation ofalpha activity, attentiveness and cognitive aware-ness of perceptual effectiveness. Neuropsychologia20, 317–325.

Kobatake, E., Wang, G., and Tanaka, K. (1998). Effectsof shape-discrimination training on the selectivityof inferotemporal cells in adult monkeys. J. Neurophysiol. 80, 324–330.

Kohler, S., Kapur, S., Moscovitch, M., Winocur, G.,and Houle, S. (1995). Dissociation of pathways forobject and spatial vision: a PET study in humans.NeuroReport 6, 1865–1868.

Kosslyn, S. M. (1988). Aspects of a cognitive neuro-science of mental imagery. Science 240, 1621–1626.

Kosslyn, S. M., Ganis, G., and Thompson, W. L.(2001). Neural foundations of imagery. Nat. Rev. 2,635–642.

Kosslyn, S. M., Thompson, W. L., and Alpert N. M.(1997). Neural systems shared by visual imageryand visual perception: A positron emission tomog-raphy study. NeuroImage 6, 320–334.

Kosslyn, S. M., Thompson, W. L., Kim, I. J., andAlpert, N. M. (1995). Topographical representa-tions of mental images in primary visual cortex.Nature 378, 496–498.

Koyama, S., Kakigi, R., Hoshiyama, M., andKitamura, Y. (1998). Reading of Japanese Kanji(morphograms) and Kana (syllabograms): A mag-netoencephalographic study. Neuropsychologia 36,83–98.

Kreiman, G., Koch, C., and Fried, I. (2000). Imageryneurons in the human brain. Nature 408, 357–361.

Kuriki, S., Hirata, Y., Fujimaki, N., and Kobayashi, T.(1996). Magnetoencephalographic study on thecerebral neural activities related to the processingof visually presented characters. Cogn. Brain Res.4, 185–199.

Lachica, E. A., Beck, P. D., and Casagrande, V. A.(1992). Parallel pathways in macaque monkeystriate cortex: Anatomically defined columns inlayer III. Proc. Natl. Sci. U.S.A. 89, 3566–3570.

REFERENCES 135


Lagae, L., Raiguel, S., and Orban, G. A. (1993). Speedand direction selectivity of macaque middletemporal neurons. J. Neurophysiol. 69, 19–39.

Lam, K., Kaneoki, Y., Gunji, A., Yamasaki, H.,Matsumoto, E., Naito, T., and Kakigi, R. (2000).Magnetic response of human extrastriate cortex inthe detection of coherent and incoherent motion.Neuroscience 97, 1–10.

Lamme, V. A. F., Zipser, K., and Spekreijse, H. (1998).Figure–ground activity in primary visual cortex issuppressed by anesthesia. Proc. Natl. Acad. Sci.U.S.A. 95, 3263–3268.

Leahy, R. M., Mosher, J. C., Spencer, M. E., Huang, M.X., and Lewine, J. D. (1998). A study of dipolelocalization accuracy for MEG and EEG using ahuman skull phantom. Electroencephalogr. Clin.Neurophysiol. 107, 159–173.

Leinonen, L., and Nyman, G. (1979). Functional pro-perties of cells in antero-lateral part of area 7 asso-ciative face area of awake monkeys. Exp. Brain.Res. 34, 321–333.

Lesevre, N., and Joseph, J. P. (1979). Modifications ofthe pattern-evoked potentials (PEP) in relation tothe stimulated part of the visual field (clues for themost probable origin of each component).Electroencephalogr. Clin. Neurophysiol. 47, 183–203.

Leventhal, A. G., Rodieck, R. W., and Dreher, B.(1981). Retinal ganglion cell classes in the Old World Monkey: morphology and centralprojections. Science 213, 1139–1142.

Linkenkaer-Hansen, K., Palva, J. M., Sams, M.,Hietanen, J. K., Aronen, H., and Ilmoniemi, R.(1998). Face-selective processing in human extras-triate cortex around 120 msec after stimulus onsetrevealed by magnetoencephalography and elec-troencephalography. Neurosci. Lett. 253, 147–150.

Liu, A. K., Belliveau, J. W., and Dale, A. M. (1998).Spatiotemporal imaging of human brain activityusing functional MRI constrained magneto-encephalography data: Monte Carlo simulations.Proc. Natl. Acad. Sci. U.S.A. 95, 8945–50.

Liu, L., Ioannides, A. A., and Streit, M. (1999). Singletrial analysis of neurophysiological correlates ofthe recognition of complex objects and facialexpressions of emotion. Brain Topogr. 11, 291–303.

Liu, J., Higuchi, M., Marantz, A., and Kanwisher, N.(2000). The selectivity of the occipitotemporalM170 for faces. NeuroReport 11, 337–341.

Livingstone, M. S., and Hubel, D. H. (1987).Psychophysical evidence for separate channels forthe perception of form, color, movement, anddepth. J. Neurosci. 7, 3416–3468.

Livingstone, M. S., and Hubel, D. H. (1988).Segregation of form, color, movement, and depth:Anatomy, physiology, and perception. Science 240,740–749.

Logothetis, N. K., and Sheinberg, D. L. (1996). Visualobject recognition. Annu. Rev. Neurosci. 19,577–621.

Lu, S. T., Hämäläinen, M. S., Hari, R., Ilmoniemi, R. J.,Lounasmaa, O. V., Sams, M., and Vilkman, V.(1991). Seeing faces activates three separate areasoutside the occipital visual cortex in man.Neuroscience 43, 287–290.

Lu, Z. L., Williamson, S. J., and Kaufman, L. (1992).Behavioral lifetime of human auditory sensorymemory predicted by physiological measures.Science 258, 1668–1670.

Luber, B., Kaufman, L., and Williamson, S. J. (1989).Brain activity related to spatial visual attention. In“Advances in Biomagnetism” (S. J. Williamson,M. Hoke, G. Stroink, and M. Kotani, eds.),pp. 213–216. Plenum, New York.

Luck, S. J., Chelazzi, L., Hillyard, S. A., and Desimone,R. (1997). Neural mechanisms of spatial selectiveattention in areas V1, V2, and V4 of macaquevisual cortex. J. Neurophysiol. 77, 24–42.

Maclin, E., Okada, Y. C., Kaufman, L., andWilliamson, S. J. (1983). Retinotopic map on thevisual cortex for eccentrically placed patterns: Firstnoninvasive measurement. Nuovo Cimento 2,410–419.

Maguire, E. A., Frith, C. D., and Cipolotti, L. (2001).Distinct neural systems for the encoding andrecognition of topography and faces. NeuroImage13, 743–750.

Maier, J., Dagnelie, G., Spekreijse, H., and Van Dijk, B.W. (1987). Principal components analysis forsource localization of VEPs in man. Vis. Res. 27,165–177.

Malach, R., Reppas, J. B., Benson, R. R., Kwong, K. K.,Jiang, H., Kennedy, W. A., Ledden, P. J., Brady, T.J., Rosen, B. R., and Tootell, R. B. H. (1995). Object-related activity revealed by functional magneticresonance imaging in human occipital cortex. Proc.Natl. Acad. Sci. U.S.A. 92, 8135–8139.

Malonek, D., and Grinvald, A. (1996). Interactionsbetween electrical activity and cortical microcircu-lation revealed by imaging spectroscopy:Implications for functional brain mapping. Science272, 551–554.

Mangun, G. R., and Hillyard, S. A. (1988). Spatial gra-dients of visual attention: behavioral and electro-physiological evidence. Electroencephalogr. Clin.Neurophysiol. 70, 417–428.

Martin, A., and Chao, L. L. (2001). Semantic memoryand the brain: Structure and processes. Curr. Opin.Neurobiol. 11, 194–201.

Martin, A., Haxby, J. V., Lalonde, F. M. Wiggs, C. L.,and Ungerleider, L. G. (1995). Discrete corticalregions associated with knowledge of color andknowledge of action. Science 270, 102–105.

Mast, J., and Victor, J. D. (1991). A new statistic forsteady-state evoked potentials. Electroencephalogr.Clin. Neurophysiol. 78, 389–401.

Maunsell, J. H. R. (1995). The brain’s visual world:Representation of visual targets in cerebral cortex.Science 270, 764–768.



Maunsell, J. H. R., and Newsome, W. T. (1987). Visualprocessing in monkey extrastriate cortex. Annu.Rev. Neurosci. 10, 363–401.

Maunsell, J. H. R., and Van Essen, D. C. (1983).Anatomical connections of the middle temporalvisual area in the macaque monkey and theirrelationship to a hierarchy of cortical areas. J. Neurosci. 3, 2563–2586.

Maunsell, J. H. R., Nealey, T. A., and DePriest D. D.(1990). Magnocellular and parvocellular contribu-tions to responses in the middle temporal visualarea (MT) of the Macaque monkey. J. Neurosci. 10,3323–3334.

McAdams, C. J., and Maunsell, J. H. R. (1999). Effectsof attention on the reliability of individual neuronsin monkey visual cortex. Neuron 23, 765–773.

Meadows, J. C (1974). The anatomical basis ofprosopagnosia. J. Neurol. Neurosurg. Psychiatr. 37,489–501.

Mehta, A. D., Ulbert, I., and Schroeder, C. E. (2000a).Intermodal selective attention in monkeys. I.Distribution and timing of effects across visualareas. Cereb. Cortex 10, 343–358.

Mehta, A. D., Ulbert, I., and Schroeder, C. E. (2000b).Intermodal selective attention in monkeys. II.Physiological mechanisms of modulation. Cereb.Cortex 10, 359–370.

Mellet, E., Tzourio-Mazoyer, N., Bricogne, S.,Mazoyer, B., Kosslyn, S. M., and Denis, M. (2000).Functional anatomy of high-resolution visualmental imagery. J. Cogn. Neurosci. 12, 98–109.

Merigan, W. H., and Maunsell J. H. R. (1993). Howparallel are the primate visual pathways? Annu.Rev. Neurosci. 16, 369–402.

Mesulam, M.-M. (1998). From sensation to cognition.Brain 121: 1013–1052.

Michael, W. F., and Halliday, A. M. (1971). Differencesbetween the occipital distribution of upper andlower field pattern evoked responses in man. BrainRes. 32, 311–324.

Michel, C. M., Kaufman, L., and Williamson, S. J.(1994). Duration of EEG and MEG suppressionincreases with angle in a mental rotation task. J. Cogn. Neurosci. 6, 139–150.

Milner, B. (1963). Effects of different brain lesions oncard sorting: The role of the frontal lobes. Arch.Neurol. 9, 100–110.

Mishkin, M. (1982). A memory system in the monkey.Philos. Trans. R. Soc. Lond. 298, 85–95.

Moran, J., Desimone, R. (1985). Selective attentiongates visual processing in the extrastriate cortex.Science 229, 782–784.

Morrow, M. J., and Sharpe, J. A. (1995). Deficits ofsmooth-pursuit eye movement after unilateralfrontal lobe lesions. Ann. Neurol. 37, 443–451.

Mosher, J. C., and Leahy, R. M. (1998). RecursiveMUSIC: A framework for EEG and MEG sourcelocalization. IEEE Trans. Biomed. Eng. 45,1342–1354.

Motter, B. C. (1993). Focal attention produces spatiallyselective processing in visual cortical areas V1, V2,and V4 in the presence of competing stimuli. J. Neurophysiol. 70, 909–919.

Motter, B. C. (1994). Neural correlates of feature selec-tive memory and pop-out in extrastriate Area V4.J. Neurosci. 14, 2190–2199.

Movshon, J. A., and Newsome, W. T. (1996). Visualresponse properties of striate cortical neuronsprojecting to area MT in macaque monkeys. J. Neurosci. 16(23), 7733–7741.

Naito, T., Kaneoke, Y., Osaka, N., and Kakigi, R.(2000). Asymmetry of the human visual field inmagnetic response to apparent motion. Brain Res.865, 221–226.

Nakamura, M., Kakigi, R., Okusa, T., Hoshiyama, M.,and Watanabe, K. (2000). Effects of check size onpattern reversal visual evoked magnetic field andpotential. Brain Res. 872, 77–86.

Narici, L., and Peresson, M. (1995). Discriminationand study of rhythmical brain activities in thealpha band: A neuromagnetic frequency respon-siveness test. Brain Res. 703, 31–44.

Narici, L., Pizzella, V., Romani, G. L., Torrioli, G.,Traversa, R., and Rossini, P. M. (1990). Evokedalpha- and mu-rhythm in humans: A neuro-magnetic study. Brain Res. 520, 222–231.

Narici, L., Portin, K., Salmelin, R., and Hari, R. (1998).Responsiveness of human cortical activity torhythmical stimulation: A three-modality, whole-cortex neuromagnetic investigation. NeuroImage 7,209–223.

Nealey, T. A., and Maunsell, J. H. R. (1994).Magnocellular and parvocellular contributions tothe responses of neurons in macaque striatecortex. J. Neurosci. 14, 2069–2079.

Neville, H. J., and Lawson, D. (1987). Attention tocentral and peripheral visual space in a movementdetection task: An event-related potential andbehavioral study. I. Normal hearing adults. BrainRes. 405, 253–267.

Nishitani, N., and Hari, R. (2000). Temporal dynamicsof cortical representation for action. Proc. Natl.Acad. Sci. 97, 913–918.

Nishitani, N., Uutela, K., Shibasaki, H., and Hari, R.(1999). Cortical visuomotor integration during eyepursuit and eye-finger pursuit. J. Neurosci. 19,2647–2657.

Nobre, A. C., Sebestyn, G. N., Gitelman, D. R.,Mesulam, M. M., Frackowiak, R. S. J., and Frith,C. D. (1997). Functional localization of the systemfor visuospatial attention using positron emissiontomography. Brain 120, 515–533.

Novak, G. P., Wiznitzer, M., Kurtzberg, D., Giesser, B. S., and Vaughan, Jr., H. G. (1988). Theutility of visual evoked potentials using hemifieldstimulation and several check sizes in the eva-luation of suspected multiple sclerosis. Electro-encephalogr. Clin. Neurophysiol. 71, 1–9.

REFERENCES 137


Nowak, L. G., and Bullier, J. (1997). The timing ofinformation transfer in the visual system. Cereb.Cortex. 12, 205–232.

O’Craven, K. M., and Kanwisher, N. (2000). Mentalimagery of faces and places activates corres-ponding stimulus-specific brain regions. J. Cogn.Neurosci. 12, 1013–1023.

Ojemann, G., Ojemann, J., Lettich, E., and Berger, M.(1989). Cortical language localization in left, domi-nant hemisphere. J. Neurosurg. 71, 316–326.

Okada, Y. C., Kaufman, L., Brenner, D., andWilliamson, S. J. (1982). Modulation transfer func-tions of the human visual system revealed bymagnetic field measurements. Vis. Res. 22, 319–333.

Okada, Y., Wu, J., and Kyuhou, S. (1997). Genesis ofMEG signals in a mammalian CNS structure.Electroencephalogr. Clin. Neurophysiol. 103, 474–485.

Okada, Y., Lähteenmäki, A., and Xu, C. (1999a).Comparison of MEG and EEG on the basis ofsomatic evoked responses elicited by stimulationof the snout in the juvenile swine. Clin.Neurophysiol. 110, 214–229.

Okada, Y., Lahteenmäki, A., and Xu, C. (1999b).Experimental analysis of distortion of magneto-encephalography signals by the skull. Clin.Neurophysiol. 110, 230–238.

Okusa, T., Kakigi, R., and Osaka, N. (2000). Corticalactivity related to cue-invariant shape perceptionin humans. Neuroscience. 98, 615–624.

Ossenblok, P., and Spekreijse, H. (1991). The extrastri-ate generators of the EP to checkerboard onset. Asource localization approach. Electroencephalogr.Clin. Neurophysiol. 80, 181–193.

Parra, J., Meeren, H. K., Kalitzin, S., Suffczynski, P., deMunck, J. C., Harding, G. F., Trenite, D. G., andLopes da Silva, F. H. (2000). Magnetic sourceimaging in fixation-off sensitivity: Relationshipwith alpha rhythm. J. Clin. Neurophysiol. 17,212–223.

Pascual-Marqui, R. D., Michel, C. M., and Lehmann,D. (1994). Low resolution electromagnetic tomo-graphy: A new method for localizing elec-trical activity of the brain. Int. J. Psychophysiol. 18,49–65.

Patzwahl, D. R., Elbert, T., Zanker, J. M., andAltenmuller, E. O. (1996). The cortical representa-tion of object motion in man is interindividuallyvariable. NeuroReport 7, 469–472.

Paus, T. (1996). Location and function of the humanfrontal eye-field: A selective review. Neuro-psychologia 34, 475–483.

Paus, T., Kalina, M., Patockova, L., Angerova, Y.,Cerny, R., Mecir, P., Bauer, J., and Krabec, P. (1991).Medial vs lateral frontal lobe lesions and differen-tial impairment of central-gaze fixation mainte-nance in man. Brain 114, 2051–2067.

Perrett, D. I., Rolls, E. T., and Caan, W. (1982). Visualneurons responsive to faces in the monkey tempo-ral cortex. Exp. Brain. Res. 47, 329–342.

Perry, V. H., and Cowey, A. (1985). The ganglion celland cone distributions in the monkey’s retina:Implication for central magnification factors. Vis. Res. 25, 1795–1810.

Perry, V. H., Oehler, R., and Cowey, A. (1984). Retinalganglion cells that project to the dorsal lateralgeniculate nucleus in the Macaque monkey.Neuroscience 12, 1101–1123.

Petit, L., and Haxby, J. V. (1999). Functional anatomyof pursuit eye movements in humans as revealedby fMRI. J. Neurophysiol. 81, 463–471.

Petit, L., Dubois, S., Tzourio, N., Dejardin, S., Crivello,F., Michel, C., Etard, O., Denise, P., Roucoux, A.,and Mazoyer, B. (1999). PET study of the humanfoveal fixation system. Human Brain Mapping 8,28–43.

Pigarev, I. N., Rizzolatti, G., and Scandolara, C. (1979).Neurones responding to visual stimuli in thefrontal lobe of macaque monkeys. Neurosci. Lett.12, 207–212.

Pigeau, R. A., and Frame, A. M. (1992). Steady-statevisual evoked responses in high and low alphasubjects. Electroencephalogr. Clin. Neurophysiol. 84,101–109.

Ploner, C. J., Rivaud-Pechous, S., Gaymard, B. M.,Agid, Y., and Pierrot-Deseilligny, C. (1999). Errorsof memory-guided saccades in humans withlesions of the frontal eye field and the dorsolateralprefrontal cortex. J. Neurophysiol. 82, 1086–1090.

Polyak, S. I. (1957). The Vertebrate Visual system.University of Chicago Press, Chicago.

Portin, K., and Hari, R. (1999). Human parieto-occipi-tal visual cortex: Lack of retinotopy and fovealmagnification. Proc. R. Soc. Lond. B 266, 981–985.

Portin, K., Salenius, S., Salmelin, R., and Hari, R.(1998). Activation of the human occipital andparietal cortex by pattern and luminance stimuli:neuromagnetic measurements. Cereb. Cortex. 8,253–260.

Posner, M. I., Snyder, C. R., and Davidson, B. J. (1980).Attention and the detection of signals. J. Exp.Psychol. 109, 160–174.

Puce, A., Allison, T., Gore, J. C., and McCarthy, G.(1995). Face-sensitive regions in human extra-striate cortex studied by functional MRI. J. Neuro-physiol. 74, 1192–1199.

Raij, T. (1999). Patterns of brain activity during visualimagery of letters. J. Cogn. Neurosci. 11, 282–299.

Regan, D. (1966). Some characteristics of averagesteady-state and transient responses evoked by modulated light. Electroencephalogr. Clin.Neurophysiol. 20, 238–248.

Regan, D. (1978). Assessment of visual acuity byevoked potential recording: Ambiguity caused by temporal dependence of spatial frequencyselectivity. Vis. Res. 18, 439–443.

Regan, D. (1989). “Human Brain Electrophysiology:Evoked Potentials and Evoked Magnetic Fields inScience and Medicine.” Elsevier, New York.



Reynolds, J. H., and Desimone, R. (1999). The role ofneural mechanisms of attention in solving thebinding problem. Neuron 24, 19–29.

Richmond, B. J., Wurtz, R. H., and Sato T. (1983).Visual responses of inferior temporal neurons in awake Rhesus monkey. J. Neurophysiol. 50,1415–1432.

Rizzolatti, G., Fadiga, L., Gallese, V., and Fogassi, L.(1996). Premotor cortex and the recognition ofmotor actions. Cog. Brain Res. 3, 131–141.

Robson, J. G. (1966). Spatial and temporal contrast-sensitivity functions of the visual system. J. Opt.Soc. Am. 56, 1141–1142.

Rockland, K. S. (1995). Morphology of individualaxons projecting from area V2 to MT in themacaque. J. Comp. Neurol. 355, 15–26.

Rodieck, R. W. (1979). Visual Pathways. Annu. Rev.Neurosci. 2, 193–225.

Roelfsema, P. R., Engel, A. K., König, P., and Singer,W. (1997). Visuomotor integration is associatedwith zero time-lag synchronization among corticalareas. Nature 385, 157–161.

Roelfsema, P. R., Lamme, V. A. F., and Spekreijse, H.(1998). Object-based attention in the primaryvisual cortex of the Macaque monkey. Nature 395,376–381.

Roland, P. E., and Gulyas, B. (1994). Visual imageryand visual representation. Trends Neurosci. 17,281–296.

Rolls, E. T. (1984). Neurons in the cortex of the tem-poral lobe and in the amygdala of the monkeywith responses selective for faces. Hum. Neurobiol.3, 209–222.

Rolls, E. T., Judge, S. J., and Sanghera, M. K. (1977).Activity of neurons in the inferotemporal cortex ofalert monkey. Brain Res. 130, 229–238.

Rovamo, J., and Virsu, V. (1979). An estimation andapplication of the human cortical magnificationfactor. Exp Brain Res. 37, 495–510.

Rugg, M. D., Milner, A. D., Lines, C. R., and Phalp, R.(1987). Modulation of visual event-related po-tentials by spatial and non-spatial visual selectiveattention. Neuropsychologia 25, 85–96.

Saghal, A., and Iversen, S. D. (1978). The effects of foveal prestriate and inferotemporal lesions on matching to sample behavior in monkeys.Neuropsychologia. 16, 391–406.

Saleem, K. S., and Tanaka, K. (1996). Divergent projections from the anterior inferotemporal area TE to the perirhinal and entorhinal corticesin the macaque monkey. J. Neurosci. 16, 4757–4775.

Salenius, S. (1999). ‘Human see, human do’ cortex.Mol. Psychiatr. 4, 307–309.

Salenius, S., Kajola, M., Thompson W. L., Kosslyn, S.,and Hari, R. (1995). Reactivity of magnetic parieto-occipital alpha rhythm during visual imag-ery. Electroencephalogr. Clin. Neurophysiol. 95, 453–462.

Salmelin, R., and Hari, R. (1994). Characterization of spontaneous MEG rhythms in healthy adults. Electroencephalogr. Clin. Neurophysiol. 91,237–248.

Salmelin, R., Hari, R., Lounasmaa, O. V., and Sams,M. (1994). Dynamics of brain activation duringpicture naming. Nature 368, 463–465.

Sams, M., Hietanen, J. K., Hari, R., Ilmoniemi, R. J.,and Lounasmaa, O. V. (1997). Face-specific res-ponses from the human inferior occipito-temporalcortex. Neuroscience 77, 49–55.

Sanghera, M. K., Rolls, E. T., and Roper-Hall, A.(1979). Visual responses of neurons in the dorso-lateral amygdale of the alert monkey. Exp. Neuro.163, 610–626.

Sary, G., Vogels, R., and Orban, G. A. (1993). Cue-invariant shape selectivity of macaque inferiortemporal neurons. Science 260, 995–997.

Sato, N., Nakamura, K., Nakamura, A., Sugiura, M.,Ito, K., Fukuda, H., and Kawashima, R. (1999).Different time course between scene processingand face processing: a MEG study. NeuroReport 10,3633–3637.

Schein, S. J., and Desimone, R., (1990). Spectral pro-perties of V4 neurons in the macaque. J. Neurosci.10, 3369–3389.

Scherg, M. (1990). Fundamentals of dipole sourcepotential analysis. In “Advances in Audiology,Vol. 6, Auditory Evoked Magnetic Fields andElectric Potentials” (F. Grandori, M. Hoke, and G. L. Romani, eds.), pp. 40–69 Karger, Basel.

Schiller, P. H., and Malpeli, J. G. (1978). Functionalspecificity of lateral geniculate nucleus laminae of the Rhesus monkey. J. Neurophysiol. 41, 788–797.

Schmidt, D. M., George, J. S., and Wood, C. C. (1999).Bayesian inference applied to the electromagneticinverse problem. Human Brain Mapping 7P,195–212.

Schneider, G. E. (1969). Two visual systems. Science163, 895–902.

Schroeder, C. E., Mehta, A. D., and Foxe, J. J. (2001).Determinants and mechanisms of attentionalmodulation of neural processing. Front. Biosci.6, D672–684.

Seidemann, E., and Newsome, W. T. (1999). Effect ofspatial attention on the responses of Area MTneurons. J. Neurophysiol. 81, 1783–1794.

Seki, K., Nakasato, N., Fujita, S., Hatanaka, K.,Kawamura, T., Kanno, A., and Yoshimoto, T.(1996). Neuromagnetic evidence that the P100component of the pattern reversal visual evokedresponse originates in the bottom of the calcarinefissure. Electroencephalogr. Clin. Neurophysiol. 100,436–442.

Seltzer, B., and Pandya, D. N. (1978). Afferent corticalconnections and architectonics of the superiortemporal sulcus and surrounding cortex in rhesusmonkey. Brain Res. 149, 1–24.

REFERENCES 139


Sereno, M. I., Dale, A. M., Reppas, J. B., Kwong, K. K.,Belliveau, J. W., Brady, T. J., Rosen, B. R., andTootell, R. B. H. (1995). Borders of multiple visualareas in humans revealed by functional magneticresonance imaging. Science 268, 889–893.

Sergent, J., and Poncet, M. (1990). From covert to overtrecognition of faces in a prosopagnosic patient.Brain 113, 989–1004.

Sergent, J., Ohta, S., and MacDonald, B. (1992).Functional neuroanatomy of face and object pro-cessing. Brain 115, 15–36.

Shah, N. J., Aine, C. J., Schlitt, H., Krause, B. J.,Ranken, D., George, J., and Müller-Gärtner, H.-W.(1998). Activation of the visual ventral stream inhumans: An fMRI study. In “Positron EmissionTomography: A Critical Assessment of RecentTrends” (B. Gulya and H. W. Müller, eds). KluwerAcademic Publishers, Dordrecht.

Shen, L., Hu, X., Yacoub, E., and Ugurbil, K. (1999).Neural correlates of visual form and visual spatialprocessing. Human Brain Mapping. 8, 60–71.

Shiffrin, R. M., and Schneider, W. (1984). Automaticand controlled processing revisited. Psychol. Rev.91: 269–276.

Shigeto, H., Tobimatsu, S., Yamamoto, T., andKobayashi, M. (1998) Visual evoked cortical mag-netic responses to checkerboard reversal stimula-tion: A study on the neural generators of N75 andN145. J. Neurol. Sci. 156, 186–194.

Shipp, S., and Zeki, S. (1985). Segregation of pathwaysleading from area V2 to areas V4 and V5 of macaquemonkey visual cortex. Nature 315, 322–325.

Shipp, S., Blanton, M., and Zeki, S. (1998). A visuo-somatomotor pathway through superior parietalcortex in the Macaque monkey: Cortical connec-tions of areas V6 and V6A. Euro. J. Neurosci. 10,3171–3193.

Shulman, G. H. M., Corbetta, M., Buckner, R. L.,Raichle, M. E., Fiez, J. A., Miezen, F. M., andPeterson, S. E. (1997). Top-down modulation ofearly sensory cortex. Cereb. Cortex 7, 193–206.

Singer, W. (1995). Visual feature integration and thetemporal correlation hypothesis. Annu. Rev.Neurosci. 18, 555–586.

Singer, W. (1999). Neuronal synchrony: A versatilecode for the definition of relations? Neuron 24,49–65.

Sokolov, A., Lutzenberger, W., Pavlova, M., Preissl, H.,Braun, C., and Birbaumer, N. (1999). Gamma-bandMEG activity to coherent motion depends on task-driven attention. NeuroReport 10, 1997–2000.

Somers, D. C., Dale, A. M., Seiffert, A. E., and Tootell,R. B. (1999). Functional MRI reveals spatiallyspecific attentional modulation in human primaryvisual cortex. Proc. Natl. Acad. Sci. U.S.A. 96,1663–1668.

Spitzer, H., Desimone, R., and Moran, J. (1988).Increased attention enhances both behavioral andneuronal performance. Science 240, 338–340.

Squire, L. R., (1986). Mechanisms of memory. Science232, 1612–1619.

Srinivasan, R., Russell, D. P., Edelman, G. M., andTononi, G. (1999). Increased synchronization ofneuromagnetic responses during consciousperception. J. Neurosci. 19(13), 5435–5448.

Stensaas, S. S., Eddington, D. K., and Dobelle, W. H.(1974). The topography and variability of theprimary visual cortex in man. J. Neurosurg. 40,747–755.

Stephen, J. M., Aine, C. J., Christner, R. F., Ranken, D.,Huang, M., and Best, E. (2002). Temporal fre-quency sensitivity of the human visual systemmeasured with MEG: I. Central versus peripheralfield stimulation (submitted).

Stone, J., and Johnston, E. (1981). The topography ofprimate retina: A study of the human, bushbaby,and new and old-world monkeys. J. Comp. Neurol.196, 205–223.

Streit, M., Ioannides, A. A., Liu, L., Wölwer, W.,Dammers, J., Gross, J., Gaebel, W., and Müller-Gärtner, H.-W. (1999). Neurophysiological cor-relates of the recognition of facial expressions ofemotion as revealed by magnetoencephalography.Cogn. Brain Res. 7, 481–491.

Supek, S., and Aine, C. J. (1993). Simulation studies ofmultiple dipole neuromagnetic source localization:Model order and limits of source resolution. IEEETrans. Biomed. Eng. 40, 529–540.

Supek, S., and Aine, C. J. (1997). Spatio-temporalmodeling of neuromagnetic data: I. Multi-sourcelocation versus time-course estimation accuracy.Human Brain Mapping 5, 139–153.

Supek, S., Aine, C. J., Ranken, D., Best, E., Flynn, E. R.,and Wood, C. C. (1999). Single vs. paired visualstimulation: Superposition of early neuromagneticresponses and retinotopy in extrastriate cortex inhumans. Brain Res. 830, 43–55.

Swithenby, S. J., Bailey, A. J., Bräutigam, S., Josephs, O. E., Jousmäki, V., and Tesche, C. D.(1998). Neural processing of human faces: Amagnetoencephalographic study. Exp. Brain Res.118, 501–510.

Tallon-Baudry, C., and Bertrand, O. (1999). Oscillatorygamma activity in humans and its role in objectrepresentation. Trends Cogn. Sci. 3, 151–162.

Tallon-Baudry, C., Bertrand, O., Wienbruch, C., Ross,B., and Pantev, C. (1997). Combined EEG andMEG recordings of visual 40 Hz responses toillusory triangles in human. NeuroReport 8,1103–1107.

Tanaka, K. (1997). Mechanisms of visual object recog-nition: Monkey and human studies. Curr. Opin.Neurobiol. 7, 523–529.

Tesche, C. D., and Kajola, M. (1993). A comparison of the localization of spontaneous neuro-magnetic activity in the frequency and timedomains. Electroencephalogr. Clin. Neurophysiol. 87, 408–416.



Tesche, C. D., and Karhu, J. (2000). Theta oscillationsindex human hippocampal activation during aworking memory task. Proc. Natl. Acad. Sci. U.S.A.97(2), 919–924.

Tesche, C. D., Krusin-Elbaum, L., and Knowles, W. D.(1988). Simultaneous measurement of magneticand electrical responses of in vitro hippocampalslices. Brain Res. 462, 190–193.

Tesche, C. D., Uusitalo, M. A., Ilmoniemi, R. J., andKajola, M. J. (1995). Characterizing the local oscil-latory content of spontaneous cortical activityduring mental imagery. Brain Res. Cogn. Brain Res.2, 243–249.

Teyler, T. J., Cuffin, B. N., and Cohen, D. (1975). Thevisual evoked magnetoencephalogram. Life Sci. 17,683–692.

Tononi, G., Srinivasan, R., Russell, D. P., and Edelman,G. M. (1998). Investigating neural correlates ofconscious perception by frequency-tagged neuro-magnetic responses. Proc. Natl. Acad. Sci. U.S.A. 95,3198–3203.

Tootell, R. B. H., Hamilton, S. L., and Switkes, E.(1988). Functional anatomy of macaque striatecortex. IV. Contrast and magno-parvo streams. J. Neurosci. 8, 1594–1609.

Tootell, R. B., Reppas, J. B., Dale, A. M., Look, R. B.,Sereno, M. I., Malach, R., Brady, T. J., and Rosen, B. R. (1995a). Visual motion aftereffect in humancortical area MT revealed by functional magneticresonance imaging. Nature 375, 139–141.

Tootell, R. B., Reppas, J. B., Kwong, K. K., Malach, R.,Bom, R. T., Brady, T. J., Rosen, B. R., and Belli-veau, J. W. (1995b). Functional analysis of humanMT and related visual cortical areas using magnetic resonance imaging. J. Neurosci. 15,3215–3230.

Tootell, R. B. H., Hadjikani, N. K., Mendola, J. D.,Marrett, S., and Dale, A. M. (1998). From retino-topy to recognition: fMRI in human visual cortex.Trends Cogn. Sci. 2, 174–183.

Towle, V. L., Carder, R. K., Khorasani, L., andLindberg, D. (1999). Electrocorticographic coher-ence patterns. J. Clin. Neurophysiol. 16, 528–547.

Treisman, A., and Geffen, G. (1967). Selective atten-tion: Perception or response? Q. J. Exp. Psychol. 19,1–17.

Treue, S., and Maunsell, J. H. R. (1996). Attentionalmodulation of visual motion processing in areasMT and MST. Nature 382, 538–541.

Treue, S., and Martinez-Trujillo, J. C. (1999). Feature-based attention influences motion processing gainin macaque visual cortex. Nature 399, 575–579.

Ungerleider, L. G. (1995). Functional brain imagingstudies of cortical mechanisms for memory. Science270, 769–775.

Ungerleider, L. G., and Desimone, R. (1986).Projections to the superior temporal sulcus fromthe central and peripheral field representations ofV1 and V2. J. Comp. Neurol. 248, 147–163.

Ungerleider, L. G., and Haxby, J. V. (1994). “What”and “where” in the human brain. Curr. Opin.Neurobiol. 4, 157–165.

Ungerleider, L. G., and Mishkin, M. (1982). Twocortical visual systems. In “Analysis of Visual Behavior” (D. J. Ingle, M. A. Goodale, andR. J. W. Mansfield, eds.), pp. 549–586 MIT Press,Cambridge.

Uusitalo, M. A., Williamson, S. J., and Seppa, M. T.(1996). Dynamical organization of the humanvisual system revealed by lifetimes of activationtraces. Neurosci. Lett. 213, 149–152.

Uusitalo, M. A., Jousmaki, V., and Hari, R. (1997).Activation of trace lifetime of human corticalresponses evoked by apparent motion. Neurosci.Lett. 224, 45–48.

Uutela, K., Hämäläinen, M., and Somersalo, E. (1999).Visualization of magnetoencephalographic datausing minimum current estimates. NeuroImage 10,173–180.

Vanderwolf, C. H. (1969). Hippocampal electricalactivity and voluntary movement in the rat.Electroencephalogr. Clin. Neurophysiol. 26(4), 407–18.

Van Essen, D. C. (1979). Visual areas of the mam-malian cerebral cortex. Annu. Rev. Neurosci. 2,227–263.

Van Essen, D. C. (1985). Functional organization ofprimate visual cortex. In “In Cerebral Cortex” A.Peters and E. G. Jones (eds.), Vol. 3, pp. 259–329.Plenum, New York.

Van Essen, D. C., and Maunsell, J. H. R. (1983).Hierarchical organization and functional streams in the visual cortex. Trends Neurosci. 6,370–375.

Van Essen, D. C., Newsome, W. T. N., and Bixby, J. L.(1982). The pattern of interhemispheric connec-tions and its relationship to extrastriate visualareas in the macaque monkey. J. Neurosci. 2,265–283.

Vanni, S., and Uutela, K. (2000). Foveal attentionmodulates responses to peripheral stimuli. J. Neurophysiol. 83, 2443–2452.

Vanni, S., Revonsuo, A., and Hari, R. (1997).Modulation of the parieto-occipital alpha rhythmduring object detection. J. Neurosci. 17, 7141–7147.

Vanni, S., Portin, K., Virsu, V., and Hari, R. (1999). Murhythm modulation during changes of visual per-cepts. Neuroscience. 91, 21–31.

Vanni, S., Tanskanen, T., Seppä, M., Uutela, K., andHari, R. (2001). Coinciding early activation of the human primary visual cortex and antero-medial cuneus. Proc. Natl. Acad. Sci. U.S.A. 98,2776–2780.

Van Voorhis, S., and Hillyard, S. A. (1977). Visualevoked potentials and selective attention to pointsin space. Percept. Psychophys. 22, 54–62.

Vidyasagar, T. R. (1998). Gating of neuronal responsesin macaque primary visual cortex by an atten-tional spotlight. NeuroReport 9, 1947–1952.

REFERENCES 141


Walla, P., Hufnagl, B., Lindinger, G., Imhof, H.,Deecke, L., and Lang, W. (2001). Left temporal andtemporoparietal brain activity depends on depthof word encoding: A magnetoencephalographicstudy in healthy young subjects. NeuroImage 13,402–409.

Wang, J. Z., Williamson, S. J., and Kaufman, L. (1992).Magnetic source images determined by a lead-field analysis: The unique minimum-norm. IEEETrans. Biomed. Eng. 39, 665–675.

Wang, L., Kakigi, R., and Hoshiyama, M. (2001).Neural activities during Wisconsin Card SortingTest-MEG observation. Cogn. Brain Res. 12, 19–31.

Watanabe, T., Sasaki, Y., Miyauchi, S., Putz, B.,Fujimaki, N., Nielsen, M., Takino, R., and Miya-kawa, S. (1998). Attention-regulated activity inhuman primate visual cortex. J. Neurophysiol. 79,2218–2221.

Watson, J. D., Myers, R., Frackowiak, R. S., Hajnal, J.V., Woods, R. P., Mazziotta, J. C., Shipp, S., andZeki, S. (1993). Area V5 of the human brain:Evidence from a combined study using positronemission tomography and magnetic resonanceimaging. Cereb. Cortex 3, 79–94.

Weinberg, H., Brickett, P., Coolsma, F., and Baff, M.(1986). Magnetic localization of intracranialdipoles: Simulation with a physical model. Electro-encephalogr. Clin. Neurophysiol. 64, 159–170.

Whiteley, A. M., and Warrington, E. K. (1977).Prosopagnosia: A clinical, psychological, andanatomical study of three patients. J. Neurol.Neurosurg. Psychiatr. 40, 394–430.

Wiesel, T. N., and Hubel, D. H. (1966). Spatial and chro-matic interactions in the lateral geniculate body ofthe rhesus monkey. J. Neurophysiol. 29, 1115–1156.

Williamson, S. J., Kaufman, L., and Brenner, D. (1978).Latency of the neuromagnetic response of thehuman visual cortex. Vis. Res. 18, 107–110.

Williamson, S. J., Kaufman, L., Curtis, S., Lu, Z. L.,Michel, C. M., and Wang, J. Z. (1996). Neural sub-strates of working memories are revealed mag-netically by the local suppression of alpha rhythm.

Electroencephalogr. Clin. Neurophysiol. (Suppl.) 47,163–180.

Wilson, M. (1978). Visual system: Pulvinar-extrastriatecortex. In R. B. Masterton (ed) “Handbook ofBehavioral Neurobiology” (R. B. Masterton, ed.),Vol. 1, pp. 209–247. Plenum, New York.

Worden, M. S., Wellington, R., and Schneider, W.(1996). Determining the locus of attentional selec-tion with functional magnetic resonance imaging.NeuroImage 3, 244.

Wright, M. J., and Ikeda, H. (1974). Processing ofspatial and temporal information in the visualsystem. In “The Neurosciences (Third StudyProgram)” (F. O. Schmitt and F. G. Worden, eds.),pp. 115–122. MIT Press, Cambridge.

Wu, J., and Okada, Y. C. (1998). Physiological bases ofthe synchronized population spikes and slowwave of the magnetic field generated by a guinea-pig longitudinal CA3 slice preparation. Electro-encephalogr. Clin. Neurophysiol. 107, 361–373.

Wurtz, R. H., and Mohler, C. W. (1976). Enhancementof visual responses in monkey striate cortex andfrontal eye fields. J. Neurophysiol. 39, 766–772.

Zeki, S. M. (1969). Representation of central visualfields in prestriate cortex of monkey. Brain Res. 14,271–291.

Zeki, S. M. (1973). Colour coding in rhesus monkeyprestriate cortex. Brain Res. 53, 422–427.

Zeki, S. (1978). Functional specialisation in the visualcortex of the rhesus monkey. Nature 274, 423–428.

Zeki, S. M. (1980). A direct projection from area V1 toarea V3A of rhesus monkey visual cortex. Proc. R.Soc. Lond. 207, 499–506.

Zeki, S., Watson, J. D. G., Lueck, C. J., Friston, K. J.,Kennard, C., and Frackowiak, R. S. J. (1991). A direct demonstration of functional specializa-tion in human visual cortex. J. Neurosci. 11,641–649.

Zouridakis, G., Simos, P. G., Breier, J. I., andPapanicolaou, A. C. (1998). Functional hemi-spheric asymmetry assessment in a visual lan-guage task using MEG. Brain Topogr. 11, 57–65.




C H A P T E R

6

Aligning Linguisticand Brain Views on

Language ComprehensionKara D. Federmeier, Robert Kluender, and Marta Kutas

INTRODUCTION

Language affords human beings anincredible degree of representational flexi-bility within the limits of highly restrictiveconstraints. For example, despite theremarkable ability of the human vocal tractto produce all kinds of sounds, only alimited subset of these is actually used inthe world’s languages. A native speaker ofany of the world’s languages will have anintuitive sense for what kinds of soundscan and cannot (e.g., coughs and raspberrynoises) figure as possible sounds in ahuman language, even if none of thesounds in question occurs in his or herown particular language. Individual lan-guages exhibit an even more restrictedinventory of speech sounds, the lowestattested number being eleven. However,this small repertoire of sounds is groupedand ordered to create a very much largerset of words in a language, termed itslexicon. Likewise, the entries in thislexicon, although large in number—typi-cally consisting of many thousands ofentries—can be combined to form literallyan infinite number of sentences describingreal, imaginary, and impossible objects andevents, not to mention emotional statesand abstract concepts.

Especially striking is that almost allhumans learn this complicated codingsystem early in life and use it throughouttheir life span with ease. Every day,humans produce and comprehend com-pletely new strings of words, at a rate ofabout 150 words per minute (Maclay andOsgood, 1959). No other species is capableof this tremendous versatility, either natu-rally in the wild or when trained inhumanlike communication systems forexperimental purposes in the laboratory.Our linguistic ability is one of the manysalient characteristics that distinguishhumans from other species. Another is therelative size and complexity of our brains,and surely these two features are not unre-lated or logically independent. In fact, wecan say that the degree of flexibility andefficiency we exhibit in this cognitivedomain is a consequence of the structure oflanguage, together with the structure of theentity that represents it and mediates itsprocessing, the human brain. In thischapter we examine how these cometogether in neural imaging studies of lan-guage comprehension relying on physio-logical measures, with an emphasis onelectrical brain activity (an equally com-pelling story could be told for languageproduction, but that tale will not be toldhere).

LANGUAGE STRUCTURE: THE LINGUIST’S VIEW AND

THE BRAIN’S VIEW

The language system is structured atmultiple levels, ranging from its physicalform to its referential meaning in context.Phonetics and phonology are the study oflanguage sounds. Phonetics describes howthe speech sounds utilized by all humanlanguages are produced, transmitted, andperceived. Within any given language,sounds (and hand shapes in signed lan-guages) come to be systematically orga-nized and categorized (into “phonemes”).For example, various combinations of dif-ferent actual sound patterns (mediated bymeasurably different vocal tract configura-tions) may all yield something that anEnglish speaker interprets as a “t”—the(different) sounds in the words “top,”“stop,” “pot,” and “button,” for example.Phonology is the study of such soundsystems and the kind of knowledge thatpeople have about the sound patterns oftheir particular language.

“Morphemes” are combinations ofphonemes that have come to have theirown meaning. Some are whole words (e.g.,“cat”), whereas others are affixes that mod-ulate the meaning of whole words (e.g., the/s/, which, when added to the end of anEnglish word, makes that word plural).Morphology, then, is the study of the pat-terns that govern word formation, includ-ing both how morphemes combine to yieldnew meanings or parts of speech (deriva-tional morphology), and how theycombine to create different forms of thesame word (inflectional morphology). Justas morphemes are combined to create newwords and new forms of words, wholewords are combined to make larger unitsof language—phrases, clauses, sentences,and discourses. Within and across lan-guages, the way in which certain wordsand types of words come to be puttogether to create these larger language

units is patterned. Phrases, for example,are built around particular types of words.Noun phrases may contain several differ-ent types of words but must contain atleast one noun and must not contain averb. Phrases act as units that can be foundin multiple places in a sentence—forinstance, noun phrases may be subjects,objects, or parts of prepositional phrases.The study of language grammars—of howwords (and affixes) pattern across phrases,sentences and discourses—is known assyntax.

Ultimately, humans use language totransmit information—meaning—that de-pends not just on the general pattern ofsounds or words, but on the specific wordsused, their specific pattern, and the specificcontext (linguistic, social, environmental)in which they occur. The study of languagemeaning in general, known as semantics,and of meaning in its larger context,known as pragmatics, asks how languageis used to transmit and, in some cases,affect or even distort reality.

From a linguist’s perspective, then, lan-guage is a highly structured system, andthis structure is important for understand-ing how language can be used so readilyand efficiently. However, it cannot be thestructure of language alone that makes itsuch a useful tool, for if that were the caseit would be difficult to understand whyhumans alone come to have fully devel-oped language skills. Rather, it must be thestructure of language in combination withthat of the human brain that explains howhumans acquire, use, and create language.A question then arises: Does the humanbrain “see” language the way that linguistssee language?

As just discussed, linguists have uncov-ered language patterns at various levels—phonological, morphological, syntactic,semantic, pragmatic. Cognitive neuroscien-tists and psycholinguists have long soughtto determine whether these regularitiesarise from some aspect of sensory and cog-nitive processing. It seems likely that at

144 6. LANGUAGE COMPREHENSION


least some of these distinctions are alsoimportant to the brain. At some level, forexample, the brain probably does processphonological patterns differently fromsemantic or syntactic ones, and there islikely to be some difference between thebrain’s processing of two different soundsthat are ultimately treated alike and twothat are ultimately distinguished. How-ever, linguists have typically focused ongeneral principles of language organiza-tion and function that cut across individuallanguages, across individual languagespeakers, and even across individualinstances of language performance. Inother words, they are generally not con-cerned with processing issues and thusoften examine patterns collapsed acrosstime and space. However, the brain’s pro-cessing of language necessarily takes placein time and space, and brain scientists arededicated to delineating the importance ofboth. For example, linguistic inputs thatare separated by different stretches of time(e.g., different numbers of words) or thatrequire different numbers/sizes of saccadiceye movements are liable to be treated dif-ferently by the brain, though not, perhaps,by linguists. At the same time, not all dif-ferences noted by linguists are likely to bemeaningful to all brain areas at all times.Early in visual processing, for instance, thebrain responds similarly to letter stringsthat can be pronounced (i.e., are phono-logically legal) and those that cannot.

Clearly then, any processing account oflanguage must reconcile the categories thatlinguists have inferred from analysis of theworld’s languages (competence) with theprocesses that brain scientists have inferredfrom various neurobiological measures ofbrain activity during language perfor-mance; thus cognitive neuroscientists andpsycholinguists have long been interestedin the brain’s “view.” However, the issue ofwhether the brain sees language as lin-guists do (and why the answer shouldmatter) has been of some debate amonglinguists. To a large extent, the debate has

played out along the lines of functionalistversus reductionist views of the study ofmind (Churchland, 1984; Fodor, 1981). Thestrongest functionalist stance on this issuecame from Noam Chomsky. With a pro-nounced emphasis on mental phenomenaover and above mere observable linguisticbehavior, Chomsky firmly establishedmodern linguistic science as practiced inthe latter half of the twentieth century as afunctionalist enterprise par excellence:

Mentalistic linguistics is simply theoreticallinguistics that uses performance as data(along with other data, for example, the dataprovided by introspection) for the determina-tion of competence, the latter being taken asthe primary object of its investigation. Thementalist, in this traditional sense, need makeno assumptions about the possible physiologi-cal basis for the mental reality that he studies.In particular, he need not deny that there issuch a basis. One would guess, rather, that it isthe mentalistic studies that will ultimately beof greatest value for the investigation of neuro-physiological mechanisms, since they alone areconcerned with determining abstractly theproperties that such mechanisms must exhibitand the functions they must perform.[Chomsky, 1965, p. 193, fn. 1]

From this point of view we can proceed toapproach the study of the human mind muchin the way that we study the physical structureof the body. In fact, we may think of the studyof mental faculties as actually being a study ofthe body—specifically the brain—conducted ata certain level of abstraction. [Chomsky, 1980,p. 2]

In essence, Chomsky’s original answerto this question was that ultimately, thebrain would indeed have to see languagethe way the linguist does, because only thelinguist is in a position to determine theprimitives and operations in need ofneural implementation. Chomsky latermodified this position by acknowledgingthat primary linguistic data provided byintrospection were no longer able to decideamong competing linguistic theories, andthat linguists would therefore need to takeinto consideration other sources of evi-

LANGUAGE STRUCTURE: THE LINGUIST’S VIEW AND THE BRAIN’S VIEW 145


dence derived from performance mea-sures, including neural imaging data.

At the opposite extreme, perhaps thestrongest reductionist answer to the ques-tion of the appropriate characterization ofthe brain–language relationship came froma structural linguist, Charles Hockett:

The essential difference between the [ana-lytical] process in the child and the procedureof the linguist is this: the linguist has to makehis analysis overtly, in communicable form, inthe shape of a set of statements which can beunderstood by any properly trained person,who in turn can predict utterances not yetobserved with the same degree of accuracy ascan the original analyst. The child’s ‘analysis’consists, on the other hand, of a mass ofvarying synaptic potentials in his centralnervous system. The child in time comes tobehave the language; the linguist must come tostate it… . [The linguistic scientist’s] purpose inanalyzing a language is not to create structure,but to determine the structure actually createdby the speakers of the language. For the scien-tist, then, ‘linguistic structure’ refers to some-thing existing quite independently of theactivities of the analyst: a language is what itis, it has the structure it has, whether studiedand analyzed by a linguist or not. [Hockett,1963, p. 280]

Although this was originally publishedin 1948 and reflects the behaviorism of thetime, it nonetheless seems not all that farremoved from a present-day neoempiristposition on this issue.

Nowadays, both psychological function-alists and reductionists are interested inhow the brain “sees” language. Forexample, one of the characteristics thatFodor (1983) assigns to mental modules isassociation with fixed neural architecture.In part because of this, much of the currentinterest in neural imaging techniquesamong linguists and psycholinguistscomes from those with functionalist lean-ings. Thus, taking the brain’s perspectiveon language will likely yield useful datafor scientists of any theoretical persuasion,and that is what we will attempt to dohere.

METHODS FOR EXAMININGBRAIN FUNCTION

The brain not only represents languagebut also is involved in its creation and itsreal-time use. To understand how requiresknowing something about the brain andabout what regularities in language thebrain notices and under what circum-stances. Thus, cognitive neuroscientistsinterested in language processing haveturned to a number of noninvasive brainimaging techniques in order to get variousmutually constraining pictures of the brainin action as it processes language. Theseinclude direct measures of brain electricalactivity and measures of metabolicprocesses that support such activity.

Comprehending and producing lan-guage are brain functions that require thecoordinated activity of large groups ofneurons. This neural communication iselectrochemical in nature, involving themovement of electrically charged elementsknown as ions. Under normal (nonstimu-lated) conditions, each neuron has a“resting” electrical potential that arises dueto the distribution of ions inside andoutside it. Stimulation of the neuronchanges the permeability of the neuralmembrane to these charged elements,thereby altering the electrical potential. Atransient increase in potential (depolariza-tion) at the cell body can cause an all-or-none wave of depolarization that movesalong the cell’s axon, known as an “actionpotential.” The action potential thenspreads to other neurons via the release ofneurotransmitters from the axon terminals;these neurotransmitters diffuse acrossextracellular space (synaptic cleft) andcause permeability changes in the den-drites of nearby neurons. These permeabil-ity changes may cause an action potentialin the receiving cell as well, or may simplyalter the electrical potential of that cell suchthat it will be more or less sensitive toensuing stimulation.



Neural communication thus involveswavelike changes in the electrical potentialalong neurons and their processes. Thesecurrent flows are the basis for electrophysi-ological recordings in the brain and at thescalp surface, because changes in electricalpotential can be monitored by placing atleast two electrodes somewhere on thehead (or in the brain) and measuring the voltage difference between them. Theresulting electroencephalogram (EEG)observed at the scalp is due to the summedpotentials of multiple neurons acting inconcert. In fact, much of the observedactivity at the scalp likely arises from corti-cal pyramidal cells whose organization andfiring satisfy the constraints for an observ-able signal (for details, see, e.g., Allison,et al., 1986; Kutas and Dale, 1997; Nunezand Katznelson, 1981).

The EEG measures spontaneous, rhyth-mic brain activity occurring in multiple fre-quency bands. For the purposes ofunderstanding the neural basis of languageprocessing, however, cognitive neuroscien-tists are often interested in the brain’sresponse to a particular event or kind ofevent, such as the appearance of a word ona computer screen. To examine event-related activity in particular, one canaverage the EEG signal time-locked to thestimuli of interest to create an “event-related potential” or ERP. The ERP is awave form consisting of voltage fluctua-tions in time, one wave form for eachrecording site. This wave form consists of aseries of positive- and negative-goingvoltage deflections (relative to some baseline activity prior to event onset). Underdifferent experimental conditions, one canobserve changes in wave form morphology(e.g., presence or absence of certain peaks),the latency, duration, or amplitude (size) ofone or more peaks, or their distributionover the scalp. Until recently, electrophysi-ological investigations of language havefocused on relatively fast (high-frequency),transient ERP responses; more recently,however, slower potentials that develop

over the course of clauses and sentenceshave also been monitored.

ERPs are informative indices of lan-guage-related processes because they are acontinuous, multidimensional signal.Specifically, ERPs provide a direct estimateof what a significant part of the brain isdoing just before, during, and after anevent of interest, even if it is extended intime. And, they do so with millisecond res-olution. ERPs can indicate not only thattwo conditions are different, but alsohow—whether, for example, there is aquantitative change in the timing or size ofa process or a qualitative change in thenature of processing or involvement of dif-ferent brain generators as reflected in a dif-ferent morphology or scalp distribution. Toa limited extent, ERPs can also be used toinfer where in the brain processes takeplace [via source modeling techniques andin combination with other neuroimagingtechniques; for more information see Daleand Halgren (2001) and Kutas et al.,(1999)].

Because it is difficult to localize pre-cisely the neural source of electrical signalsrecorded at the scalp from the electricalrecordings alone, other types of brainimaging methods have been brought tobear on attempts to link specific brainareas with cognitive processes. The brainrequires a constant supply of blood tomeet its metabolic demands, which in turnchange with neural activity. With increasedneuronal activity, glucose consumptiongoes up and there are concomitant localincreases in blood flow and changes inoxygen extraction from the blood. Thesehemodynamic (and metabolic) changescan be monitored with techniques such aspositron emission tomography (PET) andfunctional magnetic resonance imaging(fMRI). Local changes in blood flowduring a cognitive task, for example, canbe followed with PET using 15O-labeledwater. Because these increases in bloodflow exceed increases in local oxygenextraction, blood near regions of neural

METHODS FOR EXAMINING BRAIN FUNCTION 147


activity eventually has higher concentrationsof oxygenated hemoglobin compared toblood near inactive regions. Such differencescan be measured with fMRI, because ashemoglobin becomes deoxygenated itbecomes more paramagnetic than the sur-rounding tissue, thereby creating a magneti-cally inhomogeneous environment. Underthe right circumstances, these hemodynamicimaging methods can localize regions ofneural activity with high spatial resolution.Their temporal resolution, however, is poor-er, because hemodynamic responses typi-cally lag the electrical signal by 1–2 sec anddo not track activity on a millisecond-by-millisecond basis. Combinations of thesemeasures of brain activity thus seem to offerthe most complete picture of where, when,and how language is processed in the brain.

Using neuroimaging techniques,researchers have looked at language pro-cessing from early stages of word recogni-tion through the processing of multisentencediscourses, from the planning of a speech actto its articulation (e.g., Kutas and VanPetten, 1994; Osterhout, 1994; Osterhout andHolcomb, 1995). So doing reveals that thebrain’s processing of language involvesmany different kinds of operations takingplace at different times and different tempo-ral scales and in multiple brain areas. Theseoperations differ in the extent to which theyare general purpose versus language spe-cific, in the extent to which they are affectedby context (and to what types of contextsthey are sensitive), and in the extent towhich they interact with one another inspace and time.

LANGUAGE COMPREHENSION

Initially, the brain cannot know whetheran incoming stimulus is linguistic or not.Thus, its first task when confronted with awritten, spoken, or signed word—as withany external, perceptual stimulus—is todetermine what it is, or at least to what cat-egories it might belong. This (unconscious)

decision is crucial and difficult; in order toprocess a stimulus effectively, attention mustbe distributed over the stimulus appropri-ately, certain kinds of feature informationmust be extracted and possibly stored inmemory, information needed to interpret thestimulus must be accessed from long-termmemory, and so on. Because the braincannot always know what kind of stimulusit will encounter at any given moment, someaspects of (especially early) perceptual pro-cessing are likely to be similar regardless ofthe nature of the stimulus. At times, process-ing decisions may also be guided byguesses—based on frequency, recency, andother predictive regularities gained fromexperience with the sensory world—aboutwhat the stimulus is likely to be. When itcan, it seems that the brain makes use ofboth top-down (expectancy or context-based) and bottom-up (stimulus-based)information to guide its analysis of input.Thus, if someone has been reading or listen-ing to a stream of linguistic stimuli, theirbrain might be biased to treat incominginput as linguistic; in other contexts, thesame input may initially be interpreted asnonlinguistic (e.g., Johnston and Chesney,1974). To the extent that the context allows,the brain might also form expectations aboutthe physical nature of the stimulus—color,size, font, loudness, voice, etc. Modulationof attention to such stimulus parameters isreflected in variations in the amplitude ofearly sensory ERP components that origi-nate from primary and secondary sensory-processing areas in the brain (e.g., P1, N1,mismatch negativity, Nd, processing nega-tivity; see relevant chapters in this book).Depending on the task demands, there may also be various kinds of effects on laterERP components such as N2, P3, RP, etc.,shown to vary systematically with cognitivevariables.

From Perception to Language

Regardless of the nature or degree ofavailable top-down information, however,



the first task for successful language com-prehension involves early sensory classi-fication of the input. In the visual modality,for example, this might include differenti-ating single objectlike stimuli from strings,orthographically legal words from illegalwords, or pseudowords from nonwords.Schendan and colleagues (1998) examinedthe time course of this type of classificationby comparing the ERP responses to objectlike (real objects, pseudoobjects), wordlike(words, letter strings, pseudofont strings),and intermediate (icon strings) stimuli.Around 95 msec a negativity (N100) overmidline occipital sites distinguished singleobjectlike stimuli from strings (see Fig. 1).This differentiation is important because,as supported by the neuropsychological lit-erature, different attentional resources arerequired to process sets of spatially distinctobjects as opposed to a single, spatiallycontiguous form, and these processes aremediated by different brain areas (e.g.,Farah, 1990). This classification was fol-lowed shortly by a distinction betweenstrings made from real letters (words andpseudowords) and those made from other

characters (icon strings, pseudofont), sug-gesting that the visual system of experi-enced readers has developed the abilityrapidly to detect physical stimuli with theproperties of real letters. Results fromintracranial recording and fMRI studiessuggest that such differentiations may beoccurring in areas in the posterior fusiformgyrus (Allison et al., 1994) and the occipi-totemporal and inferior occipital sulci(Puce et al., 1996). Finally, the ERPsshowed a distinction between words andpseudowords, beginning approximately200 msec poststimulus onset. Similar timecourses of analyses and categorizationsseem to hold for auditory inputs as well;for example, the ERPs to meaningful andnonsense words are very similar withinthe first 150 msec of processing and beginto be distinguishable by 200–250 msec(Novick et al., 1985).

Although ERPs provide a very tempo-rally precise means of determining anupper limit on the time by which the brainmust have appreciated the differencebetween two conditions or stimuli, they donot explicitly tell us either what that differ-

LANGUAGE COMPREHENSION 149


FIGURE 1 ERPs to visual stimuli. Sample stimuli (A) including (1) words, (2) nonwords, (3) pseudofont, (4) icon strings, (5) objects, (6) and pseudoobjects and the associated grand average ERPs (B) from a midlinecentral (Cz) and occipital (Oz) electrode site. The P150 is large for stringlike stimuli (words, nonwords, andpseudofont), small for objectlike stimuli (objects and pseudoobjects), and intermediate for icon strings. FromSchendan et al. (1998); reprinted with the permission of Cambridge University Press.

ence means or the extent to which informa-tion about that difference will be availablefor or actually used in further processing.So, the fact that the processing of realwords and pseudowords is differentiatedat some level by 200–250 msec does notnecessarily mean that the brain hasidentified one type of stimulus as a wordand the other as not a word (in the sameway that a linguist or psycholinguistmight). It may just reflect the brain’sgreater exposure to one class of stimulithan the other or its sensitivity to unusual(infrequent) letter combinations that char-acterize one class of stimuli more than theother. In fact, pronounceable pseudowordscontinue to be processed much like realwords (in terms of the components elicited,though not necessarily in their size andlatency) for several hundred millisecondsmore. Unlike nonwords, but like stimulibearing meaning, including real words,pronounceable pseudowords elicit a nega-tivity peaking approximately 400 msecpoststimulus onset (N400). Thus, it wouldseem that at least some of the processingcircuits of the brain deal with pseudo-words, which have no particular learnedmeaning, no differently than these circuitsdo with real words for some time after aninitial differentiation. Perhaps the early dif-ferentiation has less to do with whether

any item is or is not a word and more to dowith the extent of prior exposure. ERPresearch with children just acquiring lan-guage and/or reading skills as well as withadults learning a second language mayprovide a means for examining thishypothesis (Mills et al., 1997; Neville et al.,1997; Weber-Fox and Neville, 1996).Indeed, answering such questions posesone of the major challenges in cognitiveneurolinguistics.

It is around the time that the brain’sresponse to words seems to first divergefrom that to pseudowords that the ERPalso first shows a sensitivity to a word’sfrequency of occurrence in a given lan-guage (Francis and Kucera, 1982)—or, fromthe brain’s point of view, the context-inde-pendent probability of encountering a par-ticular word. King and Kutas (1998b)found that the latency of a left anteriornegativity (which they labeled the lexicalprocessing negativity, or LPN) occurringbetween 200 and 400 msec poststimulusonset is strongly correlated with a word’sfrequency of occurrence in the language(see Fig. 2). In short, the brain seems toprocess more rapidly words that it has hadmore experience processing. This kind ofearly difference in the speed with whichwords are processed can have large conse-quences later in the processing stream.



FIGURE 2 The lexical processing negativity is sensitive to word frequency. Grand average ERPs in response towords presented one at a time in sentences read for comprehension. Overlapped are the ERPs (digitally high-passfiltered at 4 Hz) to words sorted as a function of their frequency of occurrence in the English language. Thelatency of the negative peak is longest (~340 msec) for low-frequency (·····) and shortest (~280 msec) for high-frequency (—-) words. Medium-frequency (----) words peak at ~300 msec. Data from King and Kutas (1998b).

King and Kutas (1998b) suggested that atleast some of the reported differencesbetween the processing of “open class”(nouns, verbs, adjectives, adverbs) and“closed class” (determiners, articles, prepo-sitions) words were due to differences intheir average frequency and the conse-quences this had on their early neural pro-cessing (also see Brown et al., 1999; Münteet al., 2001; Osterhout et al., 1997a).

It is important to point out, however,that there is no single time or place where“word frequency” is processed and/orstored. Rather, word frequency affects mul-tiple stages of processing, including wordidentification, access of associated phono-logical or semantic information from long-term memory, maintenance of word formor associated information in workingmemory, etc. In fact, ERP results clearlydemonstrate that word frequency has dif-ferent effects later in a word’s processing.For example, with all other factors heldconstant (especially in the absence ofsemantic context), N400 amplitude is aninverse function of word frequency (VanPetten and Kutas, 1991). As will be dis-cussed later, the N400 seems to be relatedto the access of semantic information fromlong-term memory and/or the integrationof this information into a larger context.This aspect of processing is also affected bymore “immediate” or local frequency infor-mation—namely, repetition in the experi-mental context (e.g., Rugg, 1985). Similar tothe effects of global frequency information,repetition reduces the amplitude of theN400 activity, among the effects it has onother components (Van Petten et al., 1991).

Processing Patterns

The fact that a word is encountered fre-quently or was just encountered thusaffects the way it is processed by the brain.Moreover, it affects the processing at differ-ent times and most likely in different ways:the time interval since the last repetition,the number of repetitions, and the context

within which the repetition occurs all seemto matter, albeit differently as a function ofthe individual’s age (Besson and Kutas,1993; Besson et al., 1992; Kazmerski andFriedman, 1997; Nagy and Rugg, 1989;Rugg et al., 1997; Young and Rugg, 1992).More specifically, words repeated in thecontext of a word list are typically charac-terized by an enhanced positivity; theeffects of repetition overlap but are notlimited to the region of the N400 and arethought to comprise multiple components.Repetition effects are large on immediaterepetition (with no delay lag) in young andolder adults. At longer lags, the pattern ofeffects is more variable in general andapparently smaller, if present at all, inolder individuals. Although multiple repe-titions of a word progressively diminishthe amplitude of the N400 component, thisreduction is modulated by the nature ofthe context in which the word reappears; aword repeated in an altered context seemsto show little signs of N400 reduction(Besson and Kutas, 1993). However, whena word is repeated in the same context, itsN400 amplitude is progressively reducedby repetition, even if it is semanticallyanomalous within its context, such that bya third presentation, the N400 region ischaracterized by a large posterior positivecomponent. In fact, the repetition effect ismore pronounced for semantically anom-alous than for semantically congruous sen-tence endings.

Effects like these are likely to hold forlanguage units larger than words as well—e.g., frequent and infrequent word com-binations and frequent or infrequentsyntactic structures. Indeed, ERPs revealthe importance of probability in the brain’sprocessing of syntactic aspects of a sen-tence. A late positivity, variously called theP600 or “syntactic positive shift” (SPS), hasmost commonly been elicited in responseto dispreferred, low-frequency, but possi-ble continuations of sentences, as well as tooutright syntactic violations (e.g., Coulsonet al., 1998b; Hagoort et al., 1993; Neville



et al., 1991; Osterhout and Holcomb, 1992).This positivity has a variable onset latency(generally late, but sometimes as early as200 msec, depending on stimulus onsetasynchrony) and a midpoint around600 msec—though this may vary with thecomplexity of the linguistic structureinvolved (Münte et al., 1997b). Its scalpdistribution is most often posterior, thoughanterior effects have also been reported(see Fig. 3).

The P600 is typically observed whensome aspect of a sentence violates struc-tural (as opposed to semantic) expecta-tions. For example, the P600 is reliablyelicited when low-level morphosyntacticviolations occur in a sentence, as when asubject does not agree in number with itsverb (“Every Monday he mow the lawn”),when a noun phrase is incorrectly markedfor case (“Ray fell down and skinned heknee”) or number (“It has a nasty tempersand bites”), or when the second verb of acompound form is incorrectly inflected(“Dew does not fell like rain does”). Inaddition, late positivity is also seen in the

ERP response when the expected canonicalword order of a phrase is disrupted(“Max’s of proof the theorem”) or when averb’s argument structure requirements arenot met (“The broker persuaded to sell thestock”). It is important to note, however,that the P600 is not contingent on the pres-ence of a grammatical violation; it is alsoelicited by points of processing difficulty,when the difficulty stems from processingat a grammatical or structural level (Kaanet al., 2000; Osterhout and Holcomb, 1992).Although these manipulations are all “syn-tactic” to linguists, they differ significantlyfrom one another in ways that are likely tomatter to the brain. For example, some,such as word order violations, dependalmost exclusively on position in the linearstring, whereas others, such as morphosyn-tactic violations, depend instead on therelationship between words in a sentence,relatively independent of their linear sen-tence position. Still others, such as verbargument structure violations, depend notonly on the relationship between sentenceelements, but also on the relationship of



FIGURE 3 The P600. Grand average ERPs to target words in grammatical (solid) and ungrammatical (dotted)sentences at one anterior (left) and one posterior (right) electrode site. Compared to grammatical controls, ERPsto ungrammatical stimuli (here, case violations) are associated with enhanced positivity between 600 and 800 msec over posterior scalp sites.

those sentence elements to requirementsspecified in the lexical entry of the verb.

So what might the P600 be indexing? Aclue comes from work by Coulson and colleagues (1998b), who examined theresponse to morphosyntactic violations ofsubject–verb agreement and case markingwhen these violations were either frequentor infrequent in an experimental run. Theyobserved a P600 response to ungrammati-cal as compared with grammatical trials,although infrequent ungrammatical eventselicited larger P600s than did frequentlyoccurring ungrammatical events. More-over, even grammatical events elicitedsome P600 activity when they occurredinfrequently among many ungrammaticalsentences (for further discussion seeCoulson et al., 1998a; Gunter et al., 1997;Münte et al., 1998a; Osterhout andHagoort, 1999; Osterhout et al., 1996).

It seems, then, that the part of the brainthat is sensitive to syntactic violations isalso sensitive to the subjective probabilityof those violations. Although the P600 isnot typically elicited by semanticallyimprobable events, it can be elicited sub-sequent to N400 effects in such contexts,and can even be elicited in response tononstandard but orthographically andphonologically licit spellings of words(Münte et al., 1997b). This may suggestthat, at least at some point, the processingof syntax takes place by reference to therelative (perceived?) frequency and relia-bility of various expected regularities inthe language, a frequency that is continu-ously updated with experience. However,much work still remains to be done detail-ing the sensitivity of P600 amplitude tononlinguistic variables, and understandingthe nature of its relationship (identity,overlap, independence) from the group ofpositivities variously called the P3, P3b,P300, and late positive component, whichlikely encompass several distinct subcom-ponents.

The nature of early, sometimes left-later-alized frontal negativities that frequently

precede the P600 in the ERPs to syntacticviolations is somewhat less well under-stood. These negativities have beenreported in two latency ranges: about 100to 300 msec postword onset, the early leftanterior negativity (ELAN), and, morecommonly, about 300 to 500 or 600 msecpostword onset, the left anterior negativity(LAN). These negativities have thus fareluded definitive identification as to theireliciting conditions, presumably becausethe experimental designs across studieshave manipulated different variables andbecause, within individual studies, themanipulations undertaken have often beensubject to more than one interpretation.

The left anterior negativity elicitedbetween 300 and 600 msec has basicallybeen interpreted in one of two differentways. One is as a direct and immediateresponse to syntactic or morphosyntacticill-formedness (e.g., Münte et al., 1993;Neville et al., 1991; Osterhout andHolcomb, 1992), and the other is as anindex of working memory processesduring sentence comprehension (Coulsonet al., 1998b; King and Kutas, 1995;Kluender and Kutas, 1993). The problem indeciding between these two alternatives isthat manipulations of syntactic well-formedness have often occurred in sen-tences that incorporate long-distancerelationships, which tax working memoryresources, and working memory manipu-lations of long-distance sentence relation-ships have often resulted in—or beenconfounded with—less than completewell-formedness, as measured by eithergrammaticality or acceptability. Moreover,there is additional evidence in support ofboth interpretations. In the case of mor-phosyntactic and word order violations,LAN effects have been dissociated fromP600 effects when such violations occur injabberwocky sentences containing pseudo-words: in these cases, the P600 effects canbe suppressed but the LAN effect persists(Canseco-Gonzalez et al., 1997; Münte et al.,1997a; but see also Hahne and Jescheniak,



2001). This dissociation has been taken tomean that the LAN is the true marker ofill-formedness, whereas the P600 is merelyan index of attempts on the part of thebrain to recompute the sentence and makesense of the faulty input it is getting; thisattempt at recomputation purportedlydoes not occur in jabberwocky sentencesbecause no sense is to made of them in thefirst place. On the other hand, in supportof the working memory interpretation,LAN effects have also been observed intandem with P600 effects in response tolong-distance, purely semantic violationsof hyponymy that crucially range acrosstwo separate clauses (Shao and Neville,1996). It may well be the case that bothinterpretations of the LAN are correct, i.e.,that it is influenced both by syntactic ill-formedness and by working memory load(Kluender et al., 1998; Vos et al., 2001).Because both syntactic processing andverbal working memory are known func-tions of left frontal cortex, and becauseboth tend to activate Broca’s area in neuralimaging studies (e.g., Dapretto and

Bookheimer, 1999; Embick et al., 2000; Kanget al., 1999; Ni et al., 2000), this could be oneclear instance in which the brain is notstrictly respecting linguistic analysis (seeFig. 4).

The early left anterior negativity elicitedbetween 100 and 300 msec postword onsetis of slightly more recent vintage. TheELAN has most reliably been elicited inresponse to word category violations usingauditory presentation (Friederici et al.,1993; Hahne and Friederici, 1999) and hasbeen interpreted as an index of a first-stagesyntactic parser sensitive only to word cat-egory (i.e., part of speech) information inbuilding an initial phrase structure tree.Because of its early latency and variability,it has been highly controversial. The mor-phology of the component varies fromstudy to study in an as yet unpredictablemanner: sometimes it is part of a broadnegativity extending throughout the entireepoch (Friederici et al., 1993, 1996), whereasat other times it is a phasic componentwith a clear peak (Hahne and Jescheniak,2001). The relation of the ELAN to the



FIGURE 4 The left anterior negativity (LAN). Grand average ERPs to target sentence intermediate words(shown in bold type in the sentences) at left and right prefrontal (top) and left and right parietal (bottom) elec-trode sites. Responses at the main clause verb of object-relative sentences with an inanimate subject (solid lines)are compared with object-relative sentences with an animate subject (dotted lines). Animate subjects are harder toprocess in this construction because, although they tend to be subjects, they are here being used as objects of therelative-clause verb. In response to this increased ambiguity in syntactic processing, one observes increased neg-ativity between 300 and 500 msec over frontal sites, with a left-lateralized distribution (LAN). Over parietal sites,the beginning of a P600 response can also be observed. Data from Weckerly and Kutas (1999).

LAN is also unclear. With low-contrastvisual input, the latency of the componentfalls within that of the LAN (Gunter et al.,1999); in jabberwocky studies, very similarmanipulations have resulted in ELANeffects in one case (Hahne and Jescheniak,2001) and in LAN effects in another(Münte et al., 1997a); sometimes the ELANis followed by a LAN, usually as a continu-ing negativity (Friederici et al., 1993, 1996;Hahne and Friederici, 1999), and some-times it is not (Hahne and Friederici, 1999;Hahne and Jescheniak, 2001).

More generally, the relation of the ante-rior phasic negativities to the P600 isunclear—at times they precede it, at othertimes they do not. Is this due to an inter-action with verbal working memory,activated in some cases more than others?Moreover, the relation of the anteriorphasic negativities to the slow anteriornegative potentials indexing verbalworking memory load (as discussed in thenext section) is equally unclear. These areissues that will need to be sorted out infuture research. What such results do makeclear, however, is that the brain is sensitiveto the frequency and recency of exposureto particular patterns. Its sensitivities rangefrom the probability of encountering a par-ticular physical stimulus to the probabilityof those stimuli patterning in a particularway with respect to one another in aphrase or sentence.

Working Memory

The brain’s sensitivity to linguistic pat-terns of various types highlights anotherimportant aspect of language, namely, theneed to process relations between items, atdifferent levels of abstraction. Many lin-guistic patterns emerge over the course ofmultiple words separated by time and/orspace, depending on the modality of pre-sentation. Processing relations betweenthese items necessitates that the brainmaintain them in some kind of temporarystore or “working memory.” Even simple,

declarative sentences (e.g., “John reallylikes his pet dog”) require workingmemory resources. At minimum, “John”must be held in memory so that thereader/listener knows who is beingreferred to when the pronoun “his” isencountered. Indeed, ERP data show thatthe brain is sensitive to the relationshipbetween a pronoun and its antecedent.When an occupational title (e.g., “secre-tary”) is paired with the more “probable”pronoun “she” (based on United Statescensus data), less negativity is observedaround 200 msec over left anterior sitesthan when the same occupation is pairedwith the less probable pronoun “he” (Kingand Kutas, 1998a). In the latter case, thebrain may assume that the “he” refers to anew participant because the pronoun–antecendent pair seems less likely; theincreased negativity may then reflect theworking memory load associated withstoring and/or holding onto informationabout two participants as opposed to onlyone. In a somewhat similar design withreflexive pronouns, Osterhout and co-workers (1997b) found that pronouns thatdisagreed with the gender definition orgender stereotype of an antecendent nounelicited a large positivity (i.e., the P600 typ-ically associated with syntactic violations).The important point, however, is that pro-nouns elicit reliable ERP effects that can beused to investigate the link between themand the nouns to which they refer—a linkthat clearly relies on working memory.

Although all sentences tap into workingmemory, some clearly absorb moreworking memory resources than others(see Fig. 5). For instance, a sentence con-taining a relative clause (e.g., “The reporterwho followed the senator admitted theerror”) typically requires more workingmemory resources than a simple declara-tive sentence, in part because a participant(“the reporter”) is involved in twoclauses/actions (“following” and “admit-ting”). These “subject-relative clauses,”however, are presumed to require fewer



working memory resources than are object-relative clauses, such as “The reporter whothe senator followed admitted the error”.In object-relative clauses, the subject of themain clause (“the reporter”) must be keptdistinct from the subject of the relativeclause (“the senator”). By examining sen-tences that vary in the extent to which theyrequire working memory resources, onecan examine the nature of the brain’sresponse to working memory load (e.g.,Friederici et al., 1998; King and Kutas, 1995;Kutas and King, 1996; Mecklinger et al.,1995; Muller et al., 1997). In addition, onecan assess individual variation in thebrain’s response to sentences of varyingstructural complexity as a function of theamount of working memory resourcesavailable (e.g., comparing individuals with high working memory “spans” withthose who have fewer working memoryresources). For example, King and Kutas(1995) compared ERP responses to subject-and object-relative sentences read one

word at a time. As soon as the sentencestructures varied, good comprehenderselicited greater left, frontal negativities inthe object-relative as compared with thesubject-relative clauses. This is the point in the sentence where, in the case of object relatives, a second participant (“thesenator”) must be stored in workingmemory (along with “the reporter”). In contrast, the response of poor compre-henders (with fewer working memoryresources) was quite negative to both typesof sentences; thus, both types of sentencesseemed to tax working memory resourcesfor poorer comprehenders. Similar effectswere observed for these same sentencespresented as natural speech (Muller et al.,1997). These results led to the hypothesisthat the left anterior effect reflects general,as opposed to modality-specific, workingmemory operations. A similar left anteriornegativity effect has also been observed forwh-questions (Kluender and Kutas, 1993).In English wh-questions (e.g., “Who did



FIGURE 5 Working memory and sentence processing. Comparison of grand average ERPs to subject-relative(solid lines) and object-relative (dotted lines) sentences from a left anterior site. On the left are the unfiltereddata and on the right are the same data after they have been low-pass filtered to highlight slowly developingresponses. The visual sentences were presented one word at a time, whereas the auditory sentences were pre-sented as natural speech. The shading represents the area where object-relative sentences are reliably morenegative than are subject-relative sentences. Visual data from Kutas and King (1996) and auditory data fromMuller et al. (1997).

the doctor cure __?”), the wh-element (the“filler,” in this case the word “who”)appears at the beginning of the sentence,leaving a “gap” in the canonical wordorder (which in English is subject–verb–object). Another example comes fromuncommon (and therefore difficult) wordorders in German (Roesler et al., 1998). Therole of working memory operations in sen-tence processing can also be examined bysimply adding an irrelevant or elaborativeclause to a simple transitive sentence(Gunter et al., 1995).

The extended nature of various workingmemory operations is also manifest in lesstransient, slow potential effects (long-lasting potentials on the order of seconds).For example, in response to the subjectversus object-relative clauses discussedabove, good comprehenders show a slowpositive shift to the subject-relative sen-tences over frontal sites that lasts for theduration of the relative clause and beyond;poor comprehenders do not show eitherthis slow positivity or this difference(Kutas and King, 1996). This comprehen-sion-related ERP difference shows up evenfor simple transitive sentences, with goodcomprehenders generating much more of afrontal positive shift compared to poorercomprehenders. At the same time, poorercomprehenders show enhanced earlysensory visual components such as theP1–N1–P2 relative to the better com-prehenders. This suggests that poorercomprehenders (as compared to good com-prehenders) may have devoted moreresources to lower level perceptual pro-cessing, thereby having fewer resources todevote to higher order (possibly working-memory demanding) language processes.The potentials in normal elderly individu-als for both simple transitive and object-relative sentences most resemble those of the poorer comprehending youngerindividuals (Kutas and King, 1996).

A number of PET and fMRI studies alsohave compared subject-relative and object-relative clauses, in the interest of localizing

verbal working memory processes withinthe human brain. Because the left inferiorfrontal gyrus is a well-established lan-guage area commonly implicated inaspects of syntactic processing and pro-duction (i.e., Broca’s area), it is perhaps notsurprising that in all such studies to date,the left inferior frontal gyrus has emergedas a reliable locus of activation when thehemodynamic response to object-relativeclauses has been compared to that forsubject-relative clauses. This is also consis-tent with the finding that the neural cir-cuitry for visuospatial working memory inthe rhesus monkey involves extensive net-works in prefrontal cortex (e.g., Goldman-Rakic, 1990). Thus in an fMRI study, Just etal. (1996) compared center-embeddedsubject- and object-relative clauses likethose above with active conjoined clauses(e.g., “The reporter followed the senatorand admitted the error”), and foundincreased activation in both Broca’s andWernicke’s areas for object-relative clausescompared to subject-relative clauses andfor subject-relative clauses as compared toactive conjoined clauses. On the otherhand, a series of PET studies of center-embedded object relatives (e.g., “The juice[that the child spilled] stained the rug”) vs.right-branching subject relatives (e.g., “Thechild spilled the juice [that stained therug]”) by Caplan and colleagues hasconsistently shown increased activationonly in Broca’s area, though the exact locus of activation within Broca’s area has shifted slightly from study to study:the pars opercularis (Brodmann’s area 44)in Stromswold et al. (1996) and Caplanet al. (1998), and the pars triangularis(Brodmann’s area 45) in Caplan et al. (1999,2000).

However, a Japanese fMRI study thatmanipulated center-embedded vs. left-branching structures using only subject-relative clauses in Japanese, a verb-finallanguage, yielded a widespread increase inactivation in response to the center-embed-ded subject relatives (Inui et al., 1998). This



increase in activation appeared in both BA44 and BA 45 of the left inferior frontalgyrus (i.e., Broca’s area), as well as in theposterior portion of the left superior tem-poral gyrus (BA 22, i.e., Wernicke’s area),and in left dorsolateral prefrontal cortex(the posterior part of BA 9). What is inter-esting about this comparison is that inorder to maintain strict left branching inJapanese, the canonical subject–object–verb(SOV) word order of Japanese, preservedin the center-embedded condition, must bedisrupted: the object noun plus its (preced-ing) relative clause must be moved to thefront of the sentence, resulting in relativelyrare OSV word order (corpus studies ofboth spoken and written Japanese textshow that the OSV pattern occurs less than1% of the time) (Yamashita, 2002). Notethat this emulates the word order of objectrelatives in English (“…who the senatorfollowed,” “…that the child spilled,” etc.).Thus at least in Japanese, center embed-ding with canonical word order seems topresent a greater load for working memorythan fronting an object in noncanonicalword order. Naturally, more cross-linguis-tic studies of this nature are needed totease apart conclusively the effects ofcenter embedding, object fronting, basicword order, and neural imaging technique(PET vs. fMRI) in these working memorystudies.

In general, neuroimaging resultssupport claims originally made in thebehavioral literature that successful lan-guage comprehension involves the storageand retrieval of information in workingmemory (e.g., Carpenter and Just, 1989;Daneman and Carpenter, 1980; Danemanand Merikle, 1996). Only through the useof working memory can the brain processcritical relationships between sensorystimuli distributed over time and space. Inaddition, these results suggest that success-ful relational processing may call for moregeneral, attentional resources. If moreattention must be paid to lower level per-ceptual processes necessary for language

comprehension, less attentional resourcesare available for the working memoryoperations especially critical for the pro-cessing of complex language structures.

Long-Term Memory

Although the processing of relationsbetween items is crucial for successful lan-guage comprehension, at its heart lan-guage involves the processing of adifferent kind of relation—the relationbetween language elements and real-worldknowledge stored in long-term memory(see McKoon and Ratcliff, 1998). Words aresymbols—that is, they are associated withinformation that is not contained in thephysical form of the word. It has been sug-gested that the human ability to remember,transform, and flexibly combine thousandsof symbols is what especially sets us apartfrom other species (e.g., Deacon, 1997).Early in their processing, words are butperceptual objects with visual or acousticproperties that must be processedsufficiently to allow categorization andidentification. Eventually, however, wordsserve as entry points into vast amounts ofinformation stored in long-term memory.This associated information has beenderived from many modalities (e.g., theshape and color of a carrot, its smell, itstaste, its firmness, and smoothness; thecrunching sound made when eating it) andhas come to be associated with the wordform through experience. The nature of the organization of long-term memory, the types of information that are stored,and the extent to which different informa-tion types are accessed under various conditions are all highly controversialissues.

Mirroring the concerns of psycholin-guistics in general, many ERP investiga-tions have been aimed at determining whatkinds of information about words are typi-cally retrieved during reading and listen-ing and the time courses with which thisinformation is retrieved. Moreover, given



its unique ability to track word, sentence,and discourse-level processing with equalresolution, the ERP technique has also beendirected at determining how informationretrieved from the various words in a sen-tence is ultimately combined into a singlemessage. ERP data suggest that the brain isclearly sensitive to some aspects ofmeaning by at least 250–300 msec poststimulus onset. In this time window, thebrain’s response to words (and pronounce-able pseudowords) in all modalities(spoken, printed, signed) (e.g., Holcomband Neville, 1990; Kutas and Hillyard,1980a, b; Kutas et al., 1987), to pictures(Ganis et al., 1996; Nigam et al., 1992) andfaces (Barrett and Rugg, 1989; Bobes et al.,1994; Debruille et al., 1996), and to mean-ingful environmental sounds (Chao et al.,1995; Van Petten and Rheinfelder, 1995)contains a negativity with a posterior,slightly right hemisphere distribution atthe scalp. Potentials at the same latencyand sensitive to these same semantic vari-ables are observed in the fusiform gyrus ofpatients with electrodes implanted forlocalizing seizure activity (e.g., McCarthyet al., 1995; Nobre and McCarthy, 1995);note that the polarity of a recorded poten-tial depends on the location of the activeelectrode and reference, such that theintracranially recorded “N400s” are notalways negative. This so-called N400 com-ponent was mentioned previously in thediscussion of frequency and repetitioneffects, because its amplitude varies withboth. In children and intact adults, theN400 seems to be the normal response tostimuli that are potentially meaningful.Some have suggested that the N400 reflectssome kind of search through long-term,semantic memory; indeed N400 amplitudedoes vary with factors that also influencememory, such as the number of items to beremembered (Stuss et al., 1986) and thelength of the delay between presentationsof an item (e.g., Chao et al., 1995). Its ampli-tude is diminished and its latency is pro-longed with normal aging, and even more

so with various dementias (e.g., Iragui et al., 1993, 1996).

We have suggested that the N400indexes some aspect of meaning becauseits amplitude is modulated by semanticaspects of a preceding context, be it asingle word, a sentence, or a multisentencediscourse. For instance, the amplitude ofthe N400 to a word in a list is reduced ifthat word is preceded by one with asimilar meaning (e.g., N400 amplitude to“dog” is reduced when preceded by “cat”compared to “cup”) (Brown and Hagoort,1993; Holcomb and Neville, 1990; VanPetten et al., 1995). Brain activity in thesame time region is also sensitive tophonological and orthographic relationsbetween words (Barrett and Rugg, 1990;Polich et al., 1983; Praamstra et al., 1994;Rugg, 1984a; Rugg and Barrett, 1987).Similarly, the amplitude of the N400 to aword in a sentence is reduced to the extentthat the word is compatible with theongoing semantic context. An anomaly(e.g., “He takes his coffee with cream anddog”) elicits the largest N400 response.Nonanomalous but less probable words(e.g., “He takes his coffee with cream andhoney”) generate less N400 activity thando anomalies but N400s of greater ampli-tude than more probable completions (e.g.,“He takes his coffee with cream andsugar”) (Kutas and Donchin, 1980; Kutasand Hillyard, 1980ab, 1984; Kutas et al.,1984). Discourse-level factors may alsoaffect the magnitude of the N400 response.As single sentences, both “the mousequickly went into its hole” and “the mouseslowly went into its hole” are congruous.However, in a larger discourse context(e.g., “Prowling under the kitchen table,the cat surprised a mouse eating crumbs.The mouse … “), the two adverbs (quicklyand slowly) are no longer equallyexpected; in fact, the N400 response to“slowly” in this type of context is largerthan the response to “quickly” (vanBerkum et al., 1999). Thus, at least around400 msec, lexical, sentential, and discourse



factors seem to converge to influence lan-guage comprehension and do so in a fairlysimilar manner. When both lexical and sen-tential factors are present, they seem toinfluence the N400 amplitude indepen-dently (Kutas, 1993; Van Petten, 1993, 1995;see also Fischler et al., 1985, for a similarconclusion). The relation of the N400 tosemantic integrative processes is furthersupported by the observation that itsamplitude is greatly attenuated and itslatency is delayed in aphasic patients withmoderate to severe comprehension prob-lems, but not in patients with equivalent

amounts of damage to the right hemi-sphere (Swaab et al., 1997).

The N400 is thus sensitive to the rela-tionship between a word and its immedi-ate sentential context and to that between aword and other words in the lexicon (seeFig. 6). Insofar as N400 indexes someaspect of search through memory, it seemsthen that the brain uses all the informationit can as soon as it can to constrain itssearch. How does context serve to guidethis search? We can think of informationabout word meaning as existing in a kindof space, structured by experience. Thenature of this structure is often inferred



FIGURE 6 The N400 responses to various experimental manipulations, all shown here at a representativeright posterior site. Incongruous words elicit large N400 amplitudes relative to congruous words, whether theseitems are in midsentence position (top left of figure) or in sentence final position. As shown in the center, thiseffect can be observed in all modalities, including written words, spoken words, and line drawings (here, allusing the same experimental materials). As seen at bottom left, the N400 is similarly sensitive to varying degreesand types of semantic relations in more minimal contexts, including highly constrained antonyms (e.g., “Theopposite of black … white”; solid line), and category membership relations (e.g., “A type of bird …) with high(e.g., “robin”; dashed line) or low (e.g., “turkey”; dot–dash line) typicality, as compared with unrelated items(dotted line). N400 amplitudes also vary with factors such as repetition, word frequency, and word position (seeright side of figure).

from the outcome of various categoriza-tion or sentence verification tasks (e.g.,Kounios, 1996; Kounios and Holcomb,1992; Kounios et al., 1994). Context (as wellas the other factors known to influenceN400 amplitude such as frequency or repe-tition) may serve to direct processing intodifferent parts of this space—usually partsthat render subsequent searches easier bybringing the processor into a state “closer”to the meaning of upcoming words. Wehave examined this hypothesis in a studyin which participants were asked to readthe following types of pairs of sentences:

Ann wanted to treat her foreignguests to an all-American dessert.She went out in the back yard andpicked some apples.

These sentence pairs were terminated with either the contextually expected item(“apples”), a contextually unexpected itemthat came from the same semantic categoryas the expected item (e.g., “oranges,”another fruit), or an unexpected item froma different semantic category (e.g.,“carrots”). Both types of unexpectedendings elicited an N400 relative to con-gruent endings. However, even thoughboth kinds of unexpected endings wereequally inappropriate and implausible inthe context, the unexpected item from theexpected category elicited a smaller N400than did the one from a different category.Moreover, the N400 reduction to such“within-category” violations was largest inhighly constraining sentence contexts,where these violations were most implau-sible. The N400 does more than simplyindex the semantic fit between an item andits local context, therefore. Rather, this datapattern shows that the organization ofsensory, motor, and higher order featuresin the brain built up of years of experiencewith the world (the fact that apples andoranges share more features in commonthan apples and carrots) has an inevitableimpact on the neural processes (here seenin the N400 response) by which brains

make sense of language in real time(Federmeier and Kutas, 1999b; Kutas andFedermeier, 2001).

An integral part of language compre-hension, therefore, involves retrievingworld knowledge associated with wordsand groups of words from long-termsemantic memory. ERP data in conjunctionwith neuropsychological data and datafrom other neuroimaging techniquessuggest that this meaning-related informa-tion resides in featural mosaics distributedacross multiple brain areas, includinghigher order perceptual and motor-pro-cessing areas. fMRI studies (in accord withneuropsychological findings), for example,have shown that different brain areasbecome active in response to words(and/or pictures) representing differentkinds of information (e.g., actions vs.colors) (Martin et al., 1995) and that pat-terns of activation within general brainareas, such as the ventral temporal cortex,vary as a function of semantic category aswell (e.g., tools, animals) (see reviews byHumphreys et al., 1999; Martin and Chao,2001). N400 data also reveal that the natureof the meaning information retrieved fromlong-term memory differs—even withinthe same linguistic context—for differenttypes of stimuli (e.g., a picture vs. wordrepresenting the same concept) (Feder-meier and Kutas, 2001) and as a functionof which cerebral hemisphere preferen-tially (based on location of presentation)processes that information (Federmeierand Kutas, 1999a, 2002). The ERP datafurther suggest that meaning emergesfrom these distributed systems by virtue oftemporally coincident and functionallysimilar activity within a number of brainareas [see also intracranial recordingstudies by Halgren et al. (1994a,b),McCarthy et al. (1995), and Nobre andMcCarthy (1995)].

Language meaning thus emerges froman interaction between structure in thebrain, built out of experience, and struc-ture in the language stream. Context



serves to shape not only the ease withwhich information can be found but alsothe nature of the information that isretrieved. Conceptual information alsoserves to shape language processing byproviding a structure (“frame” or“schema”) within which details beyond thelevel of individual words can be fit andrelated to one another. These “schemas”can be thought of as the brain’s generalexpectations about the nature of informa-tion that will be retrieved and the order inwhich it will come. These schemas mightwell influence the extent to which variousaspects of information are attended, howthey are stored in working memory, andthe ease with which they are compre-hended. A study by Münte et al. (1998b)examined how people’s schemas abouttime (built of daily experience) may affectthe brain’s processing of sentences andthus interact with working memory vari-ables. People read sentences describing thetemporal order of two events; the sentencesdiffered only in whether their initial wordwas “before” or “after” (e.g., “Before/afterthe students took the exam the teachercalled the parents”). Although these sen-tence types are otherwise identical inlexical content and syntactic structure, theydiffer in the extent to which they fit withour schema of time as a dimension movingfrom past to future. In “after” sentences,the two events are mentioned in accor-dance with this conception—the tempo-rally earlier event coming first and thetemporally later event coming second. Bycontrast, “before” sentences reverse thisnatural order. Münte et al. found that start-ing within 300 msec of the initial word (thetemporal term), “after” sentences showeda larger sustained positivity than did“before” sentences; this positivity wassimilar to that described for the relative-clause (object vs subject) contrast. This dif-ference was, again, most pronounced forindividuals with high working memoryspans. The data suggest that our knowl-edge of the world (in this case, about time)

has an immediate, lasting effect on process-ing, and that this impact is modulated byworking memory capacity and/or avail-ability. Words such as “before” and “after”serve as cues about the relationshipbetween elements to come. These relations,in turn, are easier to process if theyconform to general conceptual patternsderived from experience.

CONCLUSIONS

Comprehending language thus entails anumber of different kinds of brainprocesses, including perceptual analysis,attention allocation, retrieval of informa-tion from long-term memory, storage ofinformation in working memory, and com-parisons between/transformations ofinformation contained in working memory.These processes take place at multiplelevels for different types of information(orthographic/phonological word forminformation, morphological/syntacticinformation, conceptual/semantic infor-mation) and unfold with different timecourses; they are thus reflected in differentelectrophysiological processes with differ-ent time-courses, mediated by differentbrain areas.

Understanding language processing,therefore, demands that we apprehendhow the multiple subprocesses involvedinteract over time and space. This, in turn,compels us to appreciate how the brain’sprocessing of language interacts with moregeneral processing demands. For example,both N400 and P600 amplitudes areresponsive to attentional manipulations.The N400, for instance, is not observedwhen the priming context is masked(Brown and Hagoort, 1993), and N400effects in word pair tasks are larger whenthe prime target interval is short and theproportion of related word pairs is high(Chwilla et al., 1995; Holcomb, 1988).Similarly, the P600 to verb inflection errorsis greatly attenuated if not absent when



people are asked to scan sentences merelyto determine whether a word in a sentenceis printed in upper case (Gunter andFriederici, 1999). Orthographic, phono-logical, morphological, syntactic, and prag-matic priming and context ERP effectsseem to overlap temporally between 200and 400 msec. Various and sundrymemory-related and some attention-related ERP effects are observed in thisvery same interval. Moreover, the transientERPs elicited during the analysis of avisual stimulus as a word are super-imposed on the slower potentials that seemto be elicited during the processing of sen-tences and during various tasks requiringthat information be retrieved from longerterm memory. Indeed, the languagespecificity of any of these processesremains unknown to date.

What we do know is that language pro-cessing is a complex skill engaging thewhole brain. The goal of electrophysiologi-cal investigations of language, as well asthe goal of research exploring languageprocessing with other tools, is to fashion anunderstanding of how the variousprocesses involved in language compre-hension and production are coordinated toyield the message-level apprehension weattain from reading or listening to speech.Linguists, psycholinguists, and neurolin-guists alike strive to understand how thebrain “sees” language—because, in turn,language is such an important facet of howhumans “see” their world.

ReferencesAllison, T., Wood, C. C., and McCarthy, G. (1986).

The central nervous system. In “Psycho-physiology: Systems, Processes, and Applications”(M. G. Coles, E. Donchin, and S. W. Porges, eds.),pp. 5–25. Guilford Press, New York.

Allison, T., McCarthy, G., Nobre, A., Puce, A., andBelger, A. (1994). Human extrastriate visual cortexand the perception of faces, words, numbers, andcolors. Cereb. Cortex 4(5), 544–554.

Barrett, S. E., and Rugg, M. D. (1989). Event-relatedpotentials and the semantic matching of faces.Neuropsychologia 27(7), 913–922.

Barrett, S. E., and Rugg, M. D. (1990). Event-relatedpotentials and the phonological matching ofpicture names. Brain Lang. 38(3), 424–437.

Besson, M., and Kutas, M. (1993). The many facets ofrepetition: A cued-recall and event-related po-tential analysis of repeating words in same versusdifferent sentence contexts. J. Exp. Psycho.:Learning Memory Cogn. 19(5), 1115–1133.

Besson, M., Kutas, M., and Van Petten, C. (1992). Anevent-related potential (ERP) analysis of semanticcongruity and repetition effects in sentences. J. Cogn. Neurosci. 4(2), 132–149.

Bobes, M. A., Valdes-Sosa, M., and Olivares, E. (1994).An ERP study of expectancy violation in face per-ception. Brain Cogn. 26(1), 1–22.

Brown, C., and Hagoort, P. (1993). The processingnature of the N400: Evidence from maskedpriming. J. Cogn. Neurosci. 5(1), 34–44.

Brown, C. M., Hagoort, P., and ter Keurs, M. (1999).Electrophysiological signatures of visual lexicalprocessing: Open- and closed-class words. J. Cogn.Neurosci. 11(3), 261–281.

Canseco-Gonzalez, E., Love, T., Ahrens, K., Walenski,M., Swinney, D., and Neville, H. (1997). Processingof grammatical information in Jabberwockysentences: An ERP study. Paper presented at theCognitive Neuroscience Society, Fourth AnnualMeeting, Boston, Massachusetts.

Caplan, D., Alpert, N., and Waters, G. (1998). Effectsof syntactic structure and propositional num-ber on patterns of regional cerebral blood flow. J. Cogn. Neurosci. 10(4), 541–552.

Caplan, D., Alpert, N., and Waters, G. (1999). PET studies of syntactic processing with audi-tory sentence presentation. Neuroimage 9(3), 343–351.

Caplan, D., Alpert, N., Waters, G., and Olivieri, A.(2000). Activation of Broca’s area by syntactic processing under conditions of concurrent articulation. Human Brain Mapping 9(2), 65–71.

Carpenter, P. A., and Just, M. A. (1989). The role ofworking memory in language comprehension. In“Complex Information Processing: The Impact ofHerbert A. Simon” D. Klahr and K. Kotovsky,eds.), pp. 31–68. Lawrence Erlbaum Assoc.,Hillsdale, New Jersey.

Chao, L. L., Nielsen-Bohlman, L., and Knight, R. T.(1995). Auditory event-related potentials dis-sociate early and late memory processes.Electroencephalogr. Clin. Neurophysiol., 96(2),157–168.

Chomsky, N. (1965). “Aspects of the Theory ofSyntax.” MIT Press, Cambridge, Massachusetts.

Chomsky, N. (1980). “Rules and representations. B.Blackwell, London.

Churchland, P. M. (1984). “Matter and Consciousness:A Contemporary Introduction to the Philosophyof Mind.” MIT Press, Cambridge, Massachusetts.

REFERENCES 163


Chwilla, D. J., Brown, C. M., and Hagoort, P. (1995).The N400 as a function of the level of processing.Psychophysiology 32(3), 274–285.

Coulson, S., King, J. W., and Kutas, M. (1998a). ERPsand domain specificity: Beating a straw horse.Lang. Cogn. Processes 13, 653–672.

Coulson, S., King, J. W., and Kutas, M. (1998b). Expectthe unexpected: Event-related brain response tomorphosyntactic violations. Lang. Cogn. Processes13(1), 21–58.

Dale, A. M., and Halgren, E. (2001). Spatiotemporalmapping of brain activity by integration of multi-ple imaging modalities. Curr. Opin. Neurobiol.11(2), 202–208.

Daneman, M., and Carpenter, P. A. (1980). Individualdifferences in working memory and reading. J. Verbal Learn. Verbal Behav. 19(4), 450–466.

Daneman, M., and Merikle, P. M. (1996). Workingmemory and language comprehension: A meta-analysis. Psychonom. Bull. Rev. 3(4), 422–433.

Dapretto, M., and Bookheimer, S. Y. (1999). Form andcontent: Dissociating syntax and semantics in sentence comprehension. Neuron 24(2), 427–432.

Deacon, T. W. (1997). “The Symbolic Species.” W. W. Norton, New York.

Debruille, J. B., Pineda, J., and Renault, B. (1996).N400-like potentials elicited by faces and know-ledge inhibition. Cogn. Brain Res. 4(2), 133–144.

Embick, D., Marantz, A., Miyashita, Y., O’Neil, W.,and Sakai, K. L. (2000). A syntactic specializationfor Broca’s area. Proc. Natl. Acad. Sci. U.S.A. 97(11),6150–6154.

Farah, M. J. (1990). “Visual Agnosia: Disorders ofObject Recognition and What They Tell Us aboutNormal Vision.” MIT Press, Cambridge.

Federmeier, K. D., and Kutas, M. (1999a). Right wordsand left words: Electrophysiological evidence forhemispheric differences in meaning processing.Cogn. Brain Res. 8(3), 373–392.

Federmeier, K. D., and Kutas, M. (1999b). A rose byany other name: Long-term memory structure and sentence processing. J. Mem. Lang. 41(4),469–495.

Federmeier, K. D., and Kutas, M. (2001). Meaning andmodality: Influences of context, semantic memoryorganization, and perceptual predictability onpicture processing. J. Exp. Psychol.: Learning Mem.Cogn. 27(1), 202–224.

Federmeier, K. D., and Kutas, M. (2002). Picture thedifference: Electrophysiological investigations ofpicture processing in the cerebral hemispheres.Neuropsychologia 40, 730–747.

Fischler, I., Childers, D. G., Achariyapaopan, T., andPerry, N. W. (1985). Brain potentials during sen-tence verification: Automatic aspects of compre-hension. Biol. Psychol. 21(2), 83–105.

Fodor, J. A. (1981). The mind–body problem. Sci. Am.244(1), 114–123.

Fodor, J. A. (1983). “The Modularity of Mind: AnEssay on Faculty Psychology.” MIT Press,Cambridge, Massachusetts.

Francis, W. N., and Kucera, H. (1982). “FrequencyAnalysis of English Usage.” Houghton MifflinCompany, Boston.

Friederici, A. D., Pfeifer, E., and Hahne, A. (1993).Event-related brain potentials during naturalspeech processing: Effects of semantic, morpholog-ical and syntactic violations. Cogn. Brain Res. 1(3),183–192.

Friederici, A. D., Hahne, A., and Mecklinger, A.(1996). Temporal structure of syntactic parsing:Early and late event-related brain potential effects.J. Exp. Psychol.: Learning Memory Cogn. 22(5),1219–1248.

Friederici, A. D., Steinhauer, K., Mecklinger, A., andMeyer, M. (1998). Working memory constraints onsyntactic ambiguity resolution as revealed by electrical brain responses. Biol. Psychol. 47(3),193–221.

Ganis, G., Kutas, M., and Sereno, M. I. (1996). Thesearch for ‘common sense’: An electrophysiologi-cal study of the comprehension of words and pic-tures in reading. J. Cogn. Neurosci. 8, 89–106.

Goldman-Rakic, P. S. (1990). Cortical localization ofworking memory. In “Brain Organization andMemory: Cells, Systems, and Circuits” J. L.McGaugh and N. M. Weinberger et al., (eds.), pp.285–298. Oxford University Press, New York.

Gunter, T. C., and Friederici, A. D. (1999). Concerningthe automaticity of syntactic processing.Psychophysiology 36, 126–137.

Gunter, T. C., Jackson, J. L., and Mulder, G. (1995).Language, memory, and aging: An electrophysio-logical exploration of the N400 during reading ofmemory-demanding sentences. Psychophysiology32(3), 215–229.

Gunter, T. C., Stowe, L. A., and Mulder, G. (1997).When syntax meets semantics. Psychophysiology34(6), 660–676.

Gunter, T. C., Friederici, A. D., and Hahne, A. (1999).Brain responses during sentence reading: Visualinput affects central processes. Neurorep.: RapidCommun. Neurosci. Res. 10(15), 3175–3178.

Hagoort, P., Brown, C., and Groothusen, J. (1993). Thesyntactic positive shift (SPS) as an ERP measure ofsyntactic processing. Special Issue: Event-relatedbrain potentials in the study of language. Lang.Cogn. Processes 8(4), 439–483.

Hahne, A., and Friederici, A. D. (1999). Electro-physiological evidence for two steps in syntacticanalysis: Early automatic and late controlledprocesses. J. Cogn. Neurosci. 11(2), 194–205.

Hahne, A., and Jescheniak, J. D. (2001). What’s left ifthe Jabberwock gets the semantics? An ERP inves-tigation into semantic and syntactic processesduring auditory sentence comprehension. Cogn.Brain Res. 11(2), 199–212.



Halgren, E., Baudena, P., Heit, G., Clarke, J. M.,Marinkovic, K., Chauvel, P., and Clarke, M.(1994a). Spatio-temporal stages in face and wordprocessing. 2. Depth-recorded potentials in thehuman frontal and Rolandic cortices [publishederratum appears in J. Physiol. Paris 88(2) (1994),following p. 151]. J. Physiol. 88(1), 51–80.

Halgren, E., Baudena, P., Heit, G., Clarke, J. M.,Marinkovic, K., and Clarke, M. (1994b). Spatio-temporal stages in face and word processing. I.Depth-recorded potentials in the human occipital,temporal and parietal lobes [corrected] [publishederratum appears in J. Physiol. Paris 88(2) (1994),following p. 151]. J. Physiol. 88(1), 1–50.

Hockett, C. F. (1963). A note on structure. In “Readings in Linguistics” (M. Joos, ed.), pp. 279–280. American Council of LearnedSocieties, New York.

Holcomb, P. J. (1988). Automatic and attentional pro-cessing: An event-related brain potential analysisof semantic priming. Brain Lang. 35(1), 66–85.

Holcomb, P. J., and Neville, H. J. (1990). Auditory andvisual semantic priming in lexical decision: A com-parison using event-related brain potentials. Lang.Cogn. Processes 5(4), 281–312.

Humphreys, G. W., Price, C. J., and Riddoch, M. J.(1999). From objects to names: A cognitive neuro-science approach. Psychol. Res./Psychol. Forsch.62(2-3), 118–130.

Inui, T., Otsu, Y., Tanaka, S., Okada, T., Nishizawa, S.,and Konishi, J. (1998). A functional MRI analysisof comprehension processes of Japanese sentences.Neurorep.: Int. J. Rapid Commun. Res. Neurosci. 9(14),3325–3328.

Iragui, V. J., Kutas, M., Mitchiner, M. R., and Hillyard,S. A. (1993). Effects of aging on event-related brainpotentials and reaction times in an auditoryoddball task. Psychophysiology 30(1), 10–22.

Iragui, V., Kutas, M., and Salmon, D. P. (1996). Event-related potentials during semantic categorizationin normal aging and senile dementia of theAlzheimer’s type. Electroencephalogr. Clin.Neurophysiol. 100, 392–406.

Johnston, V. S., and Chesney, G. L. (1974). Electro-physiological correlates of meaning. Science186(4167), 944–946.

Just, M. A., Carpenter, P. A., Keller, T. A., Eddy, W. F.,and Thulborn, K. R. (1996). Brain activationmodulated by sentence comprehension. Science274(5284), 114–116.

Kaan, E., Harris, A., Gibson, E., and Holcomb, P. (2000).The P600 as an index of syntactic integrationdifficulty. Lang. Cogn. Processes 15(2), 159–201.

Kang, A. M., Constable, R. T., Gore, J. C., and Avrutin,S. (1999). An event-related fMRI study of implicitphrase-level syntactic and semantic processing.Neuroimage 10(5), 555–561.

Kazmerski, V. A., and Friedman, D. (1997). Old/newdifferences in direct and indirect memory tests

using pictures and words in within- and cross-form conditions: Event-related potential andbehavioral measures. Cogn. Brain Res. 5(4),255–272.

King, J. W., and Kutas, M. (1995). Who did what andwhen? Using word- and clause-level ERPs tomonitor working memory usage in reading. J. Cogn. Neurosci. 7(3), 376–395.

King, J. W., and Kutas, M. (1998a). He really is anurse: ERPs and anaphoric coreference. Psycho-physiology 35(Suppl. 1), S47.

King, J. W., and Kutas, M. (1998b). Neural plasticity inthe dynamics of human visual word recognition.Neurosci. Lett. 244(2), 61–64.

Kluender, R., and Kutas, M. (1993). Bridging the gap:Evidence from ERPs on the processing ofunbounded dependencies. J. Cogn. Neurosci. 5(2),196–214.

Kluender, R., Münte, T., Cowles, H. W., Szentkuti, A.,Walenski, M., and Wieringa, B. (1998). Brainpotentials to English and German questions. J. Cogn. Neurosci. (Suppl.) 1, 24.

Kounios, J. (1996). On the continuity of thought and the representation of knowledge: Electro-physiological and behavioral time-course measures reveal levels of structure in semantic memory. Psychonom. Bull. Rev. 3(3),265–286.

Kounios, J., and Holcomb, P. J. (1992). Structure andprocess in semantic memory: Evidence fromevent-related brain potentials and reaction times.J. Exp. Psychol. Gen. 121(4), 459–479.

Kounios, J., Montgomery, E. C., and Smith, R. W.(1994). Semantic memory and the granularity ofsemantic relations: Evidence from speed-accuracydecomposition. Memory Cogn. 22(6), 729–741.

Kutas, M. (1993). In the company of other words:Electrophysiological evidence for single-word andsentence context effects. Special Issue: Event-related brain potentials in the study of language.Lang. Cogn. Processes 8(4), 533–572.

Kutas, M., and Dale, A. (1997). Electrical and mag-netic readings of mental functions. In “CognitiveNeuroscience” (M. D. Rugg, ed.), pp. 197–242.Psychology Press, Hove, East Sussex.

Kutas, M., and Donchin, E. (1980). Preparation torespond as manifested by movement-related brainpotentials. Brain Res. 202(1), 95–115.

Kutas, M., and Federmeier, K. D. (2001).Electrophysiology reveals semantic memory usein language comprehension. Trends Cogn. Sci.4(12), 463–470.

Kutas, M., and Hillyard, S. A. (1980a). Event-relatedbrain potentials to semantically inappropriate andsurprisingly large words. Biol. Psychol. 11(2),99–116.

Kutas, M., and Hillyard, S. A. (1980b). Reading sense-less sentences: Brain potentials reflect semanticincongruity. Science 207(4427), 203–205.

REFERENCES 165


Kutas, M., and Hillyard, S. A. (1984). Brain po-tentials during reading reflect word expectancyand semantic association. Nature 307(5947), 161–163.

Kutas, M., and King, J. W. (1996). The potentials forbasic sentence processing: Differentiating inte-grative processes. In “Attention and Performance16: Information Integration in Perception andCommunication” (T. Inui and J. L. McClelland,eds.), pp. 501–546. MIT Press, Cambridge, Massa-chusetts.

Kutas, M., and Van Petten, C. K. (1994). Psycho-linguistics electrified: Event-related brain potentialinvestigations. In “Handbook of Psycho-linguistics” (M. A. Gernsbacher, ed.), pp. 83–143.Academic Press, San Diego.

Kutas, M., Lindamood, T. E., and Hillyard, S. A.(1984). Word expectancy and event-related brain potentials during sentence processing. In“Preparatory States and Processes” (S. Kornblumand J. Requin, eds.), pp. 217–237. LawrenceErlbaum Assoc., Hillsdale, New Jersey.

Kutas, M., Neville, H. J., and Holcomb, P. J. (1987). Apreliminary comparison of the N400 response tosemantic anomalies during reading, listening, andsigning. Electroencephalogr. Clin. Neurophysiol.(Suppl.) 39, 325–330.

Kutas, M., Federmeier, K. D., and Sereno, M. I. (1999).Current approaches to mapping language in electromagnetic space. In “The Neurocognition of Language” (C. M. Brown and P. Hagoort, eds.), pp. 359–392. Oxford University Press,Oxford.

Maclay, H., and Osgood, C. E. (1959). Hesitation phe-nomena in spontaneous English speech. Word 15,19–44.

Martin, A., and Chao, L. L. (2001). Semantic memoryand the brain: Structure and processes. Curr. Opin.Neurobiol. 11(2), 194–201.

Martin, A., Haxby, J. V., Lalonde, F. M., and Wiggs, C. L. (1995). Discrete cortical regions associated with knowledge of color and knowledge of action. Science 270(5233), 102–105.

McCarthy, G., Nobre, A. C., Bentin, S., and Spencer, D.D. (1995). Language-related field potentials in theanterior-medial temporal lobe: I. Intracranial dis-tribution and neural generators. J. Neurosci. 15(2),1080–1089.

McKoon, G., and Ratcliff, R. (1998). Memory-basedlanguage processing: Psycholinguistic research inthe 1990s. Annu. Rev. Psychol. 49, 25–42.

Mecklinger, A., Schriefers, H., Steinhauer, K., andFriederici, A. D. (1995). Processing relative clausesvarying on syntactic and semantic dimensions: Ananalysis with event-related potentials. MemoryCogn. 23(4), 477–494.

Mills, D. L., Coffey-Corina, S., and Neville, H. J.(1997). Language comprehension and cerebral

specialization from 13 to 20 months. Dev.Neuropsychol. 13(3), 397–445.

Muller, H. M., King, J. W., and Kutas, M. (1997).Event-related potentials elicited by spoken relativeclauses. Cogn. Brain Res. 5(3), 193–203.

Münte, T. F., Heinze, H.-J., and Mangun, G. R. (1993).Dissociation of brain activity related to syntacticand semantic aspects of language. J. Cogn.Neurosci. 5(3), 335–344.

Münte, T. F., Matzke, M., and Johannes, S. (1997a).Brain activity associated with syntactic incongru-encies in words and pseudo-words. J. Cogn.Neurosci. 9(3), 318–329.

Münte, T. F., Szentkuti, A., Wieringa, B. M., Matzke,M., and Johannes, S. (1997b). Human brain poten-tials to reading syntactic errors in sentences of dif-ferent complexity. Neurosci. Lett. 235(3), 105–108.

Münte, T. F., Heinze, H. J., Matzke, M., Wieringa, B.M., and Johannes, S. (1998a). Brain potentials andsyntactic violations revisited: No evidence forspecificity of the syntactic positive shift.Neuropsychologia 36(3), 217–226.

Münte, T. F., Schiltz, K., and Kutas, M. (1998b). Whentemporal terms belie conceptual order: An electro-physiological analysis. Nature 395, 71–73.

Münte, T. F., Wieringa, B. M., Weyerts, H., Szentkuti,A., Matzke, M., and Johannes, S. (2001). Dif-ferences in brain potentials to open and closedclass words: Class and frequency effects.Neuropsychologia 39(1), 91–102.

Nagy, M. E., and Rugg, M. D. (1989). Modulation ofevent-related potentials by word repetition: Theeffects of inter-item lag. Psychophysiology 26(4),431–436.

Neville, H. J., Coffey, S. A., Lawson, D. S., Fischer, A.,Emmorey, K., and Bellugi, U. (1997). Neuralsystems mediating American Sign Language:Effects of sensory experience and age of acquisi-tion. Brain Lang. 57(3), 285–308.

Neville, H. J., Nicol, J. L., Barss, A., Forster, K. I., andGarrett, M. F. (1991). Syntactically based sentenceprocessing classes: Evidence from event-related brain potentials. J. Cogn. Neurosci. 3(2),151–165.

Ni, W., Constable, R. T., Mencl, W. E., Pugh, K. R.,Fulbright, R. K., Shaywitz, S. E., Shaywitz, B. A.,Gore, J. C., and Shankweiler, D. (2000). An event-related neuroimaging study distinguishing formand content in sentence processing. J. Cogn.Neurosci. 12(1), 120–133.

Nigam, A., Hoffman, J. E., and Simons, R. F. (1992).N400 to semantically anomalous pictures andwords. J. Cogn. Neurosci. 4(1), 15–22.

Nobre, A. C., and McCarthy, G. (1995). Language-related field potentials in the anterior-medial tem-poral lobe: II. Effects of word type and semanticpriming. J. Neurosci. 15(2), 1090–1098.

Novick, B., Lovrich, D., and Vaughan, H. G. (1985).Event-related potentials associated with the dis-



crimination of acoustic and semantic aspects ofspeech. Neuropsychologia 23(1), 87–101.

Nunez, P. L., and Katznelson, R. D. (1981). “ElectricFields of the Brain: The Neurophysics of EEG.”Oxford University Press, New York.

Osterhout, L. (1994). Event-related brain potentials as tools for comprehending language compre-hension. In “Perspectives on sentence processing” (J. Charles Clifton, L. Frazier and K. Rayner, eds.), .pp. 15–44. Lawrence Erlbaum Assoc., Hillsdale,New Jersey.

Osterhout, L., and Hagoort, P. (1999). A superficialresemblance doesn’t necessarily mean you’re partof the family: Counterarguments to Coulson, King,and Kutas (1998) in the P600/SPS-P300 debate.Lang. Cogn. Processes 14, 1–14.

Osterhout, L., and Holcomb, P. J. (1992). Event-relatedbrain potentials elicited by syntactic anomaly. J. Memory Lang. 31(6), 785–806.

Osterhout, L., and Holcomb, P. J. (1995). Event relatedpotentials and language comprehension. In“Electrophysiology of Mind: Event-Related BrainPotentials and Cognition” (M. D. Rugg and M. G. H. Coles, eds.), Vol. 25, pp. 171–215. OxfordUniversity Press, Oxford.

Osterhout, L., McKinnon, R., Bersick, M., and Corey,V. (1996). On the language specificity of the brainresponse to syntactic anomalies: Is the syntactic J. Cogn. Neurosci. 8(6), 507–526.

Osterhout, L., Bersick, M., and McKinnon, R. (1997a).Brain potentials elicited by words: Word lengthand frequency predict the latency of an earlynegativity. Biol. Psychol. 46(2), 143–168.

Osterhout, L., Bersick, M., and McLaughlin, J. (1997b).Brain potentials reflect violations of gender stereo-types. Memory Cogn. 25(3), 273–285.

Polich, J., McCarthy, G., Wang, W. S., and Donchin, E.(1983). When words collide: Orthographic andphonological interference during word processing.Biol. Psychol. 16(3-4), 155–180.

Praamstra, P., Meyer, A. S., and Levelt, W. J. M. (1994).Neurophysiological manifestations of phono-logical processing: Latency variations of a negativeERP component time-locked to phonologicalmismatch. J. Cogn. Neurosci. 6(3), 204–219.

Puce, A., Allison, T., Asgari, M., Gore, J. C., andMcCarthy, G. (1996). Differential sensitivity ofhuman visual cortex to faces, letterstrings, and tex-tures: A functional magnetic resonance imagingstudy. J. Neurosci. 16(16), 5205–5215.

Roesler, F., Pechmann, T., Streb, J., Roeder, B., andHennighausen, E. (1998). Parsing of sentences in alanguage with varying word order: Word-by-wordvariations of processing demands are revealed byevent-related brain potentials. J. Memory Lang.38(2), 150–176.

Rugg, M. D. (1984a). Event-related potentials and thephonological processing of words and nonwords.Neuropsychologia 22(4), 435–443.

Rugg, M. D. (1984b). Event-related potentials inphonological matching tasks. Brain Lang. 23(2),225–240.

Rugg, M. D. (1985). The effects of semantic primingand word repetition on event-related potentials.Psychophysiology 22(6), 642–647.

Rugg, M. D., and Barrett, S. E. (1987). Event-relatedpotentials and the interaction between ortho-graphic and phonological information in a rhyme-judgment task. Brain Lang. 32(2), 336–361.

Rugg, M. D., Mark, R. E., Gilchrist, J., and Roberts, R.C. (1997). ERP repetition effects in indirect anddirect tasks: Effects of age and interitem lag.Psychophysiology 34(5), 572–586.

Schendan, H. E., Ganis, G., and Kutas, M. (1998).Neurophysiological evidence for visual perceptualcategorization of words and faces within 150 ms.Psychophysiology 35(3), 240–251.

Shao, J., and Neville, H. (1996). ERPs elicited by seman-tic anomalies: Beyond the N400. Paper presented atthe Cognitive Neuroscience Society, Third AnnualMeeting, San Francisco, California.

Stromswold, K., Caplan, D., Alpert, N., and Rauch, S.(1996). Localization of syntactic comprehension bypositron emission tomography. Brain Lang. 52(3),452–473.

Stuss, D. T., Picton, T. W., and Cerri, A. M. (1986).Searching for the names of pictures: An event-related potential study. Psychophysiology 23(2),215–223.

Swaab, T., Brown, C., and Hagoort, P. (1997). Spokensentence comprehension in aphasia: Event-relatedpotential evidence for a lexical integration deficit.J. Cogn. Neurosci. 9(1), 39–66.

van Berkum, J. J. A., Hagoort, P., and Brown, C. M.(1999). Semantic integration in sentences and dis-course: Evidence from the N400. J. Cogn. Neurosci.11(6), 657–671.

Van Petten, C. (1993). A comparison of lexical andsentence-level context effects in event-relatedpotentials. Special Issue: Event-related brainpotentials in the study of language. Lang. Cogn.Processes 8(4), 485–531.

Van Petten, C. (1995). Words and sentences: Event-related brain potential measures. Psychophysiology32(6), 511–525.

Van Petten, C., and Kutas, M. (1991). Influences ofsemantic and syntactic context in open- andclosed-class words. Memory Cogn. 19(1), 95–112.

Van Petten, C., and Rheinfelder, H. (1995). Conceptualrelationships between spoken words and environ-mental sounds: Event-related brain potential mea-sures. Neuropsychologia 33(4), 485–508.

Van Petten, C., Kutas, M., Kluender, R., Mitchiner, M.,and McIsaac, H. (1991). Fractionating the wordrepetition effect with event-related potentials. J. Cogn. Neurosci. 3(2), 131–150.

REFERENCES 167


Van Petten, C., Reiman, E., and Senkfor, A. (1995). PETand ERP measures of semantic and repetitionpriming. Poster, Soc. Psychophysiol. Res.

Vos, S. H., Gunter, T. C., Kolk, H. H. J., and Mulder, G.(2001). Working memory constraints on syntacticprocessing: An electrophysiological investigation.Psychophysiology 38(1), 41–63.

Weber-Fox, C., and Neville, H. J. (1996). Maturationalconstraints on functional specializations for lan-guage processing: ERP and behavioral evidence inbilingual speakers. J. Cogn. Neurosci. 8(3), 231–256.

Weckerly, J., and Kutas, M. (1999). An electro-physiological analysis of animacy effects in the processing of objective relative sentences.Psychophysiology 36(5), 559–570.

Yamashita, H. (2002). Scrambled sentences in Japanese.Linguistic properties and motivations for pro-duction (In press).

Young, M. P., and Rugg, M. D. (1992). Word frequencyand multiple repetition as determinants of themodulation of event-related potentials in a semanticclassification task. Psychophysiology 29(6), 664–676.




C H A P T E R

7

Episodic Memory Encoding andRetrieval: Recent Insights from

Event-Related PotentialsEdward L. Wilding and Helen Sharpe

INTRODUCTION

Episodic memory is memory of person-ally experienced events in which the“rememberer” was involved as a partici-pant, an observer, or both. Successfulepisodic retrieval can—and frequentlydoes—involve memory for the context inwhich an event or events occurred. Theterm recollection is often employed to refer to this kind of episodic retrieval. Theprocesses that are engaged when an epi-sode was experienced (Craik and Lockhart,1972), as well as those that are engagedduring an attempt to retrieve the epi-sode influence conjointly the likelihood of remembering. An additional influentialfactor is the degree of correspondencebetween those processes that are active atencoding and those that are active atretrieval (Tulving and Thomson, 1973;Morris et al., 1977). The focus of thischapter is on episodic memory retrievalprocessing, and is concerned primarilywith insights into the neural and functionalbases of episodic memory encoding andretrieval that have been garnered fromstudies in which event-related potentials(ERPs) have been acquired.

The view that neurally and function-ally distinct processes underpin episodic

memory is supported by the findings instudies of patients with circumscribedbrain damage, wherein the memoryimpairment varies according to the loca-tion as well as to the extent of the lesion(for review, see Gabrieli, 1993). Althoughthis form of evidence is important for iden-tifying neural structures that are necessaryfor the normal operation of episodicmemory, delineation of the functional rolesplayed by such structures is less straight-forward. This observation is especially rel-evant with respect to encoding andretrieval operations, because on the basisof behavioral manipulations alone it isdifficult to determine whether a memoryimpairment results from a processingdeficit that is restricted to encoding or toretrieval, or is in fact due to deficits thatoccur at both stages (Mayes and Roberts,2001).

Measures that permit neural activity tobe recorded during the encoding andretrieval phases of episodic memory tasksare less subject to these concerns, and arethus well suited to questions about theprocesses and brain regions that may beengaged only at encoding or retrieval, or atboth stages. In this context, there are threeprincipal ways in which electrophysiologi-cal measures can inform about the neural(hence cognitive) processes that support

episodic memory operations. They canprovide information about (1) the timecourse of cognitive processes, (2) the extent(or degree) to which such processes areengaged as a result of experimental manip-ulations, and (3) whether qualitatively dis-tinct processes are engaged within and/oracross experimental conditions of interest.With respect to this final observation, itshould be emphasized that inferencesabout the engagement of distinct processesneed not include specification of the neuralstructures that support those processes.The limited spatial resolution of ERPsmeans that inferences about the neuralgenerators responsible for indices ofmemory-related processes acquired viaelectrodes placed on the scalp can be madeonly when supported by converging formsof evidence, a point addressed in greaterdepth elsewhere in this volume (also seeRugg, 1998; Wilding, 2001). In keepingwith this observation, discussion of thelikely neural basis of scalp-recorded neuro-electric activity in this chapter reliesheavily on related findings taken from neu-ropsychological studies, intracranialrecordings, and functional neuroimaging,which is the term used here to refer tostudies in which blood-flow correlates ofneural activity have been obtained usingeither positron emission tomography (PET)or functional magnetic resonance imaging(fMRI).

The bulk of this review is concernedwith ERP studies of retrieval from episodicmemory. The review of effects obtained byrecording ERPs evoked by items atretrieval is preceded by a brief commen-tary on ERP studies of encoding. Theemphasis on ERP studies of episodicretrieval reflects the greater proportion ofresearch that has been devoted to thisaspect of memory retrieval processing—atleast insofar as this can be gauged from thepublished literature. Recent developmentsin ERP studies of memory encodingwarrant some comment, however, becausethey mark an important shift in the field.

This is due in part to the impetus towardunderstanding memory-encoding opera-tions that has been provided by the appli-cation of event-related fMRI to the study ofmemory encoding. The ability to separateand classify the blood-flow correlates ofneural activity on a trial-by-trial basisbecame available in the past decade (Daleand Buckner, 1997; Josephs et al., 1997), andin a short space of time has been appliedwidely to questions concerning the neuraland functional basis of episodic memoryencoding as well as episodic retrieval(Wagner, et al. 1999; Rugg and Henson,2002).

EPISODIC ENCODING

Not everything that we encounter isremembered. Although we know thatmanipulations such as levels of processingand full versus divided attention influencethe likelihood of subsequent remembering(e.g., Richardson-Klavehn and Bjork, 1988),we know less about the time course ofeffective encoding into episodic memory,and about the neural substrates responsi-ble for effective encoding of different kindsof information. Event-related potentialstudies of memory encoding have focusedon modulations of the electrical record thatare referred to as subsequent memory or Dmeffects (Paller et al., 1987). Subsequentmemory effects are differences betweenevent-related potentials evoked by itemspresented during memory-encoding tasks,and separated afterward according towhether the items were remembered orforgotten on a subsequent memory test.The logic underpinning this approach isthat reliable differences between ERPs sep-arated according to this criterion mayreflect processes that are influential in theeffective encoding of information intomemory.

The likelihood that ERP subsequentmemory effects will be observed dependson at least two factors. First, among the set

170 7. ERPs AND EPISODIC MEMORY


of ERPs recorded at encoding there mustbe sufficient variability to support two (ormore) reliably different subsets. Second,performance on the subsequent memorytest must depend on or be correlated withat least some of the processes that areresponsible for that variability. Con-sequently, if a given encoding task resultsin little intertrial variability, there will beno subsequent memory effects, irrespectiveof the form of the following memory test orthe level of memory performance. Further-more, even if there is sufficient variabilityat encoding to support a subsequentmemory effect, such an effect will not beobserved if the memory test can be com-pleted on the basis of processes that do notcontribute to that variability. This charac-terization of ERP subsequent memoryeffects—and the implications that it has fornull results—underlies a number of dis-cussions of the likely functional sig-nificance of the effects (for example, seePaller et al., 1988; Van Petten and Senkfor,1996), although the assumptions are statedexplicitly only rarely.

Excellent and comprehensive reviews ofERP studies of subsequent memory effectsare provided by Johnson (1995), Friedmanand Johnson (2000), and Wagner and col-leagues (1999). Our purpose here is tohighlight a small number of recent findingsthat we believe to be of particular impor-tance. They relate to two key questionsconcerning the nature of encoding of infor-mation into episodic memory: first,whether the neural correlates of successfulretrieval vary qualitatively according to thetype of retrieval task that is completed, andsecond, whether the correlates vary quali-tatively according to the operations towhich items are subjected at the time ofencoding.

Although visual inspection of the ERPsubsequent memory effects obtained instudies of episodic encoding up to the mid-1990s is suggestive of differences in the dis-tribution of the effects according toencoding task and retrieval task, there is

little statistical support for this impression(Johnson, 1995). There is some evidencethat for verbal stimuli the scalp distribu-tion of subsequent memory effects changeswith time (Fernandez et al., 1998), althoughin this study there was no evidence thatthese electrophysiologically separableeffects had distinct functional correlates.Other findings, however, speak to theissues raised by both of the aforemen-tioned questions. For the latter—whethersubsequent memory effects differ qualita-tively according to encoding operations—an important study is due to Sommer andcolleagues (Sommer et al., 1997), whoreported that the subsequent memoryeffect for unfamiliar faces has a scalpdistribution that differs reliably from thedistribution of the effect for words [forprecursors and related findings, seeSommer et al. (1991, 1995)]. In the 1997study by Sommer et al., participants wereexposed to study stimuli comprisingwords (unfamiliar names) and unfamiliarfaces. In separate test blocks they wererepresented with words or faces andrequired to rate the items on a 4-point scalefrom “not at all familiar” to “very famil-iar.” For both stimulus types the scalp dis-tributions of the ERP subsequent memoryeffects were obtained by subtracting thestudy ERPs attracting “not at all familiar”ratings at test from those attracting “veryfamiliar” ratings.

The effect for words was in line withprevious findings, with the effect beinglargest at frontocentral electrode sites, anddisplaying a slight tendency to be larger atright- than at left-hemisphere sites(Johnson, 1995). The effect for faces, bycontrast, was relatively more symmetric,and was largest at central scalp sites. Thisis one of the first studies to demonstratethat the scalp distribution of subsequentmemory effects varies according to stimu-lus type. An important caveat, however,follows from the observation that thenames-versus-faces contrast involvedmanipulations at encoding as well as at

EPISODIC ENCODING 171


retrieval. As a consequence, it is notstraightforward to determine whether theoperations engaged at one or both of thesephases contribute to the different scalp dis-tributions that were observed. It may bethat successful encoding of faces versusnames came about because different brainregions are important for the successfulencoding of these two stimulus types. Analternative interpretation is that the dis-tinct ERP subsequent memory distribu-tions arose because at the time of retrievalparticipants interrogated their memoriesfor different attributes of the memory trace.To cast this in terms of the earlier abstractcharacterization, due to the fact that sepa-rate “face” and “name” blocks wereemployed at test, participants may haveexploited different sources of variation inostensibly the same set of evoked poten-tials that were recorded during the twostudy tasks.

This criticism cannot be leveled at astudy by Otten and Rugg (2001a), whorecorded ERPs during an encoding task inwhich participants were cued on a trial-by-trial basis as to whether they should makean animacy (living/nonliving) or an alpha-betic (order of first and last letter) judg-ment to each study word. In a subsequenttest phase, participants were presentedwith an equal number of old (previouslystudied) words from the alphabetic andanimacy conditions, alongside the samenumber of new words. A modified recog-nition memory procedure was employed inwhich participants were asked to makeold/new judgments to test items and toindicate whether the judgment was madewith high or low confidence. The subse-quent memory effects obtained by sub-tracting the ERPs evoked by subsequentlyforgotten items from those evoked byitems that were remembered confidentlydiffered qualitatively according to theencoding operations to which they weresubjected (see Fig. 1). These findings indi-cate that successful episodic encodingdepends on neural systems and cognitive

operations that vary according to thenature of the processing to which wordsare subjected. Interesting questions forfuture research include whether the sameor different subsequent memory effects areobserved when encoding (but not retrieval)tasks are blocked rather than interleaved,and whether the effects are also influencedby the form of the subsequent retrievaltask.

Finally, on a methodological pointSommer et al. (1997) as well as Otten andRugg (2001a) employed variants on stan-dard recognition tasks in which it was pos-sible to select classes of responses such thatthe ERPs contributing to subsequentmemory effects did not include what couldbe classed as nonconfident or “unsure”responses. In subsequent memory studiesin which only a binary distinction isrequired—as has been the case in mostERP subsequent memory studies thatemploy recognition as the retrieval task—these omitted responses would have beendistributed in some ratio between the ERPsevoked by the remembered and the forgot-ten (or missed) items. The likely outcomeof this is a reduction in the magnitude ofany modulations that distinguish remem-bered from forgotten items. These observa-tions have two implications. First, theymay go some way to explaining the factthat the number of reports failing to revealreliable ERP subsequent memory effectsfor recognition outweighs the number inwhich reliable effects have been obtained(Johnson, 1995; Wagner et al., 1999).Second, they suggest that in future ERPstudies employing variants on recognitionmemory tasks it may be beneficial to incor-porate as part of the procedure a testmanipulation that enables more than abinary separation between rememberedand forgotten items [for related com-ments and findings in the context of event-related fMRI studies, see Otten andRugg, 2001b)].

The second question outlined at the startof this section concerned whether mani-



pulations at test influenced the scalp distri-bution of ERP subsequent memory effects.To our knowledge, there have been nostudies to date in which this has beendemonstrated. One reason for this may bethat the focus in some studies has been onpaired contrasts in which recognition wasone of the retrieval tasks (e.g., Paller et al.,1988). As noted previously, standard recog-nition memory tasks have not yielded sub-sequent memory effects in a number ofstudies, and in those in which they havebeen obtained the effects have not beenparticularly robust (Johnson, 1995).

An issue somewhat related to this point,however, is the question of the brainregions that are responsible for supportingdifferent classes of phenomenal experiencethat may accompany recognition judg-ments. According to dual-process accountsof recognition memory, correct “old” judg-ments can be made on the basis of two dis-sociable processes—recollection andfamiliarity (Mandler, 1980; Jacoby andDallas, 1981). Operationally, the distinctionbetween the two is that although both cansupport the ability to determine whether atest item is old or new, only the former is

EPISODIC ENCODING 173


FIGURE 1 Group-average ERPs elicited by remembered and forgotten words in the study of Otten and Rugg(2001a), separated according to encoding task. The data are shown for three midline sites and four paired left- and right-hemisphere sites. Reprinted from Cognitive Brain Research, 12; L. J. Otten and M. D. Rugg;Electrophysiological correlates of memory encoding are task-dependent, pp. 11–18. Copyright, 2001, withpermission from Elsevier Science.

accompanied by memory for contextualdetails of a prior encounter with the item.One procedure that may capture to somedegree the recollection/familiarity distinc-tion was introduced by Tulving (1985) andhas since been developed and exploredextensively by Gardiner and colleagues(for reviews, see Gardiner and Java, 1993;Gardiner, 2001). In this procedure, partici-pants are asked at test to denote whethertheir “old” judgment on a recognition taskwas accompanied by memory for anyaspects of the context of the previous pre-sentation of the item. If it was, they areasked to assign a “remember” (R) responseto the item, and otherwise to assign a“know” (K) response.

The fact that the probabilities of R and Kresponses can be manipulated indepen-dently is consistent with the view that dis-tinct processes underlie these two classesof response (Gardiner, 2001) [for somecaveats, see Donaldson (1996)]. Further-more, a number of variables that affect R orK responses selectively have also beenfound to affect recollection or familiarityselectively (Gardiner, 1988; Gardiner andJava, 1990; Jacoby and Kelley, 1992;Gardiner and Java, 1993), suggesting that,as long as appropriate transformations ofthe data are performed (Yonelinas andJacoby, 1995), there is a relatively directmapping between the two kinds of subjec-tive reports and the processes proposed bydual-process theorists.

What are the implications of theseaccounts for ERP studies of subsequentmemory? To the extent that dissociablecognitive (hence neural) processes supportR and K responses, then one predictionwould be that the subsequent memoryeffect accompanying R responses shoulddifferent qualitatively from that whichaccompanies K responses. If this was notthe case, then the findings would be con-sistent with the view that K responses areessentially weak R responses. That is, thetwo kinds of phenomenal experience arenot dissociable (see Donaldson, 1996;

Hirshman and Master, 1997; Hirshman andHenzler, 1998). Given this perspective, itmight be regarded as surprising that in atleast two studies there has been no evi-dence for qualitative differences betweenthe ERP subsequent memory effects thataccompanied R and K judgments at test(Smith, 1993; Friedman and Trott, 2000). Inthe earlier of these, Smith (1993) reportedthat the subsequent memory effects for Rand K responses were equivalent, whereasFriedman and Trott (2000) observed thesame pattern for elderly participants butevidence for a reliable subsequent memoryeffect for R responses only for a young par-ticipant group. The reasons for the dispar-ity across studies are not clear, but inneither case was there evidence consistentthe view that the neural activity promotingsubsequent R and K judgments differedqualitatively.

These findings contrast, however, withthose obtained in a study by Mangels andcolleagues (2001). They identified reliablesubsequent memory effects for “remem-ber” and “know” responses, with the latterbeing a subset of the former. Thesefindings indicate that there are encodingprocesses common to items that will subse-quently attract R or K judgments, as wellas additional processes that are correlatedonly with the likelihood of a subsequent Rjudgment. Mangels et al. (2001) went on tosuggest that the subsequent memory effectcommon to R and K responses is an indexof processes that would support subse-quent judgments based on familiarity,whereas the anteriorly distributed modula-tions restricted to predicting R responsesreflect study processing necessary for sub-sequent recollection.

An important first step in substantiatingthese claims will be identifying the reasonsfor the disparities between the findings inthis study and those of Smith (1993) andFriedman and Trott (2000). This is unlikelyto be a straightforward exercise becausethe three studies differed in a number ofways, including the electrode montage that



was used, the manipulations employed atencoding, the precise instructions given attest, and the specific test requirements.These comments notwithstanding, thefindings of Mangels and colleagues areconsistent with those obtained in an fMRIstudy of memory encoding in which theR/K procedure was employed (Henson et al., 1999). As in the electrophysiologicalstudy, the findings of Henson and col-leagues are consistent with the view thatthe brain regions engaged at encoding arenot equivalent for items attracting an R ora K response at test. It will be interesting toknow whether findings permitting thesame conclusion will be obtained whenforced-choice context judgments are usedinstead of the remember/know procedure.

To summarize, recent findings in ERPstudies of subsequent memory have pro-vided data that speak to the two questionsidentified at the outset of this section:namely, do the neural correlates of success-ful episodic encoding vary according tostudy and/or test tasks? The answer to thefirst question is in the affirmative, and thestudies of Sommer et al. (1997) and Ottenand Rugg (2001a) provide some indicationsas to the kinds of paradigms that might beemployed fruitfully in order to determinethe classes of encoding manipulations thatyield distinct subsequent memory effects.The answer to the second question is notclear, but there is some evidence to suggestthat R and K responses are associated withdistinct subsequent memory effects(Mangels et al., 2001).

It is worthwhile considering thefindings of Mangels et al. (2001) alongsidetwo other observations. The first is the sug-gestion that robust ERP subsequentmemory effects for recognition memorymay be obtained if confidence judgments(Otten and Rugg, 2001a) or some variantthereon (Sommer et al., 1997) are employedto identify low-confidence (or midrangefamiliarity) test judgments. The ERPsrecorded at study with respect to thoseitems can then be discarded in order to

permit a cleaner separation betweenremembered and forgotten items. Thesecond observation is that recall is presum-ably less dependent on familiarity than isrecognition. In combination these observa-tions suggest that one direction for futurework is to revisit, in the light of currentknowledge, a question that was prevalentin the ERP subsequent memory literaturemore than a decade ago—namely, whetherthe brain regions that support subsequentrecall are isomorphic with those thatsupport recognition memory (Paller et al.,1988; Johnson, 1995).

EPISODIC RETRIEVAL

Event-related potential studies ofretrieval from episodic memory havegained prominence largely over the past20–25 years. Prior to this, the primaryfocus of ERP memory studies was on theSternberg memory-scanning paradigm, inwhich participants were asked to remem-ber a small set of items (Sternberg, 1966;for a review see Kutas, 1988). Much of thecontemporary focus in ERP studies ofretrieval from long-term memory has beenon differences between the ERPs that areevoked by old (previously studied)items—most commonly words—and anappropriate base line. In recognitionmemory tasks and variants the commonbase line is the electrical activity evoked bycorrectly identified new (unstudied) items,and the differences between these classesof ERPs are hereafter referred to as ERPold/new effects. The logic behind this con-trast is straightforward: reliable differencesbetween the ERPs evoked by these twoclasses of test item are candidate electro-physiological indices of processes thatreflect or are contingent on successfulmemory retrieval.

It is now accepted that there is a familyof old/new effects (Donaldson et al., 2002),which are distinguishable on the basis oftheir time courses, scalp distributions, and

EPISODIC RETRIEVAL 175


sensitivity to experimental variables. Someof the old/new effects have a longerhistory than do others, and the effects alsovary considerably according to the degreeof consensus that exists concerning theirlikely functional significance. Here wereview old/new effects that have beenobserved in recognition memory tasks andin tasks in which retrieval of contextualinformation is required. We make noclaims about completeness [for example,see Friedman and Johnson (2000)]. Rather,we have selected these effects on thegrounds that there is sufficient evidence tosupport the view that they index function-ally distinct processes, and recent develop-ments provide good reason to comment. Inthis review we restrict ourselves to effectsthat have been observed in direct memorytasks, meaning tasks in which participantsare asked explicitly to retrieve informationthat was encoded in a prior study phase.Indirect memory tasks, by contrast, arethose in which no reference to a priorstudy phase is made. The findings fromstudies in which indirect tasks wereemployed are referred to only when theyrelate to the old/new effects that have beenobtained using direct tasks [for reviews ofthe ERP findings in indirect tasks, see Ruggand Doyle (1994) and Rugg and Allan(2000)]. At this point it is also worth notinga comment concerning terminology. Theterm repetition effects will be employed torefer to differences between ERPs evokedby old and new words in indirect tasks.The term old/new effects will be used only torefer to differences between ERPs evokedby old and new items to which accurateold/new judgments have been made.

The Left-Parietal ERP Old/New Effect

Sanquist and colleagues (1980) were thefirst to report that ERPs differentiated oldand new words to which correct judgmentswere made on a test of recognitionmemory. It is instructive to read the origi-nal report of Sanquist and colleagues for a

number of reasons. One striking aspect isthat the majority of the paper is concernedwith the ERPs that were recorded duringan encoding task that involved participantsmaking judgments about the orthographic,phonemic, or semantic similarities betweenpairs of stimuli that were presentedsequentially. The relatively small propor-tion of the paper that is devoted to theERPs recorded at test is perhaps surprisinggiven that contrasts between the ERPsevoked by classes of old and new teststimuli are now probably the most widelyreported data type in ERP studies ofepisodic memory.

The principal finding for the ERPsrecorded at test was that the amplitude of apositive-going wave form was greater forhits (studied words identified correctly)than it was for three other classes of ERPs:correct rejections (unstudied wordsidentified correctly), as well as misses andfalse alarms (incorrectly identified old andnew words, respectively). The amplitudedifferences between correct rejections,misses, and false alarms were markedlysmaller than the differences that separatedthese three measures from the correspond-ing mean amplitude measure for hits.These findings were important, becausethey suggested that the relative positivityelicited by hits was not due simply to repe-tition of a studied item, to processes thatare related to the act of making an “old”response, or simply to the belief that anitem was “old.”

The relative positivity elicited by hits inthe study of Sanquist and colleagues (1980)is likely the first report of what is nowtermed the left-parietal ERP old/new effect.Subsequent studies have revealed that theeffect onsets 400–500 msec poststimulus isoften larger at left than at right parietalscalp sites, and typically has a duration of500–800 msec (Karis et al., 1984; Johnson etal., 1985; Friedman and Sutton, 1987; Ruggand Nagy, 1989; Smith and Halgren, 1989;Friedman, 1990a,b; Noldy et al., 1990;Bentin et al., 1992; Potter et al., 1992; Rugg



et al., 1992; Smith and Guster, 1993). Thelink between this effect and memoryretrieval operations is strengthened by thefact that the absence of the effect for missesand false alarms reported initially bySanquist and colleagues (1980) has beenreplicated in a number of studies (Nevilleet al., 1986; Rugg and Doyle, 1992; Wildinget al., 1995; Van Petten and Senkfor, 1996;Wilding and Rugg, 1996). In combination,these findings are consistent with the viewthat the effect is an electrophysiologicalindex of veridical retrieval of informationfrom memory.

Mnemonic functional interpretations ofthe parietal old/new effect can be dividedinto those that have linked the effect to rec-ollection (e.g., Smith and Halgren, 1989;Van Petten et al., 1991; Paller and Kutas,1992; Smith, 1993; Paller et al., 1995; Rugget al., 1995), and those that have linked itwith fluency-based recognition or familiar-ity (Johnson et al., 1985; Friedman, 1990b;Potter et al., 1992; Rugg and Doyle, 1992,1994). The distinction between recognitionand familiarity has been introducedalready in the discussion of subsequentmemory effects, and it is now generallyaccepted that the parietal old/new effectindexes processes that are related moreclosely to recollection than to familiarity(Allan et al., 1998).

Arguably the strongest evidence insupport of this view has come from studiesin which recollection has been operational-ized as the ability to retrieve contextual (orsource) details of the prior occurrence of atest item—such as where on the study listit occurred, or what associations wereevoked by the stimulus at study. Forexample, Smith (1993) used the remem-ber/know paradigm in order to distin-guish recollected from familiar test items(Tulving, 1985; Gardiner and Java, 1993). Inthe study of Smith (1993), a larger parietalold/new effect was associated with R thanwith K responses that were made to oldtest words (also see Düzel et al., 1997).Complementary findings stem from two

studies by Wilding and colleagues(Wilding et al., 1995; Wilding and Rugg,1996), who used a forced-choice paradigmin which participants made an initialold/new judgment to test words, and, forthose judged to be old, a second judgmentdenoting in which of two contexts thewords had been presented at study. In thefirst report, the context manipulation wasmodality (auditory versus visual) (Wildinget al., 1995). In the second, it was speakervoice (male versus female) (Wilding andRugg, 1996). In a total of four experiments,the parietal old/new effect was largerwhen participants made correct sourcejudgments compared to when they madeincorrect source judgments following acorrect “old” judgment. In combination,these findings argue strongly for the linkbetween the parietal effect and recollec-tion, because the magnitude of any ERPold/new effect that reflected familiarityshould not vary with the success or other-wise of retrieval of contextual information.

On the basis of these and similarfindings, it has been proposed that theamplitude of the left-parietal old/neweffect is correlated positively with eitherthe quality or the amount of information ofinformation that is retrieved from memory(Rugg, 1995; Wilding and Rugg, 1996).How to distinguish between these twopossibilities is unclear; nonetheless, com-mon to them both is the prediction that theeffect should be of greater amplitude whenevoked by test items that are associatedwith accurate rather than inaccurate sourcejudgments, or R rather than K judgments.This is, of course, the pattern of data thathas been reviewed in the previous para-graph. Further consideration of thefindings in these studies, however, sug-gests an alternative interpretation.

The starting point for this argument isthe observation that the use of forced-choice measures for source judgmentsmeans that when ERPs are separatedaccording to the accuracy of the sourcejudgment, both resulting categories will



contain a proportion of trials on which rec-ollection did not occur. In more detail, theold/new effect for words associated withincorrect source judgments may havearisen as a result of averaging two types oftrials: those not associated with recollec-tion (which do not evoke parietal old/neweffects), and those associated with recollec-tion of information that was not diagnosticfor the source judgment that was requiredin the task [e.g., voice in the study ofWilding and Rugg (1996)]. Second, thelarger old/new effect for words associatedwith correct source judgments may havearisen because, in addition to containingtrials of the two types described above, onsome trials recollection of voice informa-tion would have occurred. A consequenceof this is that the findings in the studiesreviewed above cannot rule out the possi-bility that the parietal old/new effectindexes recollection in an “all-or-none”fashion. According to this account, the dif-ferences between the two critical responsecategories (correct vs. incorrect sourcejudgments) in the studies of Wilding andcolleagues (1995; Wilding and Rugg, 1996)arose because the category associated withincorrect source judgments containedhigher proportions of trials on which recol-lection did not occur. Broadly similar argu-ments can be applied to comparablefindings in other ERP studies of memorythat have required source memory judg-ments (Smith, 1993; Senkfor and VanPetten, 1998). Furthermore, the results instudies in which there is a positive correla-tion between recognition memory perfor-mance and the magnitude of theleft-parietal ERP old/new effect (e.g.,Paller and Kutas, 1992; Johnson et al., 1998)are consistent with an all-or-none account,because the level of recognition perfor-mance may well vary inversely with theproportion of trials in which recollectiondoes not occur.

In order to disentangle the graded andall-or-none accounts of the parietalold/new effect, Wilding (2000) designed an

experiment in which the larger proportionof trials not associated with recollectionwould be associated with the response cat-egory that should, according to the gradedaccount, evoke the largest left-parietalold/new effect. The experiment consistedof study phases in which each word wasassociated with two aspects of sourceinformation. These were followed by testphases in which two forced-choice sourcedecisions were required for items judged tobe old. This design permitted a contrastbetween the old/new effects evoked byitems that attracted either one or two correctsource judgments. In this design the proba-bility of one correct source judgment neces-sarily exceeds the probability of twocorrect judgments (the actual probabilitiesin the experiment, collapsed across studytask were as follows: 1 correct, 0.75; 2correct, 0.57). Assuming that trials in whichrecollection did not occur are distributedequally between the 2-correct and 1-correctresponse categories, then the relative atten-uating influence exerted by these trialsshould be greater in the 2-correct case, dueto the fact that this category contains lesstrials. The experiment thus constitutes astrong test of the graded hypothesisbecause the likely influence of guesseswould be to reduce the amplitude of the 2-correct parietal effect to a greater degreethan the amplitude of the 1-correct cate-gory.

The results shown in Fig. 2 providestrong support for the graded hypothesis,because the magnitude of the left-parietalold/new effect was correlated positivelywith the number of correct source judg-ments. At first pass this finding may seemto run counter to at least one dual-processaccount of recognition memory, accordingto which recollection is modeled as an all-or-none process, where the success orfailure of recollection is defined relative toa fixed threshold (Yonelinas, 1994;Yonelinas et al., 1996; Jacoby, 1998). Thisdisparity is more apparent than real,however, because threshold accounts make



no predictions as to how far above orbelow threshold the signal strength for agiven item will be.

In summary, the link between the left-parietal ERP old/new effects and recollec-tion is well established, and recent findingshave contributed to a more detailed under-standing of the way in which recollectionis indexed. A final comment is that the left-parietal effect has been evoked instudies in which numerous different kindsof source material have been retrieved(Donaldson et al., 2002). The fact that theeffect is observed consistently, irrespectiveof the contents of retrieval, suggests that it indexes what might be considered“central” or “core” aspects of recollectionthat are shared across content domains(Allan et al., 2000; Donaldson et al., 2002).

A Putative ElectrophysiologicalCorrelate of Familiarity

An important aspect of the studiesreviewed in the previous section is thatthey provide little or no evidence that theERP old/new effects evoked by correct

versus incorrect source judgments, or Rversus K judgments,1 are associated withscalp distributions that differ reliably. Tothe extent that any ERP index of familiar-ity would be relatively more prominent inthe ERPs evoked by incorrect source or K judgments, these findings provide little support for dual-process accounts ofrecognition memory (Mandler, 1980;Jacoby and Dallas, 1981). That is, the ERPfindings are consistent with the view thatthe difference between recognition with orwithout retrieval of source is one of degreerather than one of kind (Wilding andRugg, 1996). There are two (not mutuallyexclusive) reasons for treating this conclu-sion somewhat cautiously. The first is thedomain-general conclusion that nullresults in cognitive electrophysiological



1Düzel and colleagues (1997) reported differentialscalp distributions of the neural activity associated withcorrect R and K responses. Because their claims were notbased on analyses of appropriately rescaled data(McCarthy and Wood, 1985), their results must be treatedcautiously at present.

FIGURE 2 Grand-average ERPs elicited by words attracting either one (1-Corr) or two correct (2-Corr) sourcejudgments (collapsed across material retrieved), contrasted with the ERPs evoked by correct rejections (C Rej).Data from Wilding (2000) for left- and right-hemisphere frontal (AF7/AF8) and parietal (P5/P6) electrode sites.

studies must be interpreted with a gooddegree of circumspection, by virtue of thefact that ERPs provide only a partialmeasure of the neural activity that isevoked by stimuli in cognitive tasks (Ruggand Coles, 1995; Rugg, 1998; Wilding,2001). The second observation is that weconcur with the authors of an earlierreview, who, the aforementioned findingsnotwithstanding, find compelling theevidence from behavioral studies that rec-ollection and familiarity are indeed inde-pendent bases for recognition-memoryjudgments (Allan et al., 1998). These obser-vations are particularly apposite in light ofthe fact that the question of whether ERPsare in fact sensitive to familiarity as well asto recollection has been addressed in recentstudies. These studies suggest that thereasons for the absence of putative indicesof familiarity in the electrical record in pre-vious studies of recognition memory wasthat the paradigms employed were notoptimal for revealing a correlate of thisprocess, although other results, describedbelow, qualify this statement.

In an influential study, Rugg and col-leagues (1998) identified putative corre-lates of recollection and familiarity in asingle experiment. In their study, theauthors compared old/new effects follow-ing deep (semantic) and shallow (ortho-graphic) encoding tasks. Over the timewindow of 500–800 msec, the parietalold/new effect was larger for deeply thanfor shallowly studied words, in keepingwith previous findings (e.g., Paller andKutas, 1992). At frontocentral sites over the300- to 500-msec time period, both cate-gories of old item were more positive-going than correct rejections, and of equalamplitude. Rugg and colleagues proposedthat this earlier modulation indexes famil-iarity, a proposal supported by two aspectsof their data. First, the fact that the putativeindex of familiarity precedes the index ofrecollection is consistent with findings thatthe time after stimulus presentation atwhich familiarity is most influential for

recognition judgments is earlier than thecomparable time point for recollection(Yonelinas and Jacoby, 1994). Second, thefrontocentral effect was of equivalent sizein the deep and shallow conditions, whichconcurs with findings that depth of pro-cessing manipulations using the R/K para-digm has little influence on the probabilityof a K response, whereas the proportion ofR responses increases with depth of encod-ing (Gardiner, 1988; Gardiner et al., 1996;although see Toth, 1996; Yonelinas, 1998).

The possible link between this fronto-centrally distributed old/new effect andfamiliarity has been pursued by Curran(1999, 2000). In the earlier of these twostudies a frontocentral repetition effectanalogous to the old/new effect reportedby Rugg and colleagues was evident forboth words and pronounceable nonwords(pseudowords) when the task was eitherlexical decision or recognition memory.2 Arepetition effect analogous to the left-pari-etal ERP old/new effect was also evidentand this effect varied according to lexicalstatus, being larger for words than forpseudowords. Drawing on findings thatwords are more likely to be associated withrecollection than are pseudowords, Curranproposed, in keeping with the proposal ofRugg et al. (1998), that the frontocentraland left-parietal old/new effects indexfamiliarity and recollection, respectively.His findings in a subsequent studyrevealed the same one-way dissociationbetween the two effects. In this case, themanipulation involved presentation of sin-gular and plural words and the ERPs wereseparated according to response accuracyat test. In two experiments the left-parietalERP old/new effect was larger when plu-rality matched across study and testphases, whereas the frontocentral effect



2 Old and new words were not separated according toresponse accuracy in the recognition task so as to avoidbiasing the contrast between the repetition effectsobtained in lexical decision and old/new recognition.

was insensitive to changes in pluralitybetween study and test. To the extent that re-presentation of words in the same pluralityis associated with a greater likelihood of rec-ollection than is re-presentation of words inthe opposite plurality, these findings are alsoconsistent with the view that the frontocen-tral modulation indexes familiarity.

A similar between-groups dissociationwas reported by Tendolkar and colleagues(1999), who contrasted the ERPs evoked by correctly identified old and new stimuliin patients with Alzheimer disease (AD) with those evoked by matched controls. TheAD patients were impaired on judgmentsrequiring recollection relative to their con-trols, and a parietal old/new effect wasevident for controls only, whereas a fronto-central effect was evident for both groups(see Fig. 3). Finally, Mecklinger (2000)

reports patient data as well as data fromstudies in which participants had noneurological disorder that are consistentwith the view that a frontocentral effectindexes familiarity, but the left-parietal ERPold/new effect indexes recollection (also seeMecklinger, 1998; Mecklinger and Meins-hausen, 1998). In combination, the findingsreviewed to this point in this section arethus consistent with dual-process accountsof recognition memory (Rugg et al., 1998;Curran, 1999, 2000).

The findings in an additional study,however, suggest that the frontocentraleffect does not index familiarity directly. Inthe study of Tsivilis et al. (2001) partici-pants saw stimuli that comprised picturesof everyday objects set against backgroundscenes. The experiment was designed suchthat during old/new recognition for the



FIGURE 3 Group-average ERPs elicited by old and new words attracting correct recognition judgments forAlzheimer disease patients (A) and matched controls (B). ERPs are shown for left- and right-hemisphere frontal(F3/F4), central (T3/T4), and parietal (T5/T6) electrode sites. Note that negative is plotted upward on this figure.Reprinted from Neuroscience Letters, 263; I. Tendolkar, A. Schoenfeld, G. Golz, G. Fernandez, K.-P. Kuhl, R. Ferszt,and H.-J. Heinze; Neural correlates of recognition memory with and without recollection in patients withAlzheimer’s disease and healthy controls, pp. 45–48. Copyright 1999, with permission from Elsevier Science.

objects the object/background pairingscould be one of five types: studied objectsagainst the same background or against adifferent background (which could be pre-viously seen or novel), and novel objectsagainst a novel background or one that hadbeen seen at study. The putative index offamiliarity was evident only when bothobject and background were old (relativeto a novel object/novel background baseline), leading Tsivilis and colleagues tosuggest that the putative correlate of famil-iarity is in fact a negative-going index ofnovelty, whereby the modulation will beevident when any aspect of task-relevantinformation is new to the experiment. Werethe effect to be considered an index offamiliarity, then the relative positivity seenfor object/background conjunctions whenboth elements were old should also havebeen observed when an old object wasseen against a new background.

The findings of Tsivilis and colleaguessuggest that ERPs are not sensitive tofamiliarity directly, but remain consistentwith the view that ERPs index multipledistinct processes during recognitionmemory and variants thereon. Tsivilis et al.(2001) also observed a left-parietal old/new effect that was dissociable from theputative index of novelty, as well as anearlier anteriorly distributed componentthat was sensitive to any pairing thatinvolved at least one component (object orbackground) that was encountered atstudy. A commentary on the likely func-tional significance of this early frontal com-ponent is deferred until more is knownabout its antecedents. In combination,however, the findings in this study extendthose in previous studies. Common to themajority of findings reviewed in thissection is that they support dual-processaccounts of recognition memory to theextent that they indicate there are at least two neurally and functionally disso-ciable processes engaged during recogni-tion tasks. The findings of Tsivilis et al.,however, suggest that the mapping

between the memory-related processesevident in the electrical record and theprocesses postulated in dual-processaccounts is not a direct one.

Electrophysiological Indices of Implicit Memory

Another important result from the studyof Rugg and colleagues (1998), reviewedabove, stemmed from the fact that, forshallowly studied words, it was possible toform ERPs evoked by old words that werejudged incorrectly to be new (misses). Atfrontal sites over the time window of300–500 msec, these ERPs did not differfrom those for correct rejections, whereasat parietal sites during the same timeperiod the misses were associated with apositive-going modulation of magnitudeequivalent to those observed for correct oldjudgments to deeply and shallowly studiedwords. This early parietal effect is thereforefunctionally dissociable from the fronto-central effect, which was evident for bothclasses of old words, and the left- parietalold/new effect, which was evident princi-pally for old words studied in the deepencoding task. On the basis of the facts thatthis modulation discriminates old fromnew items, but does not vary in amplitudeaccording to the accuracy of old judg-ments, Rugg and colleagues proposed thatit is an electrophysiological index ofimplicit memory (Rugg et al., 1998).

The fact that ERPs differentiate old fromnew words over this early time period atparietal scalp sites has been documented ina number of indirect memory tasks (e.g.,Rugg, 1987; Bentin and Peled, 1990), aswell as in continuous recognition memory(e.g., Friedman, 1990a,b) and study-testrecognition tasks (see especially Smith andHalgren, 1989). In addition, the linkbetween this effect and processes related toimplicit memory has been noted in someprevious studies (e.g., Bentin et al., 1992;Friedman et al., 1992), and is suggested byvisual inspection of ERPs elicited by misses



and correct rejections in other reports (forexample, see Johnson et al., 1998). None-theless, the combination of functional dissociations reported by Rugg and col-leagues represents the first convincingdemonstration that ERPs index implicitmemory, and perhaps more importantlythe first demonstration, in normal partici-pants, that implicit and explicit memoryare associated with distinct neural sub-strates. These findings provide a poten-tially important springboard to investigateissues that include, but are not restrictedto, the sensitivity of ERPs to differentforms of implicit memory (e.g., conceptualversus perceptual priming) and the ques-tion of the degree to which implicit andexplicit memory are independent pro-cesses. Strong evidence in support of inde-pendence would accrue from findings thatthe putative electrophysiological correlatesof implicit and explicit memory processescan be manipulated such that acrossexperimental conditions both occur in theabsence of the other (Rugg et al., 2000).

Old/New Effects at Frontal Scalp Sites

Neuropsychological findings indicate thatthe functional integrity of both the medialtemporal lobes and the prefrontal cortex isnecessary for veridical source memory.Patients with extensive damage to medialtemporal lobe structures commonly exhibita marked inability to acquire and/or bring to mind new information (Scoville andMilner, 1957; Mayes, 1988). By contrast,patients with damage restricted to prefrontalcortex exhibit a more selective impairment.While often able to make simple judgmentsof prior occurrence, impairments are com-monly observed on tasks that require morecomplex judgments that involve usingcontextual details from a prior episode(Schacter et al., 1984; Shimamura and Squire,1987; Janowsky et al., 1989; Glisky et al.,1995).

On the basis of these findings it has beenproposed that the medial temporal lobes

and prefrontal cortex have distinct func-tional roles in recollection. According toone class of accounts, the former is respon-sible for information retrieval, whereas thelatter is involved in postretrieval processesthat operate on the products of retrieval(Moscovitch, 1992, 1994; Squire et al., 1993).For example, Moscovitch (1992) hasreferred to the prefrontal cortex as a“working-with-memory” structure, whichincorporates the idea that this structure isinvolved in coordinating information necessary for veridical recollection, as well as being influential in the ways inwhich memories are used to guidebehavior.

The starting point for this discussion ofERP studies that are relevant to the multi-component nature of recollection is thestudy of Wilding and Rugg (1996), whichwas introduced earlier. To recap, in thatstudy participants completed an initiallexical decision task, in which equalnumbers of word and pronounceablenonword stimuli were spoken in a male ora female voice. In a subsequent test phase,participants made initial old/new judg-ments to old and new words. For thosewords judged old, participants made asecond judgment, denoting whether theold words had been spoken in a male orfemale voice at study. This allowed thecomparison of three classes of ERPs: thoseto correct rejections, and those to wordsjudged old correctly that were assignedeither correctly or incorrectly to studyvoice.

As discussed previously, the left-pari-etal old/new effect was larger following acorrect source judgment than it was fol-lowing an incorrect source judgment. Inaddition, this study revealed a secondold/new effect that differentiated correctlyidentified old and new words. The effectwas largest over frontal scalp sites, andlarger over the right hemisphere than overthe left. The effect onset was at around thesame time as the parietal effect but had aduration that extended until at least 1400



msec poststimulus. This right-frontalold/new effect was dissociable from theleft-parietal effect in terms of time courseand scalp distribution, thereby allowingthe conclusion that two distinct neuralpopulations are engaged during retrievalof source information. An example of theright-frontal old/new effects is shown inFig. 4, which also provides an illustrationof amplitude differences between ERPsevoked by words attracting correct orincorrect source judgments.

Like the parietal old/new effect, theright-frontal effect was larger for correctthan for incorrect source judgments,leading Wilding and Rugg to suggest thatthe effect played a functional role in sourcememory judgments—a conclusion sup-ported by the fact that equivalent effectsover frontal scalp were not observed forfalse alarms or for misses (Wilding andRugg, 1996, 1997b). Consequently, follow-ing the neurologically inspired models ofMoscovitch and Squire (also see Stuss et al.,1994), Wilding and Rugg proposed that theleft-parietal effect reflected informationretrieval, whereas the right-frontal effectreflected processes necessary for integrat-ing retrieved information into a coherentrepresentation of a prior episode. Johnsonand colleagues (Johnson et al., 1996) cameto a similar conclusion on the basis offindings that activity at right-frontal elec-trode sights was greater in tasks requiringsource judgments than in tasks requiring

recognition memory judgments (also seeSenkfor and Van Petten, 1998).

Although Wilding and Rugg made thisproposal in the absence of direct electro-physiological evidence to support the viewthat the left-parietal and right-frontalold/new effects index distinct processes,subsequent findings support the claim(e.g., Wilding and Rugg, 1997a,b). Thesesubsequent findings also, however, indi-cated that the initial functional interpreta-tion of the right-frontal effect requiredmodification. The effect does not alwaysvary in magnitude according to the accu-racy of source judgments (Trott et al., 1997;Senkfor and Van Petten, 1998; Van Petten et al., 2000; Cycowicz et al., 2001), it is on occasions evident in recognition tasks in which there is no overt source retrieval requirement (Allan and Rugg,1997; Donaldson and Rugg, 1998; Rugg et al., 2000), it is not an obligatory correlateof successful source retrieval (Rugg et al.,1996; Wilding and Rugg, 1997b), it varies inmagnitude according to the type of sourceinformation that is retrieved (Wilding,1999; Donaldson et al., 2002), and it hasbeen statistically indistinguishable for Rand K judgments to old items (Düzel et al.,1997). In addition, it is not always evidentthat the same effects are in fact beingobserved across studies (in particular, seeTendolkar and Rugg, 1998). Visual inspec-tion suggests that under certain retrievalconditions frontal old/new effects vary



FIGURE 4 Grand-average ERPs elicited by old and new words to which either correct or incorrect source judg-ments were made (collapsed across retrieval task), contrasted with those evoked by correct rejections. Data are shownfor left- and right-hemisphere frontal (D7/D8) and parietal (P5/P6) electrode sites. Data from Wilding (1999).

with respect to the degree of laterality aswell as with the relative anterior/centraldistribution of the effects (Trott et al., 1997;Tendolkar and Rugg, 1998; Cycowicz et al.,2001). Furthermore, under some circum-stances the effect can be dissociated into anearly bilateral component (600–900 msec)and a longer duration right-frontal compo-nent (Wilding and Rugg, 1997a; Donaldsonand Rugg, 1998). The conditions underwhich these two components are evident,either separately or in conjunction, are notwell established. A general rule of thumb isthat the earlier of the two effects tends tobe more prominent in tasks in which anold/new judgment precedes the sourcejudgment than in studies in which itemand source judgments are combined (cf.Wilding and Rugg, 1996, 1997b), as is thecase in the recognition memory exclusiontask procedure (Jacoby, 1998).

Rather than reassessing ground that has been reviewed in detail elsewhere(Friedman and Johnson, 2000; Rugg andAllan, 2000; Van Petten et al., 2000), werestrict discussion here to a couple of broadobservations. First, the weight of evidencesupports the view that these frontal effectsare likely an index of processes thatoperate in some way on retrieved informa-tion in pursuit of task-specific goals. Amore precise characterization is elusive atpresent [for related comments, see VanPetten et al. (2000)]. Second, a number ofrelated points stem from the view that thefrontal old/new effects described aboveare likely to be generated in the frontalcortex. This claim is supported by the neu-ropsychological evidence mentioned previ-ously (Stuss et al., 1994), and the consistentfindings from functional neuroimagingstudies that frontal brain regions areengaged during episodic encoding andretrieval (Cabeza and Nyberg, 2000). In areview of the functional neuroimaging lit-erature relevant to the question of the func-tional roles played by frontal cortex inepisodic memory, Fletcher and Henson(2001) proposed that three separate regions

of lateral frontal cortex—anterior, dorsolat-eral and ventrolateral—support distinctcognitive operations. The specification ofthese operations was derived in part fromprotocols reported by participants ineffortful episodic retrieval tasks, as well asconsideration of the more domain-generalroles that frontal cortex plays in cognition(Burgess and Shallice, 1996; Shallice andBurgess, 1998). Fletcher and Henson (2001)proposed that anterior frontal cortex sup-ports processes involved in the selection ofgoals and subgoals, whereas the integrityof ventrolateral and dorsolateral frontalcortex is necessary for updating workingmemory and monitoring as well as select-ing among the contents of workingmemory, respectively.

As the authors acknowledge, this tripar-tite division is unlikely, ultimately, to havesufficient anatomical or cognitive resolu-tion. Models of this type, however (alsosee Wagner, 1999), emphasize that subtledifferences between retrieval tasks arelikely to result in engagement of somecombination of the control processes thatcan in principle be required duringepisodic retrieval. The differential engage-ment of these processes across differentretrieval tasks may go some way toexplaining the disparities across studies inthe ERP episodic memory literature. Adetailed examination of this possibility isbeyond the scope of this review. In closing,though, it is perhaps worth noting whatmay turn out to be a more fundamentalconcern. Although the existing data per-mit some conclusions about the relativeengagement of right- versus left-frontalcortex, more fine-grained delineationwithin each hemisphere is speculative atthis stage. It may be the case, that it willnot be possible to separate reliably the con-tributions of distinct frontal corticalregions with scalp-recorded event-relatedpotentials. If this turns out to be the case, then event-related potentials may not be particularly influential in develop-ing either cognitive or functional neuro-



anatomical accounts of the controlprocesses that are engaged during episodicretrieval. This conclusion is premature atpresent, however, and a resolution to thisissue will stem at least in part from studies that incorporate two improvements on the majority of studies to date. The first isthe use of electrode montages that havebetter spatial resolution than those thathave been employed to date. The second isa tighter specification of the cognitiveprocesses that are engaged during retrievaltasks.

ELECTROPHYSIOLOGICALINDICES OF

RETRIEVAL ATTEMPTS

In this final section, we review recentstudies in which contrasts have not beenrestricted to those between the ERPsevoked by correct rejections and thoseevoked by various classes of responses toold items. Rather, they have been extendedto include contrasts between classes of new(unstudied) test items that have beenobtained in different retrieval tasks. Thelogic for this contrast is that unstudieditems on a given direct memory task willpresumably be subject to processes that can broadly be defined as constituting a retrieval attempt (Wilding, 1999). Ofcourse, old items will also be subject to thisclass of retrieval processes, but these maybe confounded in the electrical record withactivity that reflects retrieval success, or thepostretrieval processing of retrieved infor-mation (Rugg and Wilding, 2000). Theactivity elicited by unstudied test items isassumed to be less subject to this confounddue to the fact that these items were notpresented in the relevant prior studyphase.

Rugg and Wilding (2000; Rugg et al.,2000) distinguished two classes of processthat might be considered to form part of aretrieval attempt. The first is retrievaleffort, defined as the recruitment of

resources in pursuit of retrieval. Thesecond is retrieval orientation, whichdetermines the task-specific processes thatmay be engaged in pursuit of retrieval ofspecific mnemonic contents. Rugg andWilding argued that separating these twoclasses of retrieval process requires them tobe manipulated independently. This hasbeen achieved in some studies but not inothers.

To our knowledge, the initial report of acontrast between ERPs evoked by classesof new items is due to Johnson and col-leagues (1996). In that study, two groups ofparticipants completed incidental encodingtasks on an equal number of words andpictures. One group was asked to think ofuses for each object designated by theword or picture. The other was asked torate how easy it would be to draw eachstimulus (either the picture or the imageevoked by each word). In the retrieval taskparticipants were shown an equal numberof old and new words. They were asked todistinguish old from new words, and forthe old words, to indicate whether theyhad been encountered as a word or as apicture at study. Distinct modulations overfrontal and posterior scalp sites differenti-ated the ERPs that were evoked by correctrejections for the two groups. The samemodulations were evident in the ERPs thatwere evoked by old words associated witha correct picture/word designation.Johnson and colleagues (1996) proposedthat the differences reflected shifts in evaluative operations that were engaged in the two cases, depending on the em-phasis at study. They argued that newitems would be subject to the same eva-luative operations as old items and it was these processes that were indexed by the modulations differentiating theERPs evoked by new items from the twogroups.

A similar line of enquiry was pur-sued by Wilding (1999) using a within-participants design. Whereas Johnson andcolleagues manipulated the processes



engaged at retrieval indirectly via anencoding manipulation, in the study ofWilding (1999) all participants completedthe same encoding task and then differentretrieval tasks in separate blocks. ERPswere recorded to visual presentations ofold and new words in separate test blocksthat required source judgments basedeither on voice or task information. Thevoice and task information had beenencoded in a task in which participantsheard words, an equal number of whichwere spoken by a male/female voice. Toeach word participants made one of twojudgments—an active/passive judgment(action task) or a pleasant/unpleasantjudgment (liking task). The type of judg-ment varied across trials, and was deter-mined by a cue that preceded each spokenword. In one test condition, participantsmade a three-way distinction between newwords and old words spoken at study bythe male or female voice. In the secondcondition, participants distinguished newwords from old words, to which either anactive/passive or a pleasant/unpleasantjudgment had been made at study. Therewas no systematic relationship at studybetween task and voice, thereby preclud-ing successful retrieval task performancebeing accomplished by focusing on oneaspect of context information only. Par-ticipants were informed of the retrievalorientation for each test block only aftercompletion of the immediately precedingstudy phase.

The likelihood of a correct rejection wasequivalent in both retrieval tasks, as werethe reaction times for these items. TheERPs evoked by correct rejections in thetwo retrieval tasks were distinguished by apolarity reversal at left- and right-hemi-sphere frontal sites, with the differencesextending to central scalp locations. Inlight of the behavioral data, Wilding pro-posed that the effects were unlikely toreflect processes tied closely to taskdifficulty, hence retrieval effort, and inkeeping with the interpretation offered by

Johnson et al. (1996) proposed that theprocesses reflected differential monitoringor evaluation of the mnemonic informa-tion that was activated by new test items.

Contrasts between classes of correctrejections were also reported in twostudies by Ranganath and Paller (1999,2000). In the earlier of these two studies(Ranganath and Paller, 1999), each partici-pant completed the same picture-encodingtask followed by two different retrievaltasks. At test each picture took one of threeforms: old, new, or new but perceptuallysimilar to old pictures (previously pre-sented pictures were rescaled, resulting insmall changes to their height and width).In one test condition (general retrieval),participants made old/new recognitionjudgments to pictures, responding old topreviously studied pictures as well as toperceptually similar pictures. In the othertest condition (specific retrieval), partici-pants responded old only to previouslystudied pictures. These two conditionswere designed to differ in the degree towhich participants were required toprocess perceptual details of the test items.

The differences between the ERPs thatwere evoked by the two classes of newitems were most evident over left-frontalscalp, where those from the specificretrieval condition (respond old only tostudied pictures) were more positive-goingfrom approximately 400 to 1200 msec poststimulus. The authors reasoned that,in contrast to the general retrieval testcondition, the specific retrieval conditionrequired participants to attend moreclosely to perceptual features of thestimuli, and to engage in more evaluativeoperations before making an old/newjudgment. They proposed that the differ-ences over left-frontal scalp reflected thegreater demands that these information-processing operations imposed on atten-tion and working memory in pursuit ofretrieval. This amounts to a retrieval effortinterpretation of the findings, which mightbe considered to run counter to the behav-

ELECTROPHYSIOLOGICAL INDICES OF RETRIEVAL ATTEMPTS 187


ioral findings: there were no reliable differ-ences between reaction times for the twoclasses of new items, and no differencesbetween the probabilities for correct “new”responses.

The probability of a correct judgment toan old item was, however, lower in thespecific than in the general retrieval task.This raises an interesting question, becausein this experiment, as well as that ofWilding (1999), task difficulty can be con-sidered constant across the retrieval tasksof interest only if the data supporting theclaim for parity are restricted to responsesattracted by new items. If task difficulty isassessed in relation to the likelihood ofcorrect responses to old items, or perhapsin terms of the discrimination measurep(hit) minus p(false alarm) (Snodgrass andCorwin, 1988), then the specific task in thestudy of Ranganath and Paller and thevoice retrieval task in the study of Wildingare more difficult. Which aspects of thebehavioral data to include when determin-ing task difficulty is not at all clear.

Two studies in which there is little argu-ment with the claim that effort and orienta-tion are confounded are due to Rugg et al.(2000) and Wilding and Nobre (2001).Rugg and colleagues also revealed differ-ences at left-frontal scalp locations betweenthe ERPs that were evoked by classes ofunstudied test items. In this case, however,the differences were related to an encodingmanipulation rather than to differentinstructions at test. Participants completedencoding tasks in which visually presentedlow-frequency words were processed withrespect to either their semantic or theirorthographic characteristics (hereafter thedeep and shallow encoding tasks, respec-tively). In subsequent old/new recognitionblocks, each block contained words thathad been processed in only one of the twoencoding tasks. For unstudied items theERPs at left-frontal scalp locations weremore positive-going for the blocks thatcontained shallowly encoded old words.Because memory was poorer in the

shallow than in the deep retrieval task, it isreasonable to assume that greater demandswere placed on attention and workingmemory in pursuit of recognition decisionsin this task. The findings are, therefore,consistent with the interpretation offeredby Ranganath and Paller (1999, 2000).

The findings of Rugg et al. for writtenwords also indicate that the left-frontalmodulation is not a consequence of usingpicture stimuli at test, and that the differ-ences observed by Ranganath and Paller(1999, 2000) likely do not reflect processesrelated to differential inspection of thesurface features of test stimuli. Whilenoting that their results were consistentwith a working memory-load interpreta-tion, however, Rugg et al. (2000) also dis-cussed an alternative account of theleft-frontal effect that can be definedbroadly as an “effort” interpretation. Theyobserved that participants adopted differ-ent response criteria in the deep andshallow retrieval blocks, and that a differ-ence in response criterion across conditionswas also evident in the behavioral datafrom the study of Ranganath and Paller(1999). In both cases, a more stringent (con-servative) criterion was adopted in themore demanding task. The left-frontaleffect could therefore reflect processesrelated to criterion setting rather than tothe differential demands placed onworking memory and/or attention (Rugget al., 2000).

In addition to this left-frontal modula-tion, Rugg et al. (2000) also observed asecond modulation that differentiated theERPs elicited by unstudied items in thedeep and shallow retrieval conditions. Thiseffect was largest at right-hemisphere cen-troparietal scalp locations, and compriseda greater negativity in the ERPs evoked byunstudied items from the deep retrievalcondition. The authors suggested that thismodulation likely indexes processes thatare distinct from those indexed by the left-frontal effect. The principal support for thisproposal was drawn from the similarity



between this modulation and the N400ERP component. This negative-going com-ponent was identified initially in studies oflanguage processing (Kutas and Hillyard,1980) and is larger in tasks that requiresemantic rather than nonsemantic process-ing of stimuli (Rugg et al., 1988; Chwilla etal., 1995). On the basis of this similarity,Rugg et al. (2000) proposed that partici-pants employed retrieval strategies at testthat varied according to their experiencesat the time of encoding, with the N400-likemodulation reflecting the greater emphasison semantic retrieval processing in theeasier of the two retrieval tasks. Whetherthe N400-like modulation indexes pro-cesses that are in fact distinct from thoseindexed by the left-frontal modulation wasnot clear, however, because there was nostatistical evidence to support the viewthat the two effects were either neurally orfunctionally dissociable.

Their claims for functionally separableprocesses draw support from a study inwhich participants were directed at test toretrieve either semantically or phonologi-cally encoded material (Wilding andNobre, 2001). Participants initially com-pleted two different encoding tasks. Thefirst involved generating a word andsaying aloud a word that rhymed with it.The second involved generating a mentalimage of a concept denoted by a word andsaying aloud a word denoting a conceptwith a similar mental image. These encod-ing tasks were blocked and each block wasfollowed by a corresponding retrievalphase in which participants madeR/K/new judgments to old and newwords. They were asked to make an Rresponse only to those words for whichthey could remember the associate (eitherimage based or phonological) that wasgenerated at study. In keeping with thefindings of Rugg and colleagues (Rugg etal., 2000), memory performance was supe-rior following image-based encoding(Craik and Lockhart, 1972). The ERPsevoked by correct rejections in this task

were also associated with a larger N400-like effect than those evoked in the phono-logical retrieval task. To the extent that theimage-based retrieval task is likely toencourage greater emphasis on semanticprocessing, compared to the phonologicaltask, these findings are consistent withthose of Rugg et al. (2000).

Wilding and Nobre (2001) included asecond experiment in their study, againincorporating the image-based/phonologi-cal manipulation. The principal differencebetween this experiment and the onedescribed already was that at test partici-pants were cued on a trial-by-trial basis asto which retrieval task to complete, andfrequent switches between tasks wererequired. Memory accuracy was no worsein this experiment than when the tworetrieval tasks were blocked. By contrast,the modulations that differentiated theERPs evoked by the two classes of correctrejections in the blocked design were atten-uated in the mixed-trial design, as isshown in Fig. 5.

Wilding and Nobre argued that thereason for this attenuation was that partici-pants in the mixed-trial design were notprovided with sufficient opportunity toengage in task-specific processing [and/orto disengage from processing relevant onprevious trials: see Rogers and Monsell(1995) and Allport and Wylie (2000)].Furthermore, they observed that the corre-spondence across studies at the level of thebehavioral data was consistent with theview that the processes reflected in the dif-ferences between the ERPs evoked bycorrect rejections in the blocked designwere not influential in determining thesuccess or failure of memory retrieval.Rather, they argued that the differenceslikely reflected processes that optimizedthe evaluation of task-relevant aspects of amemory trace. Although this interpretationmay turn out to be correct, it presumesthat the differences observed in theblocked design reflect orientation ratherthan effort. The behavioral data indicate

ELECTROPHYSIOLOGICAL INDICES OF RETRIEVAL ATTEMPTS 189


that memory accuracy was superior in theimage-based tasks, and it remains to bedetermined whether effort and orienta-tion are influenced equivalently by therequirement to switch frequently betweenretrieval tasks.

To our knowledge, retrieval effort andorientation have been manipulated inde-pendently in only one study. Robb andRugg (2002) recorded ERPs while partici-pants completed four separate recognitionmemory tasks. Old/new judgments tovisually presented words were made ineach. In two of the blocks words were pre-sented at study; in the remainder, pictureswere presented. One block in each paircomprised an easy recognition task, andthe other a difficult one. This was accom-plished by manipulating study-list lengthand the study–test interval. The designtherefore enabled identification of neuralactivity predictive of effort and activitypredictive of orientation.

Indices of both classes of processes wereevident in the electrical record, althoughthe authors encourage caution concerningthe putative index of processes related to

difficulty, given the size of the effect and itslevel of significance. The effect of orienta-tion, by contrast, was robust, extendingfrom 300 to 1900 msec poststimulus with abroad scalp distribution. Whatever theeventual status of the putative correlate ofdifficulty, these findings are importantbecause they are consistent with the viewthat effort, at least as operationalized interms of task difficulty, is not simply mani-fest as changes in activity in the same brainregions that support retrieval orientations(Rugg and Wilding, 2000).

In summary, the deployment of ERPs inorder to index processes that are engagedin pursuit of retrieval is a relatively recentendeavour. The findings to date providesome support for the distinction betweenorientation and effort, but require general-ization. This will involve the use of a widerrange of stimuli and task types, as well asimplementation in designs when electro-physiological correlates of these twoclasses of process can be disentangled. Todate, attention to processes that form partof a retrieval attempt has been somewhatlacking in ERP studies in comparison to



FIGURE 5 Group-average ERPs elicited by new words attracting correct judgments in the blocked(Experiment 1) and mixed-trial (Experiment 2) designs for frontal (F3, Fz, F4), central (C3, Cz, C4), and parietal(P3, Pz, P4) electrode sites. The ERPs elicited by new items are separated according to the retrieval task (imagebased or phonological) in which they were presented. Data from Wilding and Nobre (2001).

studies in which other functional neu-roimaging modalities have been employed(Rugg and Wilding, 2000). It is to be hopedthat this situation will change in the nearfuture.

CONCLUDING REMARKS

In this review we have focused onrecent developments in electrophysiologi-cal studies of episodic memory processes.In the case of both episodic encoding andepisodic retrieval there are good groundsto believe that important advances andinsights are being—and will continue tobe—made. Although this review hasfocused on processes that are time lockedto the presentation of discrete task stimuli,it may turn out that in future studies it willbe important to supplement theseapproaches with designs that permit themonitoring of activity that is engagedbefore stimuli are presented. There are twoways in which this could be achieved. Thefirst is to employ designs in which partici-pants are cued as to which encoding orretrieval task to complete on a trial-by-trialbasis (e.g., Otten and Rugg, 2001a; Wilding,2001; Morcom and Rugg, 2002), and torecord the activity that is evoked by thecues. Such activity will presumably reflectpreparatory processes that are set in trainby the cue and may well influence theaccuracy and/or time course of successfulepisodic encoding and retrieval.

A second approach is to record directcurrent (DC) potentials during encodingand retrieval tasks. These recordings can beemployed to monitor slow (low-frequency)changes in neural activity that can be sus-tained for several seconds and may reflectthe adoption and maintenance of a cogni-tive set. A set is a preparatory state thatdetermines the processing to which taskstimuli will be subjected (Gibson, 1941).The extent to which the success or other-wise of episodic encoding and retrieval areinfluenced by the adoption of a set or

mode (Tulving, 1983) remains to be deter-mined, but it is an issue that ERPs are wellsuited to address. This is due to the factthat by adopting a suitably broad fre-quency range at acquisition it is in princi-ple possible to index two classes ofprocess. The first is state-related activitythat is time locked to task onset, may bemaintained for the duration of the task,and is likely reflected in low-frequencymodulations of the electrical record. Thesecond is item-related activity that is set intrain by stimuli that are presented duringthe task. The activity engaged by thesestimuli is assumed to have a time coursethat is shorter than the interstimulus inter-val, and will be indexed by higher fre-quency components of the electrical recordthan those indexing state-related activity.The results of one study in which state-and item-related measures of episodicretrieval operations were obtained (Düzelet al., 1999) suggest strongly that the use ofthis approach will provide a valuableadjunct to the methods that are employedmost often at present in event-relatedpotential studies of episodic memoryencoding and retrieval.

ReferencesAllan, K., and Rugg, M. D. (1997). An event-related

potential study of explicit memory on tests ofcued recall and recognition. Neuropsychologia 35,387–397.

Allan, K. A., Wilding, E. L., and Rugg, M. D. (1998).Electrophysiological evidence for dissociableprocesses contributing to recollection. ActaPsychol. 98, 231–252.

Allan, K., Robb, W. G. K., and Rugg, M. D. (2000). Theeffect of encoding manipulations on neural corre-lates of episodic retrieval. Neuropsychologia 38,1188–1205.

Allport, A., and Wylie, G. (2000). Task-switching:Positive and negative priming of task-set. In“Attention, Space and Action: Studies inCognitive Neuroscience” (G. W. Humphreys, J.Duncan, and A. M. Treisman, eds.), pp. 273–296.Oxford University Press, Oxford.

Bentin, S., and Peled, B. S. (1990). The contribution oftask-related factors to ERP repetition effects atshort and long lags. Mem. Cogn. 18, 359–366.

REFERENCES 191


Bentin, S., Moscovitch, M., and Heth, I. (1992).Memory with and without awareness: Per-formance and electrophysiological evidence ofsavings. J. Exp. Psychol. Learn. Mem. Cogn. 18,1270–1283.

Burgess, P. W., and Shallice, T. (1996). Confabulationand the control of recollection. Memory 4, 359–411.

Cabeza, R., and Nyberg, L. (2000). Imaging cognitionII: An empirical review of 275 PET and fMRIstudies. J. Cogn. Neurosci. 12, 1–47.

Chwilla, D. J., Brown, C. M., and Hagoort, P. (1995).The N400 as a function of the level of processing.Psychophysiology 32, 274–285.

Craik, F. I. M., and Lockhart, R. S. (1972). Levels ofprocessing: A framework for memory research.J.Verbal Learn. Verbal Behav. 16, 519–533.

Curran, T. (1999). The electrophysiology of incidentaland intentional retrieval: ERP old/new effects inlexical decision and recognition memory.Neuropsychologia 37, 771–785.

Curran, T. (2000). Brain potentials of recollection andfamiliarity. Mem. Cogn. 28, 923–938.

Cycowicz, Y. M., Friedman, D., and Snodgrass, J. G.(2001). Remembering the colour of objects: An ERPinvestigation of source memory. Cereb. Cortex 11,322–334.

Dale, A. M., and Buckner, R. L. (1997). Selective aver-aging of rapidly presented trials using fMRI. Hum.Brain Mapping 5, 329–340.

Donaldson, W. (1996). The role of decision processesin remembering and knowing. Mem. Cogn. 24,523–533.

Donaldson, D. I., and Rugg, M. D. (1998). Recognitionmemory for new associations: Electrophysiologicalevidence for the role of recollection.Neuropsychologia 36, 377–395.

Donaldson, D. I., Wilding, E. L., and Allan, K. (2002).Fractionating retrieval from episodic memoryusing event-related potentials. In “The CognitiveNeuroscience of Memory: Episodic Encoding andRetrieval” (A. E. Parker, E. L. Wilding, and T. J.Bussey, eds.). Psychology Press, Hove.

Düzel, E., Yonelinas, A.P., Mangun, G.R., Heinze, H.J.,and Tulving, E. (1997). Event-related brain poten-tial correlates of two states of conscious awarenessin memory. Proc. Natl. Acad. Sci. U.S.A 94,5973–5978.

Düzel, E., Cabeza, R., Picton, T. W., Yonelinas, A. P.,Scheich, H., Heinze, H. J., and Tulving, E. (1999).Task-related and item-related brain processes ofmemory retrieval. Proc. Natl. Acad. Sci. U.S.A. 96,1794–1799.

Fernandez, G., Weyerts, H., Tendolkar, I., Smid, H. G.,Scholz, M., and Heinze, H. J. (1998). Event-relatedpotentials of verbal encoding into episodicmemory: Dissociation between the effects of sub-sequent memory performance and distinctiveness.Psychophysiology 35, 709–720.

Fletcher, P. C., and Henson, R. N. A. (2001). Frontallobes and human memory: Insights from func-tional neuroimaging. Brain 124, 849–881.

Friedman, D. (1990a). Cognitive event-related poten-tial components during continuous recognitionmemory for pictures. Psychophysiology 27, 136–148.

Friedman, D. (1990b). Endogenous event-related brainpotentials during continuous recognition memoryfor words. Biol. Psychol. 30, 61–87.

Friedman, D., and Johnson, R. J. (2000). Event-relatedpotential (ERP) studies of episodic memory encod-ing and retrieval: A selective review. Micros. Res.Techniq. 51, 6–28.

Friedman, D., and Sutton, S. (1987). Event-relatedpotentials during continuous recognition mem-ory. In “Current Trends in Event-Related Poten-tial Research” (Supplement 40 to “Electro-encephalography and Clinical Neurophysiology”)(R. Johnson, J. W. Rohrbaugh, and R. Parasuraman,eds.), pp. 316–321. Elsevier, Amsterdam.

Friedman, D., and Trott, C. (2000). An event-relatedpotential study of encoding in young and olderadults. Neuropsychologia 38, 542–557.

Friedman, D., Hamberger, M., Stern, Y., and Marder,K. (1992). Event-related potentials during repeti-tion priming in Alzheimer’s patients and youngand older controls. J. Clin. Exp. Neuropsychol. 14,448–462.

Gabrieli, J. D. E. (1993). Disorders of memoryin humans. Curr. Opin. Neurol. Neurosurg. 6, 93–97.

Gardiner, J. M. (1988). Functional aspects of recollec-tive experience. Mem. Cogn. 16, 309–313.

Gardiner, J. M. (2001). Episodic memory and auto-noetic consciousness: A first-person approach.Philos. Trans. Roy. Soc. (Lond.) 356, 1351–1361.

Gardiner, J. M., and Java, R.I. (1990). Recollectiveexperience in word and non-word recognition.Mem. Cogn. 18, 23–30.

Gardiner, J. M., and Java, R. I. (1993). Recognising andremembering. In “Theories of Memory” (A.Collins, M. A. Conway, S. E. Gathercole, and P. E. Morris, eds.), pp. 163–188. Erlbaum, Assoc.Hillsdale, New Jersey.

Gardiner, J. M., Java, R. I., and Richardson-Klavehn,A. (1996). How level of processing reallyinfluences awareness in recognition memory. Can.J. Exp. Psychol. 50, 114–122.

Gibson, J. J. (1941). A critical review of the concept ofset in contemporary experimental psychology.Psychol. Bull. 38, 781–817.

Glisky, E. L., Polster, M. L., and Routhieaux, B. C.(1995). Double dissociation between item andsource memory. Neuropsychology 9, 229–235.

Henson, R. N. A., Rugg, M. D., Shallice, T., Josephs,O., and Dolan, R. J. (1999). Recollection and famil-iarity in recognition memory: An event-relatedfunctional magnetic resonance imaging study.J. Neurosci. 19, 3962–3972.



Hirshman, E., and Henzler, A. (1998). The role of deci-sion processes in conscious recollection. Psychol.Sci. 9, 61–65.

Hirshman, E., and Master, S. (1997). Modelling theconscious correlates of recognition memory:Reflections on the remember–know paradigm.Mem. Cogn. 25, 345–351.

Jacoby, L. L. (1998). Invariance in automatic influencesof memory: Toward a user’s guide for theprocess–dissociation procedure. J. Exp. Psychol.Learn. Mem. Cogn. 24, 3–26.

Jacoby, L. L., and Dallas, M. (1981). On the relation-ship between autobiographical memory and per-ceptual learning. J. Exp. Psychol. Gen. 3, 306–340.

Jacoby, L. L., and Kelley, C. (1992). Unconsciousinfluences of memory: Dissociations and auto-maticity. In “The Neuropsychology ofConsciousness” (A. D. Milner and M. D. Rugg,eds.), pp. 201–233. Academic Press, London.

Janowsky, J. S., Shimamura, A. P., and Squire, L. R.(1989). Source memory impairment in patientswith frontal lobe lesions. Neuropsychologia 27,1043–1056.

Johnson, R. (1995). Event-related potential insightsinto the neurobiology of memory systems. In “Handbook of Neuropsychology” (F. Boller and J. Grafman, eds.), pp. 135–164. Elsevier,Amsterdam.

Johnson, R., Pfefferbaum, A., and Kopell, B. S. (1985).P300 and long-term memory: Latency predictsrecognition performance. Psychophysiology 22,497–507.

Johnson, M. K., Kounios, J., and Nolde, S. F. (1996).Electrophysiological brain activity and memorysource monitoring. NeuroReport 8, 1317–1320.

Johnson, Jr., R., Kreiter, K., Russo, B., and Zhu, J.(1998). A spatio-temporal analysis of recognition-related event-related brain potentials. Int. J.Psychophysiol. 29, 83–104.

Josephs, O., Turner, R., and Friston, K. J. (1997). Event-related fMRI. Hum. Brain Mapping 5, 243–248.

Karis, D., Fabiani, M., and Donchin, E. (1984). P300and memory: Individual differences in the VonRestorff effect. Cogn. Psychol. 16, 177–216.

Kutas, M. (1988). Review of event-related potentialstudies of memory. In “Perspectives in MemoryResearch” (M.S. Gazzaniga, ed.), pp. 182–217. MITPress, Cambridge, Massachusetts.

Kutas, M., and Hillyard, S. A. (1980). Reading sense-less sentences: Brain potentials reflect semanticincongruity. Science 207, 203–205.

Mandler, G. (1980). Recognising: The judgment of pre-vious occurrence. Psychol. Rev. 87, 252–271.

Mangels, J. A., Picton, T. W., and Craik, F. I. M. (2001).Attention and successful episodic encoding: anevent-related potential study. Cogn. Brain Res. 11,77–95.

Mayes, A. R. (1988). “Human Organic Memory Dis-orders.” Cambridge University Press, Cambridge.

Mayes, A. R., and Roberts, N. (2001). Theories ofepisodic memory. Philos. Trans. Roy. Soc. (Lond.)356, 1395–1408.

McCarthy, G., and Wood, C. C. (1985). Scalp dis-tributions of event-related potentials: An am-biguity associated with analysis of variancemodels. Electroencephalogr. Clin. Neurophysiol. 62, 203–208.

Mecklinger, A. (1998). On the modularity of recogni-tion memory for object form and spatial location:A topographic ERP analysis. Neuropsychologia 36,441–460.

Mecklinger, A. (2000). Interfacing mind and brain: Aneurocognitive model of recognition memory.Psychophysiology 37, 565–582.

Mecklinger, A., and Meinshausen, R. (1998).Recognition memory for object forms and spatiallocations: An event-related potential study. Mem.Cogn. 26, 1068–1088.

Morcom, A., and Rugg, M. D. (2002). Getting ready toremember: The neural correlates of task set duringrecognition memory. Neuroreport 13, 149–152.

Morris, C. D., Bransford, J. D., and Franks, J. J. (1977).Levels of processing versus transfer appropriateprocessing. J. Verbal Learn. Verbal Behav. 16,519–533.

Moscovitch, M. (1992). Memory and working-with-memory: A component process model basedon modules and central systems. J. Cogn. Neurosci.4, 257–267.

Moscovitch, M. (1994). Models of consciousness andmemory. In “The Cognitive Neurosciences” (M.S.Gazzaniga, ed.), pp. 1341–1356. MIT Press,Cambridge, Massachusetts.

Neville, H. J., Kutas, M., Chesney, G., and Schmidt, A.L. (1986). Event-related brain potentials duringinitial encoding and recognition memory of con-gruous and incongruous words. J. Mem. Lang. 25,75–92.

Noldy, N. E., Stelmack, R. M., and Campbell, K. B.(1990). Event-related potentials and recognitionmemory for pictures and words: The effects ofintentional and incidental learning. Psycho-physiology 27, 417–428.

Otten, L. J., and Rugg, M. D. (2001a). Electro-physiological correlates of memory encoding aretask-dependent. Cogn. Brain Res. 12, 11–18.

Otten, L. J., and Rugg, M. D. (2001b). Task-depen-dency of the neural correlates of episodic encodingas measured by fMRI. Cereb. Cortex 11, 1150–1160.

Paller, K. A., and Kutas, M. (1992). Brain potentialsduring retrieval provide neurophysiologicalsupport for the distinction between consciousrecollection and priming. J. Cogn. Neurosci. 4,375–391.

Paller, K. A., Kutas, M., and Mayes, A. R. (1987).Neural correlates of encoding in an incidentallearning paradigm. Electroencephalogr. Clin.Neurophysiol. 67, 360–371.

REFERENCES 193


Paller, K. A., McCarthy, G., and Wood, C. C. (1988).ERPs predictive of subsequent recall and recogni-tion performance. Biol. Psychol. 26, 269–276.

Paller, K. A., Kutas, M., and McIsaac, H. K. (1995).Monitoring conscious recollection via the electricalactivity of the brain. Psychol. Sci. 6, 107–111.

Potter, D. D., Pickles, C. D., Roberts, R. C., and Rugg,M. D. (1992). The effects of scopolamine on event-related potentials in a continuous recognitionmemory task. Psychophysiology 29, 29–37.

Ranganath, C., and Paller, K. A. (1999). Frontal brainpotentials during recognition are modulated byrequirements to retrieve perceptual details. Neuron22, 605–613.

Ranganath, C., and Paller, K. A. (2000). Neural corre-lates of memory retrieval and evaluation. Cogn.Brain Res. 9, 209–222.

Richardson-Klavehn, A., and Bjork, R. A. (1988).Measures of memory. Annu. Rev. Psychol. 39,475–543.

Robb, W. G. K., and Rugg, M. D. (2002). Electro-physiological dissociation of retrieval orientationand retrieval effort. Psychonom. Bull. Rev. (inpress).

Rogers, R. D., and Monsell, S. (1995). Costs of a pre-dictable switch between simple cognitive tasks. J.Exp. Psychol. Gen. 124, 207–231.

Rugg, M. D. (1987). Dissociation of semantic priming,word and non-word repetition by event-relatedpotentials. Q. J. Exp. Psychol. 39A, 123–148.

Rugg, M. D. (1995). ERP studies of memory. In“Electrophysiology of Mind: Event-Related BrainPotentials and Cognition” (M. D. Rugg and M. G. H. Coles, eds.), pp. 132–170. Oxford Univ.Press, Oxford.

Rugg, M. D. (1998). Functional neuroimaging in cognitive neuroscience. In “The Neurocognition of Language” (C.M. Brown and P. Hagoort, eds.), pp. 15–36. Oxford University Press, Oxford.

Rugg, M. D., and Allan, K. (2000). Event-relatedpotential studies of Memory. In “The OxfordHandbook of memory” (E. Tulving and F. I. M. Craik, eds.), pp. 521–537. OxfordUniversity Press, Oxford.

Rugg, M. D., and Coles, M. G. H. (1995). The ERP andcognitive psychology: Conceptual issues. In“Electrophysiology of Mind: Event-Related BrainPotentials and Cognition” (M. D. Rugg and M. G.H. Coles, eds.), pp. 27–39. Oxford University Press,Oxford.

Rugg, M. D., and Doyle, M. C. (1992). Event-relatedpotentials and recognition memory for low- andhigh-frequency words. J. Cogn. Neurosci. 4, 69–79.

Rugg, M. D., and Doyle, M. C. (1994). Event-relatedpotentials and stimulus repetition in direct andindirect tests of memory. In “CognitiveElectrophysiology” (H. J. Heinze, T. Munte, and G.R. Mangun, eds.), pp. 124–148. Birkhauser, Boston.

Rugg, M. D., and Henson, R. N. A. (2002). Episodicmemory retrieval: An (event-related) functionalneuroimaging perspective. In “The CognitiveNeuroscience of Memory: Episodic Encoding andRetrieval” (A. E. Parker, E. L. Wilding, and T. J.Bussey, eds.). Psychology Press, Hove.

Rugg, M. D., and Nagy, M. E. (1989). Event-relatedpotentials and recognition memory for words.Electroencephalogr. Clin. Neurophysiol. 72, 395–406.

Rugg, M. D., and Wilding, E. L. (2000). Retrieval pro-cessing and episodic memory. Trends Cogn. Sci. 4,108–115.

Rugg, M. D., Furda, J., and Lorist, M. (1988). Theeffects of task on the modulation of event-relatedpotentials by word repetition. Psychophysiology 25,55–63.

Rugg, M. D., Brovedani, P., and Doyle, M. C. (1992).Modulation of event-related potentials by word repetition in a task with inconsistentmapping between repetition and response.Electroencephalogr. Clin. Neurophysiol. 84, 521–531.

Rugg, M. D., Cox, C. J. C., Doyle, M. C., and Wells, T.(1995). Event-related potentials and the recollec-tion of low and high frequency words.Neuropsychologia 33, 471–484.

Rugg, M. D., Schloerscheidt, A. M., Doyle, M. C., Cox,C. J. C., and Patching, G. R. (1996). Event-relatedpotentials and the recollection of associative infor-mation. Cogn. Brain Res. 4, 297–304.

Rugg, M. D., Mark, R. E., Walla, P., Schloerscheidt, A.M., Birch, C. S., and Allan, K. (1998). Dissociationof the neural correlates of implicit and explicitmemory. Nature 392, 595–598.

Rugg, M. D., Allan, K., and Birch, C. S. (2000).Electrophysiological evidence for the modulationof retrieval orientation by depth of study process-ing. J. Cogn. Neurosci. 12, 664–678.

Sanquist, T. F., Rohrbaugh, J. W., Syndulko, K., andLindsley, D. B. (1980). Electrocortical signs of levels of processing: Perceptual analysis and recognition memory. Psychophysiology 17, 568–576.

Schacter, D. L., Harbluk, J. L., and McLachlan, D. R.(1984). Retrieval without recollection: An experi-mental analysis of source amnesia. J. Verbal Learn.Verbal Behav. 23, 593–611.

Scoville, W. B., and Milner, B. (1957). Loss of recentmemory after bilateral hippocampal lesions. J.Neurol. Neurosurg. Psychiatr. 20, 11–21.

Senkfor, A. J., and Van Petten, C., V. P. (1998). Whosaid what: An event-related potential investigationof source and item memory. J. Exp. Psychol. Learn.Mem. Cogn. 24, 1005–1025.

Shallice, T., and Burgess, P. (1998). The domain ofsupervisory processes and the temporal organisa-tion of behaviour. In “The Prefrontal Cortex:Executive and Cognitive Functions” (A. C. Roberts,



T. W. Robbins, and L. Weiskrantz, eds.), pp. 22–35.Oxford University Press, Oxford.

Shimamura, A. P., and Squire, L. R. (1987). A neuro-psychological study of fact memory and sourceamnesia. J. Exp. Psychol. Learn. Mem. Cogn. 13,464–473.

Smith, M. E. (1993). Neurophysiological manifesta-tions of recollective experience during recognitionmemory judgements. J. Cogn. Neurosci. 5, 1–13.

Smith, M. E., and Guster, K. (1993). Decomposition ofrecognition memory event-related potentialsyields target, repetition, and retrieval effects.Electroencephalogr. Clin. Neurophysiol. 86, 335–343.

Smith, M. E., and Halgren, E. (1989). Dissociation ofrecognition memory components following tem-poral lobe lesions. J. Exp. Psychol. Learn. Mem.Cogn. 15, 50–60.

Snodgrass, J. G., and Corwin, J. (1988). Pragmatics ofmeasuring recognition memory: Applications todementia and amnesia. J. Exp. Psychol. Gen. 117,34–50.

Sommer, W., Schweinberger, S. R., and Matt, J. (1991).Human brain potential correlates of face encodinginto memory. Electroencephalogr. Clin. Neurophysiol.79 (6), 457–463.

Sommer, W., Heinz, A., Leuthold, H., Matt, J., andSchweinberger, S. R. (1995). Metamemory, distinc-tiveness, and event-related potentials in recogni-tion memory for faces. Mem. Cogn. 23, 1–11.

Sommer, W., Komoss, E., and Schweinberger, S. R.(1997). Differential localisation of brain systemssubserving memory for names and faces in normalsubjects with event-related potentials. Electro-encephalogr. Clin. Neurophysiol. 102, 192–199.

Squire, L. R., Knowlton, B., and Musen, G. (1993). Thestructure and organization of memory. Annu. Rev.Psychol. 44, 453–495.

Sternberg, S. (1966). High-speed scanning in humanmemory. Science 153, 652–654.

Stuss, D. T., Eskes, G. A., and Foster, J. K. (1994).Experimental neuropsychological studies offrontal lobe functions. In “Handbook ofNeuropsychology, Volume 10” (F. Boller and J. Grafman, eds.), pp. 149–183. Elsevier, New York.

Tendolkar, I., and Rugg, M. D. (1998). Electro-physiological dissociation of recency and recog-nition memory. Neuropsychologia 36, 477–490.

Tendolkar, I., Schoenfeld, A., Golz, G., Fernandez, G.,Kuhl, K.-P., Ferszt, R., and Heinze, H.-J. (1999).Neural correlates of recognition memory with andwithout recollection in patients with Alzheimer’sdisease and healthy controls. Neurosci. Lett. 263,45–48.

Toth, J. P. (1996). Conceptual automaticity in recogni-tion memory: Levels of processing effects on famil-iarity. Can. J. Exp. Psychol. 50, 123–128.

Trott, C. T., Friedman, D., Ritter, W., and Fabiani, M.(1997). Item and source memory: Differential age

effects revealed by event-related potentials.NeuroReport 8, 3373–3378.

Tsivilis, D., Otten, J., and Rugg, M. D. (2001). Contexteffects on the neural correlates of recognitionmemory: An electrophysiological study. Neuron31, 1–20.

Tulving, E. (1983). “Elements of Episodic Memory.”Oxford University Press, Oxford.

Tulving, E. (1985). Memory and Consciousness. Can.Psychol. 26, 1–12.

Tulving, E., and Thomson, D.M. (1973). Encodingspecificity and retrieval processes in episodicmemory. Psychol. Rev. 80, 353–373.

Van Petten, C., and Senkfor, A. J. (1996). Memory forwords and novel visual patterns: Repetition, recog-nition, and encoding effects in the event-relatedbrain potential. Psychophysiology 33, 491–506.

Van Petten, C., Kutas, M., Kluender, R., Mitchiner, M.,and McIsaac, H. (1991). Fractionating the wordrepetition effect with event-related potentials.J. Cogn. Neurosci. 3, 129–150.

Van Petten, C., Senkfor, A. J., and Newberg, W. M.(2000). Memory for drawings in locations: Spatialsource memory and event-related potentials.Psychophysiology 4, 551–564.

Wagner, A. D. (1999). Working memory contributionsto human learning and remembering [Review].Neuron 22, 19–22.

Wagner, A. D., Koutstaal, W., and Schachter, D. L.(1999). When encoding yields remembering:insights from event-related neuroimaging. Philos.Trans. Roy. Soc. (Lond.) 354, 1307–1324.

Wilding, E. L. (1999). Separating retrieval strategiesfrom retrieval success: An event-related potentialstudy of source memory. Neuropsychologia 37,441–454.

Wilding, E. L. (2000). In what way does the parietalERP old/new effect index recollection? Int. J.Psychophysiol. 35, 81–87.

Wilding, E. L. (2001). Event-related functional imagingand episodic memory. Neurosci. Biobehav. Rev. 25,545–554.

Wilding, E. L., and Nobre, A. C. (2001). Task-switching and memory-retrieval processing: Electro-physiological evidence. NeuroReport 12, 3613–3617.

Wilding, E. L., and Rugg, M. D. (1996). An event-related potential study of recognition memorywith and without retrieval of source. Brain 119,889–905.

Wilding, E. L., and Rugg, M. D. (1997a). An event-related potential study of memory for spoken and heard words. Neuropsychologia 35, 1185–1195.

Wilding, E. L., and Rugg, M. D. (1997b). Event-relatedpotentials and the recognition memory exclusiontask. Neuropsychologia 35, 119–128.

Wilding, E. L., Doyle, M. C., and Rugg, M. D. (1995).Recognition memory with and without retrievalof study context: An event-related potential study.Neuropsychologia 33, 743–767.

REFERENCES 195


Yonelinas, A. P. (1994). Receiver-operating characteris-tics in recognition memory: Evidence for a dual-process model. J. Exp. Psychol. Learn. Mem. Cogn.20, 1341–1354.

Yonelinas, A. P. (1998). Recollection and familiarity in recognition memory: Con-vergence of remember/know, process dis-sociation and ROC data. Neuropsychology 12,323–329.

Yonelinas, A. P., and Jacoby, L. L. (1994). Dissociationsof processes in recognition memory: Effects of

interference and of response speed. Can. J. Exp.Psychol. 48, 516–534.

Yonelinas, A. P., and Jacoby, L. L. (1995). The relation-ship between remembering and knowing as basesfor recognition: Effects of size congruency. J. Mem.Lang. 34, 622–643.

Yonelinas, A. P., Dobbins, I., Szymanski, M. D.,Dhaliwal, H. S., and King, L. (1996). Signal-detection, threshold, and dual-process models of recognition memory: ROCs and consciousrecollection. Conscious. Cogn. 5, 418–441.




C H A P T E R

8

Self-Regulation and theExecutive Functions:

Electrophysiological CluesPhan Luu and Don M. Tucker

INTRODUCTION

Cognitive models typically describeexecutive functions as higher level pro-cesses that exert control over elementarymental operations (Norman and Shallice,1986). Executive control includes super-visory functions that must be engaged insituations in which well-learned behavioris inadequate. These situations includedemands for planning, error correction,execution of novel actions, inhibition ofroutinized behavior, and alerting todanger. When executive functions areimpaired, such as by frontal lobe lesions,the patient’s actions may be guided bydemand characteristics of the environmentwithout regard for social or practicalappropriateness (Eslinger and Damasio,1985; Lehrmitte, 1986; Lehrmitte et al.,1986).

As recognized in the cognitive analysisof attention, concepts of executive func-tions run the risk of invoking a homuncu-lus (Posner, 1978). The requirement for anexternal or supervisory control of informa-tion processing naturally leads to theassumption of an external agent that mustdirect the cognitive traffic. Because execu-tive deficits are particularly apparent withfrontal lesions, it has been natural for clini-

cians and researchers to assume, tacitly ofcourse, that the homunculus resides in thefrontal lobe.

A more attractive scientific model wouldexplain how executive psychological oper-ations might emerge from more elementaryadaptive mechanisms. Since the middle ofthe twentieth century, neuroanatomicalstudies have provided important clues tosuch mechanisms, showing dense connec-tions of the primate frontal lobe with thevisceral and emotional functions of limbiccortex (MacLean and Pribram, 1953;Pribram and MacLean, 1953). Developingthe anatomical evidence further, Nauta(1971) emphasized that the frontal lobe’sconnectivity with limbic circuits impliesprimary roles for emotion and motivationin self-regulation. For Nauta, the frontallobe lesion syndromes reflect the lack ofcontrol by affective, subjective evaluationin action planning. Even though a patientwith a prefrontal lobe lesion can accuratelydescribe the environmental situation,behavior occurs as if there is no evaluativecontrol (Nauta’s limbic “set points”). Inaddition to the evaluation of action plans,the limbic set points provide ongoing mon-itoring in order for the plan to be main-tained in relation to adaptive goals.Nauta’s reasoning has been influential onmodern concepts, and it remains consistent

with the more recent emphasis on affectivecues in frontal self-regulation by Damasioet al. (1990).

In this chapter, we begin with theassumption that an adequate scientifictheory of human self-regulation mustexplain how higher executive functionsemerge from more elementary mechanismsof learning. These mechanisms mustachieve the adaptive control of actions incontexts. We review both hemodynamicand electrophysiological evidence suggest-ing that activity in regions of limbic cortex,particularly the cingulate gyrus, may beintegral to the frontal executive functions.Neurophysiological studies of learning andmemory in animals have provided impor-tant models of the corticolimbic and corti-cothalamic mechanisms of self-regulation.Findings of basal ganglia mechanisms arealso instructive for understanding motiva-tional control of action, and we attempt topoint out how these might be relevant tocorticolimbic and corticothalamic mecha-nisms. We review several interesting cluesthat scalp electrophysiological measures inhumans could be related to the essentiallimbic, striatal, and thalamic circuits. Oneof the clues is the oscillatory nature of cor-ticolimbic activity that appears to underlieconventional electrophysiological mea-sures (such as the error-related negativity,N2, and P300). By considering current elec-trophysiological findings in humans inrelation to neurophysiological findings inanimal research, we propose that there areimportant new perspectives to be gainedon the intrinsic mechanisms of human self-regulation.

ANTERIOR CINGULATE CORTEX AND THE

EXECUTIVE FUNCTIONS

Broca identified the limbic lobe at thecore of the cerebral hemisphere. In addi-tion to the medial temporal cortex, thelimbic lobe is composed of the cingulate

gyri, bordering the corpus callosum on themedial surface of each hemisphere and setoff from the callosum by the callosalsulcus. Historically, the cingulate cortexwas also referred to as the gyrus fornicatus,emphasizing its anatomical relation withthe fornix. From its initial identification,the cingulate’s relation to the primitivelimbic regions was a subject of debate, witha number of early workers suggesting thatit belonged to the olfactory system. Withthe advent of more precise methods forstudying anatomical connectivity andcytoarchitecture, it is now clear that theentire cingulate gyrus was derived fromthe archicortex (i.e., the hippocampus) andnot the olfactory cortex (Pandya et al., 1988;Sanides, 1970). This relation with thehippocampus explains the dense connec-tions of the cingulate with the hippocam-pus and other dorsal cortical structuresderived from archicortex.

Prior to functional studies, little wasknown about the functional significance ofthe cingulate cortex. However, there weremany speculations based on its anatomicalconnections and its comparatively oldanatomical position. As a result of its inclu-sion with olfactory structures, it wasbelieved to be involved in olfactory func-tion, which was considered to be unimpor-tant in humans (MacLean, 1993). Butstudies by Ward (1948a,b) demonstratedthat the anterior cingulate cortex (ACC)was important to autonomic and motorfunctions. This is clearly demonstrated inpatients with lesions to the ACC, whoshow a host of symptoms, includingapathy, inattention, dysregulation of auto-nomic functions, akinetic mutism, andemotional instability (Barris and Schuman,1953).

Modern cognitive neuroimaging methodshave allowed researchers to study the rela-tion between executive functions and theACC. Hemodynamic studies identifiedACC activation in tasks that required sub-jects to make responses that are eithernovel or that required them to overcome

200 8. SELF-REGULATION AND THE EXECUTIVE FUNCTIONS


competing responses (George et al., 1994;Pardo, et al., 1990; Posner et al., 1988).These results were interpreted to reflect therole of the ACC in attention, particularlyattention for action (Posner and Dehaene,1994). According to this view, the ACCprovides top-down control to resolvecompeting neural activations (Posner andDiGirolamo, 1998). More recent theories ofthe ACC state that rather than strategicallyimplementing attention allocation-likefunctions, the ACC is involved in evaluat-ing conflicting demands (i.e., competingneural activation) (Carter et al., 1998, 1999).According to this theory, the ACC signalswhen executive control, i.e., functions con-trolled by the dorsolateral prefrontal cortex(MacDonald et al., 2000), is required.

It is well known that the ACC is not ahomogeneous structure, both in terms ofcytoarchitectonics and function (Devinskyet al., 1995; Vogt et al., 1993a). A review byBush et al. (2000) noted that cognitive taskstend to activate the dorsal ACC and deacti-vate the rostroventral ACC. Conversely,emotional tasks activate the rostroventralACC and deactivate the dorsal ACC. Thesetwo subdivisions of the ACC appear to bewarranted on neuroanatomical grounds aswell (Pandya et al., 1981; Vogt et al., 1987).The findings that both cognitive and emo-tional tasks activate different regions of theACC have proved to be difficult to recon-cile for pure cognitive theories of executivefunctions and the ACC. However, they areconsistent with theories that emphasize thecentral role of affect in self-regulation(Nauta, 1971), and this integration of emo-tions with cognition in the ACC is now rec-ognized to be important (Allman et al.,2001; Paus, 2001).

Executive Control and Action Monitoring

Another aspect of executive function isthe ability to integrate negative feedback,that is, to monitor for the occurrence oferrors and to adjust behavior accordingly.

This self-regulatory function is so impor-tant to adaptive behavior that failure toappreciate the significance of mistakesoften leads to the inability to manage one’slife, as is often observed in frontal lobepatients (Eslinger and Damasio, 1985;Rylander, 1947). Frontal lobe patients areoften capable of recognizing errors.However, as described in classic studies(Rylander, 1947), they are often remarkablyuntroubled by their mistakes.

Using scalp electrophysiological methodsin normals, two research groups ident-ified an event-related potential (ERP) thatappears to index the recognition of erro-neous responses (Falkenstein et al., 1991;Gehring et al., 1993). This ERP component,termed the error negativity (Ne) or the error-related negativity (ERN), has a negativedistribution over mediofrontal scalp sitesand it peaks approximately 100 msec afterresponse onset (see Fig. 1). The scalp distri-bution of the ERN suggests a mediofrontalneural generator. Dehaene et al. (1994)source analyzed the ERN with dense-arrayelectroencephalogram (EEG) data andfound a generator that lies in the region ofthe ACC. This result has been corroboratedin subsequent studies (Holroyd et al., 1998;Luu et al., 2000a; Miltner et al., 1997) and isconsistent with results of error-relatedactivity recorded in the ACC in monkeys(Gemba et al., 1986; Niki and Watanabe,1979).

The ERN is seen after hand (Falkensteinet al., 1991), feet (Holroyd et al., 1998),vocal (Masaki et al., 2001), and saccadic eyemovement errors (Van’t Ent and Apkarian,1999), findings that are consistent with an output-independent system of actionmonitoring. The ERN was originallythought to index the activity of an errordetection system that compared the actualresponse with an internal representation ofthe correct response (Falkenstein et al.,1991; Gehring et al., 1993; Scheffers et al.,1996). According to this proposal, ERNamplitude should be largest when theerror response is substantially different

ANTERIOR CINGULATE CORTEX AND THE EXECUTIVE FUNCTIONS 201


from the correct response, and this wasconfirmed in a study by Bernstein et al.(1995). Although some have proposed thatconscious awareness of having made anerror is required for the manifestation ofthe ERN (Dehaene et al., 1994; Luu et al.,2000b), a study using an antisaccade testindicates that this may not be the case(Nieuwenhuis et al., 2001). That is, the com-parison of an error response with theexpected response need not reach the levelof awareness to elicit an ERN.

Recordings of error potentials in theACC of the monkey support the errordetection view of the ERN. Gemba et al.(1986) found that error-related potentialsare not observed in the ACC prior to theanimal evincing behavioral evidence thatindicates that insight into the nature of thetask (i.e., what constitutes a correctresponse) has been grasped. It should benoted that the ERN appears to be unrelatedto other error-related processes, such asinhibiting or correcting the erroneousresponse. For example, the ERN still occursin situations in which errors cannot be cor-rected, such as pushing a button when a

response is not required (Scheffers et al.,1996) or when the response is erroneous bya speed criterion (Luu et al., 2000b).

This error detection view of the ERN hasbeen questioned because of severalfindings. First, mediofrontal negativitiescan be found after correct responses. Thesmaller negativity associated with correctresponses has been labeled the correct-related negativity (CRN) by Ford (1999).Ford noted that the ERN and CRN differedin their topographic distribution, but this isin contrast to what was found by Vidalet al.(2000) and Luu et al. (2000b), who useddense-array EEGs to compare the scalp dis-tribution of the CRN and ERN. Scheffersand Coles (2000) have found that the ERNcan be observed after correct responses ifsubjects believe that a response is in error.This finding is consistent with an errordetection view of the ERN and has beenused by Coles et al. (2001) to explain whyERN-like negativities are observed aftercorrect responses. Another hypothesis putforth is that the CRN reflects the monitor-ing process and that the ERN reflects anadditional signal, in response to errors, that



FIGURE 1 Response-locked ERP wave form for correct and error responses at the Fcz electrode (left). Thescalp distribution of the ERN at its maximum is shown on the right.

is overlaid on top of the CRN (Falkensteinet al., 2000).

Second, it has been suggested that theERN, rather than indexing the output of anerror detection system, reflects the conflict-monitoring responses of the ACC (Carteret al., 1998). According to this theory, errorsare instances in which response conflict ishigh. The conflict-monitoring hypothesis isattractive in that it also explains ACC activ-ity in a host of other tasks that requireexecutive control. Using functional mag-netic resonance imaging (fMRI) methods,Carter et al. found the ACC to be active forerror responses and for correct responsesthat had high response competitiondemands. This proposal has stimulated anumber of studies because it is clear andtestable. Falkenstein et al. (2000) found theERN amplitude to be of equivalent size fortasks that have strong response conflictand those that have no response conflict.Similarly, when responses are sortedaccording to degree of response conflict, asmeasured by activation of motor cortices,the ERN was largest for errors with thewrong response hand, which had little orno response conflict (Luu et al., 2000b). Incontrast, Gehring and Fencsik (2001) con-ducted a study in which they found errorsthat were similar to the correct response interms of motor output produced largerERN amplitudes than those that were dis-similar. They interpreted these findings tobe consistent with the conflicting monitor-ing theory of the ACC. Other fMRI studieshave found activity in the rostroventralACC to be specific to errors and the dorsalACC to be common to both errors andresponse conflict manipulation (Kiehl et al.,2000; Menon et al., 1997). Thus, ERNresponses may be influenced by conflictingresponse demands in some experimentsand not others, and there may be regionaldifferentiation within the ACC in theconflict-monitoring function.

One intriguing property of the ERN isthat it can be elicited by a feedback inform-ing subjects of the accuracy of their

response. In time-interval tasks, subjectsare required to produce a response withina certain time interval. In these tasks, acorrect response cannot immediately bedetermined because a correct response isnot defined a priori by an internal repre-sentation of the action. Therefore, the ERNis not observed until a feedback is pre-sented to the subject (Badgaiyan andPosner, 1998; Miltner et al., 1997). Thisobservation has proved difficult for theconflict-monitoring theory to reconcile, butit suggests interesting relations to affectivereactions in response to errors (Tuckeret al., 1999). Tucker et al. observed thatduring the time window of the late-positive complex (LPC, approximately472 msec) to targets, a medial prefrontalnegativity differentiated between goodand bad targets; bad targets elicited alarger negativity than did good targets.This “evaluative negativity” effect wasfound to be superposed over the topogra-phy of the LPC. The time course of thiseffect and its superposition with the LPCis very reminiscent of the stimulus-lockederror-related negativity (Falkenstein et al.,1991). Moreover, this effect was againobserved when a feedback was presentedto subjects informing them of their perfor-mance; a negative feedback elicited largermediofrontal negativity than did a positivefeedback. The results from this studysuggest that the observed evaluativenegativity is separable from the responsebecause the effect was observed inresponse to the target and the feedback.

Luu et al. (2000a) proposed that evalua-tion of action occurs along the affectivedimension of distress. These authors foundthat subjects who are high on the dimen-sion of personality known as negativeemotionality (Tellegen and Waller, 1996),which describes the tendency to experi-ence subjective distress, should show exag-gerated ERN responses because of theiraffective reactions to errors. Indeed, earlyin the experiment subjects with high nega-tive emotionality subjects displayed larger



ERN amplitudes compared to subjects withlow negative emotionality. Anxiety disor-ders, such as obsessive–compulsive disorder(OCD), are related to negative emotionalitythrough the dimension of subjective distress.Gehring et al. (2000) found that ERN ampli-tudes were larger for OCD subjects in com-parison to controls. Moreover, the ERNamplitude was correlated with symptomseverity. In a similar study, Johannes et al.(2001b) confirmed these findings. In addi-tion, these researchers found latency andtopographic differences of the ERN for OCDsubjects when compared to controls. At thelow end of the distress dimension are thosepeople who are not prone to feeling anxious.Dikman and Allen (2000) found that subjectswho are low on a scale measuring sociability(which indicates low levels of anxiety)exhibited small ERN amplitudes when theirerror responses were punished. A moredirect test of the relation between distressand ERN amplitude was demonstrated byJohannes et al. (2001a). These researchersadministered oxazepam and found that theamplitude of the ERN was reduced in thosesubjects who received this anxiolytic drugcompared to those subjects who received aplacebo. This reduction of the ERN with ananxiolytic is thus consistent with increasedERNs in subjects high in negative affect(Luu et al., 2000a).

Theta Dynamics and the ERN

Examining the morphology of the ERNwave form, researchers have noted that theERN is often preceded and followed byother negative deflections. Taking this as a

clue that the ERN is actually part of anongoing oscillatory process, Luu and Tucker(2001) found that the ERN emerges as onecomponent of a midline oscillation whoseactivity is at the theta frequency. Moreover,this midline theta oscillation was interposedwith sensorimotor oscillations, which arealso at the theta frequency. Figure 2 showsthe series of events recorded in a typical(Eriksen flanker task) ERN paradigm (Luuand Tucker, 2001). At approximately 80 msecprior to the button press, there is a negativepotential along the midline that occurs againafter a response (at approximately 88 msec).When this negativity occurs after an erro-neous button press, it is much larger com-pared to a correct button press and isrecognized as the ERN. Prior to the buttonpress (at approximately –32 msec), a nega-tive potential is observed over the contralat-eral sensorimotor recording sites and it isobserved again at approximately 136 msecafter the button press. When an error is com-mitted, the ERN becomes correlated withthese sensorimotor potentials.

Applying independent componentsanalysis (ICA) (Makeig et al., 1997) to datafrom individual subjects, Makeig et al.(2002a) discovered that the ERN is actuallycomposed of multiple independent compo-nents with maximal power at the theta fre-quency. The trial-by-trial phase resettings ofthe ongoing theta oscillation sum up toprovide the averaged deflection recognizedas the ERN. The independent componentsdemonstrate interesting relations to the dif-ferent events that occur in the task. Figure 3shows the ERP of two independent vertexand frontal components scaled to microvolt



FIGURE 2 Voltage maps showing the oscillatory nature of the ERN and sensorimotor potentials. Time is rela-tive to a button press, which is at 0 msec. (See color plates.)

levels at channel Fz, aligned with thebutton press (solid black line), and sortedaccording to response latency. The figureon the left shows that the vertex theta com-ponent is aligned to the target onset (firstdashed line to the left of response onset)and peaks after the response deadline (firstdashed line to the right of response onset).This postresponse deadline peak is largerfor error than for correct responses.Additionally, after the feedback, this vertextheta component again shows a burst ofactivity in the theta range. The frontal thetacomponent begins to show thetalike activ-ity prior to the response and peaks justafter the response, but it does not seem toshow activity related to the target or feed-back stimuli (see Fig. 3).

In another study, Luu et al. (2002) sepa-rated the feedback and response-lockedcomponents of the ERN by using a delayed-feedback paradigm. Prior to the presentationof a target stimulus, subjects received a feed-back on their performance from five trialsprevious to the current trial. This was doneto separate the response-control value of the

feedback from the emotional value of thefeedback as a performance indicator. Thegrand-averaged data were then analyzedusing a theta source model derived from thework of Asada et al. (1999). Luu et al. foundthat the ERN was made up of two compo-nents, one component was located in therostral ACC and the other was located in thedorsal ACC (see Fig. 4). The locations ofthese two sources are similar to regionsreported, using fMRI methods, to be activewhen errors are committed (Kiehl et al., 2000;Menon et al., 2001). Luu et al. found therostral ACC source to be locked to theresponse whereas the dorsal source waslocked to the feedback (see Fig. 4). Theseresults are similar to those obtained byMakeig et al. (2002a) using ICA, and theysuggest there may be regional specializationswithin the ACC that could help explainseveral findings in the ERN literature.

First, these regional specializationscould explain why an ERN can be observedin both stimulus-locked and responselocked averages; the ventral region may bemore response-locked and the dorsal



FIGURE 3 ERP of the vertex and frontal component scaled to microvolt levels at channel Fz. Left: Vertex thetacomponent; right: frontal theta component. Plots of trials aligned to button press (solid black line). Each rasterline represents a trial, and trials are sorted according to response latency. The data were filtered with a 4-Hz highpass. First dashed line to the left of the response marks target onset. First dashed line to the right of the buttonpress marks response deadline, and second dashed line after button press marks feedback onset.

region may contribute to both stimulus-and response-locked measures. Second, thisevidence could explain why there are ERNand ERN-like deflections in response to afeedback stimulus when no overt motorresponses are made; the more dorsal regionof the ACC may provide not only responsemonitoring but also event monitoringgenerally. Third, this evidence couldexplain how current source densitymethods, which are sensitive to superficialsources, can identify a dorsal ACC (e.g.,supplementary motor area or SMA) sourceof the ERN (Vidal et al., 2000) when otherERN models have identified deep ACCsources (Dehaene et al., 1994); there areindeed multiple sources, with differentfunctional properties.

These functional properties of differentERN sources can be ascertained by consid-ering their alignment with events in theexperimental trials. In action monitoring,the execution and monitoring of theresponse can be separated from the moni-toring of the context in which the action isexecuted (i.e., the context parameters thatconstrain the action). The source analysisstudy of Luu et al. (2002) and the ICAresults reported by Makeig et al. (2002a)

suggest that the dorsal ACC (SMA region)monitors the context of the action (such asthe target, response deadline, feedbackvalue). In contrast, the rostroventral ACCmonitors the response.

A similar differentiation may have beenobserved in positron emission tomography(PET) research on decision-making. Elliotand Dolan (1998) found that the dorsalACC was active when subjects generated ahypothesis about what would constitute acorrect response. In contrast, the ventralACC was active when subjects made achoice. Elliot and Dolan proposed that theventral ACC activation reflected emotionalevaluation of the action. In action monitor-ing, both regions of the ACC must be func-tionally coupled, and it appears that thisoccurs through phase coupling at the thetafrequency. Luu et al., report that the ERNsources are at the theta frequency and areapproximately 60° out of phase, which issimilar to the phase relation of thetasources in the ACC reported in magneto-encephalography (MEG) research byAsada et al. (1999).

Thus both spatial and temporal resolu-tion may be necessary in researching themechanisms of self-regulation in medial



FIGURE 4 Left: Location of the generators of the ERN. Right: Source wave forms illustrating the relative con-tribution of each source to the scalp-recorded ERN. The box indicates the window of the ERN. (See color plates.)

frontal networks. Functional differentiationbetween nearby networks of the ACC issuggested by hemodynamic studies and bydense-array EEG studies. Temporal differ-entiation of the activity in these regions issuggested by the MEG and EEG evidence,and of course this is not apparent withhemodynamic measures. The findings ofoscillatory activity contributing to the ERNand feedback-related responses may allowinterpretation of the human findings inrelation to the intriguing animal literatureon theta dynamics that appear to coordi-nate learning and neural plasticity withinmultiple, functionally distinct corticolimbicnetworks. In the next section, we reviewmodels of self-regulation from neurophysi-ological studies that may help explain howhuman corticolimbic networks are inte-grated in motivated behavior. These neuro-physiological models of learning andaction regulation may provide new waysof thinking about a number of ERP mea-sures in human research, including the no-go N2, the P3a, the P3b, and the ERN.Whereas these measures have convention-ally been interpreted in a cognitive or men-talistic framework in relation to cognitiveoperations such as attention and memory,the neurophysiological evidence points tomore elementary operations such asarousal, alerting, and orienting. By under-standing how executive functions recruitsubcortical as well as cortical systems, itmay be possible to frame concepts of self-regulation in relation to fundamentalcontrol processes that evolved for learningand action regulation.

MECHANISMS OF ACTION REGULATION

Dopamine and Errors of Prediction

One influential model has followedfrom the remarkable specificity ofresponses in the dopamine pathway to vio-lations of expectancy for reward cues

(Schultz et al., 1995, 1998). Although manyresearchers have assumed that dopamineprojections are important to mediatingresponses to reward, Shultz and associatesfound that the dopamine response wasmore specific than a diffuse rewardresponse. A property of the dopaminergicventral tegmental area (VTA) cells is thattheir response can be transferred from aprimary reward to a conditioned stimulusafter learning. Moreover, after learning, ifthe expected reward is withheld, the VTAcells show a phasic depression relative totheir base line activity. Such an “errorsignal” is particularly attractive for theo-reticians interested in learning mecha-nisms because it is an integral componentof artificial neural network models oflearning, particularly those that requiresupervisory training to guide the learningprocess.

Integrating the Schultz evidence ofdopamine coding of prediction errors withthe human evidence of error-related nega-tivities in the anterior cingulate cortex,Holroyd and Coles (2002) have recentlyproposed that the ERN reflects the trans-mission of an error signal from brain stem,dopaminergic cells to the ACC, and theACC in turn uses this signal to selectwhich motor program has access to themotor system. The Holroyd and Colesmodel is particularly important because itoffers the opportunity to link the animallearning literature with the ACC findingsin relation to human frontal lobe function,thus providing a model of the exec-utive functions that arises not from a sepa-rate, homuncular agent, but from moreintrinsic and elementary mechanisms ofself-regulation.

Although the Schultz model has beenhighly influential in linking motivationalprocesses to striatal and dopamineinfluences on action control, Redgraveet al. (1999) have questioned whether theresponse of the dopamine system may betoo fast to integrate reward informationwith the response to prediction errors.

MECHANISMS OF ACTION REGULATION 207


Rather, Redgrave et al. suggest that there isa more elementary role of the dopamineprojections, to facilitate neostriatal behav-ioral switching operations in response tosignificant events. Others have also pro-posed a similar view of dopaminergicfunction (Spanagel and Weiss, 1999).Although the dopamine modulation isunderstood primarily in the neostriatum,this might be extended to dopaminetargets in limbic cortex, such as the ACC.The finding that patients with Parkinson’sdisease exhibit smaller ERN amplitudes(Falkenstein et al., 2001) is particularlyrelevant within this context. Gurney et al.,(2001) have argued that the circuits withinthe neostriatum may be differentiated,with certain neural populations specializedfor behavioral switching functions andothers for the control of those switchingfunctions. This sort of mechanism wouldbe well suited to motivational control ofaction selection.

Although the neostriatal and dopamin-ergic influences on the ACC are only begin-

ning to be understood, they are providingimportant theoretical models for under-standing the executive control of action.Because neostriatal loops to both limbiccortex and neocortex are closely coordi-nated with thalamolimbic circuits, a theo-retical analysis of action regulation mustextend to the limbic and thalamic mecha-nisms that are integral to memory forma-tion and the control of learned behavior.

Adaptation and Context in the Papez Circuit

A novel and intriguing model of limbicand thalamic learning mechanisms hasbeen developed by Gabriel and colleagues(Gabriel, 1990; Gabriel et al., 1986, 2002).They have argued that the ACC is involvedin associative attention, a conclusion basedon a series of neurophysiological studies ofavoidant and approach discriminant learn-ing in rabbits. Unlike cognitive models ofattention, the associative attention modelof Gabriel and colleagues proposes that



FIGURE 5 A model of the action regulation system based on Gabriel’s model of discriminant learning (see textfor details). The amygdala is presented in this model because it is critical to discriminant avoidance learning inthe ACC and to a lesser extent in the PCC.

attention is a product of learning mecha-nisms. This model can thus account for atleast certain attentional phenomenawithout recourse to an external controlagent.

Gabriel et al. have considered the rolesof both limbic and thalamic components ofthe Papez circuit in discrimination learning(see Fig. 5). The anterior and medial nucleiof the thalamus are central to discrimi-nating between predictive stimuli and nonpredictive conditioned stimuli. Thesestructures then contribute to the discrimi-nant activity of unit responses in the cingu-late cortex. The ACC is particularlyimportant to the early stages of discrimina-tion learning, when rapid encoding ofnovel reinforcement contingencies is criti-cal to adaptive action. A key fact forunderstanding ventral as well as dorsallimbic inputs is that this rapid discrimina-tive avoidant learning in the ACC isdependent on projections from the amyg-dala (Poremba and Gabriel, 1997). ACCcells are sufficiently responsive to the pre-dictive value of cues such that they are ableto compensate when the cues have lowsalience but are nonetheless predictive.Gabriel (1990) describes this as a saliencecompensation function.

In contrast to the ACC, the posterior cin-gulate cortex (PCC) is involved in laterstages of discrimination learning. PCC cellsare responsive to the novelty–familiaritydimension of the predictive conditionedstimulus. This suggested to Gabriel et al.(2002) that the PCC is involved in selectingaction based on the environmental context,and that it does so with hippocampalinput. The role of the hippocampalcomplex in action regulation is to compareinputs against an expected context for acertain action. In humans, neuroimagingstudies have specifically identified hip-pocampal activation when expectations,particularly of an affective nature, are vio-lated (Ploghaus et al., 2000). Gabriel et al.reasoned that, in human cognitive studies,task-related instructions prepare the

context for action, serving a purposesimilar to that of reinforcement training inanimal studies.

A Model of Action Regulation

In contrast to traditional Pavlovianmodels that attempted to explain learningin relation to associations linked to thereflex arc, modern concepts of learningemphasize the animal’s prediction ofbehavioral outcomes (Rescorla, 1992;Schultz and Dickinson, 2000). The studiesof Gabriel et al. suggest that the self-regulation of these predictions variesdepending on the match of behavioral pat-terns to the current environmental context.When the environment remains stable, thehippocampus and PCC provide a regularand graded updating of the neural repre-sentation of the context, allowing actionsto be determined by this representation.When more rapid changes are required,such as in emergency situations, the ACCprovides a second and more rapidly adapt-ing learning mechanism, drawing on inputfrom the amygdala.

It may be possible to relate the Gabrielet al. model of limbic mechanisms with theHolroyd and Coles (2002) model ofdopaminergic control by proposing thatthe ACC integrates striatal as well limbiccircuits in its regulation of rapid adapta-tion. It is when the implicit prediction (i.e.,the PCC context representation) is violatedby unexpected events that both the ACClimbic circuit and the DA error signal areengaged. Until that point, action plans arecontext dependent, and very likely regu-lated by the ongoing hippocampal andPCC learning process. At the point thatexpectancies are violated, the ACC has thecapacity for rapid alteration of behaviorbecause of its unique connectivity withcritical subcortical structures.

Whereas the cingulate gyrus as a wholeis derived from archicortex, and thus hasits predominant connectivity with the hip-pocampus and dorsal cortical pathways,

MECHANISMS OF ACTION REGULATION 209


the ACC is unique in integrating input fromthe amygdala, which is integral to theventral limbic pathways (Tucker et al., 2000).In addition, the ACC is closely connectedwith neostriatal circuits, and together withinsular regions of ventral limbic cortex,receives substantial dopaminergic modula-tion. This pattern of connectivity may beimportant to both motor and motivationalcontrol. The capacity for effective switchingof actions afforded by striatal circuits may beelaborated at the corticolimbic level by ACCnetworks, and may be extended to the roleof the ACC in action selection under condi-tions of response conflict. The rapid emer-gency response to fight–flight conditionsrequires not only routinized action se-quences but motivational control from theamygdala. In this way, the ACC integratesrapid adaptation and switching influencesfrom striatal and ventral limbic circuits thatact in opposition to the more gradualcontext-updating mode of self-regulation inhippocampal and posterior cingulate cir-cuits.

Although these mechanisms of motivatedaction evolved as primitive mechanisms ofaction regulation, they seem to be importantto more complex processes of human self-regulation as well. Although human exe-cutive processes may engage complexprocesses of reasoning, imagination, andemotional self-control that engage wide-spread regions of the cortex, the substantialevidence of the integral role of the ACC tothese processes suggests that they may reston a substrate of elementary mechanisms ofaction regulation. These mechanisms sup-port learning which action is relevant in agiven motivational context, monitoring theoutcome of the action, and switching to adifferent set of actions when outcomes areviolated.

Corticolimbic Integration and the Theta Rhythm

Through clarifying the elementarymechanisms of self-regulation in limbic,

thalamic, and striatal circuits, it may bepossible to build a more accurate model ofthe functions of the cortex. Even themassive human cortex does not function inisolation, but requires regulatoryinfluences from limbic cortex and dien-cephalic structures, such as in the processof memory consolidation (Squire, 1992).There are intriguing clues to the neuro-physiological mechanisms through whichlimbic circuits exert regulatory control oncortical networks. A key mechanismappears to be interregional communicationentrained by the theta rhythm. Given theobservations above showing theta oscilla-tions in the EEG measures of human self-monitoring and self-control, it may beuseful to consider the neurophysiologicalmechanisms of theta modulation.

In rats, theta is observed as a rhythmic,slow activity recorded in the hippocampusunder conditions in which the animal isexploring the environment (i.e., acquiringinformation). Cells within the hippocampalcomplex are easily activated at the thetafrequency because they have intrinsic reso-nant properties tuned to this frequency(Bland and Oddie, 1998). During thetaactivity, most pyramidal cells are hyperpo-larized (i.e., inhibited) and those that areactive seem to be coding the currently rele-vant stimuli. This finding suggested toBuzáski (1996) that the theta rhythm mayact to enhance the signal to noise ratio.Additionally, cells can be differentiallyactive at different phases of theta, pro-viding a mechanism for coding temporal patterns (Buzsáki, 1996; Lisman andOtmakhova, 2001). These properties sug-gested to Buzsáki that theta is the through-put mode of neuronal transmission, andthis type of activity is centered on the inputstages of the hippocampal circuit, such asthe enthorinal complex and dentate gyrus.Buzsáki argues that this is the labile ortemporary state of memory storage forrapidly incoming information. During theacquisition stages of discriminant learning,Gabriel and colleagues (1986) observed



that cells in the hippocampus displaybursting at the theta frequency. In contrast,Gabriel et al observed that during the inter-trial period hippocampal cells show briefbursts of activity. This appears similar tothe consolidation stage of hippocampalfunction in which irregular sharp wavesare prominent in the hippocampus(Buzsáki, 1989).

If the Papez circuit functions as an inte-grated system, in which the other struc-tures are critically dependent oninformation from the hippocampus, onewould expect that a major mode of trans-mission of neural information to and fromthe hippocampus would be at the theta fre-quency. Indeed, this is the argument putforth by Miller (1991). A key finding is thatbilateral lesions to the anterior nucleus ofthe thalamus or the cingulate gyrus reducethe duration of hippocampal theta inducedby electrical stimulation. Similarly, stimu-lation to either the cingulate or the anteriorthalamus induces theta activity in the ipsi-lateral hippocampus (Azzaroni andParmeggiani, 1967). In humans, a possibleparallel to the integrative role of theta inmemory consolidation was suggested bythe finding that coherence between differ-ent cortical regions was observed only atthe theta frequency when subjects had tohold information in short-term memory(Sarnthein et al., 1998).

Limbic Theta and Executive Control

In rats, hippocampal theta is seen whenthe animal is exploring its environment.However, in primates the most reliablemethod for eliciting hippocampal theta iswithholding an expected reward (Crowneand Radcliffe, 1975). In humans, increasedtheta activity in the region of the ACC andtemporal lobe is associated with increasesin working memory demands (Gevinset al., 1997; Raghavachari et al., 2001;Slobounov et al., 2000). A review byKahana et al. (2001) concluded that themost parsimonious description regarding

the functional significance of theta acrossspecies is that it is related to cognitivecontrol. More specifically, it may be thatthe theta rhythm reflects the control ofinformation gathering by the hippocampalsystem (Buzsáki, 1996; Miller, 1991), whichinvolves communication between multiplebrain regions but particularly within thePapez circuit.

In the findings on human self-monitor-ing and attention reviewed above, we dis-cussed evidence that the ERN emergesfrom phase alignment of the EEG at thetheta rhythm. There are also functionalsimilarities between the ERN and thetathat are important to a theory of actionregulation. First, error activity in the ACCcan also be induced by withholding anexpected reward (Niki and Watanabe,1979). This is similar to the condition forrobustly eliciting theta in the primatehippocampus (Crowne and Radcliffe,1975). Second, ACC error activity and hip-pocampal theta occur early in learning(Gemba et al., 1986; Pickenhain andKlingberg, 1967). Once learning is wellestablished, the error potential recorded inthe primate ACC and the theta rhythmobserved in the rodent hippocampus dis-appear.

MOTIVATIONAL CONTROL OFACTION REGULATION

Phase Reset of the Theta Rhythm toMotivationally Significant Events

According to the Holroyd and Coles(2002) model of the ERN, when an erroroccurs dopaminergic activity is decreased,which results in a disinhibition of neuronsin the ACC; the ERN is observed as a con-sequence of this disinhibition. As we haveobserved, however, the ERN seems to arisenot as a discrete negative transient, butrather as phase shifting of mediofrontalsources that are modulated at the thetarhythm (Luu et al., 2001). When an error is

MOTIVATIONAL CONTROL OF ACTION REGULATION 211


made, the phase of the theta rhythm is resetto the erroneous response. The question ishow this phase is reset. One possibility isthat the mechanism for the phase reset is thedopaminergic signal. This mechanism mayoccur whether the dopaminergic responsecarries information about reward error orwhether it provides the signal to reallocateresources and switch actions, as argued byRedgrave et al. (1999). What is important isthat the signal carries information aboutwhen important events occur, and thisappears to be a function of dopaminergiccells that is not disputed.

Phase resetting of the theta rhythm hasbeen observed for many years in responseto a conditioned stimulus during learningtrials. Adey (1967) found that, in the earlystages of learning, when the animal wasstill trying to learn the association betweenthe condition stimuli and reward, the thetarhythm was prominent and was phased-locked to the auditory cue. As the animal’sperformance indicated that it has learnedthe task, theta amplitude decreased, andthis was due to either a loss of phase-locked activity or a modulation of the fre-quency. However, when the cue wasreversed, theta activity reappeared andwas phase-locked to the stimulus, again.Buzsáki et al. (1979) observed similareffects. When an animal has to rememberwhether the current stimulus is the same ordifferent from a previous stimulus in orderto determine which response to makeGivens (1996) found that theta recorded in the dentate gyrus became phase-lockedto the stimulus. However, when perform-ing a much simpler, well-learned task ofstimulus–response mapping, theta was notphase locked to the stimulus.

At least two studies have shown that thetheta rhythm can be reset by stimulation ofseptohippocampal afferents. Buño et al.(1978) found that electrical stimulation ofthe fornix, septal nuclei, hypothalamus,and brain stem reticular formation pro-duced an entrainment of the hippocampaltheta rhythm to the stimulus. Interestingly,

the reset of the theta rhythm was notdependent on the theta phase at which thestimulus was delivered. That is, the stimu-lus can be delivered at any time during thetheta phase and still induce a phase resetto the stimulus. These authors noted thatin order for averaged, evoked responses toshow rhythmic activity, the theta rhythmmust be aligned at stimulus onset.Brazhnik et al. (1985) recorded the thetarhythm from the medial septum and foundthat stimulating the lateral septum resultedin a phase locking of the theta rhythm tothe stimulus. According to Brazhnik et al.,the resetting of the theta rhythm could beproduced by either a temporary pause ofcell activity, which lasts between 40 and 90msec, or a cellular “burst” that is time-locked to the stimulus.

Lisman and Otmakhova (2001) pro-posed that there are essentially two thetastates, with one engaged in learning andthe other in recall. Under the recall state,the animal is constantly predicting thefuture based on past learning. Any viola-tion of the prediction will be relevant toongoing behavior and will trigger the tran-sition to the theta-learning state. Accordingto Lisman and Otmakhova, the transitionbetween the recall and theta state is initi-ated by a dopaminergic signal. However,the evidence for dual theta states (i.e., twotheta pacemakers) is inconclusive (Miller,1991). What we have proposed above forthe dopaminergic function in action regula-tion may be similar to the model put forthby Lisman and Otmakhova. However,instead of switching theta states, wepropose that the dopaminergic responseprovides the marker that allows for thetheta phase to be reset to the relevant event(such as an error response). Input frommultiple sources may be funneled to theventral tegmental area to elicit this reseteffect, such as from the brain stem reticularformation (Brazhnik et al., 1985; Buño et al.,1978).

The observation that long-term potentia-tion depends on the theta phase (Buzsáki,



1996) may suggest that limbic thetafacilitates the coordination of synapticmodification across multiple networks of the Papez circuit. If so, then reset-ting the phase of the theta rhythm, such as through error or switching signals under dopaminergic or striatalcontrol, may coordinate the response of multiple networks to important events.

Although we have considered the possi-bility that dopaminergic activity may con-tribute to phase-resetting of the thetarhythm, the exact locus of action remainsto be clarified. One candidate is the medialseptum–diagonal band complex. Cellsfrom the VTA and substantia nigra havebeen shown to project to cholinergic cellsof the medial septum–diagonal bandcomplex (Gaykema and Zaborszky, 1996),which is believed to be the pacemaker ofthe theta rhythm. Miura et al. (1987)injected dopamine into the medial septumand found an increase in hippocampaltheta activity. Therefore, it is possible thatduring an error response, a decrease ofdopaminergic input into the medialseptum or diagonal band results in a pause(or reduction) in cholinergic activity. Thispause results in a phase resetting of thetheta rhythm to the error event.Interestingly, this mechanism wouldinvolve a dopamine influence at the basalforebrain rather than in the ACC. It isentirely possible, however, that phaseresetting can be initiated intrinsicallywithin the circuit, such as by the ACC. We suspect that signals from multiplebrain regions may also be able to reset the phase of the theta rhythm based onerror information. Shultz and Dickinson(2000) reviewed evidence that multipleregions, including the noradrenergic cellsof the locus coeruleus and the cerebellum,have been shown to be responsive toerrors. Moreover, human neuroimagingstudies have found the insula and frontaloperculum to be sensitive to errors (Menonet al., 2001).

Modulating the Amplitude of the Theta Rhythm

The hippocampal theta rhythm can bealtered by opiate activity. Cells in themedial septum are especially responsive tonoxious stimulation, whether mechanicalor thermal (Dutar et al., 1985). The recep-tive fields of these cells are broad, a charac-teristic feature of cells found in the medialpain system that encode affectivesignificance of painful stimuli (Vogt et al.,1993b). The pain-responsive cells of themedial septum project to the hippocampusto produce changes in the activity of pyra-midal neurons. Lesions of the medialseptum, or administration of a cholinergicantagonist, abolish this pain-induced effect(Khanna, 1997).

In the hippocampus, painful stimuliproduce a depression of the populationspikes with a concomitant increase in thetaactivity (Khanna, 1997). This effect appearsto result specifically from medial septalprojections, which act on the γ -aminobu-tyric acid (GABA)ergic hippocampalinterneurons. The GABAergic interneuronsinhibit most of the pyramidal cells, result-ing in broad suppression of populationspikes accompanied by increases in thetaactivity. In the midst of this inhibitoryinfluence, the activity of a small popula-tion of pyramidal neurons is enhanced,giving rise to a signal-to-noise increasethat may facilitate specific processes ofassociative learning.

An early study by Vanderwolf et al.(1978) showed that, in researching theeffects of multiple neuromodulators, onlyopiates were effective at attenuatinghippocampal theta activity [but see resultsby Miura et al. (1987)]. Khanna andcolleagues (Khanna, 1997; Khanna andZheng, 1998; Zheng and Khanna, 1999)showed that intraperitoneal injections ofopiates attenuate the hippocampal noci-ceptive response (i.e., the characteristicpopulation suppression and increase intheta activity in response to a painful

MOTIVATIONAL CONTROL OF ACTION REGULATION 213


stimulus). Furthermore, this effect wasnaloxone reversible. The opiates appear toreduce the hippocampal nociceptive eitherby inhibiting the GABAergic hippocampalinterneurons (Zieglagänsberger et al., 1979)or by inhibiting the function of the medialseptum (see Fig. 6).

Affective Modulation of Action Regulation

The findings on pain and opiate modula-tion of limbic mechanisms provide furtherperspective on the neural framework forlearning and self-regulation. From an infor-mation-processing perspective, pain is asignal that tells the organism something isamiss in the environment, such that infor-mation must be acquired to determine whatis wrong and what action is required. Thissignal demands that attention, and the asso-ciated limbic, striatal, and thalamic mecha-nisms, be engaged quickly and effectively.The evidence of opiate modulation of thetheta rhythm suggests how the organism’smotivational context frames the multipleprocesses of action regulation. We think thisneurophysiological evidence is consistentwith the proposal that the ongoing evalua-tion of action, as indexed by the ERN,reflects the individual’s subjective distress,which is in turn regulated by opiate activity

(Luu et al., 2000a). Furthermore, individualdifferences in opiate modulation and sub-jective distress may help explain why ERN amplitude varies as a function ofanxiety (Dikman and Allen, 2000; Gehringet al., 2000; Johannes et al., 2001a,b). Furtherresearch on individual differences in themechanisms of action regulation may helpexplain why individuals who have lowlevels of anxiety, such as psychopaths, areunconcerned about their errors and do notlearn from their mistakes.

In summary, it appears that the ERNreflects at least two mechanisms. One is animmediate signal for action change. Theother is a more general motivational stateof the organism. First, the ERN arises fromlimbic theta that has been phase-reset bythe process of response monitoring anddetection of a prediction discrepancy. Thisdiscrepancy could be created by errordetection, or conflict detection, but mayreflect a more elementary mechanism thanwould be supposed by attributing themonitoring process to a higher executivefunction. One candidate for the phase-resetting mechanism would be mesen-cephalic dopaminergic projections, andtheir locus of action could be in the fore-brain cholinergic nuclei (targeting the sep-tohippocampal pacemaker directly) or inthe ACC (targeting the cortical control of



FIGURE 6 A model of nociceptive influence on the functioning of the hippocampus based on the work ofKhanna (1997, 1998) (see text for details). Nociceptive input activates cells in the medial septum, which thenproject to the hippocampus. If the input is not moderated by the opiates, the net effect in the hippocampus is asuppression of pyramidal cells by GABAergic interneurons and an increase in theta activity. If the input is mod-erated by the opiates, this hippocampal nociceptive response is attenuated. MS-nDBB, Medial septum andnucleus of the diagonal band of Broca.

limbic output to the motor system), orboth. In essence, phase resetting of thetheta rhythm prepares the action regula-tion system for processing of significantevents.

Second, the ERN and related medialfrontal responses reflect the individual’scurrent motivational state. This motiva-tional state is a system-wide function, withmultiple influences, a critical one of whichis the current level of opiate modulation.The opiate activity moderates the signal-to-noise function of the theta rhythm. If therhythm is not attenuated by the opiates,the signal-to-noise ratio is high. This mayreflect the state in which the animal “cares”enough to react adaptively and learn. If thesignal-to-noise ratio is reduced because ofa high level of opiate modulation, it mayreflect the situation in which the animaldoes not “care” and the signal has littleeffect on action and learning.

ELECTROPHYSIOLOGICAL SIGNS OF HUMAN

EXECUTIVE CONTROL

Although we can assemble only a sketchyoutline of self-regulatory mechanisms at thispoint, we propose that a more completemodel of the neural mechanisms of moti-vated self-regulation will eventually explaina number of ERP phenomena that areknown to be integral to cognitive process-ing, but that cannot be understood withincurrent theory. In the following section, wewill review evidence that several key ERPcomponents can be understood to arise fromthe modulation of limbic theta activity, andthat these components may provide keyinsights into the frontolimbic mechanisms ofaction regulation.

Electrophysiology of Action Regulation

A negative deflection over frontal sites isobserved approximately 250–350 msecafter a no-go stimulus in a go/no-go para-

digm (Jodo and Kayama, 1992). In this par-adigm, subjects either make or withhold aresponse according to stimulus properties.This negative deflection is referred to asthe no-go N2. It has a mediofrontal dis-tribution and has been shown by dense-array EEG recordings to be generated by asource in the anterior cingulate cortex(Bokura, et al., 2001). The N2 has beenproposed to reflect response inhibitoryprocesses (Jackson et al., 1999; Jodo andKayama, 1992; Kopp et al., 1996b).

Kopp et al. (1996a) have proposed thatin addition to appearing over frontal sites,the N2 and ERN are also similar in waveform morphology and latency. Therefore,they conclude that the N2 and ERN mayreflect the same underlying process: motorinhibition. However, Falkenstein et al.(1999) found that whereas the latency andamplitude of N2 varied as a function ofstimulus modality, the ERN amplitude didnot. Therefore, these authors concludedthat the N2 and ERN do not reflect theactivity of a common inhibition mecha-nism. Bokura and colleagues (2001) do notbelieve that the N2, as reflected in the ACCactivity, is involved in response inhibition.Rather, they propose that it is related moregenerally to the monitoring of behaviorthat requires control, i.e., nonroutinizedbehavior.

As a result of their findings, Falkensteinet al. (1999) proposed that the N2 mightreflect stimulus-locked processes whereasthe ERN might index response-lockedprocesses. Luu et al. (2002) have observedN2-like (in terms of morphology, topogra-phy, and latency) ERP deflections for feed-back stimuli indicating that neither aresponse had to be withheld nor that theresponse was in error. The source analysisresult reported by Luu et al. is particularlyinteresting in light of the suggestion byFalkenstein et al. that the N2 reflects stimu-lus-locked activity. In the Luu et al. (2002)analysis, the N2-like deflection arose fromphase locking of dorsal ACC activity to thestimulus. In contrast, the ERN was found

ELECTROPHYSIOLOGICAL SIGNS OF HUMAN EXECUTIVE CONTROL 215


to arise primarily from phase locking ofrostral ACC activity to the response (with alesser contribution from the source indorsal ACC). Although no previous reporthas noted the frequency characteristic ofthe N2, the Luu et al. findings suggest thatit may be at the theta frequency; this sug-gestion would be consistent with thefinding that theta increases in response to ano-go stimulus in a go/no-go paradigm(Miller, 1991).

Novelty and Distraction

Another ERP component that appears toindex aspects of executive control is theP3a. The P3a is a positive deflection in theEEG that has a frontocentral distributionand peaks approximately 250–350 msecafter novel stimulus onset. Traditionally,the P3a is elicited using paradigms inwhich a distractor (or novel stimulus) ispresented within a stream of target andnontarget stimuli. It has been proposed tobe an index of a multimodal, corticolimbicorienting system that is responsive to novelevents (Knight and Scabini, 1998).However, work by Polich and colleagues(Katayama and Polich, 1998; Polich, 2002)has shown that the critical parameter forthe elicitation of the P3a may not be stimu-lus novelty per se. Rather, these researchersfind that the experimental context appearsto be the important parameter. When thediscrimination between target and nontar-gets was difficult, a distractor stimulus thatwas strongly deviant from the target andnontarget stimuli elicited a large P3a.Katayama and Polich interpreted this asdemonstrating that the context sets up theattentional requirements that must be redi-rected, by the frontal lobe (e.g., executiveprocesses), to the distractor.

Although Katayama and Polich (Kata-yama and Polich, 1998) showed thatnovelty is not the only determinant of theP3a response, novelty is still relevant tomany of the findings with this component.Demiralp et al. (2001) demonstrated that

both typical and novel distractors pre-sented within a target/nontarget stream ofstimuli elicited P3a responses with similartopographies. However, applying waveletanalysis to the P3a, these researchers founda significant power increase at the thetafrequency for the novel but not the typicaldistractors. Similar to the findings that theERN is part of an ongoing theta process,the results obtained by Demiralp et al. maysuggest that the P3a is composed of multi-ple frequencies (mainly delta and thetabands) of ongoing EEG rhythms. Animportant question for further research iswhether the P3a, like the ERN, arises fromphase resetting of limbic theta rhythms,similar to the finding for other averagedevent-related potential components(Makeig et al., 2002b).

The neural substrate of the P3a isbelieved to include multiple cortical andsubcortical structures (Knight and Scabini,1998; Polich, 2002). Of particular relevanceare findings that implicate the frontal lobe(ACC) and hippocampus in the generationof the P3a (Ebmeier et al., 1995; Knight,1984, 1996). Based on its functional charac-teristics and the underlying neural genera-tors, we propose that the P3a may beanother index of the ACC response in therapid encoding of significant and dis-crepant signals (Gabriel et al., 2002). Thus,whereas the posterior cingulate regularlyupdates its representation of the context(with a P3b-like response), the more rapidalteration by the ACC appears to signal aswitching mechanism, in response to pre-dictive control violations, possibly medi-ated by dopamine projections.

Of course, although they share a medialfrontal distribution, the P3a reflects a posi-tive-going potential in the stimulus-lockedaveraged data, whereas the ERN reflects anegative-going potential in the response-locked averaged data. Could their differentpolarities reflect phase resetting at differentphases of the theta wave? Considering thelong-term potentiation findings in the hip-pocampus, could these opposite polarities



reflect functional mechanisms such aspotentiation and depotentiation of synap-tic connections? Perhaps the enhancementof P3a to distractors with increased atten-tion to the foreground task (Katayama andPolich, 1998) could reflect dopaminergiccontrol of the ACC that inhibits the switch-ing, i.e., that avoids the distraction. Inother words, dopaminergic activity pre-serves the current context at the expense ofignoring potentially important stimuli.Such an action could be consistent with the“winner take all” mechanisms of switchingdescribed for dopaminergic modulation ofneostriatal circuits (Redgrave et al., 1999): ifit is not selected, the distractor stimulusdoes not receive consolidation withinworking memory that can be provided bymechanisms of limbic theta (Buzsáki,1996).

There is an opposite interpretation thatis consistent with what is known about thetheta rhythm. Gabriel et al. (2002) find cellsin the cingulate cortex that respond withgreater activity to condition stimuli (thosethat predict the presence or absence of areinforcer) that are of short duration whencompared to those that are of long dura-tion. They proposed that the cingulatecortex participates in a process of saliencecompensation that allows for non-salient,yet associatively significant, stimuli toreceive processing. These cells do not indexstimulus duration because they increasetheir firing during the stimulus and notafter stimulus offset. Rather, these cellsmay be prepared to process low-salientstimuli based on context information pro-vided by hippocampal input (Gabriel andTaylor, 1998). Of particular importance isthe observation that prior to enhancedfiring these cells show a brief pause (40–80msec) in their activity after presentation ofthe low-salience stimuli. Gabriel andTaylor proposed that this pause is a reset-ting mechanism that allows for the recruit-ment of neurons that would otherwise nothave been available for the processing ofthe low-salience stimulus. The pause of

these neurons is reminiscent of the pausefunction identified by Brazhnik et al. (1985)for phase resetting of the theta rhythm. As noted previously, these researchersdemonstrated that one mechanism of thetaphase resetting is a 40- to 90-msec pause in neuronal activity after an electricalstimulus delivery.

The observations on rabbit learning andtheta phase reset may provide insight intothe human electrophysiological responsesdescribed by Katayama and Polich.Because of the effort required to distin-guish between targets and standards whenthe differentiating feature is small, incom-ing distractors are not as salient. The P3amay reflect for humans a compensationprocess for the distractors that is similar towhat is described by Gabriel et al. assalience compensation in their study ofrabbits. In essence, the novel distractorsproduce a phase resetting of the ACC–hip-pocampal theta rhythm. Functionally, theeffect is that the context is overcome by aphase reset of the theta rhythm to the dis-tractor. Like phase resetting of the thetarhythm that gives rise to the ERN, phaseresetting that gives rise to the P3a may beinitiated by multiple sources, which mayinclude dopaminergic and noradrenergiccell groups or other cortical structures.

Context Updating

Although much of the interesting evi-dence seems to focus on the rapid mecha-nisms of self-regulation in the ACC, it isalso important to consider the moregradual representation of the context in thePCC and associated posterior brain net-works. The P3b may provide an index tomodification of this representation inhuman studies. The P3b component is anamodal response to voluntary detection ofan infrequent target within a stream of fre-quent nontargets. Compared to the P3a,the P3b has a longer latency and a distrib-ution over parietal rather than frontocen-tral recording sites. However, like the P3a,

ELECTROPHYSIOLOGICAL SIGNS OF HUMAN EXECUTIVE CONTROL 217


the P3b is not a unitary phenomenon.Rather, it reflects activity from multiplesources and contains EEG power in bothdelta and theta frequencies. Reviews byPolich (2002) and Knight and Scabini(1998) summarize the evidence on theneural generators of the P3b. The mostconsistent finding is that bilateral tem-poroparietal cortices are major contributorsto the scalp-recorded P3b (e.g., see Ebmeieret al., 1995; Menon et al., 1997). In the fre-quency domain, Demiralp et al. (2001)showed with wavelet analysis that the P3bcomponent also has contribution from thedelta and theta frequencies. Similar fre-quency findings for the P3b have beenfound by others using different methods(Basar, 1998; Basar-Eroglu et al., 1992;Spencer and Polich, 1999).

The most influential theory of the P3b isthat it reflects context updating processesthat are engaged when an internal modelof the context for action must be revised(Donchin and Coles, 1988). For Donchinand Coles, the P3b reflects strategicprocesses that are involved in planningaction rather than immediate “tactical”processes that are concerned with immedi-ate response control. Knight and Scabini(1998) speculated that the P3b might indexthe transfer of information from corticalstructures, such as the temporoparietaljunction, to limbic regions, such as thehippocampus, for memory updating.These proposals could align the functionalsignificance of the P3b with the PCC– hippocampal action regulation system thatis involved in context-sensitive, late-stagelearning (Gabriel et al., 2002).

CONCLUSION:ELECTROPHYSIOLOGICAL CLUES

TO EXECUTIVE CONTROL

Research on the error-related negativitycomponent has provided one of the closestlinks between data obtained from animalexperiments and electrophysiological and

hemodynamic data obtained from humanstudies. The PET and fMRI findings ofACC activity in tasks requiring motivation,attention, and self-control have empha-sized the importance of the limbic base ofthe frontal lobe for the human executivefunctions. The ERN findings not onlyprovide temporal resolution to sharpen thepsychological analysis of self-regulatoryoperations, but their dynamic nature alsoprovides important clues to system-widefunctions, including multiple subcorticalcircuits, each with unique neurophysiolog-ical properties. Because the ERN appears toarise from ongoing limbic theta activity,and because the theta rhythm reflects thefunctional activity of closely integratednetworks of the limbic system, Luu et al.(2002) proposed that the ERN indexes notjust the activity of the ACC, but the func-tion of the rapid-learning system identifiedby Gabriel et al. (2002). Among the criticalinputs to the ACC, the hippocampus andposterior cingulate are essential to the self-regulatory process, but the ventral limbicstructures, including the amygdala, tempo-ral pole, and caudal orbital frontal cortex,appear to be particularly important to fastadaptation. Similarly, we propose that theN2, P3a, and P3b reflect system-wide activ-ity involving an entire network that iscoordinated by theta activity.

From this perspective, functional inter-pretations become more complicated butperhaps more realistic. Not only is itimportant to consider the properties ofeach brain region or site, but also the func-tional operation of large-scale networks. Tounderstand such an operation, one mustunderstand how the network is coordi-nated and how this coordination can bemeasured. With these factors in mind, itbecomes difficult to develop an adequatetheory based on measuring activity in one subunit of the network (e.g., the ACC), even though important clues to thecoordination of the wider network (e.g.,corticolimbic and corticothalamic, and cor-ticostriatal systems) are clearly gained by



measuring activity in the particularsubunit. It must be recognized that activityin any subunit can be modulated throughinfluences with multiple entry points intothe system-wide network.

Because no part of the network can beunderstood in isolation, the complexity ofthe neurophysiology of mammalian self-regulation thus poses a formidable theoret-ical challenge. Nonetheless, a theory ofself-regulation built from neurophysiologi-cal mechanisms could provide uniqueinsights not available to mentalistic theo-ries that begin with constructs of cognitiveprocesses and attempt to localize them inthe brain. We propose that, when consid-ered within the framework of integratedcortical and subcortical systems, executivecontrol can be seen to arise out of primitiveprocesses of action regulation (Gabriel etal., 2002). Furthermore, these processesmay provide unique insights into electro-physiological signs, including the N2, P3a,and P3b, that have proved difficult to maponto operations of a cognitive theory, suchas attention and memory, because theyseem to point to more elementaryprocesses of orienting, alerting, andarousal. From the perspective of neuro-physiology, it is just these elementarymechanisms that are required to explainhow an organism self-regulates behavior.Considered in this way, and analyzed withmodern methods providing both temporaland spatial resolution, human scalp elec-trophysiological measures may provideimportant clues to how the brain adaptsbehavior to a rapidly changing environ-ment, while maintaining a contextreflecting both past experience and thecurrent motivational state.

Acknowledgments

This work was supported by NationalInstitues of Mental Health grants MH42129and MH42669 and an Augmented Cog-nition project grant funded by DARPA toDon M. Tucker.

ReferencesAdey, W. R. (1967). Intrinsic organization of cerebral

tissue in alerting, orienting, and discriminativeresponses. In (G. C. Quarton and T. Melnechuckand F. O. Schmitt, eds.), pp. 615–633. “TheNeurosciences” Rockefeller University Press, NewYork.

Allman, J. M., Hakeem, A., Erwin, J. M., Nimchinsky,E., and Hof, P. (2001). The anterior cingulatecortex: The evolution of an interface betweenemotion and cognition. Ann. N.Y. Acad. Sci. 935,107–117.

Asada, H., Fukuda, Y., Tsunoda, S., Yamaguchi, M.,and Tonoike, M. (1999). Frontal midline thetarhythms reflect alternative activation of prefrontalcortex and anterior cingulate cortex in humans.Neurosci. Lett. 274, 29–32.

Azzaroni, A., and Parmeggiani, P. L. (1967). Feedbackregulation of the hippocampal theta-rhythm. Helv.Physiol Pharmacol. Acta 25, 309–321.

Badgaiyan, R. D., and Posner, M. I. (1998). Mappingthe cingulate cortex in response selection andmonitoring. Neuroimage 7, 255–260.

Barris, R. W., and Schuman, H. R. (1953). Bilateralanterior cingulate gyrus lesions. Neurology 3,44–52.

Basar, E. (1998). “Brain Function and Oscillations.”Springer, Berlin.

Basar-Eroglu, C., Basar, E., Demiralp, T., andSchürmann, M. (1992). P300-response: Possiblepsychophysiological correlations in delta andtheta frequency channels. A review. Int. J.Psychophysiol. 13, 161–179.

Bernstein, P. S., Scheffers, M. K., and Coles, G. H.(1995). “Where did I go wrong?” A psychophysio-logical analysis of error detection. J. Exp. Psychol.21, 1312–1322.

Bland, B. H., and Oddie, S. D. (1998). Anatomical,electrophysiological and pharmacological studiesof ascending brain stem hippocampal synchroniz-ing pathways. Neurosci. Biobehav. Rev. 22, 259–273.

Bokura, H., Yamaguchi, S., and Kobayashi, S. (2001).Electrophysiological correlates for responseinhibition in a Go/NoGo task. Clin. Neurophysiol.112, 2224–2232.

Brazhnik, E. S., Vinogradova, O. S., and Karanov, A.M. (1985). Frequency modulation of neuronaltheta-bursts in rabbit’s septum by low-frequencyrepetitive stimulation of the afferent pathways.Neurosci. 14, 501–508.

Buño, W. J., Garcia-Sanchez, J. L., and Garcia-Austt,E. (1978). Reset of hippocampal rhythmical activi-ties by afferent stimulation. Brain Res. Bull. 3,21–28.

Bush, G., Luu, P., and Posner, M. I. (2000). Cognitiveand emotional influences in anterior cingulatecortex. Trends Cogn. Sci. 4, 215–222.

REFERENCES 219


Buzsáki, G. (1989). Two-stage model of memory traceformation: A role for “noisy” brain states. Neurosci.31, 551–570.

Buzsáki, G. (1996). The hippocampal–neocortical dia-logue. Cereb. Cortex 6, 81–92.

Buzsáki, G., Grastyán, E., Tveritskaya, I. N., andCzopf, J. (1979). Hippocampal evoked potentialsand EEG changes during classical conditioning inthe rat. Electrocencephalogr. Clin. Neurophysiol. 47,64–74.

Carter, C. S., Braver, T. S., Barch, D. M., Botvinick, M.M., Noll, D., and Cohen, J. D. (1998). Anterior cin-gulate cortex, error detection, and the online moni-toring of performance. Science 280, 747–749.

Carter, C. S., Botvinick, M. M., and Cohen, J. D. (1999).The contribution of the anterior cingulate cortex toexecutive processes in cognition. Rev. Neurosci. 10,49–57.

Coles, M. G. H., Scheffers, M. K., and Holroyd, C. B.(2001). Why is there an ERN/Ne on correct trials?Response representations, stimulus-related com-ponents, and the theory of error-processing. Biol.Psychol. 56, 173–189.

Crowne, D. P., and Radcliffe, D. D. (1975). Some char-acteristics and functional relations of the electricalactivity of the primate hippocampus and ahypothesis of hippocampal function. In “TheHippocampus: Neurophysiology and Behavior”(R. L. Isaacson and K. H. Pribram, eds.), Vol. 2,pp. 185–206. Plenum, New York.

Damasio, A. R., Tranel, D., and Damasio, H. (1990).Individuals with sociopathic behavior caused byfrontal damage fail to respond autonomically tosocial stimuli. Behav. Brain Res. 41, 81–94.

Dehaene, S., Posner, M. I., and Tucker, D. M. (1994).Localization of a neural system for error detectionand compensation. Psychol. Science, 5, 303–305.

Demiralp, T., Ademoglu, A., Comerchero, M., andPolich, J. (2001). Wavelet analysis of P3a and P3b.Brain Topog. 13, 251–267.

Devinsky, O., Morrell, M. J., and Vogt, B. A. (1995).Contributions of anterior cingulate cortex tobehavior. Brain 118, 279–306.

Dikman, Z. V., and Allen, J. J. B. (2000). Error monitor-ing during reward and avoidance learning in highand low-socialized individuals. Psychophysiology37, 43–54.

Donchin, E., and Coles, M. G. H. (1988). Is the P300component a manifestation of context updating?Behav. Brain Sci. 11, 357–374.

Dutar, P., Lamour, Y., and Jobert, A. (1985). Activationof identified septo-hippocampal neurons bynoxious peripheral stimulation. Brain Res. 328,15–21.

Ebmeier, K. P., Steele, J. D., MacKenzie, D. M.,O’Carroll, R. E., Kydd, R. R., Glabus, M. F.,Blackwood, D. H. R., Rugg, M. D., and Goodwin,G. M. (1995). Cognitive brain potentials andregional cerebral blood flow equivalents during

two- and three-sound auditory “oddball tasks.” Electroencephalogr. Clin. Neurophysiol. 95, 434–443.

Elliot, R., and Dolan, R. J. (1998). Activation of differ-ent anterior cingulate foci in association withhypothesis testing and response selection.Neuroimage 8, 17–29.

Eslinger, P. J., and Damasio, A. R. (1985). Severe dis-turbance of higher cognition after bilateral frontallobe ablation. Neurology 35, 1731–1741.

Falkenstein, M., Hohnsbein, J., Hoormann, J., andBlanke, L. (1991). Effects of crossmodal dividedattention on late ERP components. II. Error pro-cessing in choice reaction tasks. Electroencephalogr.Clin. Neurophysiol. 78, 447–455.

Falkenstein, M., Hoorman, J., and Hohnsbein, J.(1999). ERP components in Go/Nogo tasks andtheir relation to inhibition. Acta Psychol. 101,267–291.

Falkenstein, M., Hoorman, J., Christ, S., andHohnbein, J. (2000). ERP components on reactionerrors and their functional significance: A tutorial.Biol. Psychol. 51, 87–107.

Falkenstein, M., Hielscher, H., Dziobek, I.,Schwarzenau, P., Hoormann, J., Sundermann, B.,and Hohnsbein, J. (2001). Action monitoring, errordetection, and the basal ganglia: An ERP study.Neuroreport 12, 157–161.

Ford, J. M. (1999). Schizophrenia: The broken P300and beyond. Psychophysiology 36, 667–682.

Gabriel, M. (1990). Functions of anterior and posteriorcingulate cortex during avoidance learning inrabbits. Prog. Brain Res. 85, 467–483.

Gabriel, M., Sparenborg, S. P., and Stolar, N. (1986).An executive function of the hippocampus:Pathway selection for thalamic neuronalsignificance code. In “The Hippocampus” (R. L.Isaacson and K. H. Pribram, eds.), pp. 1–39.Plenum, New York.

Gabriel, M., and Taylor, C. (1998). Prenatal exposureto cocaine impairs neuronal coding of attentionand discriminative learning. Ann. N.Y. Acad. Sci.846, 194–212.

Gabriel, M., Burhans, L., and Scalf, P. (2002).Cingulate cortex. In “Encyclopedia of the HumanBrain” (V. S. Ramachandran, ed.), Academic Press,New York. (In press.)

Gaykema, R. P., and Zaborszky, L. (1996). Directcatecolaminergic-cholinergic interactions in thebasal forebrain. II. Substantia nigra-ventral teg-mental area projections. J. Compar. Neurol. 374,555–577.

Gehring, W. J., and Fensik, D. E. (2001). Functions ofthe medial cortex in the processing of conflict anderrors. J. Neurosci. 21, 9430–9437.

Gehring, W. J., Goss, B., Coles, M. G. H., Meyer, D. E.,and Donchin, E. (1993). A neural system for errordetection and compensation. Psychol. Sci. 4,385–390.



Gehring, W. J., Himle, J., and Nisenson, L. G. (2000).Action monitoring dysfunction in obsessive–com-pulsive disorder. Psychol. Sci. 11, 1–6.

Gemba, H., Sasaki, K., and Brooks, V. B. (1986).“Error” potentials in limbic cortex (anterior cingu-late area 24) of monkeys during motor learning.Neurosci. Lett. 70, 223–227.

George, M. S., Ketter, P. I., Rosinsky, N., Ring, H.,Casey, B. J., Trimble, M. R., Horwitz, B.,Herscovitch, P., and Post, R. M. (1994). Regionalbrain activity when selecting a response despiteinterference: An H2

15O Pet study of the stroop andemotional stroop. Hum. Brain Mapping 1, 194–209.

Gevins, A., Smith, M. E., McEvoy, L., and Yu, D.(1997). High-resolution EEG mapping of corticalactivation related to working memory: Effects oftask difficulty, type of processing, and practice.Cereb. Cortex 7, 374–385.

Givens, B. (1996). Stimulus-evoke resetting of thedentate theta rhythm: Relation to workingmemory. NeuroReport 8, 159–163.

Gurney, K., Prescott, T. J., and Redgrave, P. (2001). Acomputational model of action selection in thebasal ganglia. I. A new functional anatomy. Biol.Cybern. 84(6), 401–410.

Holroyd, C., and Coles, M. G. H. (2002). The basis ofhuman error processing: Reinforcement learning,dopamine, and the error-related negativity.Psychol. Rev.

Holroyd, C. B., Dien, J., and Coles, M. G. H. (1998).Error-related scalp potentials elicited by hand andfoot movements: Evidence for an output-indepen-dent error-processing system in humans. Neurosci.Lett. 242, 65–68.

Jackson, S. R., Jackson, G. M., and Roberts, M. (1999).The selection and suppression of action: ERP cor-relates of executive control in humans. NeuroReport10, 861–865.

Jodo, E., and Kayama, Y. (1992). Relation of a negativeERP component to response inhibition in aGo/No-go task. Electroencephalogr. Clin.Neurophysiol. 82, 477–482.

Johannes, S., Wieringa, B. M., Nager, W., Dengler, R.,and Münte, T. F. (2001a). Oxazepam alters actionmonitoring. Psychopharmacology 155, 100–106.

Johannes, S., Wieringa, B. M., Nager, W., Rada, D.,Dengler, R., Emrich, H. M., Münte, T. F., andDietrich, D. E. (2001b). Discrepant target detectionand action monitoring in obsessive–compulsivedisorder. Psychiatr. Res. 108, 101–110.

Kahana, M. J., Seelig, D., and Madsen, J. R. (2001).Theta returns. Curr. Opin. Neurobiol. 11, 739–744.

Katayama, J., and Polich, J. (1998). Stimulus contextdetermines P3a and P3b. Psychophysiology 35,23–33.

Khanna, S. (1997). Dorsal hippocampus field CA1pyramidal cell responses to a persistent versus anacute nociceptive stimulus and their septal modu-lation. Neurosci. 77, 713–721.

Khanna, S., and Zheng, F. (1998). Morphine reversedformalin-induced CA1 pyramidal cell suppressionvia an effect on septohippocampal neural process-ing. Neurosci. 89, 61–71.

Kiehl, K. A., Liddle, P. F., and Hopfinger, J. B. (2000).Error processing and the rostral anterior cingulate:An event-related fMRI study. Psychophysiology, 37,216–223.

Knight, R. T. (1984). Decreased response to novelstimuli after prefrontal lesions in man.Electrocencephalogr. Clin. Neurophysiol. 59, 9–20.

Knight, R. T. (1996). Contribution of human hippocam-pal region to novelty detection. Nature 383, 256–259.

Knight, R. T., and Scabini, D. (1998). Anatomic basesof event-related potentials and their relationshipto novelty detection in humans. J. Clin.Neurophysiol. 15, 3–13.

Kopp, B., Mattler, U., Goertz, R., and Rist, F. (1996a).N2, P3 and the lateralized readiness potential in anogo task involving selective response priming.Electrocencephalogr. Clin. Neurophysiol. 99, 19–27.

Kopp, B., Rist, F., and Mattler, U. (1996b). N200 in theflanker task as a neurobehavioral tool for investi-gating executive control. Psychophysiology 33,282–294.

Lehrmitte, F. (1986). Human autonomy and thefrontal lobes. Part II: Patient behavior in complexand social situations: The “environmental depen-dency syndrome.” Ann. Neurol. 19, 335–343.

Lehrmitte, F., Pillon, B., and Serdaru, M. (1986).Human autonomy and the frontal lobes. Part I:Imitation and utilization behavior: A neuropsy-chological study of 75 patients. Ann. of Neurol. 19, 326–334.

Lisman, J. E., and Otmakhova, N. A. (2001). Storage,recall, and novelty detection of sequences by thehippocampus: Elaborating on the SOCRATICmodel to account for normal and aberrant effectsof dopamine. Hippocampus 11, 551–568.

Luu, P., and Tucker, D. M. (2001). Regulating action:Alternating activation of human prefrontal andmotor cortical networks. Clin. Neurophysiol. 112,1295–1306.

Luu, P., Collins, P., and Tucker, D. M. (2000a). Mood,personality, and self-monitoring: Negative affectand emotionality in relation to frontal lobe mecha-nisms of error monitoring. J. Exp. Psychol. Gen.129, 43–60.

Luu, P., Flaisch, T., and Tucker, D. M. (2000b). Medialfrontal cortex in action monitoring. J. Neurosci. 20,464–469.

Luu, P., Tucker, D. M., Derryberry, D., Reed, M., andPoulsen, C. (2002). Activity in human medialfrontal cortex in emotional evaluation and errormonitoring. Psychol. Sci. (in press).

MacDonald, A. W., Cohen, J. D., Stenger, V. A., andCarter, C. S. (2000). Dissociating the role of thedorsolateral prefrontal and anterior cingulatecortex in cognitive control. Science 288, 1835–1838.

REFERENCES 221


MacLean, P. D. (1993). Introduction: Perspectives oncingulate cortex in the limbic system. In“Neurobiology of the Cingulate Cortex andLimbic Thalamus” (B. A. Vogt and M. Gabriel, eds.pp. 1–15). Birkhauser, Boston.

MacLean, P. D., and Pribram, K. H. (1953).Neurographic analysis of medial and basal cere-bral cortex. I. Cat. J. Neurophysiol. 16, 312–323.

Makeig, S., Jung, T.-P., Ghahremani, D., Bell, A. J., andSejnowski, T. J. (1997). Blind separation of auditoryevent-related brain responses into independentcomponents. Proc. Natl. Acad. Sci. 94, 10979–10984.

Makeig, S., Luu, P., Briggman, K., Visser, E.,Sejnowski, T. J., and Tucker, D. M. (2002a).Oscillatory sources of human error-related brainactivity. In preparation.

Makeig, S., Westerfield, M., Jung, T.-P., Enghoff, S.,Townsend, J., Courchesne, E., and Sejnowski, T. J.(2002b). Dynamic brain sources of visual evokedresponses. Science 295, 690–694.

Masaki, H., Tanaka, H., Takasawa, N., and Yamazaki,K. (2001). Error-related brain potentials elicited byvocal errors. NeuroReport 12, 1851–1855.

Menon, V., Ford, J. M., Lim, K. O., Glover, G. H., andPfefferbaum, A. (1997). Combined event-relatedfMRI and EEG evidence for temporalparietalcortex activation during target detection.NeuroReport 8, 3029–3037.

Menon, V., Adleman, N. E., White, C. D., Glover, G.H., and Reiss, A. L. (2001). Error-related brain acti-vation during a go/nogo response inhibition task.Hum. Brain Mapping 12, 131–143.

Miller, R. (1991). “Cortico-hippocampal Interplay andthe Representation of Contexts in the Brain.”Springer-Verlag, New York.

Miltner, W. H. R., Braun, C. H., and Coles, M. G. H.(1997). Event-related brain potentials followingincorrect feedback in a time-estimation task:Evidence for a “generic” neural system for errordetection. J. Cogn. Neurosci. 9, 787–797.

Miura, Y., Ito, T., and Kadokawa, T. (1987). Effects ofintraseptally injected dopamine and noradrenalineon hippocampal synchronized theta wave activityin rats. Jpn. J. Pharmacol. 44, 471–479.

Nauta, W. J. H. (1971). The problem of the frontal lobe:A reinterpretation. J. Psychiatr. Res. 8, 167–187.

Nieuwenhuis, S., Ridderinkhof, K. R., Blom, J., Band,G. P., and Kok, A. (2001). Error-related brainpotentials are differentially related to awareness ofresponse errors: Evidence from an antisaccadetask. Psychophysiology 38, 752–760.

Niki, H., and Watanabe, M. (1979). Prefrontal and cin-gulate unit activity during timing behavior in themonkey. Brain Res. 171, 213–224.

Norman, D. A., and Shallice, T. (1986). Attention toaction: willed and automatic control of behavior.In “Consciousness and Self-regulation” (R. J.Davidson and G. E. Schwartz and D. Shapiro, eds.pp. 1–18). Plenum, New York.

Pandya, D. N., Van Hoesen, G. W., and Mesulam, M.-M. (1981). Efferent connections of the cingulategyrus in the rhesus monkey. Exp. Brain Res. 42,319–330.

Pandya, D. N., Seltzer, B., and Barbas, H. (1988).Input-output organization of the primate cerebralcortex. Comp. Primate Biol. 4, 39–80.

Pardo, J. V., Pardo, P. J., Janer, K. W., and Raichle, M.E. (1990). The anterior cingulate cortex mediatesprocessing selection in the stroop attentionalconflict paradigm. Proc. Natl Acad. Sci. 87, 256–259.

Paus, T. (2001). Primate anterior cingulate cortex:Where motor control, drive and cognition inter-face. Nat. Rev. 2, 417–424.

Pickenhain, L., and Klingberg, F. (1967). Hippocampalslow wave activity as a correlate of basic behav-ioral mechanisms in the rat. Prog. Brain Res. 27,218–227.

Ploghaus, A., Tracey, I., Clare, S., Gati, J. S., Rawlins, J.N. P., and Matthews, P. M. (2000). Learning aboutpain: The neural substrate of the prediction errorfor aversive events. Proc. Natl. Acad. Sci. 97,9281–9286.

Polich, J. (2002). Neuropsychology of P3a and P3b: Atheoretical overview. In “Advances in Electro-physiology and Clinical Practice and Research”(K. Arikan and N. Moore, eds.). In press.

Poremba, A., and Gabriel, M. (1997). Amygdalarlesions block discriminative avoidance learningand cingulothalamic training-induced neuronalplasticity in rabbits. J. Neurosci. 17, 5237–5244.

Posner, M. I. (1978). “Chronometric Explorations ofMind.” Erlbaum, Hillsdale, N J.

Posner, M. I., and Dehaene, S. (1994). Attentional net-works. Trends Neurosci. 17, 75–79.

Posner, M. I., and DiGirolamo, G. (1998). Executiveattention: Conflict, target detection and cognitivecontrol. In “The attentive brain” (R. Parasuraman,ed., pp. 401–423). MIT Press, Cambridge,Massachusetts.

Posner, M. I., Petersen, S. E., Fox, P. T., and Raichle,M. E. (1988). Localization of cognitive operationsin the human brain. Science 240, 1627–1631.

Pribram, K. H., and MacLean, P. D. (1953).Neurographic analysis of medial and basal cere-bral cortex. II. Monkey. J. Neurophysiol. 16, 324–340.

Raghavachari, S., Kahana, M. J., Rizzuto, D. S.,Caplan, J. B., Kirschen, M., Bourgeois, D., Madsen,J. R., and Lisman, J. E. (2001). Gating of humantheta oscillations by a working memory task. J. Neurosci. 21, 3175–3183.

Redgrave, P., Prescott, T. J., and Gurney, K. (1999). Isthe short-latency dopamine response too short tosignal reward error? Trends Neurosci. 22, 146–151.

Rescorla, R. A. (1992). Hierarchical associative rela-tions in pavlovian conditioning and instrumentaltraining. Curr. Directions Psychol. Sci. 1, 66–70.

Rylander, G. (1947). Personality analysis before andafter frontal lobotomy. In “Research Publications



Association for Research in Nervous and MentalDisease: The Frontal Lobes” (J. F. Fulton and C. D. Aring and B. S. Wortis, eds). pp. 691–705.Williams and Wilkins, Baltimore.

Sanides, F. (1970). Functional architecture of motorand sensory cortices in primates in the light of a new concept of neocortex evolution. In “The Primate Brain: Advances in Primatology” (C. R. Noback and W. Montagna, eds.) Vol. 1,pp. 137–208. Appleton-Century-Crofts, NewYork.

Sarnthein, J., Petsche, H., Rappelsberger, P., Shaw, G.L., and von Stein, A. (1998). Synchronizationbetween prefrontal and posterior associationcortex during human working memory. Proc. Natl.Acad. Sci. U.S.A. 95, 7092–7096.

Scheffers, M. K., and Coles, M. G. H. (2000).Performance monitoring in a confusing world:error-related brain activity, judgments of responseaccuracy, and types of errors. J. Exp. Psychol.: Hum.Percept. Perform. 26, 141–151.

Scheffers, M. K., Coles, M. G. H., Bernstein, P.,Gehring, W. R., and Donchin, E. (1996). Event-related brain potentials and error-related process-ing: An analysis of incorrect responses to go andno-go stimuli. Psychophysiology 33, 42–53.

Schultz, W., and Dickinson, A. (2000). Neuronalcoding of prediction errors. Ann. Rev. Neurosci. 23,473–500.

Schultz, W., Apicella, P., Romo, R., and Scarnati, E.(1995). Context-dependent activity in primatestriatum reflecting past and future behavioralevents. In “Models of information processing inthe basal ganglia” (J. C. Houk and J. L. Davis andD. G. Beiser, eds.) pp. 11–27. MIT Press,Cambridge, Massachusetts.

Schultz, W., Tremblay, L., and Hollerman, J. F. (1998).Reward prediction in primate basal ganglia andfrontal cortex. Neuropharmacology 37, 421–429.

Slobounov, S. M., Fukada, K., Simon, R., Rearick, M.,and Ray, W. (2000). Neurophysiological andbehavioral indices of time pressure effects onvisuomotor task perfomance. Cogn. Brain Res. 9,287–298.

Spanagel, R., and Weiss, F. (1999). The dopaminehypothesis of reward: Past and current status.Trends Neurosci. 22, 521–527.

Spencer, K. M., and Polich, J. (1999). PoststimulusEEG spectral analysis and P300: Attention, task,and probability. Psychophysiology 36, 220–232.

Squire, L. R. (1992). Memory and the hippocampus: Asynthesis from findings with rats, monkeys, andhumans. Psychol. Rev. 99, 195–231.

Tellegen, A., and Waller, N. G. (1996). Exploring per-sonality through test construction: Development

of the multidimensional personality question-naire. In “Personality Measures: Development andEvaluation” (S. R. Brigg and J. M. Cheek, eds.), pp. 133–161. JAI Press, Greenwich, Connecticut.

Tucker, D. M., Hartry-Speiser, A., McDougal, L., Luu,P., and deGrandpre, D. (1999). Mood and spatialmemory: Emotion and the right hemisphere con-tribution to spatial cognition. Biol. Psychol. 50,103–125.

Tucker, D. M., Derryberry, D., and Luu, P. (2000).Anatomy and physiology of human emotion:Vertical integration of brain stem, limbic, and cor-tical systems. In “The Neuropsychology ofEmotion” (J. C. Borod, ed.), pp. 56–79: OxfordUniversity Press, Oxford.

Vanderwolf, C. H., Kramis, R., and Robinson, T. E.(1978). Hippocampal electrical activity duringwaking behavior and sleep: Analyses using cen-trally acting drugs. In “Functions of the Septo-hippocampal System,” Vol. 58, pp. 199–221.Elsevier, Amsterdam.

Van’t Ent, D., and Apkarian, P. (1999). Motoricresponse inhibition in finger movement and sac-cadic eye movement: A comparative study. Clin.Neurophysiol. 110, 1058–1072.

Vidal, F., Hasbroucq, T., Grapperon, J., and Bonnet,M. (2000). Is the “error negativity’ specific toerrors? Biol. Psychol. 51, 109–128.

Vogt, B. A., Pandya, D. N., and Rosene, D. L. (1987).Cingulate cortex of the rhesus monkey: I. Cyto-architecture and thalamic afferents. J. Comp.Neurol. 262, 256–270.

Vogt, B. A., Finch, D. M., and Olson, C. R. (1993a).Functional heterogeneity in the cingulate cortex:The anterior executive and posterior evaluativeregions. Cereb. Cortex 2, 435–443.

Vogt, B. A., Sikes, R. W., and Vogt, L. J. (1993b).Anterior cingulate cortex and the medial painsystem. In “Neurobiology of the Cingulate Cortexand Limbic Thalamus” (B. A. Vogt and M. Gabriel,eds.), pp. 314–344. Birkhäuser, Boston.

Ward, A. A. (1948a). The anterior cingular gyrus andpersonality. Res. Publ. Assoc. Res. Nervous MentalDis. 27, 438–445.

Ward, A. A. (1948b). The cingulate gyrus: Area 24. J. Neurophysiol. 11, 13–24.

Zheng, F., and Khanna, S. (1999). Hippocampal fieldCA1 interneuronal nociceptive responses: Modu-lation by medial septal region and morphine.Neuroscience 93, 45–55.

Zieglagänsberger, W., French, E. D., Siggins, G. R.,and Bloom, F. E. (1979). Opioid peptides mayexcite hippocampal pyramidal neurons by inhibit-ing adjacent inhibitory neurons. Science 205,415–417.

REFERENCES 223


CHAPTER 3, FIGURE 10 Time courses of brain activity during episodic retrieval (ER) and visual attention (VA) in a left anterior medial temporal lobe region. Reprinted from Neuropsychologia; R. Cabeza, F. Dolcos, S. Prince, H. Rice, D. Weissman, and L. Nybert; Attention-related activity during episodic memory retrieval: A cross-function fMRI study. Copyright 2002, with permission from Elsevier Science.

+3.0[

-3.0

CHAPTER 8, FIGURE 2 Voltage maps showing the oscillatory nature of the ERN and sensorimotor potentials. Time is relative to a button press, which is at 0 msec.

15.

\ \

0 -5

-10

CHAPTER 8, FIGURE 4 Left: Location of the generators of the ERN. Right: Source wave forms illustrating the relative contribution of each source to the scalp-recorded ERN. The box indicates the window of the ERN.

- - Roar~A(~ /

, /


C H A P T E R

9

Effects of Age and Experienceon the Development ofNeurocognitive Systems

Teresa V. Mitchell and Helen J. Neville

INTRODUCTION

Progress in cognitive science and thedevelopment of noninvasive techniquessuch as electrophysiology have permittedextensive, ongoing mapping and differenti-ation of sensory and cognitive systems inthe mature human brain. This progresspresents a challenging opportunity for cognitive neuroscience: to characterize the processes that lead to the develop-ment of the differentiation of the maturebrain.

The event-related potential (ERP) tech-nique has been, and will continue to be,indispensable in informing us about devel-opment and how age and experiencecontribute to the adultlike mosaic of neuro-cognitive systems. It is a powerful tech-nique to use with infants and childrenbecause, although it requires attention tothe stimuli presented, it does not requireovert responses. It is also powerful becausethere are times when evidence of cognitiveand perceptual skills can be observed inERP responses, but not in overt behavioralmeasures such as looking time. Thus, theERP technique can reveal aspects of per-ception and cognition that occur inde-pendently of response preparation andexecution.

There are important issues to take intoconsideration when using the ERP tech-nique with infants and children. Multiplefactors interact to make ERP componentsrecorded from infants and children lookvery different from those recorded fromadults. Developmental changes in anatomyhave a significant impact on both thelatency and amplitude of ERP components.For example, thickness of the skull hasbeen shown to influence the amplitude ofERPs recorded on the scalp (Frodl et al.,2001) such that a thicker skull is correlatedwith smaller amplitude. A substantialincrease in skull thickness occurs overinfancy and early childhood and thischange is likely to affect amplitudesrecorded from scalp electrodes. The devel-opmental effects of increasing skull thick-ness may not affect all ERP components inthe same way; adult ERP components aredifferently affected by scalp thickness(Frodl, et al., 2001). Another substantialanatomical change across development isthe increase in myelination of white matter.Myelin serves to increase conductionvelocities along neuronal axons. Forexample, in early infancy there is a pro-gressive shortening of the latencies of theauditory brain stem response that is linkedto an increasing myelination of the brainstem (Starr et al., 1977; Eggermont, 1988).

Myelin is present in different thicknesses indifferent brain regions and pathways, andthe development of myelination pro-ceeds along nonidentical timelines acrossregions and pathways. Because of this, theeffects of myelination on the developmentof ERP components are likely to varyacross brain regions and neurocognitivesystems. Finally, regional variability insynaptogenesis and loss of synapses occurthroughout the first two decades of life, the time period of greatest changes in ERP morphologies and latencies. Thesechanges in neural development are likelyto be an additional factor contributing tothe differences in infant and adult ERPs(Aslin, 1987; Vaughan and Kurtzberg, 1992;Huttenlocher and Dabhoklar, 1997; Neville,1998).

In addition to changes associated withthese anatomical and physiological factors,changes are observed in ERPs across thelife span that are due to the factors thatinterest psychologists the most: the devel-opment of cognitive and perceptual skills.Perceptual and cognitive development isheterochronous and research has shownthat these skills, and the neural systemsthat underlie them, develop at differentrates and have different sensitive periods(see Bailey et al., 2001). Thus, multipleinteractions between anatomical and phys-iological factors and cognitive and percep-tual factors serve to produce significantchanges in scalp ERPs from infancy toadulthood.

In this chapter we describe fourdomains in which electrophysiologicaldata illustrate that development pro-ceeds at different rates in different sys-tems and that structural and functionalplasticity as expressed by early aty-pical experience differs across neuro-cognitive systems. Within each domain webriefly describe the typical findings fromadult studies, then we review evidencedemonstrating the effects of age and ofexperience on the emergence of the adult-like state.

DEVELOPMENT OF THE DORSALAND VENTRAL VISUAL STREAMS

Within the visual system there is a struc-tural and functional segregation of motionand location information from color andform information along “dorsal” and“ventral” processing streams, respectively.Visual motion is processed principally by apopulation of cell types (magno) (M) in thelateral geniculate nucleus (LGN) thatproject to particular layers in the primaryvisual cortex (Ungerleider and Mishkin,1982; Ungerleider and Haxby, 1994). Thoselayers, in turn, project to the middle tem-poral (MT) gyrus, a cortical region thatspecializes in the processing of visualmotion (Maunsell and van Essen, 1983).Color and form information, on the otherhand, are processed by a population of celltypes (parvo) (P) in the LGN that are sepa-rate from those that process motion, andthese cells project to different layers inprimary visual cortex (Ungerleider andMishkin, 1982; Ungerleider and Haxby,1994). These layers project to corticalregions within the ventral temporal andoccipital cortices that are dedicated to pro-cessing color, form, and faces (Maunselland Newsome, 1987). Little is known aboutthe plasticity of the dorsal and ventralvisual streams in humans, but evidencefrom special populations suggests thatdorsal stream functions, such as motionprocessing, are more affected by atypicalexperience, compared to ventral streamfunctions, such as color and form process-ing. Studies of patients with glaucomareport behavioral (Anderson and O’Brien,1997) and anatomical (Quigley et al., 1989)evidence of specific magnocellular (M)-layer deficits. Similarly, behavioral, physio-logical, and anatomical studies of dyslexicindividuals report deficits in dorsal streamfunctions such as motion perception (Steinand Walsh, 1997), and functional magneticresonance imaging (fMRI) data showreduced activation of area MT by motion in

226 9. DEVELOPMENT OF NEUROCOGNITIVE SYSTEMS


dyslexic adults (Eden et al., 1996; Demb et al., 1998). Psychophysical studies ofadults with Williams Syndrome also reportspared functioning of the ventral pathwayrelative to deficits observed in dorsalpathway functions (Atkinson et al., 1997;Bellugi et al., 1999). In sum, available evi-dence is consistent with the idea that thedorsal visual pathway may be moremodifiable by altered early input than isthe ventral visual pathway.

Effects of Atypical Early Experience

In our laboratory, we have studied theeffects of congenital deafness on visualfunctions. Our functional MRI studies haveshown that auditory deprivation affectsprocessing of visual motion. Congenitally,genetically deaf and hearing participantswere presented with a moving random-dotfield and were asked to attend to changesin the brightness of the dots and changesin the velocity of the motion (Bavelier et al.,2000, 2001). When attending to the centerof the flow field, the two groups producedsimilar activation. By contrast, whenattending the periphery of the flow field,deaf adults displayed more activationwithin dorsal stream area MT, the posteriorparietal cortex (PPC), and the superiortemporal sulcus (STS) than did hearing

adults. The increased recruitment of areasMT, PPC, and STS with attention toperipheral motion appears to stem fromauditory deprivation rather than the use ofa signed language: hearing adults wholearned sign language as their first lan-guage produced similar activation in theseregions when attending to motion in thevisual periphery (Bavelier et al., 2001).

These results raise the hypothesis thatauditory deprivation specifically affectsdorsal visual functions, but not ventralstream functions. To test this hypothesis,we presented normally hearing adults andcongenitally, genetically deaf adults withstimuli designed to activate differentiallythe two visual streams (Armstrong et al.,2002). The dorsal stream or “motion” stim-ulus was a low-spatial-frequency grayscalegrating. ERPs to this stimulus were time-locked to a rightward movement of thebars that lasted 100 msec. The ventralstream or “color” stimulus was a high-spatial-frequency grating of blue and greenbars. ERPs to this stimulus were recordedwhen the green bars of the stimulus turnedred for 100 msec. Participants respondedby button press to occasional presentationsof a black square, thus color and motionwere task-irrelevant features. Two promi-nent components were observed: a P1 thatwas largest over posterior, lateral temporal

DEVELOPMENT OF THE DORSAL AND VENTRAL VISUAL STREAMS 227


FIGURE 1 Average N1 amplitudesof deaf (hatched bars) and hearing(solid bars) adults in response to colorand motion stimuli. Asterisk denotessignificant population difference inresponse to motion only (p > 0.05)(from Armstrong et al., 2002).

scalp sites and peaked at roughly 120 msec,and an N1 that was observed across thescalp and peaked at roughly 180 msec.Motion stimuli evoked a minimal P1 and aprominent N1 that was larger in responseto central than peripheral presentations.Color stimuli evoked a prominent P1 andN1. Amplitudes and latencies of the P1component were similar for deaf andhearing participants across both stimulustypes. Amplitudes and latencies of the N1were also similar for the two groups inresponse to color stimuli. By contrast, N1amplitudes recorded in response to motionstimuli were reliably larger in the deaf than in the hearing participants (see Fig. 1). Furthermore, the distribution of N1responses to motion stimuli was moremedial and anterior in deaf than in hearingparticipants. The results of this ERP studyshow that, even when participants are notrequired to attend to the features of visualmotion, motion stimuli evoke greaterneural activity in deaf than in hearingadults.

Effects of Age

An understanding of the normal devel-opment of the dorsal and ventral visualstreams could provide clues as to how andwhy auditory deprivation affects dorsalstream functions more than ventral streamfunctions. What little data exist are fairlyequivocal. Behavioral studies of infant

vision suggest that the development of thedorsal stream may outpace the develop-ment of the ventral stream cells in thelateral geniculate nucleus (Dobkins et al.,1999). By contrast, anatomical studiessuggest that dorsal stream cells have amore protracted developmental trajectorycompared to the ventral stream (Hickey,1977). Studies of macaques show that face-selective regions in temporal and occipitalcortex are form and face selective by6 months of age (Rodman et al., 1991, 1993),but do not display the specificity of tuningthat is observed in adults until roughly1 year of age (Rodman, 1994). We hypothe-sized that although basic stimulus selectiv-ity may be observed early in development,adultlike functioning and tuning in theventral stream will be observed at ayounger age than in the dorsal stream. Totest this hypothesis, normally developingchildren ages 6 to 7 years of age and 8 to 10years of age, and adults, were tested usingthe same color and motion stimulus para-digm described above. Children and adultsproduced similar componentry: a promi-nent P1 and N1 in response to colorstimuli, and a smaller P1 and prominentN1 in response to motion stimuli. Al-though age-related decreases in amplitudewere observed in response to both stimu-lus types, decreases in latency wereobserved only in response to motion (seeFig. 2). Thus, significant developmentoccurs across the early school years in both



FIGURE 2 Averaged ERP traces for age groups 6 to 7 years old, 8 to 10 years old, and adults, in response tocolor and motion stimuli. Arrows point to the N1 component. Significant reductions in amplitude with age wereobserved for both stimulus types, whereas significant changes in latency with age were observed only inresponse to motion stimuli (from Mitchell and Neville, 2002).

visual streams, but speed of processingundergoes greater changes in response tomotion as compared to response to color.The relatively longer maturation of dorsalstream pathways may render them morelikely to be changed by altered earlysensory experience.

When in development do the effects ofauditory deprivation on motion processingemerge? The few studies that have investi-gated higher order attentional differencesbetween deaf and hearing children reportmixed results. A study of texture segmen-tation and visual search revealed evidenceof enhanced featural attention in deaf ascompared to hearing adults, but no suchevidence emerged from studies on children(Rettenbach et al., 1999). Two studiesemploying continuous-performance tasksreported significantly poorer performancein deaf as compared to hearing children(Quittner et al., 1994; Mitchell and Quittner,1996). An additional study investigated theeffects of motion and color as distractors ina visual search task (Mitchell and Smith,1996; Mitchell, 1996). This study wasdesigned to test the hypothesis that ifenhanced attention to motion is obligatoryin the deaf, then suppression of attentionto task-irrelevant motion should bedifficult. On the other hand, if attention tocolor is not obligatory in this population,suppression of attention to task-irrelevantcolor should be observed. The develop-mental prediction was that population dif-ferences would not be observed duringearly school years but would emerge byadulthood. To test these hypotheses, deafand hearing children ages 6 to 9 and adultsperformed two visual search tasks thatrequired attention to shape in the presenceof both color and motion distractors. In thefirst task, stimuli were presented in a circlein the center of the visual field. In this task,deaf and hearing children performed simi-larly but deaf adults were affected more byboth distractor types compared to hearingadults. In the second task, stimuli werepresented across the central and peripheral

visual fields. In this task, both deaf chil-dren and adults were more distracted bytask-irrelevant motion and color than werehearing children and adults. Thus, deafand hearing children in this study per-formed differently only when required toattend to stimuli that extended into theperiphery (Mitchell, 1996).

To further investigate the developmentof effects of auditory deprivation on visualmotion processing, we collected data from20 congenitally, genetically deaf childrenand 20 hearing children ages 6 through 10using the same color and motion ERP par-adigm described above. Analyses of theN1 show that amplitudes and latencies inresponse to color stimuli were similaracross the two groups. In response tomotion, on the other hand, some groupdifferences were observed. For centralvisual field presentations, deaf childrenproduced larger N1 amplitudes than did their hearing age mates in right-hemisphere electrodes. For peripheralvisual field presentations, overall ampli-tudes from deaf children were marginallylarger compared to those from hearingchildren. Together, these studies suggestthat some population differences emergein early school years, but these differencesvary across spatial location and stimulusfeatures. Across behavioral and ERPstudies, it appears that population differ-ences are observed more often in responseto peripheral than to central stimulationand are observed more often in response tomotion than to color, but these groupeffects in children are not as robust asthose observed in adults (Mitchell andNeville, 1999, 2002).

Results from these studies raise thehypothesis of greater modifiability withindorsal than in ventral stream areas. Thisdifference may be due to several factors,some intrinsic to the system and someextrinsic. Results from normally develop-ing children suggest that the developmentof the ventral stream may reach adultlikelevels earlier than in dorsal stream areas. If

DEVELOPMENT OF THE DORSAL AND VENTRAL VISUAL STREAMS 229


this is so, then atypical experience acrossthe lifespan may have greater opportunityto affect the structure and function of thedorsal than the ventral stream. The specificpattern of plastic changes may also be duein part to functional demands placed onthe visual system when auditory informa-tion is absent. Dorsal stream visual areasmay become more activated in deaf indi-viduals, for example, because in theabsence of auditory input, the systemwould need to attend more to dynamicvisual information to navigate throughoutthe environment. These functional needsthen interact with intrinsic developmentaltimetables and levels of plasticity withineach neurocognitive system.

SPATIAL ATTENTION: CENTRALVERSUS PERIPHERAL VISUAL

AND AUDITORY SPACE

Selective spatial attention is the abilityto focus attention on specific locations inspace, which involves either increased pro-cessing of stimuli within the attended loca-tion or inhibition of processing of stimulioutside the attended location, or both. Bothvisual and auditory spatial attention aredistributed along a gradient, with thesharpest and most efficient localization andprocessing in the center of attention and agradual degradation of processing withincreasing distance from the center(Downing and Pinker, 1985; Mondor andZatorre, 1995; Teder-Salejarvi et al., 1999).In this section, we review studies thatinvestigate the effects of unimodal sensorydeprivation on the structure and functionof spatial attention.

Effects of Auditory Deprivation onSpatial Attention

Functional MRI data reviewed aboveshow that auditory deprivation leads toenhanced activation by motion and thatthis effect is greater for the periphery than

for the center of the visual field. Studiesemploying the ERP technique have alsoshown that, whereas centrally presentedstimuli elicit similar responses in deaf andhearing adults, peripherally presentedstimuli elicit larger responses from deafthan from hearing adults (Neville andLawson, 1987). In that study, participantswere presented with alternating smallwhite squares that displayed apparentmotion. ERPs were recorded as partici-pants attended to the central or peripheralsquares and responded to indicate theirdirection of motion. Deaf participants werefaster and more accurate in detecting thedirection of motion than were hearing par-ticipants, and amplitudes of the N1 inresponse to peripheral stimuli were reli-ably larger in deaf than in hearing partici-pants. These results together suggest thateffects of auditory deprivation on spatialattention are greater on the representationof the periphery than of the center of visualspace.

Effects of Visual Deprivation on Spatial Attention

Attention to central and peripheralauditory space was compared in sightedand blind individuals in a similar para-digm. This study tested the hypothesis thateffects of sensory deprivation on spatialattention are similar across modalities. Theprediction was that the representation ofauditory peripheral space would bemodified more than the representation ofcentral auditory space in blind individuals.In this ERP study, congenitally blind andsighted, blindfolded adults attended selec-tively to noise bursts from speakersarranged either directly in front or in an arcin the far right periphery of the participant(Röder et al., 1999). ERPs were recorded asparticipants localized the noise bursts. Aprominent N1 was elicited that was largerin amplitude to attended as compared tounattended locations. Behavioral resultsshowed that blind and sighted participants



were equally accurate at localizing soundspresented within central space, but blindparticipants were more accurate thansighted participants at localizing soundsplayed from the peripheral speakers. ERPsrecorded to the sounds further demon-strated that spatial tuning in peripheralspace was sharper in blind than in sightedparticipants. As shown in Fig. 3, N1 ampli-tude was similar across the two groupswith attention to the center, whereas withattention to the periphery, amplitudes fromblind participants displayed a steeper gra-dient compared to those from sighted par-ticipants.

Together, these studies suggest thatspatial attention to peripheral space is

more modified by sensory deprivationthan is spatial attention to central space,irrespective of the deprived sensorymodality. It may be that aspects of sensorysystems that are specialized for high acuityare less modifiable by atypical input, com-pared to those displaying less acuity andprecision (see Chalupa and Dreher, 1991).In this instance, the neural representationof central space is more precisely mappedand displays higher acuity, compared toperipheral space, and it is less affected byauditory and visual deprivation than isperipheral space. The representation ofperipheral space may also be more affectedby sensory deprivation because it receivesmore converging inputs across sensory

SPATIAL ATTENTION: CENTRAL VERSUS PERIPHERAL VISUAL AND AUDITORY SPACE 231


FIGURE 3 Gradients of mean N1 amplitudes (±SE; microvolts) to standard stimuli at speakers within theattended central and attended peripheral arrays. The early attention mechanism indexed by N1 was moresharply tuned in the blind than in the sighted subjects when they were attending to targets at the peripheralspeaker 8. Reprinted by permission from Nature, Röder et al., 1999; copyright 1999 Macmillan Publishers Ltd.

modalities than does central space. Ana-tomical studies provide evidence for thishypothesis (Rockland and Ojima, 2001;Falchier et al., 2001). In intact organisms,the senses work in concert to developmaps of extrapersonal space; input fromone modality contributes to the organiza-tion of spatial representation in the othermodality. These data show that the absenceof input in one modality produces sharper,rather than coarser, spatial tuning, particu-larly in the periphery.

THE DEVELOPMENT OF FACE PERCEPTION

Face perception calls on a set of highlycomplex skills that adult humans performrapidly and with great precision and accu-racy. It includes the recognition of individ-ual faces as well as the differentiation andinterpretation of facial expressions. Amongthe neural substrates of face processing inadults are the ventral occipital and lateraltemporal lobes. Functional MRI and sub-dural electrophysiological measures haveshown that although many categories ofobjects activate ventral temporal/occipitalcortex, a region along the fusiform gyrus isactivated primarily by face stimuli (Sergentet al., 1992; Puce et al., 1995; Kanwisher et al., 1997). Event-related potential studiesof basic face processing in adults haveshown that faces, but not other objects,evoke an N170, a negative component thatis prominent at temporal scalp electrodesites and is larger over the right than theleft hemisphere (Bentin et al., 1996). Thiscomponent is also prominent in responseto face components, especially isolatedeyes. Intracranial electrophysiology revealsan N200 component that is recordeddirectly from the posterior fusiform gyrusand from the lateral middle temporalgyrus/superior temporal sulcus (Allison etal., 1999; McCarthy et al., 1999; Puce et al.,1999). These components are recordedfrom subdural electrode arrays on the

ventral portion of the occipital lobes andlateral temporal lobes in patients withepilepsy (Allison et al, 1999). A prominentN200 can be observed only in response toface stimuli, and is observed in the fusi-form gyrus, occipitotemporal sulcus, infe-rior temporal gyrus, and the lateral middletemporal gyrus/superior temporal sulcus.This component is relatively insensitive tohabituation, and electrical stimulation ofthe region that generates it produces atransient inability to name familiar faces(Allison et al., 1994). The scalp N170 com-ponent is believed to be generated in thelateral regions (superior temporal sulcusand/or occipitotemporal sulcus), ratherthan the ventral fusiform region (Bentin etal., 1996), because of its distribution andbecause it is prominent in response to eyes,unlike the N200. Overall, both the scalpN170 and the intracranial N200 are hypo-thesized to be involved more in dis-criminating faces from other stimuli (i.e.,identification of “facedness”) and less inidentification of individual faces (McCarthy,2001).

The development of this neural sub-strate of face perception is relativelyunknown. A two-phase theory has beenproposed for the development of theneural substrate of face perception (Mortonand Johnson, 1991). During the first phaseof development, the theory proposes thatsubcortical mechanisms drive infants tofixate on faces. Newborn infants prefer tolook at faces and facelike stimuli longerthan many other visual stimuli (Fantz,1963). This tendency to fixate on faces ishypothesized to train and tune corticalregions downstream to respond to facesand eventually to play a role in distin-guishing one face from another. In thesecond phase of development, experiencein perceiving faces, learning to recognizefamiliar faces, and differentiating facialexpressions drives the establishment of theadultlike cortical substrate, including thefusiform gyrus and lateral temporalregions. This initial setting up of cortical



regions is hypothesized to take placewithin the first year of life, but the long-term developmental trajectory and plastic-ity of these regions are only beginning tobe mapped out. In this section, we reviewERP studies that shed light on the develop-ment of the neural substrates of face per-ception.

Infant Face Recognition

A basic measure of infant face recogni-tion is the ability to discriminate the face ofone’s own mother from the face of anotherwoman. In behavioral paradigms, evidenceof recognition of the mother’s face can varyboth with age and with dependentmeasure. De Haan and colleagues studiedthe ability of 6-month-old infants to recog-nize their own mother (de Haan andNelson, 1997). In a preferential lookingparadigm, 6-month-old infants spent anequivalent amount of time fixating on theirown mother’s face and on a stranger’sface. However, ERP measures in this sameage group showed that processing of themother’s face was different from that of astranger’s face. Specifically, the amplitudeof the Nc component, a negative deflectionoccurring between 400 and 800 msec post-stimulus, was larger in response to pho-tographs of the infant’s own mother,compared to a photograph of anotherinfant’s mother (de Haan and Nelson,1997). This effect was observed overmidline and right anterior electrodes.

de Haan and colleagues further investi-gated effects of category and familiarity onface processing in 6-month-old infants (deHaan and Nelson, 1999). In this study,infants were presented with two categoriesof photographs and two levels of familiar-ity: photographs of familiar and unfamiliartoys and photographs of the infant’s ownmother or another infant’s mother. Twocomponents displayed category effects, apositivity occurring around 400 msec (theP400) and a middle latency negative ERP(the Nc) occurring around 600 msec. A cat-

egory effect was observed for the P400such that it was faster in response to facesthan to objects. Both category and familiar-ity effects were observed for the Nc.Familiar stimuli, regardless of category,produced larger Nc amplitudes than didunfamiliar stimuli across frontal sites.Further, this familiarity effect displayed aright frontal distribution for faces, but wasbilaterally distributed for objects. Thisreplicates the right frontal distribution ofthe Nc, in response to faces, that wasobserved in the study described above (deHaan and Nelson, 1997). This more focaldistribution may be early evidence of abasic right hemisphere asymmetry, or itmay be due to infants’ greater experiencewith faces than with toys. The authors con-clude that basic encoding of faces ininfants occurs by 400 msec and is indexedby the P400, whereas recognition processesoccur by 600 msec and are indexed by theNc. Together, these studies demonstratethat faces evoke a different pattern ofneural activity compared to other visualobjects by 6 months of age.

Face Processing across the Early School Years

One developmental question is whetherthe mechanisms involved in face process-ing undergo qualitative or quantitativechange from early school years to adult-hood. Despite the precociousness of infantface processing, the performance of youngchildren on a variety of face recognitionand discrimination tasks is far below adult levels (Carey and Diamond, 1994;Baenninger, 1994). At issue is whether chil-dren perform poorly on these tasksbecause the way they process faces is dif-ferent from the way adults do, or whetherthe same processes are at work but aresimply slower and less proficient.

To begin addressing this developmentalissue, Taylor and colleagues have studiedthe developmental characteristics of thescalp N170 in children and adults. In an

THE DEVELOPMENT OF FACE PERCEPTION 233


early study, participants, ages 4 to adult,viewed a series of individual black andwhite photographs of faces, cars, scram-bled cars, scrambled faces, and butterflies(Taylor et al., 1999). Butterflies were thetarget stimulus to which participantsresponded with a button press. An N170was observed in response to faces in all agegroups. Latencies of this componentdecreased linearly with age, accompaniedby variable changes in amplitude. A largeincrease in amplitude occurred betweensubjects 12–14 years old and adult subjects.The typical right-greater-than-left hemi-spheric asymmetry of the N170 amplitudeobserved in adults emerged with age, froma bilateral distribution in the younger chil-dren. This was mainly due to an increase inamplitudes with age recorded from right-hemisphere sites. In sum, the N170 isprominent in response to faces even in 4-year-old children, its latency decreaseswith age, and the right hemispheric asym-metry of this component also emerges withage.

In a similar study, Taylor and colleaguesinvestigated the effects of face inversion[hypothesized to elicit “featural” process-ing of faces in contrast to the “configural”processing typical of upright faces; (Careyand Diamond, 1994)] and the presentationof isolated eyes on the N170 in participants

ages 4 to adult (Taylor et al., 2001).Participants viewed upright faces, invertedfaces, scrambled faces, eyes, and flowers(and checkerboards, to which they res-ponded as the target). The N170 wasobserved in all age groups in response toeyes, upright faces, and inverted faces.Overall latencies of the N170 decreased lin-early across age groups, with the greatestdecrease observed in response to uprightfaces. As in the study described above, theyoungest age groups displayed a bilateraldistribution in response to faces, whereasolder age groups displayed an adultlikeright-greater-than-left hemispheric asym-metry. By contrast, the N170 in response toeyes produced a right-greater-than-lefthemispheric asymmetry in all age groups.As shown in Fig. 4, amplitudes in responseto eyes decreased significantly across theearly school years; overall amplitudes inresponse to upright faces were relativelyconsistent across all age groups but ampli-tudes in response to inverted facesincreased with age. Latencies were shorterin response to inverted compared toupright faces in the younger age groups,but this reversed around the onset ofpuberty. In sum, responses to eyes reachedan adultlike profile around age 11, whereasresponses to inverted faces were not adult-like until after puberty. These differences



FIGURE 4 Average N170 amplitudes acrossage groups for eyes (•), faces (), inverted faces(), and scrambled faces (). Amplitudes inresponse to eyes decreased significantly acrossthe early school years whereas overall ampli-tudes in response to upright faces were rela-tively consistent across all age groups.Amplitudes in response to inverted facesincreased with age, and amplitudes to uprightfaces were stable across age. Reprinted fromClinical Neurophysiology, Vol. 110; M. J. Taylor, G.McCarthy, E. Saliba, and E. Degiovanni; ERP evi-dence of developmental changes in processing offaces, pp. 910–915. Copyright 1999 with permis-sion from Elsevier Science.

raise the hypothesis that there are dif-ferences in the development of the neural representation of eyes as compared tofaces, and upright versus inverted faces.

Additional research with school-agedchildren similarly reports differing devel-opmental trends for ERPs recorded inresponse to upright and inverted faces.Alvarez and colleagues recorded ERPs inresponse to upright and inverted faces inparticipants aged 9, 13, and 16 years, andadults, during a face recognition matchingtask (Alvarez and Neville, 1995). Par-ticipants were presented with one face, fol-lowed by a second, and were asked torespond to indicate whether the secondface matched the first. Behavioral dataindicated that all age groups were better atrecognizing upright compared to invertedfaces, and accuracy and reaction time (RT)improved steadily with age. Electro-physiological data revealed a negativitybetween 250 and 450 msec (the N320) inresponse to the second face that was largerfor mismatched than for matched faces atall ages. Two developmental changes wereobserved in this component. First, al-though adults produced larger N320amplitudes in the right hemisphere com-pared to the left hemisphere, children produced similar amplitudes across hemi-spheres. Second, in adults N320 ampli-tudes were larger for upright than forinverted faces, whereas in children similaramplitudes were observed across face ori-entations. Adultlike ERPs were observedonly in participants ages 16 and older.Thus, the developmental changes in theN320, an index of face recognition, mirrorthose observed in the N170: the right-hemisphere asymmetry and face inversioneffects emerge slowly with age.

Face Processing in Williams Syndrome

Williams Syndrome (WS) is a geneticdisorder that produces severe deficits inmany aspects of cognitive processing, but

leaves face processing and verbal skills rel-atively spared (Bellugi et al., 2000). In orderto investigate whether face processingrelies on the same neural substrates inadult Williams patients and controls, Millsand colleagues (2000) compared facerecognition using the same paradigm as inthe Alvarez study described above. Allparticipants were faster and more accuratein response to upright compared toinverted faces. Participants with WS wereslower and less accurate overall thancontrol subjects, but most of the individualsubjects performed within the range of thecontrols. Differences in the amplitude anddistribution of the N320 were observedbetween the two subject groups. Normaladults displayed larger amplitudes to themismatched targets compared to thematched targets for upright, but notinverted, faces. In contrast, the WS subjectsdisplayed a match/mismatch effect forboth face orientation conditions. Fur-ther, the overall match/mismatch effecttended to be larger from the right hemi-sphere in controls, but larger from the left hemisphere in WS adults. These results suggest that WS adults, like normal children and unlike normal adults,do not employ different brain systems for recognizing upright and inverted faces.

These studies show that although thereis overlap in the systems involved in faceprocessing from early school years toadulthood, age and experience affect thefunctioning and distribution of thisnetwork in significant ways. Effects of faceorientation and a right-hemisphere asym-metry can be observed in behavioral measures (de Schonen and Mathivet, 1990) as well as ERP measures early innormal development, but an adultlikeprofile is not attained until puberty acrossmany studies. This suggests that increas-ing skill and experience in perceiving faces may be one factor that drives the differentiation of the substrates for pro-cessing upright and inverted faces.

THE DEVELOPMENT OF FACE PERCEPTION 235


DEVELOPMENTALNEUROPLASTICITY OF

LANGUAGE FUNCTIONS

The neural systems important for lan-guage functions are distributed acrossseveral brain regions, and these regions aredifferentially involved in the processing ofdifferent aspects of language. Observationsof varying rates of differentiation anddegrees of specification can contribute tothe identification and characterization ofdifferent functional subsystems within lan-guage. In a series of experiments, we havestudied the development of the neuralsystems important in lexical, semantic, andgrammatical processing. In normal, right-handed, monolingual adults, nouns andverbs (“open class” words) that providelexical/semantic information elicit a differ-ent pattern of brain activity (as measuredby ERPs) than do function words, includ-ing prepositions and conjunctions (“closedclass” words), which provide grammaticalinformation in English (Neville et al., 1992;Nobre and McCarthy, 1994). In addition,sentences that are semantically nonsensicalbut are grammatically intact elicit a differ-ent pattern of ERPs compared to sentencesthat contain a violation of syntactic struc-ture but leave the meaning intact (Nevilleet al., 1991; Osterhout et al., 1997). Theseresults are consistent with other types of evidence that suggest that different neural systems mediate the processing oflexical/semantic and grammatical infor-mation in adults. Specifically, they imply agreater role for more posterior temporal-parietal systems in lexical/semantic pro-cessing and for frontal-temporal systemswithin the left hemisphere in grammaticalprocessing. This overall pattern appearsubiquitous in adults, and many investiga-tors have suggested that the central role ofthe left hemisphere in language processingis strongly genetically determined. Cer-tainly the fact that most individuals,regardless of the language they learn,

display left-hemisphere dominance for thatlanguage indicates that this aspect ofneural development is strongly biased.Nonetheless, it is likely that language-rele-vant aspects of cerebral organization aredependent on and modified by languageexperience.

Studies of Bilingual Adults

Many investigators have studied theeffects of language experience on neuraldevelopment by comparing cerebral orga-nization in individuals who learned asecond language at different times indevelopment (Dehaene et al., 1997; Kim etal., 1997; Perani et al., 1996). In general, ageof exposure to language appears to affectcerebral organization for that language.Moreover, there appears to be specificity inthese effects. In Chinese–English bilin-guals, delays of as long as 16 years in expo-sure to English had very little effect on theorganization of the brain systems active inlexical/semantics processing. In contrast,delays of only 4 years had significanteffects on those aspects of brain organiza-tion linked to grammatical processing(Weber-Fox and Neville, 1996). Theseresults and parallel behavioral results fromthe same study suggest that aspects ofsemantic and grammatical processingdiffer markedly in the degree to whichthey depend on language input. Speci-fically, grammatical processing appearsmore vulnerable to delays in languageexperience.

Studies of Deaf Adults

Further evidence of the plasticity of lan-guage systems was provided by ERPstudies of English sentence processing incongenitally deaf individuals who learnedEnglish late and as a second language[American Sign Language (ASL) was thefirst language of these individuals (Nevilleet al., 1992)]. These deaf participants dis-played ERP responses to nouns and to


II. COGNITIVE PROCESSING IN THE INTACT BRAIN

semantically anomalous sentences inwritten English that were indistinguishablefrom those of normally hearing partici-pants who learned English as a first lan-guage. These data are consistent with thehypothesis that some aspects of lexical/semantic processing are largely unaffectedby the many different aspects of languageexperience that differ between normallyhearing and congenitally deaf individuals.By contrast, deaf participants displayedaberrant ERP responses to grammaticalinformation such as that presented in func-tion words in English. Specifically, they didnot display the specialization of the ante-rior regions of the left hemisphere that ischaracteristic of native, hearing/speakinglearners. These data suggest that thesystems that mediate the processing ofgrammatical information are more modi-fiable and vulnerable in response to alteredlanguage experience than are those associ-ated with lexical/semantic processing.

Studies of ASL

We have employed the ERP and fMRItechniques to further pursue this hypothe-sis and also to obtain evidence on the ques-tion of whether the strongly biased role ofthe left hemisphere in language occursindependently of the structure and modal-ity of the language first acquired (Nevilleet al., 1997, 1998). ERPs were recorded fromhearing and deaf adults who learned ASLas a first language and from hearing sub-jects who acquired ASL late or not at all, asthey viewed ASL signs that formed sen-tences. The results were compared acrossthese groups and with those from hearingsubjects reading English sentences. ERPsrecorded to response to open and closedclass signs in ASL sentences displayedsimilar timing and anterior/posterior dis-tributions to those observed in previousstudies of English. However, whereas innative speakers of English responses toclosed class English words were largestover anterior regions of the left hemi-

sphere, in native signers closed class ASLsigns elicited activity that was bilateraland that extended posteriorly to includeparietal regions of both the left and righthemispheres. These results imply that theacquisition of a language that relies onspatial contrasts and the perception ofmotion may result in the inclusion of right-hemisphere regions into the languagesystem. Both hearing and deaf nativesigners displayed this effect. However,hearing people who acquired ASL in thelate teens did not show this effect, suggest-ing there may be a limited time (sensitive)period during which this type of organ-ization for grammatical processing candevelop (Newman et al., 2002). By contrastthe response to semantic information wasnot affected by age of acquisition of ASL,in keeping with the results from studies ofEnglish that suggest that these differentsubsystems within language display dif-ferent degrees of developmental plasticity.

In fMRI studies comparing sentenceprocessing in English and ASL, we alsoobserved evidence for biological con-straints and effects of experience on themature organization of the languagesystems of the brain. As in the studydescribed above, hearing adults with noASL background, hearing native signersand deaf native signers, were imagedwhile observing written English sentencesand while observing sentences signed inASL (Neville et al., 1998). When hearingadults read English, their first language,there was robust activation within the leftbut not the right hemisphere and in partic-ular within the inferior frontal (Broca’s)regions. When deaf people read English,their second language, learned late andimperfectly, these regions within the lefthemisphere were not activated. Is this lackof left-hemisphere activation in the deaflinked to lack of auditory experience withlanguage or to incomplete acquisition ofEnglish grammar? ASL is not sound basedbut displays each of the characteristics ofall formal languages, including a complex

DEVELOPMENTAL NEUROPLASTICITY OF LANGUAGE FUNCTIONS 237


grammar that makes extensive use ofspatial location and hand motion (Klimaand Bellugi, 1988). When deaf subjectsviewed sentences in their native ASL, weobserved activation within the same infe-rior frontal regions of the left hemispherethat become active when native speakers ofEnglish process English. These data sug-gest that there is a strong biological bias forthese neural systems to mediate gram-matical language, regardless of the struc-ture and modality of the languageacquired. However, if the language is notacquired within the appropriate timewindow this strong bias is not expressed.The fMRI data also indicate a robust rolefor the right hemisphere in processingASL. These results suggest that the natureof the language input, in this case the co-occurrence of location and motion informa-tion with language, shape the organizationof the language systems of the brain.Further research is necessary to specify thedifferent times in human developmentwhen particular types of input are requiredfor optimal development of the manysystems and subsystems important inlanguage processing.

Effects of Primary Language Acquisitionon Cerebral Organization

The research summarized above impliesthat language experience determines thedevelopment and organization of lan-guage-relevant systems of the brain. Astrong test of this hypothesis would be tochart the changes in brain organization aschildren acquire their primary languageand to separate these from more generalmaturational changes. We compared pat-terns of neural activity relevant to lan-guage processing in 13- and 20-month-oldinfants to determine whether changes incerebral organization occur as a function ofspecific changes in language developmentwhen chronological age is held constant(Mills et al., 1993, 1997; Neville and Mills,1997). ERPs were recorded as children lis-

tened to words they knew, words they didnot know, and backward words. Specificand different ERP components dis-criminated comprehended words fromunknown and from backward words. Dis-tinct lateral and anterior/posterior distrib-utions were apparent in ERP responses tothe different types of words. At 13 monthsof age, the effects of word comprehensionwere apparent over anterior and posteriorregions of both the left and right hemi-spheres. However, at 20 months of age,these effects occurred only over temporaland parietal regions of the left hemisphere.This increasing specialization of language-relevant systems was not solely dependenton chronological age. Comparisons ofchildren of the same age who differed insize of vocabulary demonstrated that lan-guage experience and knowledge werestrongly predictive of the maturity ofcerebral organization; 13-month-old infantswith large vocabularies displayed morefocal left temporal/parietal effects of wordmeaning than did those with smallvocabularies.

A similar effect is found in the develop-ment of the differential processing of openand closed class words. We comparedERPs to open and closed class words ininfants and young children from 20 to 42months of age (Mills et al., 1997; Nevilleand Mills, 1997). All children understoodand produced both the open and closedclass words presented. At 20 months, ERPsin response to open and closed class wordsdid not differ (see Fig. 5). However, bothtypes of words elicited ERPs that differedfrom those elicited by unknown and back-ward words. These data suggest that in theearliest stages of language development,when children are typically speaking insingle-word utterances or beginning to puttwo words together, open and closed classwords elicit similar patterns of brain activ-ity. At 28–30 months of age, when childrentypically begin to speak in short phrases,ERPs to open and closed class wordselicited different patterns of brain activity.



However, the more mature left-hemisphereasymmetry to closed class words was notobserved. By 36 months of age most chil-dren speak in sentences and use closedclass words appropriately to specify gram-matical relations and, like adults, ERPsfrom 36-month-old children displayed aleft-hemisphere asymmetry to closed classwords. The results across the three agegroups are consistent with the hypothesisthat open and closed class words areprocessed initially by similar brain systemsand that these systems become progres-sively specialized with increasing languageexperience. Further evidence on thishypothesis comes from an examination ofERPs from children who were the same agebut who differed in language abilities. The20-month-old children who scored below

the 50th percentile for vocabulary size didnot show ERP differences to open andclosed class words. In contrast, those withvocabulary sizes above the 50th percentiledisplayed ERP differences to open andclosed class words that were similar to the28- to 30-month-old patterns. These datastrongly suggest that the organization ofbrain activity is linked to language abilitiesrather than to chronological age.

Across these studies we have shownthat the cerebral organization of languageis the product of interactions between bio-logical constraints and language experi-ence across the life span, as has beendescribed for many other systems indevelopmental biology. Left-hemisphereinvolvement in language processing ap-pears to be highly biologically determined

DEVELOPMENTAL NEUROPLASTICITY OF LANGUAGE FUNCTIONS 239


FIGURE 5 Current source density (CSD) analyses of neural activity to closed class words at 200 msec. TheCSDs illustrate sinks (i.e., activity flowing into the head, shown in purple) and sources (i.e., activity flowing outof the head, shown in orange), at three age groups. At 36–42 months the CSD shows a sink over left anteriorregions. At 28–30 months the CSD shows sinks that are bilateral but slightly more prominent over the right com-pared to the left hemisphere. At 20 months the CSD shows sinks over both the left and the right hemispheres(from Neville and Mills, 1997). (See color plates.)

and is relatively unaffected by languagemodality. However, markedly differentlanguage experience, such as learning amanual, spatial language as one’s nativelanguage, recruits the right hemisphereinto language processing to an extent notobserved in spoken and written languages.These results suggest that the nature of thelanguage input, in this case the co-occur-rence of location and motion informationwith language, shapes the organization ofthe language systems of the brain. Finally,the developmental data illustrate the roleof language abilities in establishing thecanonical organization of the languagesystems of the brain. Thus, similar princi-ples govern the development and organi-zation of neural systems important inperception and language.

SUMMARY

Factors That Influence the Developmentand Plasticity of Neurocognitive Systems

The studies reviewed in this chapterillustrate that the development of neu-rocognitive systems is influenced byfactors both intrinsic and extrinsic to thechild. For example, evidence of a left-hemi-sphere asymmetry for language processingand a right-hemisphere asymmetry for faceprocessing is observed early in develop-ment. However, the left-hemisphere asym-metry for language appears to be driven, atleast in early development, by the accumu-lation of words in the vocabulary. Theright-hemisphere asymmetry for faces doesnot reach an adultlike profile until wellinto puberty. Although these features oflanguage and face processing may be rela-tively stable in the face of atypical experi-ence, other aspects of perception andcognition display greater plasticity. Right-hemisphere involvement is more heavilyrecruited with early acquisition of a signedlanguage. Attention to peripheral space isheightened by both auditory and visual

deprivation, but attention to central spaceis not. Processing along the dorsal visualstream is enhanced following auditorydeprivation, but processing along theventral stream is not. These complexitiesreveal the need for careful characterizationof developmental events within the specificneurocognitive systems and subsystems.

The variation in the effects of atypicalexperience may be due, in part, to differingrates of development across neural regionsand neurocognitive systems. This fact hasimportant consequences for plasticity anddevelopment because different types ofatypical early experience may affect earlydeveloping, as compared to late-develop-ing, systems differently. Systems thatundergo an early and short sensitiveperiod are strongly affected, both struc-turally and functionally, by atypical earlyexperience. For example, binocular visionrequires competing input from the twoeyes in order for ocular dominancecolumns to be set up normally in primaryvisual cortex (Wiesel and Hubel, 1965;Tychsen, 2001). If input to one eye is dis-rupted within the first months of life, thepruning and branching necessary forocular dominance columns does not occurto the extent that it does with normal,binocular input, and there are lifelong con-sequences for visual acuity even wheninput from the deprived eye is restored. Inthese systems with early and short sensi-tive periods, deprivation of the input nec-essary to lay down the basic wiring andfunction will prevent the typical develop-mental outcome from emerging. On theother hand, systems that undergo pro-tracted periods of development may beless affected by acute atypical early experi-ence if input is restored. If the develop-mental window is long enough, therestored input may be sufficient to estab-lish relatively normal developmental out-comes. However, these systems withlonger sensitive periods may be moreaffected by chronic atypical experience,even when the atypical experience is not



thought of as “necessary.” One example isthe impact of auditory deprivation onvisual functions. Although auditory inputis not thought of as necessary in establish-ing the neural substrates of visual func-tions, deprivation of this input across thelife span can have an impact on specificvisual subsystems. In this case, when atyp-ical input is provided throughout thedevelopmental time course, this may besufficient to induce plastic changes in thestructure and function of these subsystems.We hypothesize that this is the case withthe cross-modal developmental effects wereport here.

In this chapter we have presented evi-dence that biological constraints and expe-rience interact epigenetically to producethe neural substrates of perception andcognition. Further research is necessary tospecify the different times in human devel-opment when particular types of input arerequired for optimal development of themany systems and subsystems importantin cognition and perception and to under-stand to what degree these systems aremodifiable by atypical experience and themechanisms that underlie and permit thismodifiability.

ReferencesAllison, T., Ginter, H., McCarthy, G., Nobre, A. C.,

Puce, A., Luby, M., and Spencer, D. D. (1994). Facerecognition in human extrastriate cortex. J. Neuro-physiol. 71, 821–825.

Allison, T., Puce, A., Spencer, D. D., and McCarthy, G.(1999). Electrophysiological studies of human faceperception. I: Potentials generated in occipitotem-poral cortex by face and non-face stimuli. Cereb.Cortex 9, 415–430.

Alvarez, T. D., and Neville, H. J. (1995). The develop-ment of face discrimination continues into adult-hood: An ERP study. Soc. Neurosci. Abstr. 21, 2086.

Anderson, R. S., and O’Brien, C. (1997). Psycho-physical evidence for a selective loss of M ganglion cells in glaucoma. Vis. Res. 37(8),1079–1083.

Armstrong, B., Neville, H., Hillyard, S., and Mitchell, T. (2002). Effects of auditory deprivationon motion and color processing. Cogn. Brain Res.(in review).

Aslin, R. N. (1987). Visual and auditory developmentin infancy. In “Handbook of Infant Development”(J. D. Osofsky, ed.), pp. 5–97. Wiley, New York.

Atkinson, J., King, J., Braddick, O., Nokes, L., Anker,S., and Braddick, F. (1997). A specific deficit ofdorsal stream function in Williams’ syndrome.NeuroReport 8(8), 1919–1922.

Baenninger, M. (1994). The development of facerecognition: Featural or configurational process-ing? J. Exp. Child Psychol. 57, 377–396.

Bailey, D. B., Bruer, J. T., Symons, F. J., and Lichtman,J. W. (eds.) (2001). “Critical Thinking aboutCritical Periods: Perspectives from Biology,Psychology and Education.” Paul H. BrookesPubl. Co., Baltimore.

Bavelier, D., Tomann, A., Hutton, C., Mitchell, T.,Corina, D., Liu, G., and Neville, H. (2000). Visualattention to the periphery is enhanced in congeni-tally deaf individuals. J. Neurosci. 20, RC93, 1–6.

Bavelier, D., Brozinsky, C., Tomann, A., Mitchell, T.,Neville, H., and Liu, G. (2001). Impact of earlydeafness and early exposure to sign language onthe cerebral organization for motion processing.J. Neurosci. 21(22), 8931–8942.

Bellugi, U., Lichtenberger, L., Mills, D., Galaburda, A.,and Korenberg, J. R. (1999). Bridging cognition,the brain and molecular genetics: Evidence fromWilliams syndrome. Trends Neurosci. 22(5),197–207.

Bellugi, U., Lichtenberger, L., Jones, W., Lai, Z., andSt. George, M. (2000). The neurocognitive profileof Williams Syndrome: A complex pattern ofstrengths and weaknesses. J. Cogn. Neurosci.12(Suppl. 1), 7-29.

Bentin, S., Allison, T., Puce, A., Perez, E., andMcCarthy, G. (1996). Electrophysiological studiesof face perception in humans. J. Cogn. Neurosci. 8,551–565.

Carey, S., and Diamond, R. (1994). Are faces perceivedas configurations more by adults than children?Vis. Cogn. 1, 253–274.

Chalupa, L., and Dreher, B. (1991). High precisionsystems require high precision ‘blueprints’: a newview regarding the formation of connections inthe mammalian visual system. J. Cogn. Neurosci. 3,209–219.

de Haan, M., and Nelson, C. A. (1997). Recognition ofthe mother’s face by six-month-old infants: A neu-robehavioral study. Child Dev. 68(2), 187–210.

de Haan, M., and Nelson, C. A. (1999). Brain activitydifferentiates face and object processing in 6-month-old infants. Dev. Psychol. 35(4), 1113–1121.

Dehaene, S., Dupoux, E., Mehler, J., Cohen, L.,Paulesu, E., Perani, D., van de Moortele, P. F.,Lehericy, S., and Le Bihan, D. (1997). Anatomicalvariability in the cortical representation of firstand second language. NeuroReport 8(17),3809–3815.

REFERENCES 241


Demb, J. B., Boynton, G. M., and Heeger, D. J. (1998).Functional magnetic resonance imaging of earlyvisual pathways in dyslexia. J. Neurosci. 18(17),6939–6951.

de Schonen, S., and Mathivet, E. (1990). Hemisphericasymmetry in a face discrimination task in infants.Child Dev. 61, 1192–1205.

Dobkins, K. R., Anderson, C. M., and Lia, B. (1999).Infant temporal contrast sensitivity functions(tCSFs) mature earlier for luminance than for chro-matic stimuli: Evidence for precocious magnocel-lular development? Vis. Res. 39(19), 3223–3239.

Downing, C. J., and Pinker, S. (eds.) (1985). “Attentionand Performance.” Erlbaum Assoc., Hillsdale,New Jersey.

Eden, G. F., VanMeter, J. W., Rumsey, J. M., Maisog,J. M., Woods, R. P., and Zeffiro, T. A. (1996).Abnormal processing of visual motion in dyslexiarevealed by functional brain imaging. Nature382(6586), 66–69.

Eggermont, J. J. (1988). On the rate of maturation ofsensory evoked potentials. Electroencephalogr. Clin.Neurophysiol. 70, 293–305.

Falchier, A., Renaud, L., Barone, P., and Kennedy, H.(2001). Extensive projections from the primaryauditory cortex and polysensory area stp toperipheral area V1 in the macaque. Soc. Neurosci.Abstr. 27, S11.21.

Fantz, R. (1963). Pattern vision in newborn infants.Science 140, 296–297.

Frodl, T., Meisenzahl, E. M., Muller, D., Leinsinger, G.,Juckel, G., Hahn, K., Moller, H. J., and Hegerl, U.(2001). The effect of the skull on event-relatedP300. Clin. Neurophysiol. 112, 1173–1176.

Hickey, T. L. (1977). Postnatal development of thehuman lateral geniculate nucleus: Relationship toa critical period for the visual system. Science 198,836–838.

Huttenlocher, P. R., and Dabhoklar, A. S. (1997).Regional differences in synaptogenesis in humanCerebral cortex. J. Comp. Neurol. 387, 167–178.

Kanwisher, N., and McDermott, J., Chun, M. (1997).The fusiform face area: A module in human extras-triate cortex specialized for face perception.J. Neurosci. 17, 4302-4311.

Kim, K. H. S., Relkin, N. R., Lee, K.M., and Hirsch, J.(1997). Distinct cortical areas associated withnative and second languages. Nature 388, 171-174.

Klima, E. S., and Bellugi, U. (1988). “The Signs ofLanguage.” Harvard University Press, Cambridge,Massachusetts.

Maunsell, J. H. R., and Newsome, W. T. (1987). Visualprocessing in monkey extrastriate cortex. Annu.Rev. Neurosci. 10, 363-401.

Maunsell, J. H., and van Essen, D. C. (1983). The con-nections of the middle temporal visual area (MT)and their relationship to a cortical hierarchy in themacaque monkey. J. Neurosci. 3(12), 2563-2586.

McCarthy, G. (2001). Physiological studies of face pro-cessing in humans. “The New CognitiveNeurosciences” (M. S. Gazzaniga, ed.), pp. 393-409. MIT Press, Cambridge, Massachusetts.

McCarthy, G., Puce, A., Belger, A., and Allison, T.(1999). Electrophysiological studies of human faceperception. II: Response properties of face-specificpotentials generated in occipitotemporal cortex.Cereb. Cortex 9, 431-444.

Mills, D. L., Coffey-Corina, S. A., and Neville, H. J.(1993). Language acquisition and cerebral special-ization in 20-month-old infants. J. Cogn. Neurosci.5(3), 317–334.

Mills, D. L., Coffey-Corina, S., and Neville, H. J.(1997). Language comprehension and cerebral spe-cialization from 13 to 20 months. Dev.Neuropsychol. 13(3), 397-445.

Mills, D. L., Alvarez, T. D., St. George, M.,Appelbaum, L. G., Bellugi, U., and Neville, H.(2000). Electrophysiological studies of face pro-cessing in Williams Syndrome. J. Cogn. Neurosci. 12(Suppl.), 47-64.

Mitchell, T. V. (1996). “How Audition Shapes VisualAttention”. Department of Psychology, IndianaUniversity, Bloomington.

Mitchell, T., and Neville, H. (1999). The electrophysi-ology of motion and color perception from earlyschool years to adulthood. Soc. Res. Child Dev.

Mitchell, T., and Neville, H. J. (2002). Asynchrony inthe development of electrophysiological responsesto motion and color. In preparation.

Mitchell, T. V., and Quittner, A. L. (1996). A multi-method study of attention and behavior problemsin hearing-impaired children. J. Clin. Child Psychol.25, 83-96.

Mitchell, T. V., and Smith, L. B. (1996). Deafness drivesdevelopment of attention to change. “Proceedingsof the Eighteenth Annual Conference of theCognitive Science Society.” Erlbaum Assoc.,Hillsdale, New Jersey.

Mondor, T. A., and Zatorre, R. J. (1995). Shifting andfocusing auditory spatial attention. J. Exp. Psychol.Hum. Percept. Perform. 211, 387-409.

Morton, J., and Johnson, M. H. (1991). CONSPEC andCONLERN: A two-process theory of infant facerecognition. Psychol. Rev. 98(2), 164-181.

Neville, H. J. (1998). Human brain development.“Fundamental neuroscience” (M. Posner and L.Ungerleider, eds.). Academic Press, New York.

Neville, H. J., and Lawson, D. (1987). Attention tocentral and peripheral space in a movement detec-tion task: An event-related potential and behav-ioral study. II. Congenitally deaf adults. Brain Res.405, 268-283.

Neville, H. J., and Mills, D. L. (1997). Epigenesis oflanguage. Mental Retard. Dev. Disabil. Res. Rev. 3(4),282-292.

Neville, H. J., Nicol, J., Barss, A., Forster, K., andGarrett, M. (1991). Syntactically based sentence



classes: Evidence from event-related brain poten-tials. J. Cogn. Neurosci. 3: 155-170.

Neville, H. J., Mills, D. L., and Lawson, D. S. (1992).Fractionating language: Different neural subsys-tems with different sensitive periods. Cereb. Cortex2(3), 244-258.

Neville, H. J., Coffey, S. A., Lawson, D. S., Fischer, A.,Emmorey, K., and Bellugi, U. (1997). Neuralsystems mediating American Sign Language:effects of sensory experience and age of acquisi-tion. Brain Lang. 57(3), 285-308.

Neville, H. J., Bavelier, D., Corina, D., Rauschecker, J.,Karni, A., Lalwani, A., Braun, A., Clark, V.,Jezzard, P., and Turner, R. (1998). Cerebralorganization for language in deaf and hearingsubjects: Biological constraints and effects ofexperience. Proc. Natl. Acad. Sci. U.S.A 95(3), 922-929.

Newman, A. J., Bavelier, D., Corina, D., Jezzard, andP., Neville, H.J. (2002). A critical period for right hemisphere recruitment in American Sign Language processing. Nat. Neurosci. 5(1), 76-80.

Nobre, A. C., and McCarthy, G. (1994). Language-related ERPs: Scalp distributions and modulationby word type and semantic priming. J. Cogn.Neurosci. 6, 233-255.

Osterhout, L., McLaughlin, J., and Bersick, M. (1997).Event-related brain potentials and human lan-guage. Trends Cogn. Sci. 1(6), 203-209.

Perani, D., Dehaene, S., Grassi, F., Cohen, L., Cappa, S.F., Dupoux, E., Fazio, F., and Mehler, J. (1996).Brain processing of native and foreign languages.NeuroReport 7(15–17), 2439–2444.

Puce, A., Allison, T., Gore, J., andMcCarthy, G. (1995).Face-sensitive regions in human extrastriate cortexstudied by functional MRI. J. Neurophysiol. 74,1192-1199.

Puce, A., Allison, T., and McCarthy, G. (1999).Electrophysiological studies of human faceperception. III: Effects of top-down processing on face-specific potentials. Cereb. Cortex 9, 445-458.

Quigley, H. A., Dunkelberger, G. R., and Green, W. R.(1989). Retinal ganglion cell atrophy cor-related with automated perimetry in human eyes with glaucoma. Am. J. Opthalmol. 107(5), 453-464.

Quittner, A. L., Smith, L. B., Osberger, M. J., Mitchell,T. V., and Katz, D. B. (1994). The impact of audi-tion on the development of visual attention.Psychol. Sci. 5(6), 347-353.

Rettenbach, R., Diller, G., and Sireteanu, R. (1999). Dodeaf people see better? Texture segmentation andvisual search compensate in adult but not in juve-nile subjects. J. Cogn. Neurosci. 11(5), 560-583.

Rockland, K. S., and Ojima, H. (2001). Calcarine areaV1 as a multimodal convergence area. Soc.Neurosci. Abstr. 27, S11.20.

Röder, B., Teder-Salejarvi, W., Sterr, A., Rosler, F.,Hillyard, S. A., and Neville, H. J. (1999). Improvedauditory spatial tuning in blind humans. Nature400, 162-166.

Rodman, H. R. (1994). Development of inferiortemporal cortex in the monkey. Cereb. Cortex 5,484-498.

Rodman, H. R., Skelly, J. P., and Gross, C. G. (1991).Stimulus selectivity and state dependence of activ-ity in inferior temporal cortical visual areas ofinfant monkeys. Proc. Natl. Acad. Sci. U.S.A. 88,7572-7575.

Rodman, H. R., and O’Scalaidhe, S. P., and Gross,C. G. (1993). Visual response properties of neuronsin temporal cortical visual areas of infantmonkeys. J. Neurophysiol. 70, 1115-1136.

Sergent, J., Ohta, S., and MacDonald, B. (1992).Functional neuroanatomy of face and object pro-cessing: A positron emission tomography study.Brain 115, 15-36.

Starr, A., Amlie, R. N., Martin, W. H., and Sanders, S.(1977). Development of auditory function innewborn infants revealed by auditory brainstempotentials. Pediatrics 60, 831-838.

Stein, J., and Walsh, V. (1997). To see but not to read.The magnocellular theory of dyslexia. TrendsNeurosci. 20(4), 147–152.

Taylor, M. J., McCarthy, G., Saliba, E., andDegiovanni, E. (1999). ERP evidence of develop-mental changes in processing of faces. Clin.Neurophysiol. 110, 910-915.

Taylor, M. J., Edmonds, G. E., McCarthy, G., andAllison, T. (2001). Eyes first! Eye processingdevelops before face processing in children. NeuroReport 12(8), 1671-1676.

Teder-Salejarvi, W. A., Hillyard, S., Roder, B., Neville,H.J. (1999). Spatial attention to central and peri-pheral auditory stimuli as indexed by event-related potentials (ERPs). Cogn. Brain Res. 8(3),213-227.

Tychsen, L. (2001). Critical periods for developmentof visual acuity, depth perception and eye track-ing. In “Critical Thinking about Critical Periods:Perspectives from Biology, Psychology andEducation” (D. B. Bailey, J. T., Bruer, F. J., Symons,and J. W. Lichtman, eds.) pp. 67-80. Paul H.Brookes Publ. Co., Baltimore.

Ungerleider, L. G., and Haxby, J. V. (1994). “What”and “where” in the human brain. Cur. Opin.Neurobiol. 4, 157-165.

Ungerleider, L. G., and Mishkin, M. (1982). Two cortical visual systems. In “Analysis of Visual Behavior” (M. A. G. D. J. Ingle, and R. J. W. Mansfield, eds.), pp. 579-586. MIT Press,Cambridge, Massachusetts.

Vaughan, H. G., and Kurtzberg, D. (1992). Electro-physiologic indices of human brain maturationand cognitive development. In “DevelopmentalBehavioral Neuroscience: The Minnesota

REFERENCES 243


Symposia on Child Psychology” (M. A. Gunnar,and C. A. Nelson, eds.), Vol. 24, pp. 1–36. ErlbaumAssoc., Hillsdale, New Jersey.

Weber-Fox, C., and Neville, H. J. (1996). Maturationalconstraints on functional specializations forlanguage processing: ERP and behavioral evidence

in bilingual speakers. J. Cogn. Neurosci. 8(3),231–256.

Wiesel, T. N., and Hubel, D. H. (1965). Comparison ofthe effects of unilateral and bilateral eye closure oncortical unit responses in kittens. J. Neurophysiol.28(6), 1029-1040.



CLOSED CLASS WORDS

36-42 month olds

28-30 month olds

20-month olds

CHAPTER 9, FIGURE 5 Current source density (CSD) analyses of neural activity to closed class words at 200 msec. The CSDs illustrate sinks (i.e., activity flowing into the head, shown in purple) and sources (i.e., activity flowing out of the head, shown in orange) at three age groups. At 36-42 months the CSD shows a sink over left anterior regions. At 28-30 months the CSD shows sinks that are bilateral but slightly more prominent over the right compared to the left hemisphere. At 20 months the CSD shows sinks over both the left and the right hemispheres (from Neville and Mills, 1997).


C H A P T E R

10

Neural Mechanisms of Attention

George R. Mangun

INTRODUCTION

Evolution has crafted powerful brainmechanisms for attending and ignoringobjects and events in the visual environ-ment. The consequences of these mecha-nisms include alterations in the perceptualprocessing of otherwise equivalent visualinputs. Whether these mechanisms involvetop-down or bottom-up control over atten-tion, they are engaged by innate mecha-nisms shaped by natural selection, learnedstrategies for survival, and the momentarypressures to interact efficiently with acomplex world. An elementary form ofvisual selective attention is based uponlocation in the visual scene, and is termedspatial attention (e.g., Posner, 1980).

The properties of spatial attention havebeen contemplated for hundreds of years,and experimental observations can betraced back at least to the end of the nine-teenth century. Herman Von Helmholtz,the great German scientist, dabbled instudies of visual spatial attention. In aseries of studies aimed at investigating thelimits of perception, Helmholz sought todetermine how much information could bederived from the brief presentation of acomplex display. In one study he built acrude tachistoscope by using a battery tocreate a brief flash of light (spark) that illu-

minated an otherwise dark scene. Thescene Helmholtz constructed was of aseries of letters painted onto a sheet hungat one end of his lab. When he triggeredthe spark in the dark it provided a shortlived illumination of the sheet of letters.Helmholtz quickly realized that he haddifficulty accurately perceiving all theletters on the sheet during one brief illumi-nation. But he was able to perceive theletters accurately in one portion of thescreen, and moreover, he discovered that ifhe decided which portion of the sheet ofletters to attend to in advance, then hecould perceive those letters, but hadtrouble perceiving letters at other locations.Importantly, this could be done withoutmoving his eyes around the screen; hemaintained fixation on a central pinholeillumination on the screen. That is,Helmholtz invoked what we now callcovert visual attention to a region of thesheet of letters. Helmholtz wrote about hisobservations in these experiments, and inhis book “Treatise on Physiological Optics”(1924), he commented that “by a voluntarykind of intention, even without eye move-ments, … one can concentrate attention onthe sensation from a particular part of ourperipheral nervous system and at the sametime exclude attention from all otherparts.”

Helmholtz and others of his era, such asAmerican psychologist William James,were apt at describing the phenomena ofattention, but provided little support fortheir views through direct experimentaltests. It was not until the middle of the nextcentury that researchers began successfullyto quantify attention effects on stimulusprocessing. One particularly successfulapproach involves the chronometricmethod. By measuring the time it takes torespond to a target stimulus when it occursat attended versus unattended regions ofvisual space, it has been possible to quan-tify precisely spatial attention effects, asreflected in performance. For example,Posner and colleagues (Posner et al., 1980)used target-location expectancy to mani-pulate spatial attention, and then acquiredreaction times (RTs) to expected-locationtargets and compared those to the sametargets when they occurred at unexpectedlocations. The paradigm that has beenmodified and used in hundreds of studies over the past 30 years is shown inFig. 1.

In trial-by-trial spatial cuing paradigmssubjects receive a cue, in this case an arrow

at fixation, that indicates the most likelylocation (right or left field) of a subsequenttarget stimulus. The cue predicts the arrowwith some high probability, such as 0.80(valid trials). The remaining trials includeincorrect (invalid) cues/targets (e.g., thearrow points left but the target occurs onthe right), and may also include so-calledneutral trials in which the cue does notpredict the location of the target (underthese conditions the cues might consist ofdouble-headed arrows or other nonpredic-tive symbols). Posner and colleagues foundthat reaction times were significantly fasterin response to targets appearing at precuedlocations in comparison to targets atuncued locations. They interpreted theseeffects to be evidence for early selectionmodels of attention, models that posit thatselection could occur prior to completestimulus analysis. However, these methodsdo not permit the mechanisms of attentionto be unambiguously related to specificstages of information processing. In orderto investigate early stages of sensory analy-sis directly, many researchers have turnedto physiological methods in humans andanimals.

248 10. NEURAL MECHANISMS OF ATTENTION

III. NEURAL MECHANISMS OF SELECTIVE ATTENTION

FIGURE 1 Posner cuing para-digms. An arrow presented atfixation predicts the most likelylocation of an impending targetstimulus that follows severalhundred milliseconds later.Displayed is a so-called valid trialin which the cue accurately pre-dicts the location of the upcomingtarget.

ELECTROPHYSIOLOGICALMEASURES OF

SPATIAL ATTENTION

In studies in humans, event-relatedbrain potentials have been used to investi-gate what stage(s) of information process-ing could be influenced by top-downattentional factors. We investigated theneural bases of the spatial cuing effects inhealthy persons using event-related poten-tials (ERPs) in a series of studies (Mangunet al., 1987; Mangun and Hillyard, 1991).The logic was that using ERPs we couldprovide indices of information processingin the human visual system at variousearly (sensory) and later (postperceptualand decision) stages of processing, andthen could investigate whether stimuli thatwere precued (and elicited speeded reac-tion times) also showed faster or morerobust sensory responses.

The design used in our studies followedthat of Posner and colleagues (1980). Eachtrial began with an arrow, presented atfixation for 200 msec, that pointed to eitherthe left or right visual field. Following eacharrow by about 800 msec was a lateralizedtarget (a briefly flashed vertical bar). The

arrow indicated the side on which thetarget would occur with high probability(0.75), but on some (0.25) of the trials thetarget occurred in the opposite visual fieldlocation (invalid trial). This spatial primingparadigm produces an “endogenous ori-enting,” because attention is directed byvoluntary processes. Voluntary orienting isto be contrasted with “exogenous orient-ing” in which attention is capturedreflexively to the location of a sensorysignal (e.g., Klein et al., 1992). We shallreturn to a discussion of reflexive attentionbelow.

As noted, precuing by predictive cueshas been shown to lead to faster reactiontimes to precued targets. If such effects aredue to changes in the perceptual process-ing of validly and invalidly cued targets,then the early sensory-evoked ERP compo-nents should show expectancy-related(attention-related) changes in either ampli-tude or latency. If, on the other hand, thecuing effects on RT are the result ofchanges in decision and/or response bias,then one would expect stability of thoseearly ERP components that reflect thesensory-perceptual stages of processing,and instead, changes in the longer latencyERP components related to decision or

ELECTROPHYSIOLOGICAL MEASURES OF SPATIAL ATTENTION 249


FIGURE 2 Modulations of ERPs with voluntary attention, showing group-average visual ERPs recorded totarget stimuli in a trial-by-trial spatial cuing paradigm like that illustrated in Fig. 1. Modulations of the sensoryevoked P1 component can be seen at right and left occipital scalp sites. When a left field stimulus (LVF) was cor-rected, indicated by the preceding vertical bar cue, the P1 at contralateral (right) occipital scalp sites was larger(solid more positive than dotted) from 90 to 130 msec after the onset of the target stimulus. Negative voltage isplotted upward, and stimulus onset occurs at the vertical calibration bar in each plot. After Mangun and Hillyard(1991).

action. We found that when a simplespeeded response to target onset wasrequired, an early sensory response in theERPs (P1) was larger in peak amplitudewhen the evoking stimulus had beenprecued (Fig. 2). In other experiments, wealso observed that the later N1 component(180 msec) could also be modulated whenthe task required discrimination of thetarget features (i.e., height judgment of abar stimulus). These effects were not theresult of eye movements toward the cuedlocation because the eyes remained fixed ata central fixation point, and eye move-ments were rigorously monitored.

These amplitude modulations of earlyvisual ERP components produced by trial-by-trial cuing are similar, if not identical, tothose observed in tasks in which attentionis sustained on a single visual field locationthroughout a block of trials while compa-rable stimuli are flashed to that locationand other unattended locations in thevisual field (e.g., Eason et al., 1969; Eason1981; Van Voorhis and Hillyard, 1977).Such effects have been interpreted as evi-dence that early sensory processing isaffected by spatially directed attention(e.g., Eason, 1981; Mangun, 1995). Thus,the evidence obtained in the trial-by-trialcuing experiment is consistent with theproposal of Posner and others thatexpectancy-induced facilitation of reactiontime might be the result of improvementsin early sensory and perceptual processing.

These data do not, however, tell uswhether the effects observed in behavioralstudies of spatial priming are causallydependent on early selection mechanisms.For example, it is possible that the reactiontime effects of Posner and others mightresult from changes in decision andresponse criteria alone (Luck et al., 1994), orthat other factors such as perceptual loadmight influence the relationship betweenphysiological mechanisms and behavioraleffects (e.g., Handy, 2000; Handy et al.,2001; Lavie and Tsal, 1994). Nonetheless,the finding of amplitude modulations of

ERP components with as short a latency as80–100 msec poststimulus duringexpectancy-based cuing strongly suggestsa link between sensory-perceptual sensitiv-ity and RT effects. Thus, these data con-verge with behavioral findings of changesin perceptual sensitivity (d’) for precuedtargets (e.g., Hawkins et al., 1990; Luck etal., 1994) and provide information abouthuman neural mechanisms and levels ofinformation processing involved in atten-tional processing that is consistent withstudies in animals (e.g., McAdams andMaunsell, 1999; Moran and Desimone,1985).

LOCUS OF SELECTION

It remains unclear precisely how early,during visual processing, top-down atten-tion influences may be manifest; however,most selective processes appear to be oper-ating at the cortical sensory level. Whetherthis can occur at the level of primarysensory cortex to act on the incomingsensory signal is still under debate. Somestudies have reported that attention caninfluence processing in V1 (striate cortex)during spatial [e.g., McAdams andMaunsell (1999) and Motter (1993)—inmonkeys] and nonspatial attention [e.g.,Zani and Proverbio (1995)—in humans],but the electrophysiological findings todate remain inconsistent with otherworkers arguing against effects in striatecortex for visual attention (e.g., Clark andHillyard, 1996; Heinze et al., 1994; Luck etal., 1997; Mangun et al., 1993). In part, someof the difficulty in understanding where insensory processing attention may influenceperceptual analyses comes from the factthat it is difficult to localize where in thebrain a particular scalp-recorded ERP isgenerated; this will be discussed in detaillater.

Some neuroimaging data in humanssuggest that changes in neuronal process-ing in striate cortex may occur with selec-



tive attention (e.g., Tootell et al., 1998), butthese findings do not specify the timecourse of the activations. As a result it isnot clear whether evidence of neuronalactivations in striate cortex during atten-tion reflect gating of stimulus inputs orchanges in longer latency activity mediatedby reafferent (top-down) modulations ofstriate neurons (e.g., Martinez et al., 1999;see also Roelfsema et al., 1998). Similarly,there is no evidence at present for subcorti-cal modulations of ascending sensoryinputs in the lateral geniculate nucleusduring selective attention, although sub-cortical systems are clearly influenced byattention, and may participate in atten-tional control circuitry. For example, thepulvinar nucleus of the thalamus, thoughnot a sensory relay nucleus per se, isknown to have visually responsiveneurons that are affected by attention (seePosner and Petersen, 1990; LaBerge, 1995).

REFLEXIVE ATTENTIONAL ORIENTING

Attention can be directed by volun-tary, top-down processes (Kastner andUngerleider, 2000), but may also be “cap-tured” by sensory events. This automaticorienting of attention has been calledreflexive (or exogenous) to indicate thatcontrol over attention was externally con-trolled (e.g., Posner et al., 1980). These twotypes of attentional orienting have beenshown to have different characteristics.Top-down voluntary attention leads to afacilitation in response times, accuracy, andsensory-evoked ERP components forattended-location stimuli, and these are notmanifest until some time after the decisionto orient. In the case of voluntary orientingto a symbolic cue (arrow), this may takemore than 250 msec from cue onset. Someof this time involves that required todecode the cue information and then toorient attention.

Reflexive orienting also leads to fasterand more accurate behavioral responses tostimuli at previously cued locations, butthe effects occur more rapidly than for vol-untarily oriented attention. Reflexiveattention shifts may begin by 50 msec afteran attention-capturing event, and com-pared to the effects of voluntary attention,reflexive orienting is more resistant tointerference and dissipates more quickly(e.g., Cheal and Lyon, 1991; Jonides, 1981;Müller and Rabbitt, 1989). That is, the facil-itation in reaction times at a reflexivelycued location is replaced by inhibition atthat same cued location, by a few hundredmilliseconds after cue onset (e.g., Posnerand Cohen, 1984). This phenomenon,known as inhibition of return (IOR), mayfacilitate the ability efficiently to reorientattention away from potentially distractingevents (Posner and Cohen, 1984).

These differences between voluntaryand reflexive attention may indicate thatreflexive attention is controlled by separateneural systems from those controlling top-down voluntary attention (e.g., Kustovand Robinson, 1996; Rafal, 1996). Giventhat voluntary attention involves modu-lations of visual sensory processing(reviewed above), one might ask whetherthere are any changes in the perceptualprocessing of visual stimuli as a result ofreflexive shifts of attention.

We have demonstrated that reflexiveattention is indeed able to modulate earlyvisual processing at the same neural locusas voluntary attention (Hopfinger andMangun, 1998). Targets were preceded byan uninformative flash (the “cue”) at thesame location (cued-location target) oropposite field location (uncued-locationtarget). The subjects knew that the cue wascompletely uninformative of the locationof the upcoming target, and therefore didnot invoke any voluntary orienting inresponse to the cue. The cue-to-target inter-stimulus interval (ISI) ranges were eitherlong or short in order to measure the rapidand changing time course of reflexive

REFLEXIVE ATTENTIONAL ORIENTING 251


attention (“short” ISI = 34–234 msec;“long” ISI = 600–800 msec).

When the cue-to-target ISI was short(34–234 msec), targets at the cued locationelicited visual ERPs with significantlyenhanced amplitudes compared to targetsat an uncued location (Fig. 3). This enhance-ment was observed for a positive polarity

sensory ERP peak, the occipital P1 compo-nent. Topographic voltage maps during thetime period of the P1 component showedthat the location of the maximal responseat the scalp corresponding to the P1 com-ponent was highly similar for cued- versusuncued-location targets (not shown in Fig. 3). This pattern is consistent with theview that the same neural process wasinvoked in both cases, with the primarydifference being the strength of theresponse. Because the occipital P1 compo-nent represents the earliest stage of visualprocessing to be reliably modulated by voluntary spatial attention (e.g., Heinze et al., 1994; Mangun, 1995; Mangun andHillyard, 1991), the present findings indi-cate that, although separate control cir-cuitry may be involved, reflexive attentionleads to modulations at this same stage ofvisual cortical processing.

In line with the reflexive cuing literatureusing reaction times, the longer cue-to-target ISI’s (566–766 msec), showed theeffect of the cues on the P1 component wasreversed—targets at cued locations nowelicited significantly smaller responsesthan did targets at uncued-locations (seeHopfinger and Mangun, 1998). This pat-tern is similar to that of reaction time IOR;however, in this study, it was actually thecase that no differences were found in RTs,at the long ISIs. Moreover, in a follow-upstudy in which subjects speeded the reac-tions to the reflexively cued targets (ratherthan the discrimination required here), atlong ISIs, the P1 showed no IOR-like pat-tern, but reaction times did (Hopfinger andMangun, 2001). Thus, because of P1 andthe reaction time, it is premature to suggestthat IOR, as defined by reaction timeinhibition, is related to the inhibition some-times seen in the P1 at long ISIs. None-theless, the highly consistent findings forthe short ISI facilitation effects on P1 indi-cate that when attention is reflexively cap-tured by a sensory event, cortical visualprocessing is modulated at the same orsimilar stage of visual processing as the



FIGURE 3 Modulations of ERPs with reflexiveattention. Group-average visual ERPs to target stimulifrom lateral occipital scalp sites (collapsed across leftand right occipital sites when contralateral to thehemifield of the target stimulus). The P1 componentwas enhanced (solid line) when the eliciting targetstimulus had been preceded recently at the same loca-tion by a brief sensory event (four location markingdots were turned off and on surrounding the locationto which the target was subsequently presented). Theamplitude changes in the P1 component are summa-rized in bar graphs; these amplitude differences werehighly statistically significant. In order to remove theoverlapping ERP potentials from cue to target at theshort interstimulus interval (ISI) shown here, theADJAR filter of Woldorff (1993) was employed to esti-mate and subtract away the ERPs to the cue from theERPs to the target. After Hopfinger and Mangun(1998).

shortest latency effects of voluntary atten-tion.

LOCALIZATION OF VOLUNTARYATTENTION PROCESSES

The recording of ERPs from humanscalps provides a powerful means fortracking the time course and, to someextent, the functional anatomy of brainattentional processes. However, limitationsin the anatomical resolution of scalp-recorded ERPs make it difficult to inferfunctional anatomy with precision. Forexample, whether ERPs showing attentioneffects are generated in striate versusextrastriate cortex has proved to be adifficult question to answer (see sectionabove Locus of Selection). What are thelimitations in scalp recordings, and howcan these limitations be overcome?

Modeling ERP Generators

Studies of the intracranial generators ofscalp-recorded ERPs all suffer from thesame general limitation—the recordingsare being made relatively far from their siteof generation. Active neurons in the brainare causing currents to flow passivelythrough the tissues of the brain, skull, andscalp, where they can be recorded. Thus,tiny signals deep in the brain can becomevisible to us as they are passively con-ducted to the surface of the scalp.However, this fact also creates somedifficulties for estimating where in thebrain an ERP is generated. That is, onecannot assume that the brain tissue directlyunderneath a recording electrode has gen-erated the recorded signal.

Another aspect of the problem stemsfrom what is known as the “inverseproblem.” The inverse problem refers tothe difficulty that exists in inferring thebrain generators of scalp activity based onthe pattern of voltages over the scalp(given some assumptions about generator

configuration, head and brain structure,and conductivity). Although a given distri-bution of charges inside the head specifiesa unique pattern on the scalp (the so-calledforward solution), the inverse is not true.A particular pattern on the scalp might becaused by a large number of possibleconfigurations of currents flowing insidethe volume of the head. Thus, no uniquesolution can be obtained when going in theinverse direction from scalp recordings toneural generators. Although models can bederived from the exercise of inverse mod-eling, the concern remains that it isdifficult to falsify such models.

If ERPs are not the method of choice oflocalization of brain activity, how can weachieve this localization while not losingthe temporal advantage of ERPs? Oneapproach is the integration of methodshaving complementary strengths formeasuring the timing and localization ofbrain processes (e.g., Fox and Woldorff,1994). For example, we have combinedneuimaging using positron emissiontomography (PET) or functional magneticresonance imaging (fMRI) with ERPrecordings in normal human subjects in several recent studies in order toinvestigate human brain attentionmechanisms.

LOCALIZATION USINGCOMBINED ERP

AND NEUROIMAGING

In three studies combining ERP record-ing and PET imaging (Heinze et al., 1994;Mangun et al., 1997, 2001), we investigatedthe neural mechanisms of visual spatialattention. Stimuli were flashed bilaterallyin the visual field at a rate of about two persecond, and subjects were instructed tofocus attention selectively on the stimuli inone visual field in order to discriminatetheir features. The stimuli were nonsensesymbols, two in each hemifield, andmatching pairs of symbols at the attended

LOCALIZATION USING COMBINED ERP AND NEUROIMAGING 253


location were targets that required a buttonpress response.

In these studies we showed for the firsttime that spatial selective attention resultsin increased regional blood flow in extras-triate cortical areas in the hemisphere con-tralateral to the attended stimuli (Fig. 4).Modulations of occipital, sensory-evokedactivity in the ERPs were also produced byspatial selective attention, in the same sub-jects performing the same task (Fig. 5).

The PET activations provided the possi-ble locations and number of active corticalneurons that were related to attentionalselection. This information was then incor-porated as constraints for subsequent mod-eling of the ERP data, in order to permit usto ask whether electrical activity within theanatomical locus defined by PET imaging

could have generated the electrical pat-terns that were recorded from the scalp.These models are termed “seeded forwardsolutions,” given that we placed (i.e.,seeded) the model sources (equivalentcurrent dipoles) at the PET-defined brainloci in the computer models.

Based on the evidence from Mangunet al. (1997), two candidate regions in eachhemisphere were defined by functionalimaging (see Fig. 4), it was possible to seeddipoles to either the fusiform gyrus or themiddle occipital gyrus and to ask which ofthese produced the best forward-solutionmodel of the recorded ERP data for anytime range. For the modeling procedureswe used a boundary element approach in arealistic head model (rather than a simplerspherical head model). The details of the



FIGURE 4 PET activations during spatial attention. Group-averaged changes in regional cerebral blood flow(activation) during attention to left versus right visual field stimuli (see text). The activations are overlaid ontohorizontal sections of brain from MRI. The sections pass through ventral visual areas at lower slices on the left ofthe figure (z = –16), and at slightly higher slices on the right (z = 4). Activations in the hemisphere contralateral tothe attended field were observed in the posterior fusiform gyrus (FG) and middle occipital gyrus (MOG). AfterMangun et al. (1997).

FIGURE 5 Attentional modulations of ERPs to bilateral stimuli. Grand-average visual ERPs to the bilateralstimulus arrays when the subjects attended the left half of the arrays and the right half of the arrays. The P1 andthe longer latency range, the late positive component (LPC), are indicated. Attention to the left half of the arrayproduced larger P1 components over the right hemisphere (top), whereas over the left hemispheres, larger P1components were elicited by attention to the right. During the LPC latency range, attention to the left visualhemifield produces larger amplitude (more positive) components over both hemispheres. OL, Left occipital sites;OR, right occipital sites.

brain, skull, and scalp anatomy werederived from anatomical MR images of thesubjects in the experiment, and then theERP data were coregistered with the MR-derived realistic head model.

When dipoles were placed in thefusiform gyrus, the model provided abetter explanation (lower residual vari-ance) for the recorded data in the timerange from 110 to 140 msec latency (P1time range) than did placement of thedipoles in the more lateral brain PET acti-vations in middle occipital gyrus.Interestingly, the reverse pattern was truefor ERP activity in a later time range, thatbetween 260 and 300 msec latency. In thistime range of the ERP effects, the middleoccipital gyrus dipoles produced a bettermodel solution than did those in thefusiform gyrus.

The relationship between the recordedand modeled data is illustrated in Fig. 6 as

topographic maps. The topographies aredisplayed on the surface of the realistichead model viewed from the rear.Topographies of the model data are moresimilar to those of the recorded data in thetime range 110–140 msec, when the dipoleswere placed in the medial [fusiform gyrus(FG)] activation see Fig. 6; compare A(recorded) to B (modeled) data] than whenplace in the lateral [middle occipital gyrus(MOG) activation] (compare A and C, Fig. 6). Although the maps in Fig. 6C beara superficial similarity to the recordeddata, subtle differences in the magnitudesand scalp locations of maxima and minimacan be discerned. The inverse pattern is thecase for the longer latency effects, modeledhere for the 260- to 300- msec latencyrange. The topography of the model datais more similar to that of the recorded datafor dipoles place at the loci of the middleoccipital gyrus activation (compare Daud

LOCALIZATION USING COMBINED ERP AND NEUROIMAGING 255


FIGURE 6 Recorded and model topographic maps of attention effects. Group-averaged topographic mapsfor recorded data (A and D), and model data for dipoles in fusiform gyrus (FG) only (B and E) or dipoles in middle occipital gyrus (MOG) only (C and F). The maps are for the time period of the P1 component (110–140 msec) in the D–F, and for the leading edge of the late positive component (260–300 msec) in A–C. Seetext for description. The difference between the recorded and model data expressed as percent residual variance(RV) is shown below each model head. The view of the heads is from the rear (left of brain, on left). Electrodelocations are indicated by gray disks. (See color plates.)

F, Fig. 6) than when the dipoles are placed in the more medial, fusiform loca-tion (compare Daud E, Fig. 6). Thesefindings support our prior conclusion thatactivity in the region of the fusiform loca-tion is related to attentional modulations ofearly sensory processing, and extend thisresult by showing that activations in morelateral regions of visual cortex are relatedto attention modulations at longer laten-cies, in line with the organization of theventral visual processing stream (e.g.,Tootell et al., 1998).

Functional MRI

By taking advantage of the improvedspatial resolution afforded by fMRI, ourstudies now look at covariations betweenERPs and blood flow in single subjects(e.g., Mangun et al., 1998). Fig. 7 showsdata from six individual subjects in an

attention paradigm similar to thatdescribed above. Subjects were alternatelycued for 16-sec periods to attend to eitherthe left half or the right half of bilaterallyflashed stimulus arrays. The stimuli werenonsense symbols, and targets at theattended location were matching symbols.The brain sections in Fig. 7 are coronal sec-tions of high-resolution anatomical MRIswith the statistically significant activationssuperimposed (correlations of voxel timeseries with a boxcar function that definedthe attend-left and attend-right conditions).When the subjects attended the left field(keeping their eyes fixed centrally, as in allthese studies) there were activated regions(yellow color) in the right hemisphere inthe ventral visual cortex, and when theyattended right these activations were pri-marily in the left hemisphere (red color).These data replicate closely the group-averaged PET effects shown in Fig. 4,



FIGURE 7 fMRI activations during visual attention in single subjects. Data from six volunteers during selec-tive attention to bilateral stimulus arrays. See text for description. The correlation range for the correlation ofvoxel signal intensity and the attention conditions is shown below each coronal slice. Left of the brain is shownon the left of each image. (See color plates.)

although obtained in a very differentmanner. This approach could also be com-bined with electrical modeling, as we aredoing, in order to provide fully integratedspatiotemporal models of visual analysis.The individual variations in brain anatomyand physiology that plague groupapproaches actually turn out to be animportant source of information whenworking in individual subjects, becauseanatomy can be used to constrain dipolemodels for each subject as defined by theiranatomical organization.

SUMMARY

Attention involves top-down processesthat influence the gain of sensory transmis-sion early in visual cortex. These effectsobserved in ERPs in humans are highlyconsistent with studies of single neuronrecordings while monkeys attend andignore visual stimuli. In a similar fashion,though presumably via different controlcircuitry, bottom-up reflexive control overattention can modulate visual cortical pro-cessing in humans. Neuroimaging can bebrought to bear on the question of whichbrain structures are involved in attentionalprocessing of visual signals, and whencombined with ERP recordings, a detailedspatiotemporal model can be developedthat permits a finer grained analysis ofattentional selection than any singlemethod. The view that emerges from theliterature on human spatial attention indi-cates that within visual cortex, powerfulmodulations of incoming signals alter thescene one observes. Behavioral analysestell us that these changes in visual brainprocessing have significant effects on howwe perceive and respond to the worldaround us.

Acknowledgments

The author is grateful for the contribu-tions of J. Hopfinger, M. Buonocore, M.

Girelli, C. Kussmaul, M. Soltani, S. Rash,and Evan Fletcher to the work describedhere. Supported by grants from theNINDS, NIMH, and HFSP to G.R.M.

ReferencesCheal, M. L., and Lyon, D. R. (1991). Central and

peripheral precuing of forced-choice discrimina-tion. Q. J. Exp. Psychol. 43A, 859–880.

Clark, V. P., and Hillyard, S. A. (1996). Spatial selec-tive attention affects extrastriate but not striatecomponents of the visual evoked potential. J. Cogn. Neurosc. 8, 387–402.

Eason, R. G. (1981). Visual evoked potential correlatesof early neural filtering during selective attention.Bull. Psychonom. Soc. 18, 203–206.

Eason, R. G., Harter, M., and White, C. (1969). Effectsof attention and arousal on visually evoked corti-cal potentials. Physiol. Behav. 4, 283–289.

Fox, P., and Woldorff, M. (1994). Integrating humanbrain maps. Curr. Opin. Neurobiol. 4, 151–156.

Handy, T. C. (2000). Capacity theory as a model ofcortical behavior. J. Cogn. Neurosc. 12, 1066–1069.

Handy, T. C., Soltani, M., and Mangun, G. R. (2001).Perceptual load and visuocortical processing: ERPevidence for sensory-level selection. Psychol. Sci.12, 213–218

Hawkins, H. L., Hillyard, S. A., Luck, S. J., Mouloua,M., Downing, C. J., and Woodward, D. P. (1990).Visual attention modulates signal detectability. J. Exp. Psychol. Hum. Percept. Perform. 16, 802–811.

Heinze, H. J., Mangun, G. R., Burchert, W., Hinrichs,H., Scholz, M., Münte, T. F., Gös, A., Johannes, S.,Scherg, M., Hundeshagen, H., Gazzaniga, M. S.,and Hillyard, S. A. (1994). Combined spatial andtemporal imaging of spatial selective attention inhumans. Nature 392, 543–546.

Hopfinger, J., and Mangun, G. R. (1998). Reflexiveattention modulates processing of visual stimuliin human extrastriate cortex. Psychol. Sci. 9,441–447.

Hopfinger, J. B., and Mangun, G. R. (2001). Trackingthe influence of reflexive attention on sensory andcognitive processing. Cogn. Affect. Behav. Neurosci.1, 56–65.

Jonides, J. (1981). Voluntary versus automatic controlover the mind’s eye movement. In “Attention andPerformance IX” (J. B. Long, and A. D. Baddeley,eds.), pp. 187–203. Erlbaum Assoc., Hillsdale,New Jersey.

Kastner, S., and Ungerleider, L. (2000). Mechanisms ofvisual attention in the human cortex. Annu. Rev.Neurosci. 23, 315–341.

Klein, R. M., Kingstone, A., and Pontefract, A. (1992).Orienting of visual attention. In “Eye Movementsand Visual Cognition: Scene Perception and

REFERENCES 257


Reading” (A. Raynor, ed. by), pp. 46–65. Springer-Verlag, New York.

Kustov, A. A., and Robinson, D. L. (1996). Sharedneural control of attentional shifts and eye move-ments. Nature 384, 74–77.

LaBerge, D. (1995). Computational and anatomicalmodels of selective attention in objectidentification. In “The Cognitive Neurosciences”(M. Gazzaniga, ed.) , pp. 649–665. MIT Press,Cambridge, Massachusetts.

Lavie, N., and Tsal, Y. (1994). Perceptual load as amajor determinant of the locus of selection in visual attention. Percept. Psychophys. 56,183–197.

Luck, S. J., Hillyard, S. A., Mouloua, M., Woldorff, M.G., Clark, V. P., and Hawkins, H. L. (1994). Effectsof spatial cuing on luminance detectability:Psychophysical and electrophysiological evidencefor early selection. J. Exp. Psychol. Hum. Percept.Perform. 20, 887–904.

Mangun, G. R. (1995). Neural mechanisms of visualselective attention. Psychophysiology 32, 4–18.

Mangun, G. R., and Hillyard, S. A. (1991).Modulations of sensory-evoked potentials indicatechanges in perceptual processing during visual-spatial priming. J. Exp. Psychol. Hum. Percept.Perform. 17, 1057–1074.

Mangun, G. R., Hansen, J. C., and Hillyard, S. A.(1987). The spatial orienting of attention: Sensoryfacilitation or response bias? In “Current Trends inEvent-Related Potential Research” (R. Johnson, Jr., J. W. Rohrbaugh, and R. Parasuraman, eds.), pp. 118–124. Elsevier, Amsterdam.

Mangun, G. R., Hillyard, S. A., and Luck, S. J. (1993).Electrocortical substrates of visual selective atten-tion. In “Attention and Performance XIV” (D.Meyer, and S. Kornblum, eds.), pp. 219–243. MITPress, Cambridge, Massachusetts.

Mangun, G. R., Hopfinger, J., Kussmaul, C., Fletcher,E., and Heinze, H. J. (1997). Covariations in PETand ERP measures of spatial selective attention inhuman extrastriate visual cortex. Hum. BrainMapping 5, 273–279.

Mangun, G. R., Buonocore, M., Girelli, M. and Jha, A.(1998). ERP and fMRI measures of visual spatialselective attention. Hum. Brain Mapping. 6,383–389.

Mangun, G. R., Hinrichs, H., Scholz, M., Mueller-Gaertner, H. W., Herzog, H., Krause, B. J., Tellman,L., Kemna, L., and Heinze, H. J. (2001). Integratingelectrophysiology and neuroimaging of spatialselective attention to simple isolated visualstimuli. Vision Res. 41,1423–1435.

Martinez, A., Anllo-Vento, L., Sereno, M. I., Frank, L.R., Buxton, R. B., Dubowitz, D. J., Wong, E. C.,Heinze, H. J., and Hillyard, S. A. (1999).Involvement of striate and extrastriate visual corti-

cal areas in spatial attention. Nat. Neurosci. 2,364–369.

McAdams, C. J., and Maunsell, J. H. R. (1999). Effectsof attention on orientation-tuning functions ofsingle neurons in macaque cortical area V4. J. Neurosci. 19, 431–441.

Moran, J., and Desimone, R. (1985). Selective attentiongates visual processing in the extrastriate cortex.Science 229, 782–784.

Motter, B. C. (1993). Focal attention produces spatiallyselective processing in visual cortical areas V1, V2and V4 in the presence of competing stimuli. J. Neurophysiol. 70, 909–919.

Müller, H. J., and Rabbitt, P. M. (1989). Reflexive andvoluntary orienting of attention: Time course ofactivation and resistance to interruption. J. Exp.Psychol. Hum. Percept. Perform. 15, 315–330.

Posner, M. I. (1980). Orienting of attention. Q. J. Exp.Psychol. 32, 3–25.

Posner, M. I., and Petersen, S. E. (1990). The attentionsystem of the human brain. Annu. Rev. Neurosci.13, 25–42.

Posner, M. I., Snyder, C. R. R., and Davidson, B. J.(1980). Attention and the detection of signals. J. Exp. Psychol. Gen. 109, 160–174.

Posner, M. I., and Cohen, Y. (1984). Components ofvisual orienting. In “Attention and Perform-ance X” (H. Bouma, and H. Bouwhuis, eds.), pp. 531–554. Erlbaum Assoc., Hillsdale, NewJersey.

Rafal, R. (1996). Visual attention: Converging opera-tions from neurology and psychology. In “Con-verging Operations in the Study of Visual SelectiveAttention” (A. F. Kramer, M. G. H. Coles, and G. D. Logan, eds.), pp. 139–192. AmericanPsychological Association, Washington, D.C.

Roelfsema, P. R., Lamme, V. A. F., and Spekreijse, H.(1998). Object-based attention in the primaryvisual cortex of the macaque monkey. Nature 395,376–381.

Tootell, R. B., Hadjikhani, N., Hall, E. K., Marrett, S.,Vanduffel, W., Vaughan, J. T., and Dale, A. M.(1998). The retinotopy of visual spatial attention.Neuron 21, 1409–22.

Van Voorhis, S. T., and Hillyard, S. A. (1977). Visualevoked potentials and selective attention to pointsin space. Percept. Psychophys. 22, 54–62.

Von Helmholtz, H. (1924). “Treatise on PhysiologicalOptics.” Optical Society of America, Rochester,New York.

Woldorff, M. G. (1993). Distortion of ERP averagesdue to overlap from temporally adjacent ERPs:Analysis and correction. Psychophysiology 30,98–119.




CHAPTER 10, FIGURE 6 Recorded and model topographic maps of attention effects. Group-averaged topographic maps for recorded data (A and D), and model data for dipoles in fusiform gyrus (FG) only (B and E) or dipoles in middle occipital gyrus (MOG) only (C and F). The maps are for the time period of the P1 component (110-140 msec) in the D-F, and for the leading edge of the late positive component (260-300 msec) in AM2. See text for description. The difference between the recorded and model data expressed as percent residual variance (RV) is shown below each model head. The view of the heads is from the rear (left of brain, on left). Electrode locations are indicated by gray disks.

CHAPTER 10, FIGURE 7 fMRI activations during visual attention in single subjects. Data from six volunteers during selective attention to bilateral stimulus arrays. See text for description. The correlation range for the correlation of voxel signal intensity and the attention conditions is shown below each coronal slice. Left of the brain is shown on the left of each image.


C H A P T E R

11

Steady-State VEP andAttentional Visual Processing

Francesco Di Russo, Wolfgang A. Teder-Sälejärvi, and Steven A. Hillyard

INTRODUCTION

The vast majority of studies that investi-gated attentional modulation of the visualevoked potential (VEP) have been confinedto the transient responses evoked by iso-lated stimuli. This class of potentials isevoked by stimuli having an asynchronousand low repetition rate (not faster than 2stimuli per second). These potentials arecalled “transient” because the slow rate ofstimulation allows the sensory pathways torecover or “reset” before the next stimulusappears. When visual stimuli are presented

at a constant rate that is rapid enough toprevent the evoked neural activity fromreturning to base line state, the elicitedresponse becomes continuous and is calledthe steady-state visual evoked potential(SSVEP). With steady-state stimulation thetypical VEP wave form is markedlychanged. For instance, the transient VEPincludes three major early components: theC1 at 60–80 msec, the P1 at 80–120 msec,and the N1 at 120–180 msec. (see Fig. 1). Atmore rapid stimulation rates, the brainresponse to the same stimulus becomessinusoidal and is typically modulated at

FIGURE 1 Pattern-reversal VEP wave form as a function of stimulation frequency. Note that the wave form isbasically modulated at the second harmonic of the stimulus frequency. At the slowest rate (2 Hz) the componentsof the transient VEP can be seen.

the fundamental stimulus frequency in thecase of an unstructured stimulus (e.g.flash) or at the second harmonic (doublethe stimulation frequency) if the stimulusis a pattern-reversal (Regan, 1989).

Like any sinusoidal wave form, theSSVEP can be measured in terms of itsamplitude and phase. The phase is a jointfunction of the stimulus frequency and thetime delay between stimulus and brainresponse. The amplitude indicates the rela-tive magnitude of a given harmonic of theresponse and, as for transient evokedpotentials, is measured in microvolts. Theamplitude and phase of the SSVEP vary asfunction of the temporal frequency, spatialfrequency, contrast, luminance, and hue ofthe driving stimulus (Regan, 1989).

SSVEP AND COGNITIVE PROCESSES

The SSVEP offers certain advantagesover the transient VEP for the study ofsensory and cognitive processes in that itssignal is easily recorded and quantifiedand can be rapidly extracted from back-ground noise (Regan, 1989). It is somewhatsurprising, therefore, that only a fewstudies have attempted to relate SSVEPparameters to cognitive processes. One ofthe first such studies was by Wilson andO’Donnell (1986), who found that indi-vidual differences in reaction time inmental rotation and memory matchingtasks were correlated with the conductionspeed (“apparent latency”) of the SSVEPrecorded in a separate session. Theseinvestigators did not find any reliablerelationships between SSVEP latency andmental workload, however (Wilson andO’Donnell, 1986).

In a visual vigilance task, Silbersteinet al. (1990) found that the SSVEP ampli-tude to an irrelevant flicker was reducedduring a period when the subject wasactively searching for a target shape ascompared to when no target was expected.

This effect was interpreted in line with theauthors’ hypothesis that the SSVEP to suchan irrelevant probe would be reduced inbrain areas as a function of how activelyengaged those brain areas were in per-forming the ongoing task. Accordingly,they concluded that during the period ofactive vigilance there was increased brainactivity in parietooccipital regions, leavingfewer neurons available to respond to theirrelevant background 13-Hz flicker.

The use of the SSVEP as a probe of cog-nitive function was extended by Silbersteinet al. (1995) in a study of the WisconsinCard Sorting Task. They found that theSSVEP amplitude to a continuous irrele-vant background flicker was attenuatedover prefrontal, central, and right pari-etotemporal regions in the interval follow-ing a cue to change the card-sort criterion.These SSVEP reductions were interpretedas reflecting an increase in task-related cor-tical activity in those brain regions duringtask performance.

SSVEP AND SPATIAL ATTENTION

The neural mechanisms of visual–spatialattention have been studied extensively bymeans of transient VEPs (reviewed inHillyard and Anllo-Vento, 1998; Martinezet al., 2001). The general finding has beenthat paying attention to a specific region ofthe visual field is associated with increasedamplitudes of the early components ofVEPs to stimuli flashed at the attendedlocation. This attentional modulation of thetransient VEP includes amplitude enhance-ments of the sensory-evoked P1 (80–120msec) and N1 (140–200 msec) components,which have been localized by dipole mod-eling techniques to specific zones of extras-triate visual cortex (Martinez et al., 2001). Ithas been proposed that these P1 and N1modulations reflect a sensory gain controlmechanism, whereby visual informationfalling within the spotlight of spatial atten-tion is facilitated and passed along to

260 11. STEADY-STATE VEP


higher levels of processing (Hillyard andAnllo-Vento, 1998).

The effects of spatial attention on theSSVEP in response to flickering stimuli wasstudied by Morgan et al. (1996) in a task inwhich subjects were cued to attend to aletter/number sequence in one visual field and to ignore a similar, concurrentsequence in the opposite field (Fig. 2A).The letter/number sequences in the two

fields were superimposed on small back-ground squares flickering at 8.6 Hz in onefield and 12 Hz in the other. Repres-entative SSVEP waveforms (averaged inthe time domain) from one subject areshown in Fig. 2B for the condition inwhich the 12-Hz background flicker waspresented in the left visual field and the8.6-Hz flicker was presented in the rightvisual field. Recordings shown are from

SSVEP AND SPATIAL ATTENTION 261


FIGURE 2 Overview of experimental design and results from the study of Morgan et al. (1996). (A) Subjectsmonitored the character sequence in one visual field while ignoring the contralateral sequence. (B) Time domainaverages of SSVEP responses to flickering squares in the left (12 Hz) or right (8.6 Hz) visual field recorded fromthe right occipitotemporal scalp (site PO8) in a typical subject. Wave forms shown were obtained by averagingthe responses to successive flashes over the first 6 sec of the flickering sequence, time locked to either the 12- orthe 8.6-Hz flashes, and then averaging across all the trials of that type. Dashed wave forms correspond to theattend-left condition and solid waveforms to the attend-right condition. (C) Frequency domain analysis of theSSVEPs illustrated in B. Amplitude values were derived from fast Fourier transforms. Reprinted from Morgan,Hansen & Hillyard; Proc. Nat. Acad. Sci. USA 93, 4770–4774. Copyright 1996 National Academy of Sciences, USA.

the right occipital scalp, where consistentattention– related enhancement wasobserved for SSVEPs at both frequencies.The amplitude of the SSVEP elicited by the12-Hz flicker was much larger when atten-tion was directed to the left stimulussequence rather than the right, whereas theSSVEP in response to the concurrently pre-sented 8.6-Hz flicker showed the reverse.This amplitude enhancement of the SSVEPin response to the irrelevant backgroundflashes at the attended location was alsoevident in the frequency domain (fastFourier) analysis of these wave forms (Fig. 2C). These findings indicate that therelative amplitudes of the frequency-specific SSVEPs elicited by each stimulusmay index the allocation of attentionamong the flickering stimulus locations.

To gain information about the corticalregions responsible for generating theenhanced SSVEP to attended stimuli, afurther study used functional magnetic res-onance imaging (fMRI) to localize activebrain regions while subjects performed thesame task shown in Fig. 2 (Hillyard et al.,1997). Two specific zones of extrastriatevisual cortex were found to be activatedduring attention to the lateralized flicker-ing stimuli, one in the fusiform gyrus/inferior occipital area and the other in morelateral occipitotemporal cortex. Dipole mod-eling of the grand-average SSVEP that wasrecorded in the same subjects revealeddipolar sources in occipitotemporal cortexjust medial to the fMRI activations.

These findings of increased SSVEPamplitudes to irrelevant flicker at attendedversus unattended locations might at firstappear to conflict with the finding ofSilberstein et al. (1990), that the SSVEP toirrelevant flicker was decreased during aperiod of active vigilance. This difference inoutcome can be explained, however, bydifferences in the size and location of theirrelevant flickering stimuli between thetwo studies. Whereas the flickering back-grounds in the study of Morgan et al. (1996)were discrete and superimposed on the

task-relevant stimulus locations, the flic-kering background in the Silberstein et al.study was large and diffuse, subtending 30° by 80° of visual angle. Thus, focusingattention on the relevant stimulus sequencein the design of Morgan et al. would resultin enhanced processing of the discreteflicker because it fell within the attentionalspotlight, whereas in the Silberstein et al.study very little of the diffuse flicker would be included in the attentional spotlight.Indeed, if the attentional spotlight wasnarrowly focused on the relevant stimuli,the SSVEP response to the diffuse sur-rounding flicker may actually have beensuppressed

The SSVEP was also found to be a sensi-tive index of spatial attention to stimuliflickering in the range of 20–28 Hz (Mülleret al., 1998a). In this experiment, subjectswere asked to attend to a flickering light-emitting diods (LED) display in one visualfield while ignoring a similar display flick-ering at a different frequency in the othervisual field. For example, when subjectsattended to an array of LEDs flickering at27.8 Hz in one visual field and ignored anarray flickering at 20.8 Hz in the oppositefield, the SSVEP at the attended frequencywas more than doubled in amplitude (Fig. 3). Modeling of the neural generatorsof the higher frequency SSVEP using acurrent estimation technique indicatedfocal sources in dorsal occipital cortex inthe hemisphere contralateral to the stimu-lus position.

In further studies, the SSVEP responseto these high-frequency flickers was usedto provide an electrophysiological index ofthe speed of attention switching (Müller etal., 1998b). In this study, each trial beganwith concurrent flickering of LED displaysin the left (at 20.8 Hz) and right (at 27.8 Hz)visual fields. A central cue then appearedadjacent to the fixation point to indicatewhether the left or right display was to beattended on that trial. It was found that theSSVEP amplitude at the attended fre-quency (measured by moving-window



Fourier analysis) increased abruptly afterthe cue (Fig. 4); it was calculated thatsteady-state cortical activity evoked by theattended-side stimulus was facilitated byabout 500 msec after the onset of the cue toshift attention. This estimate of attentionswitching time corresponded with the timeat which behavioral target detection withinthe attended display was reaching itsmaximum. Moreover, the SSVEP rise timeswere substantially faster in those subjectswho switched attention more rapidly, asindicated by their earlier target detections.Also of interest was the finding that theattention effect on the SSVEP was purelyfacilitatory; that is, the SSVEP elicited bythe attended flicker was enhanced follow-ing the attention-directing cue, but theSSVEP to the unattended-location flicker inthe opposite visual field was not attenu-

ated. Müller et al. (1998b) proposed thatthis facilitation reflects the operation of a gain-control mechanism that booststhe discriminability of attended-locationstimuli by enhancing their signal-to-noiseratio. They concluded that the SSVEP pro-vides a continuous measure of the timecourse of attention switching and the facil-itation of the cortical processing of thenewly attended stimulus.

A further study investigated the ef-fects of spatial attention on concurrentlyrecorded transient and steady-state visualERP responses to the flickering arrays(Müller and Hillyard, 2000). Consistentwith previous findings, SSVEP amplitudewas enlarged for attended flicker stimuli atposterior electrode sites contralateral to theattended visual hemifield. Significant cor-relations were found between the N1 and

SSVEP AND SPATIAL ATTENTION 263


FIGURE 3 Schematic diagram of stimulus array and SSVEP wave forms from one subject shown for theattended (bold line) and unattended (thin line) conditions recorded from occipitotemporal sites (TO2 and TO1)contralateral to the flickering stimulus. The flicker rates were 20.8 Hz for the left row and 27.8 Hz for the rightrow of LEDs. The four possible color configurations are shown for each row, with all five LEDs being red in thestandard configuration. Target and nontarget color changes (two LEDs changed to green) occurred in randomorder on both sides with a stimulus-onset asynchrony of 400 to 700 msec (onset to onset). Gray oval is the fixationpoint. The SSVEPs were obtained by a sliding-average technique in the time domain and were time-locked toeither the left or the right flickering stimulus. From Müller et al. (1998b). Reprinted from Müller, Teder-Sälejärvi,& Hillyard; Nature Neuroscience 1, 631–634, 1998.

N2 components of the transient ERPresponse to color-change target stimuli andthe SSVEP attention effects, suggesting thatthe SSVEP and transient ERP reflect par-tially overlapping attentional mechanismsthat facilitate the discriminative processingof stimuli at attended locations.

ATTENTION EFFECTS ON SSVEP PHASE

The aforementioned studies found thatspatial attention strongly increased theamplitude of the SSVEP, but attentioneffects on response phase were not ana-lyzed in detail. Morgan and colleagues(1996), using 8.6- and 12-Hz flickeringstimuli, observed substantial phase shiftsbetween attended and unattended waveforms in some subjects (e.g., Fig. 2), but

these shifts were inconsistent across sub-jects and electrode sites. In the studies ofMüller and colleagues (1998a,b) that usedstimulus frequencies of 20.8 and 27.8 Hz,phase shifts were observed at many scalpsites between the attended and unattendedwave forms. Statistical analysis, however,failed to demonstrate any consistent phaseshifts across subjects as a function ofattention

The phase of the SSVEP depends in parton the transmission time between the stim-ulus and the evoked brain activity but doesnot give a direct measure of this transmis-sion time. The steady-state phase can beexpressed in terms of the ratio of thesine/cosine components of the Fourieranalysis (see Regan, 1989; Porciatti et al.,1992), but this solution is not unique andincludes a group of phase values separatedby multiples of 2p radians. For instance, a



FIGURE 4 Representative time- and frequency-domain wave forms from a single subject in the study ofMüller et al. (1998b). (A) Averaged time-domain wave form following the cue to attend right, time-locked to theright flickering stimulus. SSVEP activity to this attended flicker can be seen at the expanded time scale. (B) Timecourse of SSVEP amplitude in the frequency domain obtained from the wave form shown in (A) by a moving-window fast Fourier transform at the stimulus frequency; successive window steps were 4 msec. Thin horizontalline is drawn through precue base line. Bold tracing is attended wave form; thin tracing shows unattended waveform elicited by the same flickering stimulus when the cue directed the attention to the left. Note that the last 500 msec were not analyzed because the moving window reached the end of the epoch. Reprinted from Muller,Teder-Sälejärvi, & Hillyard; Nature Neuroscience 1, 631–634, 1998.

45° phase shift with respect to stimulusonset for the first harmonic of a 20-HzSSVEP corresponds to a time shift of 12.5 msec, but it is impossible to knowwhether the response is delayed by addi-tional whole cycles. Thus, in this examplethe response latency could be 12.5, 62.5 or102.5 msec, etc., depending on how manycycles may have occurred.

The true latency of the SSVEP cannotusually be determined unequivocally, butits “apparent latency” can be estimated asthe slope of the function relating phasechange to the stimulation frequency(Regan, 1989). In other words, if the SSVEPis recorded over a range of several differ-ent stimulation frequencies (for example,from 4 to 11 Hz with 0.5-Hz steps), thephase value will linearly decrease as func-tion of the frequency (see Fig. 5b). The VEPlatency for that frequency range can thenbe estimated in terms of the slope of the phase-frequency function. As demon-strated in numerous studies (e.g., Spek-reijse et al., 1977; Riemslag et al., 1982;Spinelli et al., 1994; Spinelli and Di Russo,1996; Di Russo and Spinelli, 1999a,b), theapparent SSVEP latency estimated in thisway is around 100–150 msec, which cor-

responds with the latency of the P100 component of the transient VEP.

The first study to systematically analyzethe SSVEP phase and apparent latency in avisuo spatial attention experiment was byDi Russo and Spinelli (1999a). In this studythe SSVEP was recorded in response to atask-irrelevant grating that was phase-reversed at nine temporal frequenciesranging from 5 to 9 Hz with 0.5-Hz steps.This background grating (11° wide by 18°high) was continuously displayed in theleft visual field with its medial edge 1.5°from fixation. A target (a light changingcolor) was presented either in the left or inthe right visual field (eccentricity 7°). Thetask was to count the number of targetcolor changes, without moving the eyesfrom the central fixation point. Thus, atten-tion was directed either to the left or to theright visual field. The results in a singlesubject (Fig. 6) and averaged over all sub-jects (Fig. 7) confirmed that the amplitudeof the early sensory activity was modu-lated by spatial attention. Moreover, theyshowed that the speed of stimulus process-ing in an attended region of the visual fieldwas facilitated; i.e., the SSVEPs in theattended condition had a shorter apparent

ATTENTION EFFECTS ON SSVEP PHASE 265


FIGURE 5 The dynamics of the SSVEP can be described by two graphs, one (A) representing the amplitude ofthe harmonic component analyzed as function of the stimulation frequency, and another (B) representing thephase as function of the stimulation frequency. In this example the graphs describe the second harmonic ampli-tude and phase as a function of the stimulation frequency (from 4 to 11 Hz) in response to a sinusoidal pattern-reversal grating having a spatial frequency of 0.6 cycle/degree (unpublished data).

latency than SSVEPs in the unattendedcondition. The difference in latency produced by attention ranged from 5 to 20 msec. Similar results were obtained inthis study with transient VEPs; stimuli atthe attended location elicited VEPs withshorter latencies for the N60, P100, andN140 components and larger amplitudesfor the P100 and N140

As mentioned before, the apparentlatency of the SSVEP was calculated from

the slope of the function relating phase tostimulation frequency. In control experi-ments, the effect of varying stimulus eccen-tricity on SSVEP amplitude and latency wasmeasured (Fig. 8). It was found that theamplitude was dramatically increased whenthe eccentricity was reduced, but the latencywas little affected. This ruled out the possi-bility that the attention effect on SSVEPlatency was an artifact of the subjects’ shift-ing their gaze toward the stimulus.



FIGURE 6 (A) SSVEP phase and amplitude variation as a function of stimulation frequency can be observedin recordings obtained from a typical subject in the attended and unattended conditions in the study of Di Russoand Spinelli (1999a). Stimuli were contrast-reversed at increasing frequencies (5–9 Hz) and SSVEPs were recordedover corresponding time epochs. (B) The resulting apparent latencies estimated from the slope of the phase-frequency function are shown below and the SSVEP amplitude-frequency function is shown above. Bars repre-sent the standard deviations of the amplitudes and phases. Reprinted from Vision Research 39; F. Di Russo and D. Spinelli; Electrophysiological evidence for an early attentional mechanism in visual processing in humans, pp. 2975–2985. Copyright 1999, with permission from Elsevier Science.

EFFECT OF ATTENTION ON THE MAGNOCELLULAR AND PARVOCELLULAR

VISUAL PATHWAYS

A subsequent experiment (Di Russo andSpinelli, 1999b) examined the effect ofspatial attention on the magno- and parvo-cellular components of the visual path-ways. The so-called p (parvo) pathwayoriginates predominately in the fovealregion of the retina from ganglion cells thatare characterized by low conduction veloc-ity, small receptive fields, strong center sur-round inhibition, high-contrast sensitivity,and a tendency to adapt slowly to station-ary stimuli. The optimal visual stimulusfor this system is a sinusoidally modulatedpattern having a low temporal frequency(1–5 Hz) and high spatial frequency.Another important feature of the p system

is its processing of color information(Eskin and Merigan, 1986). The second m(magno) pathway is so labeled because itsganglion cells are larger than the parvocells. The magno ganglion cells are widelydistributed throughout the retina and arecharacterized by high conduction velocity,large receptive fields, low contrast sensitiv-ity, and rapid adaptation to stationarystimuli. This system responds over a widerange of temporal frequencies (5–40 Hz),but the optimal stimulus is a sinusoidalluminance modulation with high temporalfrequency (6–10 Hz) and low spatial fre-quency (e.g., Bodis-Wollner, 1992; Spinelliet al., 1994).

As in the previous study, attention wasdirected to the left or to the right of thefixation point, but this time the stimulusgratings were modulated either in lumi-nance or color contrast. Different temporalfrequencies (from 2 to 6 Hz for color and 5

EFFECT OF ATTENTION ON THE MAGNOCELLULAR AND PARVOCELLULAR VISUAL PATHWAYS 267


FIGURE 7 SSVEP data averaged across subjects under attended and unattended conditions in the study of Di Russo and Spinelli (1999a). Left: Mean amplitudes (and standard errors) are plotted as a function of stimula-tion frequency. Right: Mean phase values in radians are plotted as a function of stimulus frequency. Note the dif-ference in phases between the two conditions. Apparent latencies are derived from the slopes of the regressionlines. Mean latencies (and standard errors) across subjects are shown in the bar graph. Reprinted from VisionResearch 39; F. Di Russo and D. Spinelli; Electrophysiological evidence for an early attentional mechanism invisual processing in humans, pp. 2975–2985. Copyright 1999, with permission from Elsevier Science.



FIGURE 8 Effect of stimulus eccentricity on the SSVEP. Top graphs show the amplitude and latency data forone subject at various eccentricities. Increasing negative degree values (in the upper labels) indicate fixationpoints closer to the stimulus. Note the systematic increment of amplitude (top left graph) and the small variationof latency (top right graph) with decreasing eccentricity. Data from three subjects are reported with differentsymbols in the lower graphs. The positions of the different fixation points are shown on the abscissa. The arrowindicates the location of the fixation point used in the attention experiment. At –1.5°, the fixation point was on theedge of the grating. The SSVEP amplitudes recorded at the nine temporal frequencies were averaged to obtain amean value for each eccentricity for each subject. The differences of the mean amplitude (bottom left graph) orthe latency (bottom right graph) with respect to the reference values of each subject are showed on the ordinate.The reference values were the amplitude and the latency recorded when the fixation point was in the same posi-tion used in the attention experiment, i.e., 0° on the abscissa. For comparison, filled symbols show the increase inamplitude and the shortening of the latency observed in the same subjects when attention was manipulated.Reprinted from Vision Research, 39, F. Di Russo and D. Spinelli; Electrophysiological evidence for an early atten-tional mechanism in visual processing in humans, pp. 2975–2985. Copyright 1999, with permission from ElsevierScience.

to 10 Hz for luminance) were used in orderto maximize the activation of parvocellularor magnocellular pathways, respectively.SSVEPs recorded in attended and unat-tended conditions were again compared.As shown in Fig. 9, both the latency andamplitude of SSVEPs to the luminance-modulated stimuli were modified by atten-tion. For the chromatically modulatedstimuli, however, attention affected onlythe amplitude and not the latency of theSSVEPs.

Di Russo and Spinelli (1999b) concludedthat spatial attention uses different mecha-nisms to affect sensory transmission in themagno and parvo systems. Attention pro-duced a decrease in latency only forevoked activity in the fast, magnocellularpathway. It was proposed that attentionuses the faster signals of the magnocellular

pathways to give priority to stimuli atattended locations and to direct resourceallocation and enhancement of activity ofthe parvosystem.

The effects of attention on SSVEP ampli-tude and latency suggest that attentionmay play a role in regulating gain controlmechanisms operating in human cortex.Automatic gain control mechanisms forcontrast are present at several levels in the visual system, from the retina to thevisual cortex (Shapley and Victor, 1981;Bernadette et al., 1992; Reid et al., 1992).This control, specific for m but not forp pathways, is mediated by feedback loops that cause a nonlinear increment of the response amplitude and phaseadvance with increasing luminance con-trast (Shapley and Victor, 1981; Lee et al.,1994; Bernadette and Kaplan, 1999).

EFFECT OF ATTENTION ON THE MAGNOCELLULAR AND PARVOCELLULAR VISUAL PATHWAYS 269


FIGURE 9 The effect of attention on SSVEPs to time-varying luminance and chromatic modulations in thestudy of Di Russo and Spinelli (1999b). Left: SSVEP amplitudes (and standard errors) are plotted as a function ofmodulation frequency. Filled symbols denote luminance gratings; open symbols denote isoluminant chromaticgratings. Attended values are shown as circles, unattended values as triangles. Right: SSVEP phases (in radians)are plotted as a function of modulation frequency. Note the differences in phase (and in apparent latency)between the attended and unattended SSVEPs for luminance gratings and the absence of such differences forchromatic gratings. From Di Russo and Spinelli (1999b).

This attentional modulation of SSVEPlatency is consistent with findings inpatients having an attentional deficit forcontralesional space (hemineglect) conse-quent to brain lesions. The VEPs responsesto stimuli located in the contralesional,neglected hemifield have latencies longerthan do those to ipsilesional, nonneglectedstimuli (Spinelli et al., 1994; Angelelli et al.,1996; Spinelli and Di Russo, 1996); thisdelay was observed only for luminance-modulated stimuli, not for chromatic-modulated stimuli (Spinelli et al., 1996).

ATTENTION EFFECT ON SSVEPCONTRAST RESPONSE

The effect of attention on the SSVEPresponse to stimuli at varying contrastlevels was investigated by Di Russo et al.,(2001). The purpose of this study was touse SSVEPs to examine how attention mayaffect the cortical mechanisms that controlcontrast gain. Both luminance-modulatedand chromatically modulated stimuli wereused in order to investigate possible differ-



FIGURE 10 The effect of attention on SSVEP contrast-response curves for luminance gratings. Averaged data(N = 11) from the study of Di Russo et al. (2001). Top: SSVEP amplitudes (and standard deviations) as a functionof luminance contrast recorded at two electrode sites (POz and PO4) in the attended (circles) and unattended (tri-angles) conditions. Noise levels recorded in the two conditions are shown as continuous (attended) and dashed(unattended) lines. Slopes of the regression lines were 0.66 (attended) vs. 0.49 (unattended) at POz, and 0.88(attended) vs. 0.62 (unattended) at PO4. Regression lines intercepted the abscissa at 0.24% (attended) vs. 0.23%(unattended) at POz, and 0.54% (attended) vs. 0.46% (unattended) at PO4. Bottom: SSVEP phases (and standarddeviations) in radians as a function of luminance contrast. The slopes of the curves were 1.09 (attended) vs. 1.42(unattended) radians/log unit of contrast at POz, and 1.13 (attended) vs. 1.51 (unattended) radians/log unit ofcontrast at PO4. In other words, the phase advance with contrast was reduced in the attended condition.Reprinted from Vision Research 41, F. Di Russo, D. Spinelli, and M. C. Morrone; Automatic gain control contrastmechanisms are modulated by attention in humans: Evidence from visual evoked potentials, pp. 2435–2447.Copyright 2001, with permission from Elsevier Science.

ences between the magnocellular and par-vocellular pathways in their control of con-trast gain (Derrington and Lennie, 1984;Merigan, 1989; Lee et al., 1990).

The SSVEP was recorded in response tocounterphased sinusoidal gratings modu-lated over a range of contrasts. The 1cycle/degree gratings were modulatedeither in luminance or chromatic (red–green) contrast and were phase reversed at9 and 2.5 Hz, respectively, to selectivelyactivate the magno- and parvocellularsystems. Attention was directed toward thegratings (displayed in the left visual field)by requiring subjects to detect and respondto randomly occurring changes in contrast.In a control condition, attention toward the

grating was minimized by requiring sub-jects to detect a target letter among dis-tracters briefly flashed in the contralateralvisual field. As shown in Figs. 10 and 11,attention increased SSVEP amplitudes forboth luminance and chromatic stimuli,moreso at high than at low contrast levels,as reflected in steeper slopes of the con-trast amplitude curves (over the nonsatu-rating range of contrasts).

The estimates of contrast thresholdobtained by extrapolation of amplitudes tothe abscissa were unaffected by attention.Attention also affected the SSVEP phases,but only for luminance gratings (Fig. 10),where it acted to reduce the magnitude ofphase advance with contrast. Attention

ATTENTION EFFECT ON SSVEP CONTRAST RESPONSE 271


FIGURE 11 The effect of attention on SSVEP contrast-response functions for chromatic gratings. Averageddata (N = 11) from the study of Di Russo et al. (2001). Top: Averaged SSVEP amplitudes (and standard deviations)as a function of chromatic contrast. Slopes of the regression lines were 1.15 (attended) vs. 0.98 (unattended) atPOz, and 1.77 (attended) and 1.26 (unattended) at PO4. Regression lines intercepted the abscissa at 0.29%(attended) vs. 0.25% (unattended) at POz, and 0.56% (attended) vs. 0.44% (unattended) at PO4. Bottom: AveragedVEP phases (and standard deviation) in radians as a function of chromatic contrast. The slopes of the curves were0.79 (attended) vs. 0.86 (unattended) radians/ log unit of contrast at POz, and 0.81 (attended) vs 0.80 (unat-tended) radians/log unit of contrast at PO4. In other words, the phase advance with contrast was not significantly changed by attention. Reprinted from Vision Research 41; F. Di Russo, D. Spinelli, and M. C.Morrone; Automatic gain control contrast mechanisms are modulated by attention in humans: Evidence fromvisual evoked potentials, pp. 2435–2447. Copyright 2001, with permission from Elsevier Science.

had no effect on the contrast-phase func-tions for chromatic gratings (Fig. 11). Theseresults are consistent with the hypothesisthat attention acts on cortical gain controlmechanisms, which are known to be differ-ent for the magno- and parvocellularsystems.

The contrast gain mechanism actingthrough a feedback loop seems to occurexclusively in the magnocellular pathway(Lee et al., 1994; Bernardete and Kaplan,1999). Only cells of this pathway arereported to change their latency and tem-poral tuning with contrast, whereas parvo-cellular latency and temporal tuningremain constant both in response to isolu-minant and luminance-modulated stimuli(Bernardete et al., 1992; Lee et al. 1994).Unfortunately, no single-cell recordingshave been made to assess the corticalresponse to equiluminant chromatic modu-lations. The findings of Di Russo et al.(2001), consistent with previous results,indicate that the human SSVEP responsesto chromatic modulations are subject tocontrast gain control but probably only atthe cortical level, although other interpre-tations are possible. For instance, differentsources with different integration timesmay contribute to the overall VEP waveform, and their relative contributions mayvary with contrast. However, whatever theexplanation of the phase advance for isolu-minant chromatically modulated gratings,attention did not affect it systematically.This result points to a possible differencebetween the attentional control mecha-nisms for the color and luminance corticalpathways.

It is interesting to compare the presentluminance data with those obtained inmasking experiments, where SSVEP con-trast-response curves were measured in thepresence of parallel or orthogonal lumi-nance-modulated stimuli (Burr andMorrone, 1987; Morrone et al., 1987). Asuperimposed mask grating that was ori-ented orthogonally to the test grating wasfound to attenuate SSVEP amplitudes mul-

tiplicatively (so-called cross-orientationinhibition) and to increase the phaseadvance. That is, the effect of the maskmimicked the effect of engaging attentionon another task, both for SSVEP amplitudeand phase. It has been proposed (Burr andMorrone, 1987) that orthogonal maskingeffects on the SSVEP are mediated by theautomatic contrast gain control mecha-nisms previously described. Accordingly,attention may use the same inhibitory cir-cuitry already in place for contrast regula-tion to increase the processing speed andstimulus discriminability. Such a mecha-nism would have the advantage of improv-ing vision without requiring anyadditional circuitry that was specificallydedicated to attentional processes.

CONCLUSIONS

The evidence indicates that the SSVEPprovides a sensitive measure of spatialattention processes and offers certainadvantages over the transient VEP. In par-ticular, because of the high rate of stimuluspresentation (4–20 times faster than fortransient VEPs), it is possible to obtain reli-able wave forms more rapidly. Second,with SSVEPs it is possible to study atten-tion to stimuli that are continuouslypresent (flickering) rather than onlyflashed occasionally, thereby yielding acontinuous measure of attentional focusingand switching processes. Third, SSVEPmeasurements can reveal how attention isallocated within a complex, multielementstimulus array, because the visual responseto each element can be measured individu-ally by examining the SSVEP at its specificflicker frequency. Fourth, because theSSVEP can be elicited by an irrelevantbackground flicker, it can be used to studyspatial attention to any type of superim-posed stimulus, whether it be a rapidsequence of visual events or a stimulusthat does not change over time. Finally,because of the different temporal response



characteristics of the magno- and parvocel-lular visual pathways, the SSVEP providesa means of studying the mechanisms ofattentional modulation of these pathwaysin relative isolation from one another.

ReferencesAngelelli, P., De Luca, M., and Spinelli, D. (1996).

Early visual processing in neglect patients: Astudy with steady state VEPs. Neuropsychologia 34,1151–1157.

Bernardete, E. A., and Kaplan, E. (1999). The dynam-ics of primate M-retinal ganglion cells. Vis.Neurosci. 16, 355–368.

Bernardete, E. A., Kaplan, E., and Knight B. W. (1992).Contrast gain control in the primate retina: P cellsare not X-like, some M cells are. Vis. Neurosci. 8,483–486.

Bodis-Wollner, I. (1992). Sensory evoked potentials:PERG, VEP, and SEP. Curr. Opin. Neurol. Neurosurg.5(5), 716–26.

Burr, D. C., and Morrone, M. C. (1987). Inhibitoryinteractions in the human visual system revealedin pattern visual evoked potentials. J. Physiol. 389,1–21.

Derrington, A. M, and Lennie, P. (1984). Spatial andtemporal contrast sensitivities of neurones inlateral geniculate nucleus of macaque. J. Physiol.357, 219–240.

Di Russo, F., and Spinelli, D. (1999a). Electro-physiological evidence for an early attentionalmechanism in visual processing in humans. Vis.Res. 39, 2975–2985.

Di Russo, F., and Spinelli D. (1999b). Spatial attentionhas different effects on the magno- and parvo-cel-lular pathways. NeuroReport. 10, 2755–2762.

Di Russo, F., Spinelli, D. and Morrone, M. C. (2001).Automatic gain control contrast mechanisms aremodulated by attention in humans: Evidence fromvisual evoked potentials. Vis. Res. 41, 2435–2447.

Eskin, T. A., Merigan, W. H. (1986). Selective acry-lamide-induced degeneration of color opponentganglion cells in macaques. Brain Res. 378(2),379–384.

Hillyard, S. A., and Anllo-Vento, L. (1998). Event-related brain potentials in the study of visualselective attention. Proc. Natl. Acad. Sc. U.S.A. 95,781–787.

Hillyard, S.A., Hinrichs, H., Tempelmann, C., Morgan,S.T., Hansen, J. C., Scheich, H., and Heinze, H. J.(1997). Combining steady-state visual evokedpotentials and fMRI to localize brain activity duringselective attention. Hum. Brain Mapping 5, 287–292.

Lee, B. B., Pokorny, J., Smith, V. C., Martin, P. R. andValberg, A. (1990). Luminance and chromatic mod-ulation sensitivity of macaque ganglion cells andhuman observers. J. Optic. Soc. Am. A 7, 2223–2236.

Lee, B. B., Pokorny, J., Smith, V. C. and Kremers, J.(1994). Responses to pulses and sinusoids inmacaque ganglion cells. Vis. Res. 34, 3081–3096.

Martinez, A., Di Russo, F., Anllo-Vento, L., Sereno, M. I.,Buxton, R. B. and Hillyard, S. A. (2001). Puttingspatial attention on the map: Timing and localiza-tion of stimulus selection processes in striate andextrastriate visual areas. Vis. Res. 41(10–11),1437–1457.

Merigan, W. H. (1989). Chromatic and achromaticvision of macaques: role of the P pathway.J. Neurosci. 9, 776–783.

Morgan, S. T., Hansen, J. C. and Hillyard, S. A. (1996).Selective attention to stimulus location modulatesthe steady-state visual potential. Proc. Natl. Acad.Sci. U.S.A. 93, 4770–4774.

Morrone, M. C., Burr, D. C. and Speed, H. (1987).Cross-orientation inhibition in cat is GABA medi-ated. Exp. Brain Res. 67, 635–644.

Müller, M. M., and Hillyard, S. A. (2000). Con-current recording of steady-state and transientevent-related potentials as indices of visual-spatial selective attention. Clin. Neurophysiol. 111,1544–1552.

Müller, M. M., Picton, T. W., Valdes-Sosa, P., Riera, P.,Teder-Sälejärvi, A. W., and Hillyard, S. A. (1998a).Effects of spatial selective attention on the steady-state visual evoked potential in the 20–28 Hzrange. Cogn. Brain Res. 6, 249–261.

Müller, M. M., Teder-Sälejärvi, A. W., and Hillyard, S.A. (1998b). The time course of cortical facilitationduring cued shifts of spatial attention. Nat.Neurosci. 1, 631–634.

Porciatti, V., Burr, D. C., Morrone, M. C. andFiorentini, A. (1992). The effects of ageing on thepattern electroretinogram and visual evokedpotential in humans. Vis. Res. 32, 1199–1209.

Regan, D. (1989). “Human Brain Electrophysiology:Evoked Potentials and Evoked Magnetic Fields inScience and Medicine”. Elsevier, New York.

Reid, R. C., Victor, J. D., and Shapley, R. M. (1992).Broadband temporal stimuli decrease the integra-tion time of neurons in cat striate cortex. Vis.Neurosci. 9, 39–45.

Riemslag, F. C., Spekreijse, H., and van Walbeek, H.(1982). Pattern evoked potential diagnosis of mul-tiple sclerosis: A comparison of various contraststimuli. Adv. Neurol. 32, 417–426.

Shapley, R. M. and Victor, J. (1981). How the contrastgain control modifies the frequency responses ofcat retinal ganglion cells. J. Physiol. (Lond.) 318,161–179.

Silberstein, R. B., Schier, M. A., Pipingas, A., Ciorciari,J., Wood, S. R., and Simpson, D. G. (1990). Steady-state visually evoked potential topography associ-ated with a visual vigilance task. Brain Topogr. 3,337–347.

Silberstein, R. B., Ciorciari, J., and Pipingas, A. (1995).Steady-state visually evoked potential topography

REFERENCES 273


during the Wisconsin card sorting test.Electroencephalogr. Clin. Neurophysiol. 96, 24–35.

Spekreijse, H., Estevez, O., and Reits, D. (1977).Visual evoked potentials and the physiologicalanalysis of visual processes in man. In “VisualEvoked Potentials in Man: New Development”(J. E. Desmedt, ed.), pp. 16–89. Clarendon Press,Oxford.

Spinelli, D., and Di Russo, F. (1996). Visual evokedpotentials are affected by trunk rotation in neglectpatients. NeuroReport 7, 553–556.

Spinelli, D., Burr, D. C., and Morrone, M. C. (1994).Spatial neglect is associated with increased laten-cies of visual evoked potentials. Vis. Neurosci. 11,909–918.

Spinelli, D., Angelelli, P., De Luca, M., and Burr, D. C.(1996). VEPs in neglect patients have longer laten-cies for luminance but not for chromatic patterns.NeuroReport 7, 553–556.

Wilson, G. F., and O’Donnell, R. D. (1986). Steady-state evoked responses: Correlations with humancognition. Psychophysiology 23, 57–61.




C H A P T E R

12

Visual Selective Attentionto Object Features

Alice Mado Proverbio and Alberto Zani

INTRODUCTION

The recent developments in neuro-imaging techniques have provided newprecious data on the cortical and sub-cortical neural circuits underlying selectiveattention mechanisms. So-called posteriorand anterior attentional systems have beendescribed. The anterior attentional system,composed of a neural network includingthe frontal and prefrontal areas, the ante-rior cingulate gyrus, and the basoganglia,is believed to regulate functionally therecruitment and control of the cerebralareas having the function of carrying outstimulus processing and complex cognitivetasks.

Unlike the anterior system, the posteriorattentional system, including the parietaland occipital-temporal cortex, the pulvinar,and the superior colliculus, has been foundto be actively involved in the selective pro-cessing of visual information. The mecha-nisms of selection are strongly dependenton the modulation of the functional activ-ity of two neural substreams that projectfrom the visual cortex to the posterior pari-etal area (the so-called dorsal stream) or tothe inferior temporal area (the so-calledventral stream). The dorsal stream, whichreceives both contra- and ipsilateral collic-ular afferents, is believed to handle infor-

mation on spatial location and movementof visual stimuli. Conversely, the ventralstream is thought to analyze stimulusfeatures such as orientation, color, spatialfrequency, and texture.

In the present chapter, findings fromboth our and other labs studying event-related potentials (ERPs) are reviewed anddiscussed in order to reveal the functionalmechanisms of the anterior attentionalsystem in regulating the selective process-ing of visual information. We also focus onneural mechanisms of the posterior systeminvolved in the selection of nonspatialfeatures as investigated by means of ERPs,with specific reference to the possibleelectrocortical generators of these poten-tials. We discuss evidence in favor of, oragainst, a possible segregation of the sub-streams of this system. Furthermore, wedeal with the possibility that the primaryvisual areas, or striate visual cortex (whoseanatomical reference in monkey is areaV1), might be recruited during attentionalsensory filtering of object features at thevery earliest stages of information pro-cessing. In this regard, different lines ofERP research have provided robust evi-dence that selection of visual informationbased on spatial location is accomplishedthrough the activation of extrastriate visualareas (Brodmann areas 18–19) of the con-

tralateral hemisphere to the attendedvisual field starting as early as 70–80 msecpoststimulus (cf. Mangun’s Chapter 10 fora thorough review of this research).

Conversely, whether the selective pro-cessing of nonspatial features is fulfilledthrough a different neural mechanismdirectly modulating V1 activation, or,whether the latter view is championed,when in time—whether at the very earlysensory stage of processing within the pro-jection areas or beyond this stage—is stillcontroversial and a matter of animateddebate.

With these goals in mind, in the follow-ing discussions, a large body of recentresearch is reviewed and discussed, report-ing ERP findings on the attentional selec-tion of single visual features, such asorientation, and spatial frequency, as wellas the conjunction of nonspatial features,such as location and frequency, and loca-tion and color. The consistency of thesefindings with other lines of research onattention in cognitive neuroscience, such ashemodynamic studies of the brain andintracerebral cell recordings, is also treatedto some extent.

Among other main conclusions, it is pro-posed that neural mechanisms of atten-tional selection for the manifold features ofvisual objects, although in part functionallyand anatomically distinct, strongly interactwith one another, as is also suggested bycomparatively recent behavioral findingsobtained in brain-lesioned patients. Basedon ERP findings, it is further suggested thatthis interaction begins at an early sensorystage of processing.

VISUAL SELECTIVE ATTENTIONAND ANTERIOR AND POSTERIOR

NEURAL SYSTEMS

Reviewing the body of evidence gath-ered so far in visual selective attentionresearch, the main conclusion seems to bethat the information selection mechanisms

based on attention perform at least twodistinct functions. In the first place, theypermit the processing of a selected stimu-lus to be preferentially increased comparedwith others present in the surroundingenvironment. Without it, in fact, all stimuliwould be processed at the same level(Hillyard et al., 1999). In the second place,they control the recruitment of functionalcircuits suitable for performing a giventask. This second, executive type, functionis added to cognitive control functions,such as decision-making and control pro-cesses that the individual possesses vis-à-vis his own behavior and the externalenvironment (Posner and Petersen, 1990;Aston-Jones et al., 1999; Gehring andKnight, 2000).

Neurophysiological research based onrecordings of single cell units (Desimoneand Duncan, 1995) in animals, as well asfunctional anatomical studies of the brain,have provided converging evidence on thecortical and subcortical nervous circuitsunderlying these selection mechanisms(Hillyard et al., 1998, 1999; Corbetta, 1999;Corbetta and Shulman, 1999; Haxby et al.,1999). These data provide direct supportfor the view of the existence of two distrib-uted functional networks: a so-called pos-terior attentional system and a so-calledanterior attentional system (Posner andPetersen, 1990; LaBerge, 1995).

The Anterior Attentional System

The anterior attentional system includesthe frontal and prefrontal areas, the ante-rior cingulate gyrus, and the basoganglia.This system is thought to be responsiblefor the functional recruitment and controlof the cerebral areas having the function of carrying out selective informationprocessing across all sensory modalities and complex cognitive tasks (Desimoneand Duncan, 1995). Several neuroimaging and clinical studies reported in the litera-ture confirm the crucial role played by thisfunctional system. In particular, it has been

276 12. ATTENTION TO OBJECT FEATURES


shown that the prefrontal cortex (PFC) isinvolved in all kinds of top-down process-ing, when behavior must be guided byinternal states or intentions (Gehring andKnight, 2000; Miller and Cohen, 2001).

Prior to the massive increase in theseneuroimaging studies, one of the mostsignificant research sectors supporting theview of the executive role played by theanterior intentional system is clinical neuro-psychology. In general, the discoveriesmade in neuropsychological clinical prac-tice have shown how the above-mentioneddeficits are linked to an important disorderof the capacity to maintain a mental repre-sentation of stimulus–response mappingstrategy or of the stimulus set–in otherwords, a combined disorder of the workingmemory system and the attentional system(Shimamura, 1995).

In this regard, evidence is available toshow that patients with lesions of the dor-solateral prefrontal (DL/PF) cortex sufferfrom a set of primary (e.g., deficits of theinhibitory control of response to, or diffi-culties in detection of, novelty), secondary(e.g., distractibility, reduced attention, etc.),and tertiary (e.g., reduced memory andorganizational planning, problems in order-ing past, present, and future events) symp-toms, linked to a frontal syndrome (Stuss etal., 1994; Knight and Grabowecky, 1995;Swick and Knight, 1999). A body of experi-mental evidence to support this model hasemerged from studies on patients withfocal brain lesions subjected to ERP record-ing, a neurophysiological method to studythe synchronized activity of the neuronassemblies. This technique makes it possi-ble to evaluate the effects of specific lesionson the ERP components and providesinteresting information concerning thefunctional role of the various brain areas ingiven tasks.

For this purpose, the long-latency “endo-genous” components of the ERPs, whichare sensitive to the cognitive and psycho-logical variables accompanying a stimulusevent (e.g., Hillyard and Kutas, 1983), were

used to investigate the neural mechanismsinvolved in attention to, as well as memoryand cognitive processing of, the stimulusmaterial in general. A wide-ranging inves-tigation was made, for example, of the so-called P3 of the ERPs, a positive potentialassociated with a number of psychologicalconstructs, including context updating,stimulus categorization, memory, checkingthe correspondence of the stimulus infor-mation with an internal representation, vol-untary orientation, and attention allocation(Coles, 1989). Robust evidence has accu-mulated to show that this componentembodies different subcomponents. Thebest known, P3b, is greatest at central-posterior locations of the scalp, and may be observed when the observer isrequested to detect relevant infrequentstimuli (Donchin, 1981) during signaldetection tasks (Sutton, 1965), wheneverthe relevant stimulus corresponds to aninternal model (Gomer et al., 1976),and more generally during attentional(Proverbio et al., 1994) and coding tasks aswell as mnemonic tasks (Fabiani et al.,1986). In addition, the P3a elicited by anew rare stimulus, irrelevant to the task,associated with automatic voluntary atten-tional orientation processes (Proverbio andMangun, 1994), arousal, and response tonovelty, is stronger at the frontocentralelectrode sites (Squires et al., 1975; Knight,1991).

By studying DL/PF cortex patientsusing cognitive brain potentials it wasshown that, compared with healthycontrol subjects, these patients displayed adramatic reduction in P3a amplitude in thecase of “novel” stimuli (that is, deviantstimuli included in a sequence of irrele-vant and relevant stimuli, with the patientsinstructed to respond to the latter by press-ing a button). This reduction was greatestat the anterior electrode sites for all thesensory modes—visual, auditory, andsomatosensory—of the stimuli used inthese studies (e.g., Knight, 1991; Knightand Grabowecky, 1995). It is very interest-

VISUAL SELECTIVE ATTENTION AND ANTERIOR AND POSTERIOR NEURAL SYSTEMS 277


ing to note that this specific reduction infrontal P3a was found to be typical only ofthese patients. Patients with damaged pari-etal and temporal lobes actually displayedP3as with amplitudes comparable to thoseobtained in healthy control subjects(Knight, 1991; Knight and Grabowecky,1995).

In general, these discoveries show howcrucial the prefrontal cortex is in the detec-tion of changes in the external environ-ment and in distinguishing derived modelsof the world both internally and externally(Knight, 1991; Knight and Grabowecky,1995; Shimamura, 1995). More recent ERPdata indicate that what is perhaps the mostimportant deficit linked to a lesion of theDL/PF cortex consists of the inability toreject or suppress irrelevant informationwithin all sensory systems, whereas themajor deficit linked to a lesion of themedial prefrontal cortex is impairment ofthe ability to monitor behavior to guideand compensate possible behavioral errorsand conflicts. These two deficits indicatethat critical functions of subareas of theprefrontal cortex are the control of neuralinformation processing through the modu-lation of activation of sensory systems, asshown by Barcelo et al. (2000) for visualextrastriate cortex, and the monitoring ofbehavior through direct connections to thecingulate cortex (Gehring and Knight,2000).

We obtained P3 data supporting theneuropsychological view that prefrontalareas might also exert a function of inhibi-tory filter in neurologically intact volun-teers (Zani and Proverbio, 1995, 1997a). Inthe first of the studies, we used a selectivevisual attention task in which the subjectswere presented with random sequences ofsix checkerboard patterns with differentcheck sizes. In different experimentalsessions, the attentional task consisted ofpaying selective attention and makingmotor responses to one checkerboardhaving checks of a given size, and to ignoreall the others (see also the section on space-

based and frequency-based attentionalselection). ERPs were recorded from theposterior left and right mesial (O1 and O2),and lateral occipital (OL and OR), besidesthe anterior F7 and F8, homologous scalpsites falling over the two hemispheres ofthe brain. At the anterior sites, the late pos-itive component P3b (latency between 350and 500 msec) was found to have greateramplitude in the case of irrelevant stimuli,compared to relevant stimuli. Previousstudies with a two-stimuli go/no-go taskdesign simpler than ours also showed thatsuppression of the response during no-gotrials elicited ERP late components, or P300waves, of the greatest amplitude at theanterior electrode locations (e.g., Robertset al., 1994). Interestingly, similar resultswere also found in more structured studiesrequiring the information to be stored inthe working memory during go/no-gotasks (e.g., Gevins and Cutillo, 1993).However, as may be inferred by observingthe grand-average ERPs recorded at ante-rior sites in response to stimuli of differentcheck sizes in our study (see Fig. 1), theamplitude of the late positive componentdisplayed a clear-cut gradient as a functionof attended size, in that it was larger forirrelevant stimuli more similar to the rele-vant target (i.e., one or more octaves withinthe frequency band—and thus more diffi-cult to ignore) and decreased as the checksizes became more unlike the target (i.e.,more octaves outside the frequency band—and thus easier to ignore). These findingsextend those made in previous studies, inthat they indicate that the larger frontal P3effect for no-go/irrelevant stimuli does notindex inhibition in simply an “all-or-none”fashion, thus enabling more than onebinary separation between “irrelevant”and “relevant” stimuli. Rather, to theextent that this larger frontal positivity isindexing a suppressive process, the view isadvanced that the extent of process modu-lation is a function of the degree of similar-ity of irrelevant stimuli to relevant ones. Inother words, it is a function of the greater



or lesser interference during the task of theformer with the latter, and, as a conse-quence, of the stronger or weaker need tosuppress neural response to irrelevantstimuli within the attentional and motorchannels so to avoid processing overloadand incorrect responses (i.e., false alarms).

To the extent that the suppression of aresponse represents a gap in the primarymapping strategy between stimulus andresponse, the anterior distribution of thisspecific potential seems to be consistentwith the role of the frontal lobes in thestructuring of temporal events, in themediation of preparatory processes, of pro-gramming and control on the allocation ofthe individual’s attentional resources. It isalso consistent with the hypothesis of thefrontal lobes being extensively involved

in the working memory, as suggested byneurophysiological studies too (Goldman-Rakic, 1987).

In another study (Zani and Proverbio,1997a) investigating neural mechanisms ofthe selective processing of multidimen-sional stimuli—that is, the conjoined andseparate selection of spatial location andspatial frequency features—21 subjectswere administered four sinusoidal grat-ings randomly flashed to the inferior andsuperior quadrants of the visual fieldwhen relevant and irrelevant. The gratingsproduced stimulation at 0.75, 1.5, 3, and6 cycles per degree (cpd) of visual angle.ERPs were recorded from homologousmesial occipital O1 and O2 and lateral-occipital OL and OR, as well as mesialfrontal F3 and F4 electrode sites. In dif-ferent runs, volunteers either engaged inpassive gazing of the gratings, or selec-tively attended and responded motoricallyto either 0.75 or 6 cpd at a relevant location(i.e., one of the visual quadrants) whileignoring all the other gratings and loca-tions. In this way, while the physicalstimuli remained unchanged, attentionshifted across spatial frequency and spatiallocation. Thus, in separate attention condi-tions, one and the same stimulus could be(1) relevant both in spatial location andspatial frequency (i.e., L+F+), (2) relevantin location but irrelevant in frequency(L+F–), (3) irrelevant in location but rele-vant in frequency (L–F+), and (4) irrele-vant in both features (L–F–). As shown inFig. 2, regardless of the spatial frequencyattended, within the 300- to 600-mseclatency range, stimuli irrelevant in one(i.e., L+F– and L–F+) or in both featuresshowed a larger P3 wave at anterior, com-pared to posterior, sites. The reverse wastrue for the targets (L+F+).

Regardless of electrode site, P3 responseto targets was larger than to gratingssharing spatial location with them (L+F–).Again, the latter condition yielded largerP3s than did stimuli not sharing spatiallocation (L–F+) or neither feature (L–F–)



FIGURE 1 Grand-average ERPs to checkerboardpatterns recorded at lateral frontal sites as a functionof attended check size (in minutes of arc). The maineffects of attention are an enhanced frontal positivityto relevant patterns, peaking on average at 225 msec,followed by a later P3b greater to irrelevant patternsfalling outside of the band, probably reflecting anactive suppression of the response to these stimuli.Reprinted from Electroencephalography and ClinicalNeurophysiology 95; A. Zani and A. M. Proverbio; ERPsigns of early selective attention effects to check size,pp. 277–292. Copyright (1995) with permission ofElsevier Science.

with targets, whereas the latter two condi-tions did not differ from each another. Alsoworth noting is that although at anteriorsites the latter two conditions were signi-ficantly much larger in amplitude, com-pared to the passive condition, at posteriorsites they were not. This finding suggeststhat higher level selective processing ofstimuli irrelevant in location was sup-pressed at some previous stage of process-ing. ERPs displayed in Fig. 2 point to apositive activation occurring within theearly latency range of 160–260 msec post-stimulus as the site of the supposedsuppression. Indeed, within the aforemen-tioned latency range a large P2 componenthaving a mean latency of 190 msec dis-played a different trend across posteriorand anterior electrode sites, thus indicatingthat this positivity might actually indexdifferent neural processes in separate dis-tricts of the brain. In actual fact, at posterior sites, P190 was largest inresponse to passively viewed stimuli (i.e.,

neutral), and decreased as a gradient fromgratings irrelevant in both features to thoseirrelevant in one feature only, and, finally,to those relevant in both features. This wasprobably due to an overlapping effect of anincreasing selection negativity as a func-tion of feature attention conditions in thesame latency range. Conversely, at anteriorsites, P190 was larger in response to stimuliirrelevant in location (i.e., both L–F+ andL–F–) than to neutral stimuli. In turn,neutral stimuli yielded a larger P190 thandid stimuli relevant in location (i.e., bothL+F+ and L+F–).

Overall, the present findings suggestthat, unlike in the passive task, duringselection tasks, prefrontal areas differen-tially are activated to control the selectiveprocessing of stimuli falling within oroutside the focus of spatial attention by theposterior areas of the brain. The view isadvanced that P190 might index an earlysuppression operated by prefrontal cortexof stimulus perceptual processing of both



µ

FIGURE 2 Grand-average ERPs to 7-cpd sinusoidal gratings collapsed across the four quadrants of the visualfield, as recorded at homologous prefrontal and occipital scalp sites as a function of attention condition in aconjoined selection task. L, Location; F, frequency; +, relevant; –, irrelevant; Pas, passive gazing.

relevant and irrelevant gratings fallingoutside the focus of spatial attention. Onthe other hand, the P3 might index abelated suppression mechanism of higherlevel stimulus processing, in particular of stimuli falling within the focus ofattention.

To the extent that the same prefrontalareas exerting an executive control on selec-tive mechanisms of attention are involved inoculomotor programming and execution, aspointed out by Corbetta and Shulman (1999), an alternative hypothesis may beadvanced to explain the P190 trend atanterior sites. Indeed, it might be arguedthat the latter possibly index the activationof prefrontal areas suppressing eye sac-cades toward stimuli falling in an un-attended point in space, no matter whetherrelevant in neither feature, or one or morefeatures.

Sufficient evidence is still unavailable todismiss either viewpoint, and, very likely,scalp-recorded ERPs, as well as fMRI orPET, are not enough to answer this ques-tion. Most probably, only a combiningstudy will come up with the timing, local-ization, and true functional nature of theprocesses reflected at the scalp by P190 andP3, respectively, in this conjoined selectiontask. It is interesting, however, in thisregard, that reviews (Knight et al., 1999;Hermann and Knight, 2001) of studies onpatients with prefrontal lesions engaged ingoal-directed tasks show that both early(P1 and N1) and late (P3) ERP componentsare modulated by excitatory and inhibitorymechanisms. Based on these findings, theconclusions advanced by the authors arethat, on one hand, this damage disruptsinhibitory modulation of irrelevant inputsto primary sensory cortex, and on theother, results in multimodal decreases inneural activity in the posterior associationcortex in the hemisphere ipsilateral todamage.

The latter findings provide some indica-tion in favor of the processing suppressionviewpoint rather than the suppression of

oculomotor saccades viewpoint in explain-ing P190 behavior in our study.

Briefly, the converging results of neuro-psychological clinical and electrophysio-logical studies suggest that the maintenanceof short-term behavior control strategies,together with the capacity to inhibit theprocessing of irrelevant stimuli or events,are among the more important functionsperformed by the frontal lobes of the ante-rior attentional system. These two functionsare closely related and can account formany of the behavioral disorders derivingfrom lesions of the prefrontal cortex, whichis part of this system.

The Posterior Attention System

Unlike the anterior system, the posteriorattentional system includes the parietal andoccipital-temporal cortex, the pulvinar andthe superior colliculus. This system isactively involved in the selective process-ing of visual information. The selectiveprocessing strongly depends on the modu-lated activity of two parallel streams, orneural subsystems, which from the pri-mary visual cortex (or V1) project to theposterior parietal area (the so-called dorsalstream) or to the inferior temporal area(the so-called ventral stream), first proposedby Ungerleider and Mishkin (1982) andlater subjected to intense investigation (forexample, see Merigan and Maunsell, 1993;Webster and Ungerleider, 1999). The dorsalstream, which receives both contra- andipsilateral collicular afferents, managesinformation on stimulus spatial locationand movement. Conversely, the ventralstream analyzes stimulus features such asorientation, color, spatial frequency, andtexture.

Dorsal and Ventral Streams of the Posterior System

As already noted, the two streams of theposterior system involved in the selectionand analysis of visual inputs project from the visual cortex onto the posterior



parietal (dorsal stream) and the inferiortemporal (ventral stream) areas, respec-tively. Evidence has accumulated to indi-cate that the dorsal stream handlesinformation on spatial position and motionof stimuli, as it possesses (also ipsilateral)collicular afferents, whereas the ventralsystem analyzes physical features such asorientation, color, spatial frequency, andtexture. (Ungerleider, and Mishkin, 1982;Webster and Ungerleider, 1999). Althoughthe former mostly receive afferent fibersfrom large magnocellular gangliar cells, thelatter receive afferences from small parvo-cellular cells. There is also further evidencethat these two systems are related to sco-topic and peripheral vision, as opposed tophotopic and foveal vision, to the vision oflow as opposed to high spatial frequenciesand, more generally, to the visual attentionmechanisms based on space rather than onthe object (Fink et al., 1997).

Hemodynamic functional anatomicalstudies have clearly shown that visualattention modulates the activity of bothsystems (Cabeza and Nyberg, 2000; Dupontet al., 1998). This modulation has beenobserved also by measuring changes inamplitude, latency, and scalp topographyof event-related potentials of the brain inresponse to visual stimuli as a function oftask relevance and attention condition (e.g.,Anllo-Vento and Hillyard, 1996; Martin-Loeches et al., 1999; Näätänen, 1992; Previc,1990; Wang et al., 1999; Zani and Proverbio,1995). [See Chapter 10, this volume, forreview of attention mechanisms based onspatial location. Several additional reviewson these mechanisms have been providedby Hillyard et al. (1995), Martinez et al.(1999), and Luck et al. (2000).] In theremainder of this chapter, we deal primar-ily with neural mechanisms underlying theselection of nonspatial features as investi-gated with ERPs. Reference is also made tosingle-unit and blood-flow studies on thesemechanisms concerned with their possiblecortical localization to corroborate ERPfindings.

DOES VISUAL SELECTIVEATTENTION MODULATE

PRIMARY VISUAL AREAS?

For a long time it was believed that theprimary projection areas of brain cortexacted as simple analyzers of input featuresand were not directly involved in the so-called top-down selection mechanisms, i.e.,those based on higher cognitive strategies.Only recently has this conception beenopposed, thanks to the new findings pro-vided by bioimaging and neurophysiologi-cal and electromagnetic techniques. Thesetechniques are able, on the one hand, todetermine the functional activation of the cortical and subcortical areas, and, onthe other, to show up the early timing ofthe attentional influences on the processingstages.

Regarding auditory modality, it hasbeen shown in selective listening tasks thatattention can modulate the informationflow captured by one of the ears whenmaterial is presented rapidly to both ears,right from the earliest processing stages.This was demonstrated by measuring thechanges in amplitude of a small positivedeflection (latency, 20–50 msec), the gener-ator of which was identified in the primaryauditory area by the combined ERP andPET study of Woldorff and colleagues(1993).

For the visual modality, there is robustevidence indicating that space-based infor-mation selection influences the extrastriatevisual areas (i.e., Brodmann areas 18 and19 of the hemisphere contralateral to theattended field) as early as 70–80 msec post-stimulus (e.g., Mangun et al., 1997, 2001;Martinez et al., 1999).

As for nonspatial features, it has beenreported that attentional selection takesplace through different neural mechanismsdirectly affecting analysis of the specificfeature (color, spatial frequency, etc.),although this view is still controversial inthe literature. These findings are illus-



trated in greater detail in the followingdiscussions.

As far as temporal onset is concerned,the attentional effect is believed to start ataround 60–70 msec, corresponding in allprobability to the activation of the primaryvisual area (V1). For example, the selectionof checkerboard patterns based on theircheck size produces an increase in ampli-tude of the sensory responses P1 and N115recorded at electrodes O1 and O2, corre-sponding to the primary visual areas (Zaniand Proverbio, 1995). Likewise, selectinggratings on the basis of their spatial fre-quency (Zani and Proverbio, 1997a–e) andorientation (Karayanidis and Michie, 1997)or selecting alphanumeric characters onthe basis of their shape (Skrandies, 1983)produces an increase in the evokedresponse at the sensory level. This meansthat the attentional strategy adopted by theobserver to identify as rapidly and effec-tively as possible an interesting object inthe visual environment is able to enhancethe response of the visual system to thatobject’s features by setting an early selec-tion sensory filter.

This type of filter is seen, as well as inthe attentional modulation of P1, also in

increases in amplitude of a precedingresponse, prominent on the hemisphereipsilateral to stimulation, known asP/N80, so named because of its averagelatency of 80 msec (see Fig. 3). It has a pos-itive or negative polarity according to thetype of stimulus (for example, increasingnegativity with increasing spatial fre-quency), type of hemifield (polarity isreversed on going from the inferiorhemifield to the upper hemifield), andtype of retinal eccentricity. The P/N80 isalso known as C1 [Component 1, Jeffreysand Axford (1972)] and its inversion inpolarity depends strongly on the crossedretinotopic organization of visual path-ways and calcarine fissure in the occipitalstriate cortex, described as the cruciformmodel. According to this model, based onthe organization of visual pathways,stimuli falling beneath the horizontalmeridian of the visual field are projected tothe superior lip of the calcarine fissure inthe hemisphere contralateral, across thevertical meridian, to the area of the visualfield affected by the stimuli. Conversely,stimuli falling above the horizontal merid-ian end up finding their representation inthe lower lip of the same fissure with thesame hemispheric logistics (see Fig. 4A–C). The neural generator of this poten-tial was identified in the calcarine fissureof the striate cortex using differentmethods, such as scalp current density(SCD) (see Appendix D for a more detailedillustration) by Proverbio et al. (1996), andthe combining of spatiotemporal dipolemodeling (this technique is described ingreater detail in Chapter 2) with corticalanatomy provided by magnetic resonance(Clark and Hillyard, 1996).

Indeed, whatever the polarity of thecomponent under passive vision condi-tions, we found that spatially directedselective attention was able to modulatethe amplitude of this potential, generallyin the direction of increased positivity(Zani and Proverbio, 1997c). The atten-tional modulation of P/N80 recorded in

DOES VISUAL SELECTIVE ATTENTION MODULATE PRIMARY VISUAL AREAS? 283


FIGURE 3 Grand-average ERPs to black andwhite luminance-modulated high-spatial-frequency(6 cpd) gratings as a function of task relevance. ERPswere recorded from left lateral occipital site (OL) tostimuli presented in the lower and upper quadrantsof left visual field (LVF).



FIGURE 4 Relationships between the so-called cruciform model and the P/N80 (or C1) component. (A) Sketchy drawing of the crossed retinotopic layout of primary visual cortex versus the visual field underlyingthe cruciform model. On the left, the central (i.e., foveal) and peripheral (i.e., perifoveal) districts of the visualfield are drawn, marked with capital letters. On the right, the occipital lobes of brain left and right hemisphereshave been depicted as virtually severed and blown up, as well as pulled apart, so to expose the calcarine fissurein the mesial surface of each hemisphere. As can be seen, the various districts of the visual field find a crossedrepresentation, across both the vertical and the horizontal meridians of the latter, in the internal and the externallips of the calcarine fissure, due to the anatomical organization of the visual pathways. (B) Downward dipole. Onthe right, a grating pattern is presented in the upper left visual hemifield. For the aforementioned organization ofthe visual pathways this stimulus is projected to the inferior lip of the contralateral calcarine fissure. This can beseen in the observer’s head profile, where the posterior portions of the hemisphere ipsilateral to the stimulushave been removed to make the mesial surface of the contralateral occipital lobe visible. This lobe and itsBrodmann areas may be observed more clearly, in the blow-up on the right. Observing this blow-up it can beeasily evinced that the dipole localized in the inferior lip of the calcarine fissure goes downward off the scalp elec-trode. For this the electrode records the currents’ flow inward from the head surface as a sink and early latencynegativity. (C) Upward dipole. The stimulus in the lower quadrant is projected to the upper lip of the contralat-eral calcarine fissure. The dipole results in a source because the currents flow outward, approaching the electrode,which records positivity.

human observers engaged in selectiveattention tasks is consistent with the neu-rophysiological evidence obtained usingcats and monkeys (see, for instance,Motter, 1993; Lamme and Spekreijse, 2000)on the modulation of neuronal populationsof V1, as well as of V2 and V4 during theselection of nonspatial features. Several different neurophysiological studies carriedout on macaques also indicated a clear-cutattentional modulation of V1 for the selec-tion of orientation (Press and Van Essen,1997; Vanduffel et al., 1997), of movement(Watanabe et al., 1998), of spatial frequencyand color (Metha et al., 1997), and of shape(Roelfsema et al., 1997, 1998). Interestinglyenough, Ito and Gilbert (1999) found that thefiring rate of cells in the primary visualcortex of alert monkeys was significantlymodulated by attentional set during aspatial attention task. Overall, these findingsprovided evidence that attentional modula-tion of sensory responses can be observed inmost areas of the visual cortex, including V1(see Treue, 2001, for a review), and perhapsearlier in the lateral geniculate nucleus of thethalamus (Vanduffel et al., 2000). Neverthe-less, the literature is still controversial on thismatter. Conflicting evidence concerning V1modulation for spatial attention has been

found, for instance, by Luck et al., (1997).Their macaque study, in which differentspatial visual attention tasks were used, didnot reveal any modulation of response ofthe neurons in visual area V1, as opposed toV2 and V4 modulation (Luck et al., 1997).The results led the authors to conclude thatthe striate visual area (V1) was not modu-lated by spatial attention. Nevertheless, thisis not a straightforward conclusion because,as the authors correctly stress, stimuliattended or ignored by the animal fell in thereceptive fields of different neurons, so thatit was impossible to determine whethertheir firing rate would also have been mod-ulated by attention if both stimuli had fallenin their receptive field.

Somers et al. (1999) carried out a 3-teslafMRI study on spatial attention modula-tion in humans. Very interestingly, theyfound a robust attentional modulation inboth striate and extrastriate cortical areasduring object-based spatial attention tasks.The authors concluded that neural pro-cessing in V1 can be strongly and speci-fically influenced by spatial attention.

Again using human subjects, a significantERP study by McCarthy and Scabini (1991)was carried out on epileptic subjects whohad microelectrodes implanted directly in

DOES VISUAL SELECTIVE ATTENTION MODULATE PRIMARY VISUAL AREAS? 285


FIGURE 4 (continued).

the striate occipital cortex for therapeuticalpurposes. Because this is a direct measure-ment, it is interesting that an altered activa-tion of the neurons in this area should befound during selective attention tasks. Theless invasive studies of Aine et al. (1995),who combined MRI measures and magne-toencephalography (MEG) measures, andthe metaanalysis by Shulmann et al. (1997),performed by making a comparative surveyof a large number of studies carried outusing this method, consistently indicate amodulation of visual area 17 during a largenumber of active tasks involving the dis-crimination of features. Likewise, recentelectrophysiological studies (Zani andProverbio, 1995, 1997a; Proverbio et al., 1998)on healthy volunteers have shown an atten-tional modulation of early evoked responsesselectively recorded at O1 and O2 scalp sitesof mesial occipital areas, which might reflectthe activity of intracranial neural generatorslocated in the primary visual area. In partic-ular, the study by Proverbio et al. (1998),using hierarchical alphanumeric stimuli thatare either congruent or incongruent at local/global level, demonstrated that whenever astimulus attended at the local level has thesame shape as the global configuration (con-gruent condition), it elicits a greater N115response over the mesial occipital area (asshown in the wave forms and activationmap in Fig. 5), compared with the incongru-ent condition.

In addition to the visual cortex, atten-tion might also modulate the functionalityof subcortical structures such as the supe-rior colliculus, and the lateral geniculatebody and the pulvinar of the thalamus.They actually receive and transmit retino-topically organized projections to thestriate and prestriate cortex. Indeed, datasupporting this view have been reportedby LaBerge and Buchsbaum (1990). Anenhanced activation of these structurescould also account for the increase invisual evoked potentials recorded as earlyas 40 msec poststimulus in an attentionaltask by Oakley and Eason (1990). This

strongly supports the hypothesis thatvisual information might be subjected to asensory filter controlled and monitored bysuperior centers such as the dorsolateralfrontal areas, which might be capable ofmodulating the earliest analysis at corticaland subcortical levels.

FEATURE-DIRECTED ATTENTION

Several ERP studies have investigatedthe selective attending of nonspatial attrib-utes of visual information. In these studiesvolunteers had to attend and select either asingle attribute or a conjunction of two ormore stimulus attributes. Although somestudies have obtained evidence for earlyselection, many others have failed to findsuch effects (Rugg et al., 1987; Hillyard et al., 1998). Indeed, this has started a now long-running controversy in attentionresearch between proponents of earlyversus late models of selection of anobject’s physical attributes. This contro-versy still forms the backdrop for somecurrent electrophysiological work in atten-tion (e.g., Martinez et al., 2001; Proverbio et al., 2002).

Below we illustrate the data in the litera-ture separately for attribute and single andconjoined selection tasks. We note here thatwe do not review all the work carried outon visual selective attention to stimulusfeatures; this work has been discussedexhaustively previously [see, example, theexcellent review by Schroeder (1995)]. Withsome exceptions, our focus is mostly veryrecent work.

Spatial Frequency

Object recognition is known to occur inthe very early stages of processing bymeans of spatial frequency analysis of theluminance configuration (see Box 1). Thisis made possible by the presence in visualcortex of specific frequency analyzers in



the form of neurons sensitive to a givenrange of frequencies and organized on thebasis of preferences for orientation andcolor. The spatial frequency channels varyin sensitivity from 0.8 to 3 octaves appro-

ximately, increasing this sensitivity onaverage to 1.8 octaves in parafovea and1.4–1.6 octaves in fovea. An octave is theinterval between a given spatial frequency(e.g., 1° 30’) and double the frequency (inthis case 3°). For example, a spatial fre-quency channel with a sensitivity band-width of 1 octave will respond significantlyto the preferred frequency (e.g., 2° 15’) andto a lesser degree to limitrophe frequencieswithin the octave.

Harter and Previc (1978) were the firstto study the spatial frequency selectionmechanisms and the variation in activityof the various channels by measuring thebrain’s bioelectrical response to stimuli ofthe attended (target) or ignored (nontar-get) frequency. They found that the selec-tion negativity (onset, 150 msec) elicited bythe visual cortex for target stimuli gradu-ally decreased with the increasing gapbetween the frequency of the target (in theform of large checks) and those of the non-targets. These authors also showed that theamplitude of the selection bandwidthgradually shrank as the stimulus analysisproceeded down to the absolutely certainidentification of the target frequency at theP300 level.

A subsequent study (Proverbio et al.,1993), using 1.5, 3, 6, and 12 cpd sinusoidalfoveal gratings, revealed that selectiveattention for spatial frequency is able tomodulate the amplitude of visual evokedresponses at the level of the sensory com-ponent P1, as well as N1 and N2. Fig. 6shows the effect of attention on the firstERP components (evoked by 6 cpd grat-ings) manifested by a greater amplitude ofthe response for relevant stimuli, ratherthan irrelevant stimuli. The specific effectof the general state of alert, rather thanthat of attentional selection, is made visibleby comparing the amplitude of theresponses evoked by relevant and irrele-vant stimuli (active condition) with thoseevoked by neutral stimuli (the samestimuli during passive vision), the latterbeing significantly lower.

FEATURE-DIRECTED ATTENTION 287


FIGURE 5 Top: Grand-average ERPs to global andlocal targets as a function of congruence with theunattended level. Recordings are from Oz electrodesite. Bottom: Isocolor voltage maps of early sensoryresponse to local targets in the latency range of N115component. (See color plates.) Reprinted fromCognitive Brain Research, 6; A. M. Proverbio, A.Minniti, and A. Zani; Electrophysiological evidence ofa perceptual precedence of global vs. local visualinformation, pp. 321–334. Copyright (1998) withpermission of Elsevier Science.

Another study carried out by ourresearch group (Zani and Proverbio, 1993,1995) identified the first effects of spatial fre-quency selection in a modulation of thesensory components P90 and N115 gener-ated on the striate and prestriate visualcortex. Fig. 7 shows the effects of attentionon ERPs to target and nontarget stimuli as afunction of the distance from the channelband. The effect of selection is seen to be

very strong and, starting from early laten-cies, becomes increasingly pronounced,especially for the selection negativity (SN),which overlaps the components N1 and N2,and P300. The similarity between targetsand neighboring band nontargets producedintermediate evoked responses and a certainnumber of false alarms (see Fig. 8).

It must be mentioned that several morerecent ERP studies also dealing with selec-



The spatial frequency of a visual stimu-lus (in cycles per degrees) defines thenumber of variations of luminance presentin a degree of visual angle. Consideringthat 1° of visual angle, seen from a distanceof 57 cm, subtends 1 cm of visual space, itis possible to compute the stimulus spatialfrequency, which will naturally vary as afunction of the viewing distance, whenstimulus size and viewing distance areknown. The equation is as follows:

(1) 57 : d = 1 : s,

where s is stimulus size in centimeters, andd is viewing distance. For example, if thestimulus size is 2 cm (e.g., the width of abar), and the distance from the observer is200 cm (2 m), the stimulus spatial fre-quency will be:

(2) 57 : 200 = 1 : 2,(3) 200 : 114 = 1.75,(4) 1° 45’’of spatial frequency.

In psychophysics and psychophysiologyof vision, simple visual configurationscharacterized by periodic variations ofluminance such as gratings and checker-boards, defined by their spatial frequency,orientation, spatial phase, size, eccentricity,and contrast, are usually adopted as experi-

mental stimuli. Gratings are defined assinusoidal if their luminance varies sinu-soidally (that is, smoothly from the darkerto the brighter area), or square wave, iftheir luminance varies abruptly from thedarker to the brighter area.

In principle, every complex luminanceconfiguration is decomposable into thesum of multiple simple configurations ofdifferent phase and spatial frequency. Thisdecomposition is based on the so-calledFourier analysis, which is a mathematicalprocedure used to determine the collectionof sine waves, differing in phase andamplitude, that makes up a complex visualor acoustic pattern. It is named after theFrench mathematician Joseph Fourier(1768–1830), known chiefly for his contri-bution to the mathematical analysis of heatflow. This procedure is widely used in theanalysis and treatment of communicationssignals, linear systems analysis, optics,antenna studies, acoustics, etc. In addition,this technique represents the fundamentaltool underlying electroencephalogramanalysis, in that it allows the raw EEGsignal to be broken down into the variousfrequencies of oscillation of spontaneousbrain bioelectrical activity (alpha, beta,theta, gamma, delta, etc).

BOX 1

S P AT I A L F R E Q U E N C Y

tive attention to spatial frequency did notcome up with the reported early attentioneffects (e.g., Martinez et al., 2001). Still, it isworth noting that there are many method-ological differences between our studiesand these others. For a review of thesesignificant differences, possibly explainingthe lack of early effects in the later studies,the reader is referred to Proverbio et al.(2002).

Orientation

As for spatial frequency, orientation is avisual attribute analyzed at the primarylevel by visual cortex neurons organizedinto columns. Campbell and Maffei (1970)provided electrophysiological evidence ofthe existence of spatial frequency channels

sensitive to stimulus orientation, anddetermined the amplitude of the sensi-tivity bandwidth in humans by measuringthe amplitudes of the evoked potentials asa function of adaptation to high-contrast



FIGURE 6 Attention to spatial frequency. Grand-average ERPs to 6-cpd luminance-modulated gratingsrecorded at left (OL) and right (OR) lateral occipitalsites as a function of attention condition: attended (—),unattended (…..), or passive (---). Stimuli were pre-sented in the central visual field. Passive refers to acondition of passive gazing.

FIGURE 7 Grand-average ERPs elicited bycheckerboards of different check sizes as a function ofattended size (either 60, 10, or 5 minutes of arc).Recordings were taken from left (OL) and right (OR)lateral occipital sites. Reprinted from Electro-encephalography and Clinical Neurophysiology, 95; A.Zani and A. M. Proverbio; ERP signs of early selectiveattention effects to check size, pp. 277–292. Copyright(1995) with permission of Elsevier Science.

FIGURE 8 Mean percentages of emitted responses(hits plus false alarms) as a function of attention con-dition and check size. Note that when attention waspaid to intermediate check sizes, there was a com-parable high percentage of emitted responses to thepatterns closer in size to the attended one. Reprintedfrom Electroencephalography and Clinical Neuro-physiology, 95; A. Zani and A. M. Proverbio; ERP signsof early selective attention effects to check size, pp. 277–292. Copyright (1995) with permission ofElsevier Science.

gratings. They showed that when the teststimulus differed in orientation at least 20°or more with respect to the adaptationgratings, the amplitude of evoked poten-tials dramatically increased compared towhen they were elicited by stimuli closerin orientation to the adaptation gratings.These findings demonstrated that the ori-entation sensitivity bandwidth in humansis about 10–20° (varying as a function ofstimulus spatial frequency).

So although the visual system can dis-criminate rather well between elementarystimuli rotated by only 20°, the vastmajority of ERP studies on orientationhave used only vertical and horizontalorientations for experimentation. The firstelectrophysiological studies on orientationselection were carried out in the early1980s by the distinguished scholar RussellHarter (who died prematurely), togetherwith his colleagues Guido and Previc. Theresults of their pioneering studies showedthat the responses evoked by target gratingorientations were characterized by a selec-tion negativity that developed on the visualcortex beginning 150 msec poststimulus,and that indicated the existence of specificsensory filters in the visual cortex (Harterand Guido, 1980). Based on these findings,Harter advanced a neural specificity theory ofattentional selection (Harter and Aine,1984), according to which the SN to a givendimension of an attended stimulus (e.g., acertain orientation or spatial frequencyband) reflected the increased response ofneural centers normally involved in theanalysis of that dimension. The lattertheory, though, encountered oppositionamong peer electrophysiologists involvedin the enterprise of investigating neuralmechanisms of attention (see Hillyard andMangun, 1986).

Although properly conducted theseearly studies nevertheless suffered fromseveral methodological weaknesses, suchas the use of fixed latency measurements of brain potentials (e.g., at 75 msec, at125 msec, etc., thus not respecting the true

latency of ERP peaks), and a nonoptimalsignal-to-noise ratio for the evokedresponses, which may have impaired theanalysis of the earliest and the weakestsensory responses.

All of the relatively few studies thatfollowed that of Harter’s group continuedto investigate only orthogonal orientations(e.g., Kenemans et al., 1994), and oddlyenough none of these studies led to knowl-edge of the scalp topography of theelectrophysiological effects of orientationselection.

A very recent whole-head (32-channel)topographic mapping study by Proverbioet al. (2002a) revealed an interesting effectof early modulation of the temporal P1,with a strong lateralization over the lefthemisphere (see Fig. 9), reflecting theattentional selection of grating orientation.In this study, participants were foveallystimulated using a random series of iso-luminant black and white gratings (3 cpd)having an orientation of 50°, 70°, 90° (verti-cal), 110°, and 130° of visual angle, and asize of 2°. The task consisted of selectivelyattending and responding to one of the fivegrating orientations, while ignoring theothers. Difference waves obtained by sub-tracting ERPs to irrelevant orientationsfrom those to relevant orientations showedthat the selection of this feature modulatedneural processing at an early poststimuluslatency within the P1 latency range. Inaddition, ERP mapping procedures yieldeda focus for this effect over the posteriortemporal regions.

It is worth noting that this area wasindicated in both neurophysiological andhemodynamic studies as the area of finalcortical projection of the ventral visualsystem having the function of analyzingthe nonspatial features (What system), andthus appears to be preferentially involvedin this type of task. Indeed, Vogels andOrban (1994) and Tanaka (2000) found thatcells of the inferior temporal cortex inmonkey are strongly activated duringorientation and/or object discrimination;



the PET study by Kawashima (1998)showed a specific activation of the left infe-rior temporal cortex during object discrimi-nation in humans.

In line with the aforementioned find-ings, our ERP results suggest that temporalarea activity may be modulated by visualnonspatial attention at a very early stage ofprocessing. Consistently, Schroeder et al.(1998) described an early modulatory effectin V4 and the inferior temporal region (IT),as revealed by current source density(CSD) analysis of brain single-cell poten-tials in awake macaques stimulated withfoveally presented light flashes and blackand white square-wave gratings (3 cpd). Inour opinion, it is probable that the findingsof our study in humans indicate that top-down attentional processes may modulate

these initial visual inputs to V4 and the ITregion found in monkeys.

Besides the left-sided hemisphericasymmetry for the early attention effect,we found similar asymmetries for theattention effects concerning the later N150and P300 components, as well as a left gen-erator for the extrastriate posterior N150,regardless of grating relevance. All in all,these findings support the hypothesis of apredominant involvement of the left hemi-sphere in object features discrimination, asalso supported by neuroimaging studies(Dupont et al., 1998; Georgopoulos et al.,2001), as well as some ERP attentionstudies (Eimer, 1996a; Martinez et al., 2001;Zani and Proverbio, 1995).

Another very interesting result of theProverbio et al. (2002a) study is the clear



FIGURE 9 Top: Isocolor maps of effects of orientation selection for vertical (90°) gratings in the P1 latencyrange. Maps were obtained by plotting the values of the difference wave (target–nontarget) computed by sub-tracting the ERP to vertical gratings while oblique orientations (i.e., 50°, 70°, 110°, or 130°) were attended from theERPs to vertical targets. Bottom: When attention was paid to oblique gratings, attention effects were stillsignificant but somewhat smaller, and had a slightly different scalp distribution. (See color plates.) Reprintedfrom Cognitive Brain Research, 13; A. M. Proverbio, P. Esposito, and A. Zani; Early involvement of temporal area inattentional selection of grating orientation: An ERP study, pp. 139–151. Copyright (2002) with permission ofElsevier Science.

difference in both performance and in theevoked responses between the attend-oblique and attend-vertical tasks (the orien-tations were 50°, 70°, 90°, 100°, and 130°).Indeed, the reaction times were much morerapid and the P300 and the SN much largerfor the vertical targets. Taking account ofthe fact that all the stimuli were isolumi-nant, and of equal spatial frequency anddimension, this result supports the hypo-thesis of the so-called oblique effect phe-nomenon [for a description of this effect,see Regan (1989), as well as Campbell et al.(1996), and Arakawa et al. (2000)], for whichthe sensory response was found to behigher at configurations set in the ortho-gonal orientations rather than in theoblique ones. This asymmetry is believed tohave a neural basis in the disproportionaterepresentation of the two orthogonal orien-tations at the level of the visual cortex. Inthis regard the hypothesis was put forwardthat this asymmetry is due to the mainlyperpendicular arrangement of the environ-mental stimuli provided by the Earth’slandscape, which is subjected to the force ofgravity (trees, horizon, sea, houses, humanbeings in erect position, etc.) The recentfMRI study by Furmanski and Engel (2000)very likely revealed the neurofunctionalbases of this effect by demonstrating thatthe activation of neurons in V1 is greater to horizontal and vertical than to obliqueorientations in humans.

SPACE-BASED AND FREQUENCY-BASED

ATTENTIONAL SELECTION

Visual attention, both voluntary andautomatic, can be directed either to singleobjects in visual space, regardless of theirlocation, and to comparatively circum-scribed regions in visual space. Oneexample of the first mechanism is the casein which we are looking for a specificobject (e.g., a pencil on our desk) on thebasis of its visual features (thin, long, made

of wood, brown, with a black point); anexample of the second mechanism is thecase in which we are monitoring (eitherovertly or covertly) an open door in orderto see when a friend of ours comes throughit. The two mechanisms of object- andspace-based selective attention normallywork in close interaction (in order to recog-nize my friend I must make a careful selec-tion based on the features of his face andappearance). Yet, because they are partlyfunctionally segregated, and probablybased on the activation of nonoverlappingvisual neural areas, to some degree it ispossible to investigate the latter areasseparately in order to unveil their neuro-functional activation.

We carried out a series of experiments(Zani et al., 1999) in which, by adopting thesame set of visual stimuli in differenttasks—lateralized isoluminant gratings oftwo spatial frequencies, namely, 1 and7 cpd—we were able to observe the twofunctionally active attentional mechanismsby inducing a different type of attentionalset in the subjects. This was obtained by suitably modifying the experimentalinstructions—that is, the task that theobservers had to perform at different times.In different sessions the same subjects wereinstructed to pay attention and respond todifferent stimulus properties while brain-evoked responses were recorded with a 32-channel montage. In this way it was pos-sible to separate the constant effect in theERPs due to the physical features of thestimuli from that due to the top-downattentional strategies used by our volun-teers. In order to compare location selectionwith frequency selection mechanisms wedevised two different paradigms. In thefirst, the subjects were requested to payselective attention to a spatial frequency,and to respond to the target frequencywhatever its spatial location, whereas in thesecond they had to attend and respond toall the gratings solely on the basis of theirspatial location, thus ignoring the fre-quency. The results revealed a very early



attentional selection effect in both cases,although with completely different mor-phology and neurofunctional activation. Inparticular, the selection of the location,which in agreement with the previousstudies of Mangun and colleagues (Heinzeet al., 1994; Mangun et al., 1997, 2001), modu-lated mainly the extrastriate area beginningat about 80 msec poststimulus (for the topo-graphic distribution of our attentional effectssee the color topographic maps in Fig. 10),was characterized by a prominent P1 with apeak latency of about 135–140 msec. Thiscomponent was followed by a compara-tively small N1 and an exceptionally earlyP300. Conversely, the selection of the spatialfrequency determined the presence of a very early P/N80 followed by a consider-able negative deflection (N1/N2 complexstrongly modulated by selection negativity)and by a somewhat delayed P300 (see Fig.11). At the same time, the response times inthe spatial selection task were about 100

msec faster than those obtained in the fre-quency selection task. These data confirmthe partial functional independence of the two visual feature selection systems,both based on a very early sensory filter(early selection) although dependent on twoanatomically and functionally separateneural streams. Observation of the topo-graphic distribution of the attentional effectof ERP differences—obtained by subtractingthe response to the nontargets from that tothe same stimuli when they were targets—for the selection of the spatial frequencyidentified a filter that is strongly linked tothe visual processing first of area 17 (at thelevel of P80) and then of areas 18 and 19 (atthe level of selection negativity; see the maps in Fig. 12), whereas the selection of the spatial position indicated it was basedon the functionality of the dorsal visualpathway: “extrastriate visual area for P1 (seeagain Fig. 10), motor cortex for N2, and pari-etooccipital area for the very fast P300.

SPACE-BASED AND FREQUENCY-BASED ATTENTIONAL SELECTION 293


FIGURE 10 Space selection. Scalp current density maps reflecting attention effects for spatial location selec-tion between 80 and 140 msec poststimulus. Maps were computed on the difference wave obtained by subtract-ing brain response to nontargets from that to targets in the P1 latency range. Stimuli were 7-cpd black and whitegratings presented in the right visual field. Note the lateral occipital-parietal flowing of currents, suggesting theactivation of the Where system. (See color plates.)

FEATURES CONJUNCTION ANDOBJECT PERCEPTION

Of the most interesting studies by theHarter’s group, mention must be made ofthat of 1982 (Previc and Harter, 1982) on themechanisms of combined selection ofspatial frequency and orientation. In thispioneering study the participants had toattend and respond to target gratingshaving each time a given frequency and agiven orientation. Spatial frequency wasfoveally projected and could be high or low

and the orientation vertical or horizontal.The results indicated a strong attentionaleffect at the level of negative deflection N2 and positive P300, the amplitudes ofwhich are shown comparatively to thepercentage of emitted responses in Fig. 13.What is interesting in this study is theattempt to gain objective measures of theallocation of attentional resources and theactivation of neural mechanisms under-lying sensory features selection by meas-uring the amplitude of the bioelectricresponse in the visual areas at the variousstages of processing. The important find-ings to note in the data presented in Fig. 13are the monotonic relations between thevolunteers’ conjoined and separate atten-tional processing of stimulus attributes andthe amplitude of ERP components. At theN2 level—i.e., the timing of the first sign of sensory gating—the orientation had alower value than spatial frequency, becausethe brain’s response to stimuli that sharedonly the orientation but not the frequencywith the target was significantly lowercompared to when stimuli shared thespatial frequency (SF) but not the orienta-tion (O). No attentional response is presentfor stimuli irrelevant in both features,whereas a large response is observable fortargets. It is very important to point outthat the amplitude of N2 for target stimuli(SF + O) was greater than the sum of thetwo responses to the single feature (SF orO), thus demonstrating a nonadditivenature and a close interaction between themechanisms of selection of frequency andorientation. This trend was apparent for theamplitude of P300, also displayed in thecentral histograms in Fig. 13. At this stageof processing the response to relevant ori-entation in the absence of the relevant fre-quency is lower; in addition, the responseto the individual features decreases: thebrain is getting ready to make a decisionconcerning the presence of the target, andthe conjoining of the two attributes takes onspecial relevance as an object. At theresponse time level (displayed in the lower



FIGURE 11 Grand-average ERPS recorded at theleft inion site (IN3) in response to gratings of 7 cpdand recorded in the attend-space and attend-frequency tasks. The frequency distribution ofreaction times in the two attention tasks is shownbelow the ERPs. RT, Response time.

block diagram, Fig. 13), this trend is evenmore apparent: the percentage of responsesemitted to stimuli having the same fre-quency as the attended frequency (i.e., SF)is slightly greater than 10%. This is mostrelevant when thinking that the amplitudeof the ERP components was as high as 50%in the case of N2 and greater than 35% inthe case of P300. All in all, these findingsalso revealed a progressive narrowing ofthe attention filter band as neural pro-cessing progresses from input to motorresponse.

In further work on feature-conjunctionselection carried out by our research groupwe investigated the mechanisms of thecombined selection of frequency andspatial location using isoluminant gratingsof variable spatial frequency (0.75, 1.5, 3and 6 cpd) laterally presented in theinferior and superior quadrants of the leftand right visual hemifields (Zani andProverbio, 1997b). In this study we used amore advanced recording and analysis

technique than in the previous study andrecorded from four occipital electrode loca-tions, namely O1, O2, and OL, OR homolo-gous sites. The results indicated that amuch earlier selection of the frequencyoccurred than previously believed—that is,within 60–130 msec poststimulus. Morespecifically, the two features influenced theamplitude of the evoked sensory responseP1, even though the effect of frequencyrelevance was felt only when the stimulifell in the attended quadrant. The latterfindings suggest that the selection mecha-nisms of the two features operate in paral-lel right from the earliest stages of analysis,and that object or feature selection, ratherthan being preceded by a space selection,is centered “on line” on precise coordi-nates of the attended space. This view issupported in the literature—as discussedlater in this section—but can be consideredstraightforwardly sound per se, from anecological perceptual viewpoint related tothe pervasive subjective phenomena we

FEATURES CONJUNCTION AND OBJECT PERCEPTION 295


FIGURE 12 Selection negativity. Realistic three-dimensional isocolor voltage maps for frequency (left) andlocation (right) selection. The maps were computed in the selection negativity latency range (i.e., 180–280 msec;peak latency, 240 msec) on difference waves obtained by subtracting brain response to nontarget gratings fromthat to targets in the two attend-object and attend-location tasks.

experience with objects in everyday life.Indeed, looking aimlessly and absentmind-edly at the outer world, one is rather often

aware of having seen something—i.e., acar, a neon sign, or whatever—automati-cally surfacing at the conscious level, butnot being able to localize it except bymeans of a visual rescanning of the sur-rounding space, looking for its location.

In a later study (Zani and Proverbio,1997e), carried out using exactly the sameparadigm and a much higher density elec-trode montage (i.e., 32 electrodes). ERP dataprovided evidence that the effect of P1 mod-ulation by the relevant spatial frequencypresented in the attended spatial quadrantmodulated the activity of the striate visualcortex, as is reported in the scalp currentdensity maps displayed in Fig. 14. Becausethis does not occur when stimuli are selectedsolely on the basis of spatial location, thisevidence indicates a strong reciprocalinteraction among the various visualmechanisms of sensory analysis.

The evidence presented so far showsthat object-based selective attention affectsvisual processing at early latency stages, asindexed by modulation of early sensory-evoked responses. To further investigatethe timing of the activation of this earlysensory gating and obtain further supportfor our findings, we have repeated thecombined space–object attentional para-digm (Zani and Proverbio, 2002), usingseveral significant methodological vari-ants. This time, instead of measuring theamplitude of sensory-evoked responses atpeak latency, we measured the meanamplitude of early potentials in four differ-ent time windows (i.e., 60–80, 80–100,100–120, and 120–140 msec poststimulus)over both mesial and lateral occipital sites,and compared the effect of frequency rele-vance (F+), location relevance (L+), and theconjunction of both features (F+L+) onevoked potentials across electrodes, hemi-spheres, and different temporal windows.Furthermore, a larger sample of volunteers(N = 21) was tested in order to obtain thebest possible signal-to-noise ratio in ERPwave forms. Results showed that stimulirelevant in frequency and/or location eli-



FIGURE 13 Comparative representation of theamplitudes of N2 and P300 components and percent-age of emitted responses to spatial frequency gratingsas a function of number of attributes shared with target(both spatial frequency and orientation, SF+O+; onlyspatial frequency, SF+; only orientation, O+; neitherattribute, N). Reprinted from Electroencephalography and Clinical Neurophysiology, 45; M. R. Harter and F. H. Previc; Size-specific information channels andselective attention: Visual evoked potentials and behav-ioural measures, pp. 628–640. Copyright (1978) withpermission from Elsevier Science and author.

cited larger evoked responses compared toirrelevant stimuli as early as 60–80 msecpoststimulus (P/N80), a latency range,reflected at scalp surface by the so-called C1component, that is acknowledged to corre-spond to sensory activity in the striatecortex.

Fig. 15 displays grand-average ERPselicited by low-frequency (0.75 cpd) and high-frequency (6 cpd) luminance-modulated gratings presented for 80 msecin the left hemifield, as a function of atten-tion condition. It is possible to see theattentional fine gradient in the form oflarger responses for gratings relevant inlocation but not in frequency (L+F–), withrespect to those relevant in frequency butnot in location (L–F+). Of great interest isthe finding of larger responses in the firsttime window (i.e., 60–80 msec) to stimulirelevant in both features compared tostimuli relevant only in location, suggest-ing that frequency relevance alone affectsvisual processing at the earliest processingstage (see Fig. 15).

So far we have seen how it is possible inthe laboratory to study the way in whichthe visual system processes and attention-ally selects one or more visual features(spatial frequency, depth, stereopsis, color,orientation, texture, luminance) of thesurrounding environment, separately. Ofcourse, in actual fact, we perceive a unitaryenvironment and not a separate series ofobjects or individual attributes [see thediscussion on this point in Previc (1990)].This perception of the unitary naturederives from the interaction between theWhere and What systems, which, althoughpartially anatomically and functionallydistinct, operate in parallel and in veryclose coordination. Clear evidence of thisinterdependence comes from neuroimag-ing and neuropsychological literature. Forexample, the clinical neuropsychologicalstudy by Friedman-Hill et al. (1995) indi-cated that patients with focused bilaterallesions of the parietal cortex are unablecorrectly to combine color and shape ofstimuli presented in the two visual hemi-



FIGURE 14 Frequency selection. Scalp current density maps reflecting attention effects for spatial frequencyselection between 80 and 140 msec poststimulus. Maps were computed on the difference wave obtained by sub-tracting brain response to nontargets from that to targets in the P1 latency range. Stimuli were 7-cpd black and white gratings presented in the right visual field. Note the leftward occipital-temporal flowing of currents,suggesting the activation of the What system. (See color plates.)

fields. This suggests that the integrity ofthe Where system is absolutely essential forthe correct recognition of objects. On theother hand, a large body of neuropsycho-logical, neurophysiological, and behavioraldata (see Olson and Gettner, 1996) indi-cates that the spatial and nonspatial

intentional mechanisms are probably notseparated at all because of the existence ofspecific selection mechanisms centered onthe object at a given spatial location (i.e.,object-centered space receptive fields). Inour view, our electrophysiological data onconjoined selection of frequency and space(Zani and Proverbio, 1997b,e) reportedabove are in line with this view derivedfrom other research lines in cognitiveneuroscience.

The extensive available neurophysio-logical and psychophysical literature hasshown that the perceptual construction ofobjects takes place through the simultane-ous analysis of a complex series of cues(namely luminance, color, texture, motion,and binocular disparity) at very early pro-cessing stages (Mountcastle, 1998; Regan,2000). We shall now see how selectiveattention is able to modulate (that is, tooptimize or ignore) the perception ofobjects thus constructed through the selec-tion of illusory subjective figures. In astudy carried out on healthy young con-trols we investigated the brain mechanismsunderlying the perception of illusory con-tours and determined the time course ofsensory and perceptual processing in theboundary completion process, analyzingthe timing of ERP responses (Proverbioand Zani, 2002). The so-called illusory con-tours are known to be perceived edges thatexist in the absence of local borders andthat determine the perception of subjectivefigures, such as the universally knownKanizsa square or triangle (Kanizsa, 1976).They are based on the peculiar boundaryalignment of inducers (simple geometricshapes in striking contrast with the homo-geneous background luminance) that elicitthe response of edge detectors in area 18(Hirsch, 1995; Larsson et al., 1999) andperhaps 17 (Grosof et al., 1993; Lee andNguyen, 2001; Olson, 2001), giving rise tothe subjective perception of an object withpartially illusory boundaries. The aim ofour research was to investigate the mecha-nisms underlying object-based selective



FIGURE 15 Grand-average ERPs to black andwhite luminance-modulated gratings of low (0.75cpd) and high (6 cpd) spatial frequency as a functionof attention condition in a conjoined selection task.ERPs were recorded from left and right lateral (i.e.,OL and OR, respectively) and mesial (i.e., O1 and O2,respectively) occipital sites to stimuli presented in thelower and upper quadrants of the left visual field.

attention by presenting target and non-target stimuli possessing the same amountof physical energy (luminance), differingonly in the presence or absence of the sub-jective figure. Indeed, they consisted of thesame elements presented in different orien-tations: the outward rotation of externalinducers in nontargets cancelled the per-ception of illusory boundaries, and there-fore of the illusory object (see Fig. 16). Theobject to be detected and attentionallyselected thus did not actually exist forretinal photoreceptors, but was really seenby the visual cortex with the same clarityas a real object. We wondered which mech-anisms underlay the attentional selectionof objects mainly perceived on the basis ofluminance contrast and boundary align-ment, and, above all, at which stage oflatency attention was able to affect visualprocessing. ERP results showed that theoccurrence of illusory contours was associ-ated with a strong bilateral activation oflateral occipital areas at about 145 msecpoststimulus as indexed by N1 component,followed by a left-sided activation of thesame region at about 250 msec of latency.Overall, our data support the view that theintegration of contours arises at very earlystages of visual processing, as proposed byHess and Field (1999). Moreover, our dataalso indicate that object-based attention isable to affect illusory contour binding by

somewhat enhancing ERPs to the illusorypercepts at lateral occipital sites.

Quite interesting, in this regard, is theGrossberg and Raizada (2000) neuralmodel—based on neurophysiologicaldata—indicating how visual objects aregrouped, or bound together, by being con-nected by regions of layer 2/3 of the visualcortex. The model also proposes detailedlaminar circuits to account for how top-down attentional task demands may affectthe activity of V1 cells (for example, in atten-tional grouping) by means of a mechanismcalled folded feedback, based both on facilita-tory ON-center attentional enhancementsand inhibitory suppressive OFF-surroundeffects.

COLOR

We have seen that object features suchas spatial frequency, orientation, or shapeare processed in the ventral stream of theoccipitotemporal visual areas (the so-called What system), unlike the movementand spatial position represented instead inthe occipitoparietal system (the Wheresystem). A separate section should bededicated to color selection as far asaspects related to the selection of objectfeatures are concerned. This is due to thefact that several studies have dealt withthe investigation of neural mechanismsunderlying the selection of this featureboth per se and in conjunction with otherfeatures, mostly finding a special status forthis nonspatial feature with respect to theothers.

Neuroimaging studies have identifiedseveral human brain areas, notably thelingual and fusiform gyrus of the extra-striate cortex, which selectively respond tocolor (e.g., Corbetta et al., 1991; Corbetta,1999). It is interesting to note that neuro-logical lesions in the same areas are associ-ated with the inability to perceive the colorof objects—a deficit reported as achroma-topsia—which may occur also in the



FIGURE 16 (A) Stimulus generated by thickinducers (packmen) producing the classical Kanizsasquare illusion. (B) Same stimulus but with outwardrotation of the packmen and no illusory perception.

absence of a deficit in object recognition(i.e., visual agnosia).

Many electrophysiological studies havealso been carried out in an attempt todetermine how conjunctions of color andother sensory features, such as shape ororientation, are coded and selected, byrecording ERPs during tasks involvingattentional selection of one or more attrib-utes of the objects. The majority of studiesinvolved the use of simple stimuli such asgeometric shapes, spatial frequency grat-ings, checkerboards, alphanumeric charac-ters, colored Mondrians, or faces. In thesestudies, the relationship between shapeand color was completely arbitrary (take,for instance, a random sequence of red orblue squares or circles), because, of course,in association with such forms, no specificrepresentation of the color of the latter isstored in the brain.

In these ERP paradigms it has been fre-quently reported that the attentional selec-tion of color takes place before that of theother nonspatial attributes such as size,shape, or orientation (e.g., Eimer, 1996b;Karayanidis and Michie, 1997). Moreover,the selection of those features oftendepended hierarchically on that of color(Wijers et al., 1989). In particular, the selec-tion negativity of ERPs reflecting the atten-tional selection at the level of N1–N2 wasabsent when the color was irrelevant evenwhen the other attribute was relevant tothe task (Smid and Heinze, 1997; Rotte et al., 1996).

Yet, the data in the literature indicatethat this neural logistic does not hold forthe selection of color in conjunction withspace location. Indeed, in an electrophysio-logical study by Anllo-Vento and Hillyard(1996), a comparison was made betweenthe color- and movement-selection systemsby recording ERPs to little squares dis-played on a computer monitor, while indi-viduals were involved in spatially directedattention tasks. ERP measurements andtopographic mapping showed that wher-ever color- and movement-selection were

topographically dissociated in the twoventral and dorsal streams at selectionnegativity (N2) level, the selection of thespatial location in any case temporallypreceded that of the two nonspatialfeatures by modulating the early P1 andN1 components. Nonetheless, the sameauthors (Anllo-Vento et al., 1998), using redand blue isoluminant checkerboards,found that, as well as space selection, colorselection modulated scalp-recorded poten-tials in the lateral occipital areas as early as100 msec poststimulus, thus further sup-porting the hypothesis advanced by Zaniand Proverbio (1993, 1995) of an earlyselection also for nonspatial attributes ofsensory inputs.

To further investigate the apparentpriority of color selection over that of theother sensory features, we adopted anexperimental paradigm different fromthose used previously (Proverbio et al.,2002b). Our aim was to determine whethercolor processing interacted with that ofshape by recording ERPs in response tofamiliar objects and common animalsrepresented in their canonical color (heredefined as “prototypical”), as well as invarious other colors.

Because the stimuli used in our testsconsisted of drawings of everyday lifeobjects and common animals, we managedto ensure that the attribute of the objects’color was in some cases strongly associatedwith their shape: the combination of thetwo features (e.g., the color yellow and theshape of a banana) was specific to andcharacteristic of that particular object. Thetask consisted in selectively attending andresponding to target images either on thebasis of their color, regardless of theirshape, or on the basis of their shaperegardless of their color. We investigatedthe timing and topographic distribution ofthe attentional selection of both features bycomparing the electrofunctional responsesto target stimuli canonical in both shapeand color (e.g., a yellow banana) withthose to noncanonical targets (e.g., a



yellow tire or swallow during color selec-tion; a pink or blue banana during shapeselection).

The results suggest that the selection ofthe two dimensions takes place in paralleland not separately, and that it modulatesthe amplitude of the occipital-temporalcomponents N1 (N166) and N2 (N278) andof the centroparietal P300. Consistent withprevious literature, both the RTs and theSN and P3 latencies were found to be morerapid in the color selection condition thanin that of shape, also given that the twoattentional components were much largerin the shape selection condition. This initialevidence suggests that shape selectionmight actually be more demanding thanselection of an object’s color. However, adetailed analysis of selection negativity (orN278 component) elicited by the differentstimuli as a function of the attributesshared with the target indicated that colorselection is affected by stimulus shape, inthat greater responses were recorded tostimuli of the relevant color having aprototypical shape (e.g., a yellow bananaor chick compared to a yellow tire orswallow), whereas shape selection does notdepend on the color of the image presented(e.g., a piglet, whatever its color: pink,yellow, or blue—see Fig. 17).

Electrocortical topographic mappingshowed that selection of the color dimen-sion of familiar images strongly activatedthe posterior temporal and extrastriatecortex of the left hemisphere. These areaswere found to be highly sensitive to theprototypical effect of shape with respect tocolor, and consequently to be able possiblyto indicate the probable locus of conjoinedprocessing of shape and color in thehuman visual cortex. It is interesting thatin our experimental paradigm color selec-tion is responsible for a strong activation ofareas not specifically related to color butrather to information concerning shape.Indeed, this fact proves extremely impor-tant when it is taken into account that inexperimental selective attentional studies

what is valid is the general assumptionthat the selection of one feature of the stim-ulus modulates the bioelectrical activity ofthe same brain regions that are apparentlyactive during the perception of this feature(as suggested also by several different ERPand PET studies). In this regard it isextremely interesting that a PET study(Chao and Martin, 1999) reported, forexample, that the cortical regions involvedin color perception are not the same asthose that convey information related tothe color of objects (object color know-ledge), which has many points in commonwith our electrophysiological resultsdescribed above.

This double dissociation between colorperception and object color knowledge hasalso been robustly supported by some veryrecent clinical cases reported in neuro-psychological literature. These cases showthat semantic information about objectcolors is apparently actually grounded in

COLOR 301


FIGURE 17 Grand-average ERPs recorded at theleft inion site in response to pictures of familiarobjects and animals as a function of the four target(prototypical, solid line; nonprototypical, dashed line)and nontarget (prototypical, dotted lines; non-prototypical, dot-dashed line) conditions in the twoattend-color and attend-shape tasks. ERPs to shapeprototypically associated to target color (such as agreen artichoke in the attend-green condition) eliciteda larger N2 component, as compared to nonprototypi-cal targets (such as a green swallow) over the left hemi-sphere.

neural systems distinct from thoseinvolved in form and function knowledge.Indeed, although previous neurologicalliterature was scarce and anecdotal, Miceliet al. (2001) reported a straightforward caseof three patients with lesions in the mesialtemporal region of the left hemisphere whowere unable to decide whether a colormatched a given familiar object (“Can alion be red?”), despite their essentiallynormal ability to name colors or to definefunctional properties of common objects(“Is a pencil made of glass”?). The authorsconcluded that mesial temporal structuresof the left hemisphere are specificallyinvolved in representing or accessing colorknowledge. These findings strongly agreewith our ERP topographical mapping data,notwithstanding the inherent limitations ofscalp-recorded ERPs for source localizationdue to their scarce spatial resolution.

CONCLUSIONS

Several conclusions may be drawn on thebasis of the ERP findings reviewed above onnonspatial feature selection mechanisms.One is that attentional selection of visualinputs is carried out by means of the parallelactivation of a distributed neural network ofthe brain, made up of the anterior and poste-rior attentional systems, whose centers andpathways play different but synergic roles inthe selection. As far as the posterior atten-tional system is concerned, the data indicatethat, as for attention to space, attention toobject features can modulate neural process-ing of visual areas at an early sensory level,very likely involving the primary striatecortex, with modalities that are still notclearly known and which must be furtherinvestigated. Furthermore, the data suggestthat the dorsal and ventral streams of thevisual system, although partially anatomi-cally segregated, may be activated in paralleland in an independent or conjoined mode,depending on the attentional demands andtask requirements. Last, but not least, there

is evidence that the perception of multi-dimensional objects in the visual field isaccomplished through a unitary activebinding process of spatial and nonspatialfeatures. This mechanism is believed not tobe based on hierarchically organized inde-pendent processes, but rather to reflect thehorizontal processing of visual cells thattakes place at very early stages of inputanalysis. This view seems to be consistentwith models assuming that different stimu-lus attributes may be initially conjoined in asingle representation, while at the same timebeing separately analyzed in parallel,dimension by dimension (cf. Treisman,1999).

ReferencesAine, C. J., Supek, S., and George, J. S. (1995).

Temporal dynamics of visual-evoked neuro-magnetic sources: Effects of stimulus parametersand selective attention. Int. J. Neurosci. 80, 79–104.

Anllo-Vento, L., and Hillyard, S. A. (1996). Selectiveattention to color and the direction of movingstimuli: Electrophysiological correlates of hier-archical feature selection. Percept. Psychophys. 58,191–206.

Anllo-Vento, L., Luck, S., and Hillyard, S. A. (1998).Spatio-temporal dynamics of attention to color:Evidence from human electrophysiology. Hum.Brain Mapping 6, 216–238.

Arakawa, K., Tobimatsu, S., Kurita Tashima, S.,Nakayama, M., Kira, J. I., and Kato, M. (2000).Effects of stimulus orientation on spatial frequencyfunction of the visual evoked potential. Exp. BrainRes. 131, 121–125.

Aston-Jones, G. S., Desimone, R., Driver, J., Luck, S. J.,and Posner, M. I. (1999). Attention. In “Funda-mental Neuroscience” (M. J. Zigmond, F. E. Bloom,S. C. Landis, J. L. Roberts, and L. R. Squire, eds.),pp.1385-1409. Academic Press, San Diego.

Barcelo, F., Suwazono, S., and Knight, R. T. (2000).Prefrontal modulation of visual processing inhumans. Nat. Neurosci. 3, 399–403.

Cabeza, R., and Nyberg, L. (2000). Imaging cognitionII: An empirical review of 275 PET and fMRIstudies. J. Cogn. NeuroSci. 12, 1–47.

Campbell, F. W., and Maffei, L. (1970). Electro-physiological evidence for the existence of orien-tation and size detectors in the human visualsystem. J. Physiol. 207, 635–652.

Campbell, F. W., Kulikowski, J. J., and Levinson, J.(1966). The effect of orientation on the visual reso-lution of gratings. J. Physiol. 187, 427–36



Chao, L., and Martin, A. (1999). Cortical regionsassociated with perceiving, naming, and knowingabout colors. J. Cogn. NeuroSci. 11, 25–35.

Clark, V. P., and Hillyard, S. A. (1996). Spatial selec-tive attention affects early extrastriate but notstriate components of the visual evoked potentials.J. Cogn. Neurosci. 8, 387–402.

Coles, M. G. (1989). Modern mind-brain reading:Psychophysiology, physiology, and cognition.Psychophysiology 26(3), 251–269.

Corbetta, M. (1999). Functional anatomy of visualattention in the human brain: Studies withpositron emission tomography. In “The Attentivebrain” (R. Parasuraman, ed.), pp. 95–122. MITPress, Cambridge, Massachusetts.

Corbetta, M., and Shulman, G. L. (1999). Human corti-cal mechanisms of visual attention during orient-ing. In “Attention, Space and Action. Studies inCognitive Neuroscience” (G. W. Humphreys, J.Duncan, and A. Treisman, eds.), pp. 183–198.Oxford University Press, Oxford.

Corbetta, M., Miezin, F. M., Dobmeyer, S., Shulman,G., and Petersen, S. E. (1991). Selective anddivided attention during visual discriminations ofshape color and speed: functional anatomy bypositron emission tomography. J. Neurosci. 11,2383–2402.

Desimone, R., and Duncan, J. (1995). Neural mecha-nisms of selective visual attention. Annu. Rev.Neurosci. 18, 193–222.

Donchin, E. (1981). Surprise! … Surprise? Psycho-physiology 9, 493–513.

Dupont, P., Vogels, R., Vandenberghe, R., Rosier, A.,Cornette, L., Bormans, G., Mortelmans, L., andOrban, G. A. (1998). Regions in the human brainactivated by simultaneous orientation discrimina-tion: A study with positron emission tomography.Eur. J. Neurosci 10, 3689–3699.

Eimer, M. (1996a). The N2pc component as an indica-tor of attentional selectivity. Electroencephalogr.Clin. Neurophysiol. 99, 225–34.

Eimer, M. (1996b). An event-related potential (ERP)study of transient and sustained visual atten-tion to color and form. Biol. Psychol. 44, 143–160.

Fabiani, M., Karis, D., and Donchin E. (1986). P300and recall in an incidental memory paradigm.Psychophysiology 23, 298–308.

Fink, G. R., Dolan, R. G., Halligan, P. W., and Marshall,J. C. (1997). Space-based and object-based visualattention: Shared and specific neural domains.Brain 120, 2013–2028.

Friedman-Hill, S. R., Robertson, L., and Treisman, A.(1995). Parietal contributions to visual featurebinding: Evidence from a patient with bilaterallesions. Science 269, 853–855.

Furmanski, C. S., and Engel, S. A. (2000). An obliqueeffect in human primary visual cortex. Nat.Neurosci. 3, 535–536.

Gehring, W. J., and Knight, R. T. (2000). Prefrontal-cingulate interactions in action monitoring. Nat.Neurosci. 3, 421–423.

Georgopoulos, A. P., Whang, K., Georgopoulos, M.A., Tagaris, G. A., Amirikian, B., Richter, W., Kim,S. G., and Ugurbil, K. (2001). Functional magneticresonance imaging of visual object constructionand shape discrimination: Relations among task,hemispheric lateralization, and gender. J. Cogn.Neurosci. 13, 72–89.

Gevins, A., and Cutillo, B. (1993). Spatiotemporaldynamics of component processes in humanworking memory. Electroencephalogr. Clin.Neurophysiol. 87, 128–143.

Goldman-Rakic, P. (1987). Circuitry of primate pre-frontal cortex and regulation of behavior by repre-sentational memory. In “Handbook of Physiology.The Nervous System—Higher Functions of theBrain” (F. Blum, ed.), Vol. 5, Part 1, pp. 373–417.American Physiology Association, Bethesda,Maryland.

Gomer, F. E., Spicuzza, R. J., and O’Donnell, R. D.(1976). Evoked potentials correlates of visual itemrecognition during memory-scanning tasks.Physiol. Psychology 4, 61–65.

Grosof, D. H., Shapley, R. M., and Hawken, M. J.(1993). Macaque V1 neurones can signal “illusory”contours. Nature 365, 550–552.

Grossberg, S., and Raizada, R. D. S. (2000). Contrast-sensitive perceptual grouping and object-basedattention in the laminar circuits of primary visualcortex. Vis. Res. 40, 10–12.

Harter, M. R., and Aine, C. J. (1984). Brain mecha-nisms of visual selective attention. In “Varieties ofAttention” (R. Parasuraman, and R. Davies, eds.),pp. 293–321. Academic Press, New York.

Harter, M. R., and Guido, W. (1980). Attention topattern orientation: Negative cortical potentials,reaction time, and the selection process. Electro-encephalogr. Clin. Neurophysiol. 49, 461–475.

Harter, M. R., and Previc, F. H. (1978). Size-specificinformation channels and selective attention:Visual evoked potentials and behavioural mea-sures. Electroencephalogr. Clin. Neurophysiol. 45,628–640.

Haxby, J. V., Courtney, S. M., and Clark, V. P. (1999).Functional magnetic resonance imaging and thestudy of attention. In “The Attentive Brain”(R. Parasuraman, ed.), pp. 123–142. MIT Press,Cambridge, Massachusetts.

Heinze, H. J., Mangun, G. R., Burchert, W., Hinrichs,H., Scholze, M., Münte, T. F., Gös, A., Johanne,S. S., Scherg, M., Hundeshagen, H., Gazzaniga,M. S., and Hillyard, S. A. (1994). Combined spatialand temporal imaging of spatial selective atten-tion in humans. Nature 372, 543–546.

Hermann, C. S., and Knight, R. T. (2001). Mechanismsof human attention: Event-related potentials andoscillations. Neurosci. Biobehav. Rev. 25, 465–476.

REFERENCES 303


Hess, R., and Field, D. (1999). Integration of contours:New insights. Trends Cogn. Sci. 3, 480–486.

Hillyard, S. A., and Mangun, G. R (1986). The neuralbasis of visual selective attention: A commentaryon Harter and Aine. Biol. Psychol. 23, 265–279.

Hillyard, S. A., and Anlo-Vento, L. (1998). Event-relatedbrain potentials in the study of visual selectiveattention. Proc. Natl. Acad. Sci. U.S.A. 95, 781–787.

Hillyard, S. A., and Kutas, M. (1983). Electro-physiology of cognitive processing. Annu. Rev.Psychol. 34, 33–61.

Hillyard, S. A., Mangun, G. R, Woldorff, M. G., andLuck, S. J. (1995). Neural systems mediating selec-tive attention. In “The Cognitive Neurosciences”(M. S. Gazzaniga, ed.), pp. 665–681. MIT Press,Cambridge, Massachusetts.

Hillyard, S. A., Teder-Sälejärvi, W. A., and Münte, T. F.(1998). Temporal dynamics of early perceptualprocessing. Curr. Opin. Neurobiol. 8, 202–210.

Hillyard, S. A., Vogel, E. K., and Luck, S. J. (1999).Sensory gain control (amplification) as a mechanismof selective attention: Electrophysiological andneuroimaging evidence. In “Attention, Space andAction. Studies in Cognitive Neuroscience” (G. W. Humphreys, J. Duncan, and A. Treisman,eds.), pp. 31–53. Oxford University Press, Oxford.

Hirsch, J., DeLaPaz, R., Relkin, R. N., Victor, J., Kim,K., Li, T., Borden, P., Rubin, N., and Shapley, R.(1995). Illusory contours activate specific regionsin human visual cortex: Evidence from functionalmagnetic resonance imaging. Proc. Natl. Acad. Sci.U.S.A. 92, 6469–6473.

Ito, M., and Gilbert, C. D. (1999). Attention modulatescontextual influences in the primary visual cortexof alert monkey. Neuron 22, 593–604.

Jeffreys, D. A., and Axford, J. G. (1972). Source loca-tions of pattern specific components of humanvisual evoked potentials. I. Components of striatecortical origin. Exp. Brain Res. 16, 1–21.

Kanizsa, G. (1976). Subjective contours. Sci. Am. 234,48–52.

Karayanidis, F., and Michie, P. T. (1997). Evidence ofvisual processing negativity with attention toorientation and color in central space. Electro-encephalogr. Clin. Neurophysiol. 103, 282–297.

Kawashima, R., Satoh, K., Goto, R., Inoue, K., et al.(1998). The role of the left inferior temporal cortexfor visual pattern discrimination-––a PET study.Neuroreport 11, 1581–1586.

Kenemans, J. L., Kok, A., and Smulders, F. T. Y. (1994).Event-related potentials to conjunctions of spatialfrequency and orientation as a function of stimulus parameters and response requirements.Electroencephalogr. Clin. Neurophysiol. 88, 51–63.

Knight, R. T. (1991). Evoked potential studies of atten-tion capacity in human frontal lobe lesions. In“Frontal Lobe Function and Dysfunction” (H.Levin, H. Eisenberg, and F. Benton, eds.), pp.139–153. Oxford Univ. Press.

Knight, R. T., and Grabowecky, M. (1995). Escapefrom linear time: Prefrontal cortex and consciousexperience. In “The Cognitive Neurosciences” (M.S. Gazzaniga, ed.), pp. 1357–1371. MIT Press,Cambridge, Massachussets.

Knight, R. T., Staines, W. R., Swick, D., and Chao, L. L.(1999). Prefrontal cortex regulates inhibition andexcitation in distributed neural networks. ActaPsychol. 101, 159–178.

LaBerge, D. (1995). “Attentional Processing. TheBrain’s Art of Mindfulness.” Harvard UniversityPress, Cambridge, Massachusetts.

La Berge, D., and Buchsbaum, M. S. (1990). Positronemission tomographic measurements of pulvinaractivity during an attention task. J. Neurosci. 10,613–619.

Lamme, V. A. F., and Spekreijse, H. (2000).Modulations of primary visual cortex activity rep-resenting attentive and conscious scene percep-tion. Frontiers Biosci. 5, 232–243.

Larsson, J., Amuntus, K., Gulys, B., Malikovic, A.,Zilles, K., and Roland, P. E. (1999). Neuronal corre-lates of real and illusory contour perception: func-tional anatomy with PET. Eur. J. Neurosci. 11,4024–4036.

Lee, T. S., and Nguyen, M. (2001). Dynamics of subjec-tive contour formation in the early visual cortex.Proc. Natl. Acad. Sci. U.S.A. 98, 1907–1911.

Luck, S. J., Chelazzi, L., Hillyard, S., and Desimone,R. (1997). Neural mechanisms of spatial selectiveattention in areas V1, V2, and V4 of macaquevisual cortex. J. Neurophysiol. 77, 24–42.

Luck, S. J., Woodman, G., and Vogel, E. K. (2000).Event-related potential studies of attention. TrendsCogn. Sci. 4, 432–440.

Mangun, G. R., Hopfinger, J. B., Kussmaul, C.,Fletcher, E. M., and Heinze, H. J. (1997).Covariations in PET and ERP measures of spatialselective attention in human extrastriate visualcortex. Hum. Brain Mapping 5, 273–279.

Mangun, G. R., Hinrichs, H., Scholz, M., Mueller-Gaertner, H. W., Herzog, H., Krause B. J., TellmanL., Kemna, L., and Heinze H. J. (2001). Integratingelectrophysiology and neuroimaging of spatialselective attention to simple isolated visualstimuli. Vis. Res. 41, 1423–35.

Martin-Loeches, M., Hinojosa, J. A., and Rubia, F. J.(1999). Insights from event-related potentials intothe temporal and hierarchical organization of the ventral and dorsal streams of the visual systemin selective attention. Psychophysiology 36, 721–36.

Martínez, A., Anllo-Vento, L., Sereno, M. I., Frank, L.R., Buxton, R. B., Dubowitz, D. J., Wong, E. C.,Hinrichs, H., Heinze, H. J., and Hillyard, S. A.(1999). Involvement of striate and extrastriatevisual cortical areas in spatial attention. Nat.Neurosci. 2, 364–369.

Martínez, A., Di Russo, F., Annlo-Vento, L., andHillyard, S. A. (2001). Electrophysiological analysis



of cortical mechanisms of selective attention to highand low spatial frequencies. Clin. Neurophysiol. 112,1980–1998.

McCarthy, G. A., and Scabini, D. (1991). Attention modi-fies event-related potentials recorded from humanvisual cortex. IBRO World Congr. in Neurosci. 3,176.

Merigan, W. H., and Maunsell, J. H. R. (1993). Howparallel are the primate visual pathways? Annu.Rev. Neurosci 16, 396–402.

Metha, A. D., Ulbert, D., Lindsley, R. W., andSchroeder, C. E. (1997). Timing and distribution ofattention effects across areas in the macaque visualsystem. Soc. of Neurosci. Abstr. 23 (121.1), 299.

Miceli, G., Fouch, E., Capasso, R., Shelton, J. R.,Tomaiuolo, F., and Caramazza, A. (2001). The dis-sociation of color from form and function knowl-edge. Nat. Neurosci. 4, 662–667.

Miller, E. K., and Cohen, J. D. (2001). An integrativetheory of prefrontal cortex function. Annu. Rev.Neurosci. 24, 167–202.

Motter, B. C. (1993). Focal attention produces spatiallyselective processing in visual cortical areas V1, V2,and V4 in the presence of competing stimuli.J. Neurophysiol. 70, 909–919.

Mountcastle, V. B. (1998). “Perceptual Neuroscience:The Cerebral Cortex.” Harvard University Press,Cambridge, Massachusetts.

Näätänen R. (1992). “Attention and Brain Function.”Lawrence Erlbaum Assoc., Hillsdale, New Jersey.

Oakley, M. T., and Eason, R. G. (1990). Subcorticalgating in the human visual system during spatial selective attention. Int. J. Psychophysiol. 9,105–120.

Olson, C. R. (2001). Object-based vision and attentionin primates. Curr. Opin. Neurobiol. 11, 171–179.

Olson, C. R, and Gettner, S. N. (1996). Brain represen-tation of object-centered space. Curr. Opin.Neurobiol. 6, 165–170.

Posner, M. I., and Petersen, S. E. (1990). The attentionsystem of the human brain. Annu. Rev. Neurosci. 13,25–42.

Press, W. A., and Van Essen, D. C. (1997). Attentionalmodulation of neuronal responses in macaquearea V1. Soc. Neurosci. Abstr. 23 (405.3), 1026.

Previc, F. H. (1990). Functional specialization in thelower and upper visual fields in humans: Itsecological origins and neurophysiological implica-tions. Behav. Brain Sci. 13, 519–575.

Previc, F. H., and Harter, M. R (1982). Electro-physiological and behavioural indicants of selec-tive attention to multifeature gratings. Percept.Psychophys. 32, 465–472.

Proverbio, A. M., and Mangun, R. G. (1994). Electro-physiological and behavioral “costs” and“benefits” during sustained visual-spatial atten-tion. Int. J. Neurosci. 79, 221–233.

Proverbio, A. M., and Zani, A. (2002). Electrophysio-logical indexes of illusory contour perception inhumans. Neuropsychologia 40, 479–491.

Proverbio, A. M., Zani A., and Mangun R. G. (1993).Electrophysiological substrates of visual selectiveattention to spatial frequency. Bull. Psychonom. Soc.31, 368.

Proverbio, A. M., Zani, A., Gazzaniga, M. S., andMangun, R. G. (1994). ERP and RT signs of a right-ward bias for spatial orienting in a split-brainpatient. Neuroreport 5, 2457–2461.

Proverbio, A. M., Zani, A., and Avella, C. (1996).Differential activation of multiple current sourcesof foveal VEPs as a function of spatial frequency.Brain Topogr. 9, 59–68.

Proverbio, A. M., Minniti, A., and Zani, A. (1998).Electrophysiological evidence of a perceptualprecedence of global vs. local visual information.Cogn. Brain Res. 6, 321–334.

Proverbio, A. M., Esposito, P., and Zani, A. (2002a).Early involvement of temporal area in attentionalselection of grating orientation: an ERP study.Cogn. Brain Res. 13 (1), 139–151.

Proverbio, A. M., Burco, F., and Zani, A. (2002b). Bluepiglets? Electrofunctional evidence of how shapecan affect color selection, and not vice versa. Cogn.Brain Res. (in revision).

Regan, D. (1989). “Human Brain Electrophysiology.”Elsevier, Amsterdam.

Regan, D. (2000). “Multiple Cues for ObjectPerception: Early Visual Processing of SpatialForm Defined by Luminance, Color, Texture,Motion and Binocular Disparity.” Sinauer Assoc.Sunderland, Massachusetts.

Roberts, L. E., Rau, H., Lutzenberger, W., andBirbaumer, N. (1994). Mapping P300 waves ontoinhibition: Go/No-go discrimination. Electro-encephalogr. Clin. Neurophysiol. 92, 44–55.

Roelfsema, P. R., Lamme, V. A. F., and Spekreijse, H.(1997). Attentive response modulation in area V1 ofthe macaque during the detection of connectedobjects. Soc. Neurosci. Abstr. 23 (603.8), 1544.

Roelfsema, P. R., Lamme, V. A. F., and Spekreijse, H.(1998). Object-based attention in the primaryvisual cortex of the macaque monkey. Nature 395,376–381.

Rotte, M., Heinze, H. J., and Smid, H. G. O. M.,(1996). Selective attention to conjunctions of colorand shape of alphanumeric versus non-alphanu-meric stimuli: A comparative electrophysiologystudy. Biol. Psychol. 46, 199–221.

Rugg, M. D., Milner, A. D., Lines, C. R., and Phalp, R.(1987). Modulation of visual event-related poten-tials by spatial and non-spatial visual selectiveattention. Neuropsychologia 25, 85–96.

Schroeder, C. E. (1995). Defining the neural bases ofvisual selective attention: Conceptual and empiri-cal issues. Int. J. Neurosci. 80, 65–78.

Schroeder, C.E., Mehta, A.D., and Givre, S. J. (1998). Aspatiotemporal profile of visual system activationrevealed by current source density analysis in theawake macaque Cereb. Cortex 8, 575–592.

REFERENCES 305


Shimamura, A. P. (1995). Memory and frontal lobefunction. In “The Cognitive Neurosciences” ( M. S.Gazzaniga, ed.), pp. 803–813. MIT Press,Cambridge, Massachusetts.

Shulman, G. L., Corbetta, M., Fiez, J. A., Buckner, R. L.,Miezin, F. M., Raichle, M. E., and Petersen, S. E.(1997). Searching for activations that generalize overtasks. Hum. Brain Mapping 5, 317–322.

Skrandies, W. (1983). Information processing andevoked potentials: Topography of early and latecomponents. Adv. Biol. Psychiatr. 13, 1–12.

Smid, H. G. O. M, and Heinze, H. J. (1997). An electro-physiological study of the selection of the colorand shape of alphanumeric characters in responsechoice. Biol. Psychol. 44, 161–185.

Somers, D. C., Dale, A. M., Seiffert, A. E., and Tootel, R.B. (1999). Functional MRI reveals spatially specificattentional modulation in human primary visualcortex. Proc. Natl. Acad. Sci. U.S.A. 96, 1663–1668.

Squires, N. K., Squires, K. C., and Hillyard, S. A. (1975).Two varieties of long-latency positive waves evokedby unpredictable auditory stimuli in man.Electroencephalogr. Clin. Neurophysiol. 38(4), 387–401.

Stuss, D. T., Eskes, G. A., and Foster, J. K. (1994).Experimental neuropsychological studies offrontal lobe functions. In “Handbook of Neuro-psychology” (F. Boller, and J. Grafman, eds.), Vol.9, pp. 149–185. Elsevier, Amsterdam.

Sutton, S., Braren, M., Zubin, J., and John, E. R. (1965).Evoked-potential correlates of stimulus uncer-tainty. Science 26, 1187–1188.

Swick, D., and Knight, R. T. (1999). Cortical lesionsand attention. In “The Attentive Brain”(R. Parasuraman, ed.), pp. 143–162. The MIT Press,Cambridge, Massachusetts.

Tanaka, K. (2000). Mechanisms of visual object recog-nition studied in monkeys. Spatial Vis. 13, 147–63.

Treisman, A. (1999). Feature binding, attention andobject perception. In “Attention, Space and Action. Studies in Cognitive Neuroscience” (G. W. Humphreys, J. Duncan, and A. Treisman,eds.), pp. 91–111. Oxford University Press, Oxford.

Treue, S. (2001). Neural correlates of attention inprimate visual cortex. Trends Neurosci. 24, 295–300.

Ungerleider, L. G., and Mishkin, M. (1982). Two corticalvisual systems. In “Analysis of Visual Behaviour”(D. G. Ingle, M. A. Goodale, and R. J. W. Mansfield,eds.), pp. 549–586. MIT Press, Cambridge, Massa-chusettes.

Vanduffel, W., Tootell, R. B. H., and Orban, G. A.(1997). State dependent modulation of visual pro-cessing in the macaque. Exp. Brain Res. 117, S70.

Vanduffel, W., Tootell, R., and Orban, G. (2000).Attention-dependent suppression of metabolicactivity in the early stages of the macaque visualsystem. Cereb. Cortex 10, 109–126.

Vogels, R., and Orban, G. A. (1994) Activity of inferiortemporal neurons during orientation discrimina-

tion with successively presented gratings.J. Neurophysiol. 71, 1428–1451.

Wang, J., Zhou, T. , Qiu, M., Du, A. , Cai, K., Wang, Z.,Zhou, C., Meng, M., Zhuo, Y., Fan, S., and Chen, L.(1999). Relationship between ventral stream forobject vision and dorsal stream for spatial vision: An fMRI + ERP study. Hum. Brain Mapping 8,170–181.

Watanabe, T., Sasaki, Y., Miyauchi, S., Putz, B.,Fujimaki, N., Nielsen, M., Takino, R., andMiyakawa, S. (1998). Attention-regulated activityin human primary visual cortex. Rapid Commun.Am. Physiol. Soc. 22, 2218–2221.

Webster, M. J., and Ungerleider, L. G. (1999).Neuroanatomy of visual attention. In “TheAttentive Brain” (R. Parasuraman, ed.), pp. 19–34.MIT Press, Cambridge, Massachusetts.

Wijers, A. A., Mulder, G., Okita, T., and Mulder, L. J. M. (1989). An ERP study on memory search andselective attention to letter size and conjunctions ofletter size and color. Psychophysiology 26, 529–547.

Woldorff, M. G., Gallen, C. C., Hampson, S. A., Hill-yard, S. A., Pantev, C., Sobel, D., and Bloom, F. E.(1993). Modulation of early sensory processing inhuman auditory cortex during auditory selectiveattention. Proc. Natl. Acad. Sci. U.S.A. 90, 8722–8726.

Zani, A., and Proverbio, A. M. (1993). ERP signs ofearly influences of selective attention on spatialfrequency channels. Twenty-fifth Annual Meetingof the European Brain and Behaviour Society,p. 278. EBBS, Madrid.


Zani, A., and Proverbio, A. M. (1997a). ERP indicantsof frontal and occipital brain mechanisms mediat-ing spatially directed visual processing. Exp. BrainRes., S68, V/18, 117.

Zani, A., and Proverbio, A. M. (1997b). Attentionmodulation of short latency ERPs by selectiveattention to conjunction of spatial frequency andlocation. J. Psychophysiol. 11, 21–32.

Zani, A., and Proverbio, A. M. (1997c). Attention mod-ulation of C1 and P1 components of visual evokedpotentials. Electroencephalogr. Clin. Neurophysiol.103, 15–3, 97.

Zani, A., and Proverbio, A. M. (1997d). ERP evidenceof attentional selection in the occipital primaryvisual areas. Brain Topogr. 10, 49–97.

Zani, A., and Proverbio, A. M. (1997e). Selective atten-tion modulates sensory activity of primary visualareas: Electrophysiological evidence in humans.Soc. Neurosci. Abstr. 23 (1), 121.2, p.299.

Zani, A., and Proverbio, A. M. (2002). In preparation.Zani, A., Avella, C., Lilli, S., and Proverbio, A. M.

(1999). Scalp current density (SCD) mapping ofcerebral activity during object and space selectionin humans. Biomed. Tech. 44 (Suppl. 2),162–165.



-4.0

-3.0

-2.0

-1.0

0.0

JJV -I.0

- GLOBALtiigels I I Oonc~ue~ t . . . . . 4 Ima~uent

" i .4,0

-3.0

-2.0

-I.0

0.0

uV -I.0 -I00

n

LOCAL targets

I I I I l i II l i i I I

0 60 140 220 300 msec

CHAPTER 12, FIGURE 5 Top: Grand-average ERPs to global and local targets as a function of congruence with the unattended level. Recordings are from Oz electrode site. Bottom: Isocolor voltage maps of early sensory response to local targets in the latency range of Nl15 component. Reprinted from Cognitive Brain Research, 6; A. M. Proverbio, A. Minniti, and A. Zani; Electrophysiological evidence of a perceptual precedence of global vs. local visual information, pp. 321-334. Copyright (1998) with permission of Elsevier Science.

V E R T I C A L

O B L I Q U E

• 1.0 pV • 0.9 • 0.8 • 0.7

0.6 0.5 0.4

~ oJ m o.n @ o.o

-0.1 • -0.2 • -0.3 • -0.4 • -0.5 pV

100/105 1051110 1101115 1151120 msec

CHAPTER 12, F I G U R E 9 Top: Isocolor maps of effects of orientation selection for vertical (90°) gratings in the P1 latency range. Maps were obtained by plotting the values of the difference wave (target-nontarget) computed by subtracting the ERP to vertical gratings while oblique orientations (i.e., 50 °, 70 °, 110 °, or 130 °) were attended from the ERPs to vertical targets. Bottom: When attention was paid to oblique gratings, attention effects were still significant but somewhat smaller, and had a slightly different scalp distribution. Reprinted from Cognitive Brain Research, 13; A. M. Proverbio, P. Esposito, and A. Zani; Early involvement of temporal area in attentional selection of grating orientation: An ERP study, pp. 139-151. Copyright (2002) with permission of Elsevier Science.

CHAPTER 12, FIGURE 10 Space selection. Scalp current density maps reflecting attention effects for spatial location selection between 80 and 140 msec poststimulus. Maps were computed on the difference wave obtained by subtracting brain response to nontargets from that to targets in the P1 latency range. Stimuli were 7-cpd black and white gratings presented in the right visual field. Note the lateral occipital-parietal flowing of currents, suggesting the activation of the Where system.

CHAPTER 12, FIGURE 14 Frequency selection. Realistic three-dimensional isocolor voltage maps for frequency (left) and location (right) selection. The maps were computed in the selection negativity latency range (i.e., 180-280 msec; peak latency, 240 msec) on difference waves obtained by subtracting brain response to nontarget gratings from that to targets in the two attend-object and attend-location tasks.

Small Medium Large

o 2

N

CHAPTER 14, FIGURE 2 Grand-averaged (n = 13) functional magnetic resonance imaging activation elicited by a small (10%), medium (30%), and large (100%) increase in frequency deviation superimposed on an individual structural MR][ in Talairach space. Images were thresholded at P < 0.01. All deviants induced significant activation in the superior temporal gyri bilaterally, whereas the opercular part of the right frontal gyrus was significantly activated only when the large and medium deviants were presented. Adapted from Opitz et al. (2002).


C H A P T E R

13

Event-Related EEGPotential Research inNeurological Patients

Rolf Verleger

INTRODUCTION

This chapter reviews studies that haveused the event-related potential (ERP)methodology to investigate patients suffer-ing from neurological diseases. Thesestudies have been conducted within theframework of two different points of view:one point of view assesses the utility ofERPs for the clinic, and the other perspec-tive evaluates neurological diseases aslesion models for understanding ERPs.This review focuses on the utility of ERPs for the clinic; neurological diseasesand principal neurological symptoms (cf.Appendix B for a synopsis of neurologicaldiseases) will be discussed from the per-spective of the degree to which ERPs haveplayed a role in diagnosis and prognosis,in delineating and understanding impactsof diseases on cognition, and in therapyand rehabilitation

DEGENERATIVE DISEASES

For effective therapy of degenerativediseases, the mechanisms of neuronaldegeneration have to be understood on amolecular level. This aim has so far notbeen achieved to such a degree that effec-

tive therapeutic treatment is able to amelio-rate symptoms to a significant degree(except for Parkinson’s disease), or even tohalt the progress of the disease. The molec-ular level is not readily accessible to ERPs,therefore ERPs have not yet played asignificant role in therapeutic research.Whether there is a role for ERPs in diagno-sis and prognosis and in understandingconsequences of diseases with respect tocognition is discussed in the followingsections.

Alzheimer’s Disease

Alzheimer’s disease (AD) is the mostcommon cause of dementia in elderlypeople. Pathological markers are plaquesand tangles within neuronal tissue, butthese markers can be identified only bypostmortem neuropatholological examina-tion. Thus, for patients, standard criteriarecommend clinical diagnosis of “proba-ble” AD by diagnosing dementia andexcluding other possible causes of demen-tia (vascular encephalopathy, certain hor-monal or vitamin deficiencies, etc.).

The Diagnostic Problem

In clinical practice, two diagnostic prob-lems may arise in evaluating dementia:Does a patient suffer from dementing

illness at all, and, if so, is it Alzheimerdisease? Since the pioneering work byGoodin et al. (1978), the first problem hasbeen tackled in ERP research by measuringdelays of P3 latency in the “oddball” task.Such delays have been found reliably(Polich, 1991) when patients were morethan mildly demented, but not in cases ofmild to very mild dementia (e.g., Kraiuhinet al., 1990; Verleger et al., 1992; Polich et al.,1986), although it is precisely in the lattercases where there is a real need for diag-nostic information that would complementneuropsychological testing [cf. also thecontroversial exchange between Goodin(1990) and Pfefferbaum et al. (1990)]. Infact, the lack of P3 latency delay in milddementia of the Alzheimer type comes asno surprise, P3 latency being a globalmeasure of psychomotor retardation, closelycovarying with response times [as long asresponse times are relatively fast; seereview by Verleger (1997)]. Because ADpatients are not mentally slow in the mild,early stage, there is no reason why their P3latency should be delayed.

It is this author’s impression that the N2component might be a more promis-ing candidate for distinguishing mildlydemented AD patients from healthypeople. This component was smaller andreliably delayed in the auditory oddballtask in our study (Verleger et al., 1992).Furthermore, a neurotic patient who wasafraid of being demented but in fact wasnot had a huge N2, larger than normal andnot delayed (unpublished). N2 amplitudeswere also reported to be smaller even in ahealthy group who had an enhancedgenetic risk for AD (Green and Levey,1999). The reason for the possible sensitiv-ity of N2 might be its close association toresponse selection (Ritter et al., 1979). ADpatients often trouble experimenters inthese tasks by continually forgetting whatthey have to do in response to some targetevent. The N2 decrease perhaps reflectssuch lapses of response selection even intrials in which the patients succeed in

overtly responding. Of course, these obser-vations will have to be pursued more sys-tematically in order to become relevant.

The second diagnostic problem, the dif-ferential diagnosis of Alzheimer disease vs.other types of dementia, has been tackledby several researchers, since Goodin et al.(1986) first explored the issue. Indeed,other etiologies of dementia can be delin-eated rather well from Alzheimer-typedementia by diagnosis-specific abnormali-ties, including vascular dementia (Yama-guchi et al., 2000) and Huntington’s disease(Goodin et al., 1986) (but conflicting evi-dence will be discussed later). However,the relevance of these findings for diagnos-ing AD is limited, because in practice thereis no problem in diagnosing Huntington’sdisease (by genetic testing), so there is noneed for additional diagnostic help byERPs; also, a positive diagnosis of reducedblood supply to the brain, leading to vas-cular dementia, is much easier (by com-puter tomography scans and transcranialDoppler measurement of cerebral bloodflow) than is the positive diagnosis of AD. What is necessary is a positive markerfor AD before the postmortem diagnosis.Positron emission tomography (PET) mea-surements of regional cerebral blood flowhave become a promising tool, showingdecreased metabolism within the temporaland parietal lobes, but PET measurementsare expensive and invasive.

Measuring Hippocampal Activity

Presumably, ERPs would easily fulfillthe task of providing a simple, positivemarker of AD if they would allow mea-surement of hippocampal activity. It is well known that the hippocampus, situ-ated in the midst of the temporal lobe, isone of the key structures for memory (seeEichenbaum, 2000), that the hippocampusis one of the structures most affected inAD, and that very large and significant P3-type potentials (peaking somewhat laterthan the P3 measured at the scalp) havebeen obtained when recordings are made

310 13. ERP RESEARCH IN NEUROLOGY

IV. CLINICAL AND APPLIED PERSPECTIVES

intracranially from the hippocampus (fromepileptic patients, to find their epilepticfocus) (e.g., Grunwald et al., 1995; Halgrenet al., 1995).

Thus, there is good reason to believethat the hippocampal potentials should beseverely altered in AD. However, becausethe hippocampi are twisted structures,remote from the scalp surface, hippo-campal potentials, whatever their strength,are unlikely to be measured by scalpelectrodes (Lutzenberger et al., 1987).Indeed, few changes in scalp-recorded P3responses to target events have been foundin patients whose hippocampi wereseverely affected (e.g., Onofrj et al., 1992;Polich and Squire, 1993; Knight, 1996).However, these latter studies had focusedon studying whether the scalp-recorded P3is abolished or severely compromised byhippocampal damage (which it is not). Asecond look at these data might be appro-priate, focusing on the questions(1)whether some reliable trace of the hip-pocampal activity can be found (mostprobably following the peak of the scalp-recorded P3, as it does in the intracranialrecordings), (2)what recording sites wouldbe most appropriate to measure such activ-ity, and (3)whether this activity would bealtered in AD (cf. Frodl et al., 2002).

Probing the Memory Deficit

Several studies have tried to focus oncortical reflections of Alzheimer patients’memory deficit. In doing so, some of thesestudies pursued the discussed diagnosticproblem, whereas others aimed at con-tributing to delineating and understandingconsequences of the disease with respect tocognition. Tools used in this research weremismatch negativity, priming positivity,N400, and the P3, measured in Sternberg’stask.

Mismatch negativity (MMN) can beevoked by deviant sounds even in theabsence of any attention paid to the soundsand therefore serves as a measure of pre-conscious auditory memory (Näätänen and

Winkler, 1999). Results of AD studies aresomewhat ambiguous: MMN was equal insize in AD patients and in healthy controls(Gaeta et al., 1999) and in the 1-sec inter-stimulus interval (ISI) condition ofPekkonen et al. (1994). On the other hand,AD patients’ MMN tended to be smaller inKazmerski et al. (1997) and was smallerthan that of healthy controls in the 3-secISI condition of Pekkonen et al. (1994), but,as Gaeta et al. (1999) pointed out, this 3-seccondition always came last, so the decreaseof MMN might also be due to AD patients’getting more tired or impatient withincreasing time spent on the task.

Priming positivity is evoked byattended items of a large set (usuallywords) being repeated, either immediatelyor (with a usually weaker effect) withsome items in between. The usual pro-cedure is to define some targets (e.g., non-words or animal names) and to measurepriming positivity by repeating some ofthe nontarget words. This positivity isthought to consist of both a decrease ofN400, reflecting facilitated (post)lexicalprocessing, and an increase of P600,reflecting some conscious recollection,though this latter component might be ofless relevance in elderly people (Rugg etal., 1997). The amount of AD patients’priming positivity was statistically indis-tinguishable from age-matched healthyparticipants in the studies of Rugg et al.(1994), Kazmerski et al. (1995), andKazmerski and Friedman (1997). Thus,these studies demonstrate preservedpriming by earlier presentation in AD.

Ford et al. (1996) and Revonsuo et al.(1998) had AD patients listen to sentences,in order to measure whether semantic pro-cessing of naturally presented speech isstill intact. This is a nearby question, inview of the patients’ severe impairments ofworking memory, needed to integrate themeaning of heard speech. In half of thesentences, the final word did not fit thepreceding words, evoking a large N400 inthe healthy group. Both studies found that

DEGENERATIVE DISEASES 311


this difference between nonfitting andfitting final words was much reduced inAD. Less marked effects were obtainedwhen words were primed by only one pre-ceding word (Schwartz et al., 1996), whichsupports the interpretation that theworking-memory deficit is the importantfactor. Somewhat in contrast to this inter-pretation, Castañeda et al. (1997) obtainedan N400 pattern similar to those of Fordet al. and Revonsuo et al. when a pictureeither did not fit or did fit the category ofthe one preceding picture.

Finally, a fourth approach to probingmemory dysfunction in AD usesSternberg’s (1966) memory search task: A“memory set” of varying size (1 to 5letters) was presented, followed by singleletters that either were or were notmembers of the memory set, requiring anappropriate choice response. As could beexpected, performance became worse inAD patients when the memory set waslarger, also reflected in a disappearance of the P3 component evoked by singleletters (De Toledo-Morrell et al., 1991), but somewhat surprisingly, this find-ing was not replicated (Swanwick et al.,1997). More data are needed to clarify thisissue.

Conclusion

To summarize, the contribution of ERPresearch to problems of AD has beenlimited. The memory task of listening tosentences has produced the most consis-tent differences between AD and healthyparticipants (Ford et al., 1996; Revonsuo etal., 1998). Possible avenues for furtherresearch include the reliability of the N2component as a diagnostic marker and thepossibility of measuring traces of hip-pocampal activity at the scalp. The issue ofdiagnostic specificity, over and beyond theinformation obtained more easily by neu-ropsychological tests, and above all theissue of finding a positive marker for AD,remain a challenge.

Parkinson’s Disease

The cardinal symptoms of Parkinson’sdisease (PD) are stiffness, lack of move-ment, and tremor during rest. There isoften a slight diffuse impairment of cogni-tive functions, reminiscent of frontal lobepathology. The main pathological mecha-nism is degeneration of dopamine-produc-ing neurons in the brain stem (substantianigra), thus the basal ganglia lack thedopamine needed for their adequate func-tioning. The basal ganglia project in differ-ent ways via the thalamus to cortical areas,one important target area being the supple-mentary motor area (SMA). There are atleast two main challenges for research inPD (apart from spectacular methods suchas grafting embryonic tissue). One isfurther precise delineation of the dopaminemetabolism in order to further refine treat-ment beyond the global replacement of themissing dopamine. ERP research has notplayed a relevant role in this respect. Theother challenge is precise description of themotor and nonmotor impairments of PDpatients, because there is still a great gapbetween what is known about the neu-roanatomical basis of PD and what thismeans in terms of behavior—in otherwords, the functions of the basal gangliaare still under debate. The contributions ofERP research to this endeavor will be high-lighted in the following discussions. Again,as with the hippocampus in AD, a method-ological obstacle for ERPs is that activity ofthe basal ganglia cannot be measured atthe scalp. What can be measured by ERPsare cortical consequences of the basalganglia deficit.

Movement-Related Activity

One of the cardinal symptoms in PD isthe patients’ problem in initiating move-ments. Although the problem is mostobvious in global movements of the wholebody, such as standing up and walking,methodological reasons (movement arti-facts and immobile recording devices) have



forced research to focus on finger move-ments, with only few exceptions (Vidailhetet al., 1993).

Self-initiated movements PD patients’Bereitschaftspotential (BP) preceding fingermovements was found to differ fromnormal values in several studies: Dick et al.(1989) obtained weaker activation about0.6 sec before movement onset, prior to thesteep rise of activation in the remainingperiod until movement onset. This resultwas in contrast to an earlier study byBarrett et al. (1986), who measured BPs in away similar to that of Dick et al. but did notobtain any differences between groups. Onthe other hand, results of Dick et al. werereplicated by subsequent work publishedfrom the same lab (Touge et al., 1995;Jahanshahi et al., 1995) and by Cunningtonet al. (1995) (“cues absent” condition),although other details of the results greatlydiffered between these studies. Touge et al.(1995) obtained no difference betweengroups for repetitive movements (incontrast to Dick et al. and Jahanshahi et al.),but did obtain a difference whenparticipants had to select randomly oneout of four movements. Further, in thestudy by Dick et al. (1989) this smaller earlyamplitude was compensated for by asubsequent steeper rise of activation,whereas BP amplitude remained smaller inPD patients throughout in Touge et al.(1995), Jahanshahi et al. (1995), andCunnington et al. (1995). Studying the samePD patients by PET, Jahanshahi et al. (1995)found weaker blood flow in the PDpatients’ SMA (among a few other areas)during self-initiated movements, thusvalidating the interpretation that thesmaller BP in PD patients reflects smallerinvolvement of the SMA. In contrast to thegeneral trend of lower BP amplitudes,Fattaposta et al. (2000) measured larger BPsin PD patients than in healthy controls,preceding two key presses performed inquick sequence by the left and right hands.An interval of 40–60 msec was required

between the two presses to be counted ascorrect, which the (mildly affected)patients often failed to achieve, thus thisinteresting, irregular result might be due toextraordinary effort taken by the patientsin order to do well.

In summary, the research on self-initi-ated movements showed that: (1) there is aslight but rather consistent deficit in PDpatients’ BP amplitude, either at an earlyphase of the BP or generally throughoutthe time course of the BP, but not focusedon the immediate motor execution part; (2)in view of the severe impairment that PDmay cause for movement initiation, thedeficit in BP amplitude is surprisinglysmall; and (3) variability of results mighthave different reasons, including variabil-ity between patients, but probably is alsodue to technical limitations. For example,Barrett et al. (1986) apparently had no soft-ware available for creating grand meansacross groups and for overlaying thosegrand means of the two groups.Consequently, the measures actually takenmight have been less than optimal forcatching some group difference. Indeed, adifference pointed out by Cunnington et al.(1995) literally remained out of sight inother studies: at least in later stages of thedisease (Cunnington et al., 1997) PDpatients’ BPs do not return to baseline asreadily as do BPs of healthy subjects. Butof course, these limitations cannot explainthe findings of Fattaposta et al. (2000) ofeven larger BPs. Further, none of thestudies used multichannel recordings inorder to find subtle topographical differ-ences. Thus, there is still need for furtherstudies on the BP before and after self-initi-ated movements.

Movement preparation before imperativesignals In recent years, the bulk ofmovement-related research in PD hasturned away from the BP to the contingentnegative variation (CNV), i.e., from self-initiated movements to movementpreparation between some announcing



signal and an imperative stimulus.Empirical reasons for this turn have notbeen convincing so far, because two of thevery few studies using both approaches[see Cunnington et al. (1995), though theydid not use an announcing signal proper;see also Ikeda et al. (1997)] did not obtain asignificant interaction, i.e., CNV wassmaller than BP throughout and PDpatients had smaller amplitudes than didhealthy participants throughout, but PDpatients did not have particularly reducedamplitudes in the CNV task. Anotherstudy, a short report by Oishi et al. (1995),even obtained no difference betweengroups in the movement-related (late) partof the CNV, in contrast to BP, which wassmaller in the patients. But certainly theCNV situation can be better experimentallycontrolled, e.g., it might be argued(Rockstroh et al., 1989) that the PD deficitin movement initiation actually gets out ofsight in self-initiated movements becausethese are actually the instances in whichPD patients succeeded in overcoming theirdeficit.

The overall finding in the studies thatmeasured CNV before the imperative stim-ulus was a reduction of amplitudes in PDpatients (Linden et al., 1990; Wright et al.,1993; Cunnington et al., 1995, 1997, 1999;Pulvermüller et al., 1996; Praamstra et al.,1996; Wascher et al., 1997; Ikeda et al., 1997;Gerschlager et al., 1999), with the exceptionof the report by Oishi et al. (1995) and of astudy on PD patients with marked hemi-Parkinsonism (Cunnington et al., 2001) (seebelow). The amount of CNV reductionmay depend on the task: larger differencesbetween patients and control group werefound by Praamstra et al. (1996) after unin-formative cues (four-choice response afterS2) than after informative cues (limitingthe response alternatives to two), but, inapparent contrast, by Linden et al. (1990)after fully informative cues (one responsealternative) than after uninformative cues(two response alternatives). Larger differ-ences were also found by Wascher et al.

(1997) in the cognitively more demanding“validity task” (brief interval between cueand imperative stimulus, possibility ofwrong preparation) compared to the “clocktask.” Also, the topographical extent ofCNV reduction on the anterior–posterioraxis differed between studies and tasks:Widespread reduction was seen in Wrightet al. (1993) and in the validity task ofWascher et al. (1997), only central and parietal, but not frontal reduction inPulvermüller et al. (1996) and in the clocktask of Wascher et al. (1997), only central inPraamstra et al. (1996), and frontally andfrontocentrally focused in Gerschlager et al. (1999). This variability of CNV reduc-tion by topography and task both betweenand within studies is hard to subsumeunder a general rule. It might be related tothe fact that CNV is a variable mixture of activations of response preparation,stimulus expectation, and effort (vanBoxtel, 1994; Verleger et al., 2000a), so each of these processes might be differ-entially affected in Parkinson’s disease,depending on the task and on the patients’status.

Effects of dopamine medication on PDpatients’ CNV have been published onlyinfrequently but Gerschlager et al. (1999)showed that PD patients’ CNV becameindistinguishable from that of normal par-ticipants when their subthalamic nucleiwere stimulated by implanted electrodes.

Of special interest are the topographicaldifferences reported in studies with goodtopographical resolution. Praamstra et al.(1996) found more widespread contralat-eral activation; when the responding handwas already cued by the warning signal,the central site contralateral to the hand(C3 or C4) became more negative prior tothe imperative stimulus than did the ipsi-lateral site in both healthy subjects and PDpatients, but it was only in PD patients that this contralateral–ipsilateral differenceextended to frontocentral and even frontalsites. Similarly, Cunnington et al. (2001)reported unilaterally enhanced fronto-



central and frontal CNV amplitudes pre-ceding bimanual responses in hemi-Parkinson patients contralateral to theirmore severely affected hand.

These results are hard to summarize,due to the large variation of the CNV dif-ference and the topography of this differ-ence between studies and between tasks.Again, as with the BP, these studies con-tribute to the overall summary that it is notmovement execution that is above allimpaired in PD; rather a number ofprocesses involved in response prepara-tion, depending on the particular taskused, appear to be vulnerable to the dys-function of the basal ganglia.

Movement preparation after ambiguousimperative signals Praamstra et al. (1998)explained their above-mentioned 1996finding of an enlarged area of contralateralactivation in PD patients by assuming that PD patients are more dependent onthe lateral premotor system, making moreuse of visual stimulation for selectingmovements, in order to compensate for thepossibly deficient mesial SMA-basedsystem or due to pathophysiologicalchanges in the lateral premotor system. Bymeasuring the time course of the differencebetween homologous sites, contralateralminus ipsilateral to the responding handwithin the short time interval betweenstimulus and response [lateralized readi-ness potential (LRP); Coles, 1989], Pra-amstra and colleagues provided relevantdata. Praamstra et al. (1998), replicated byPraamstra et al. (1999), demonstrated thatPD patients were more affected, comparedto healthy participants, by interferingstimuli that flanked the imperative sti-mulus. The LRP in PD patients went to thewrong side to a larger extent than inhealthy participants when the flankingstimuli suggested a response different fromthe imperative stimulus. [The imperativestimulus was an arrow pointing left or right, requiring a left- or right-handresponse, but was surrounded by other

arrows that also pointed left or right; cf. forLRP measurement in this task Gratton et al.(1988), Kopp et al. (1996), and Wascher et al. (1999)].

Further, Praamstra and Plat (2001)demonstrated that the fact that some rele-vant stimulus is presented laterally affectedPD patients’ motor system more than wasseen in healthy participants. Lateral poste-rior activation contralateral to the relevantstimulus spread to a larger extent to (pre)motor sites in PD patients than in healthyparticipants. [Stimuli were the letters Aand B, requiring a left- and right-handresponse, respectively. Irrelevant to thetask, but meant to induce lateral shifts ofattention and response tendencies, theletters were presented left or right fromfixation; cf. for measurement of posteriorand central contralateral–ipsilateral differ-ences in this task Wascher and Wausch-kuhn (1996), and Wascher et al. (2001)].

Findings from both paradigms convergeon the conclusion that visuomotor trans-mission is increased in PD. This might beinterpreted as learned compensatorybehavior that is applied by the patientseven in inappropriate situations or, asfavored by Praamstra and Plat (2001), as apathological alteration of executive controlover the motor system.

Conclusion Taken together, the ERPresults on movement control in PDpatients’ appear to be of relevance forunderstanding the disease. Apart fromimportant details, one general conclusionfrom these studies might be that theimpairment of movement is moreintimately linked to the general cognitivesyndrome of PD than is commonlyrealized.

Other Approaches

Oddball task Beginning with Hanschet al. (1982), more than a dozen studiesused the oddball task to investigate PD.Aims were (1) to establish differencesamong PD patients related to neuro-



psychological performance, to the presenceof dementia, or to medication status, (2) toinvestigate whether PD patients differfrom healthy participants, and (3) toinvestigate how PD patients with dementiadiffer from other demented patients. Inretrospect, these goals do not appear to beof continuing relevance. This is not to saythat they were irrelevant a priori. However,as with Alzheimer disease, interindividualvariability proved to be too large to allowERPs to be used for diagnosis. The mainresult, summarized in a review by Ebmeier(1992), was that P2, N2, and P3 peaklatencies tended to be delayed in PD, moreso with more cognitively impaired patients(the latter applying to N2 above all). Notethat this is not so dissimilar fromAlzheimer disease and, moreover, does notallow for the clinically relevant distinctionbetween PD and multisystem atrophy.

Measures of executive control As notedin the introductory remarks on PD, somediffuse cognitive impairment is common inPD, akin to frontal lobe pathology. ERPsoffer the opportunity for measuring signsof this pathology uncontaminated byimpairment in overt behavior seen in PDpatients. Tsuchiya et al. (2000) extended theauditory oddball task by interspersingoccasional novel, unusual sounds amongtarget and nontarget sounds. PD patients’orienting responses to novel sounds wereimpaired, reflected by reduced anddelayed frontal P3 specifically to novelstimuli, not to oddball targets. Becausesuch responses to novel sounds aredrastically reduced in patients with lesionsof the frontal lobe (Knight, 1984), this isevidence for impaired frontal lobefunctioning. A similar argument may bemade for the no-go P3, which is oftenlarger and more anteriorly distributed thanthe go P3 in situations in which go and no-go stimuli are equally probable (e.g.,Pfefferbaum et al., 1985; Verleger and Berg,1991), and therefore might reflect someaspect of the inhibitory function of the

frontal lobe. Indeed, the only work thatcompared PD patients’ go and no-go P3s(Pulvermüller et al., 1996) found their no-go P3s to be specifically reduced.

Stam et al. (1993), Vieregge et al. (1994),and Karayanidis et al. (1995) presentedoddball sequences to both ears separately,instructing participants to attend to soundsin one ear only and to press a key to occa-sional targets presented to that ear [taskintroduced by Hillyard et al. (1973)]. Underthese circumstances, the ERPs evoked bystandard sounds are more negative forsounds in the attended than in theunattended ear [“Nd,” Hansen and Hillyard(1980); viz. “processing negativity,”Näätänen (1990)]. Nd was much smaller inPD patients than in healthy subjects in Stamet al. (1993). This result was replicated byVieregge et al. (1994) when the interstimulusinterval was 1 sec but not when the ISI was0.5 sec and not by Karayanidis et al. (1995),where the ISI was 0.35 sec. Thus, it appearsthat PD patients have a deficit in maintain-ing the attentional trace of the standardsound whenever intervals between thesounds get too long [cf. Ravizza and Ivry(2001), for some converging evidencederived from performance errors]. Finally,measuring error negativity (Ne) in tasks ofvarying difficulty, Falkenstein et al. (2001)found Ne to be reduced in PD, except invery easy tasks (M. Falkenstein, personalcommunication), which might be inter-preted either as a deficit in the very error-monitoring process or a consequence ofbeing overloaded in moderately complextasks. Either alternative would reflect adeficit in executive control.

Memory Mismatch negativity can beevoked by deviant sounds even in theabsence of any attention paid to the sounds,and therefore serves as a measure of pre-conscious auditory memory (Näätänen andWinkler, 1999). Of particular interest to thepresent purpose, there is evidence for theinvolvement of a frontal lobe component ofMMN (Deouell et al., 1998). However, the



report of a reduced MMN in PD (Pekkonenet al., 1995) has not yet been replicated, tomy knowledge (cf. Vieregge et al., 1994;Pekkonen et al., 1998).

Tachibana and colleagues investigatedrepetition priming, reflected as reductionof N400, where repetitions were either taskrelevant (Minamoto et al., 2001) or not(Tachibana et al., 1999a). A clear differentialeffect of priming on the N400 of PDpatients and control participants could notbe established. However, even at first pre-sentation the PD patients had consistentlysmaller N400 amplitudes. Speculatively,this might reflect shallower processing of the presented words, reflecting theParkinsonian frontal-patient-like attitude.

Conclusion Taken together, of the ERPapproaches to investigate PD beyondmovement disorders, the studies reflectingimpaired frontal lobe function haveprovided the most consistent results.Pursuing this line of research further andmaking direct comparisons to patientswith frontal lesions will certainly shed newlight on the Parkinsonian syndrome ofcognitive impairment.

Cerebellar Atrophy and Other Diseasesof the Cerebellum

Cerebellar atrophy (CA) denotes a classof diseases, some idiopathic, some heredi-tary (some toxic, not considered here, e.g.,by alcohol), which are characterized byimpairments of movement precision and ofbalance, obviously related to shrinkage ofcerebellar volume. Often, in the course ofthe disease, pathology progresses from thecerebellum to neighboring structures(olivo-ponto-cerebellar atrophy; OPCA).

Fortunately, CA occurs much less fre-quently than Parkinson’s disease (PD), butsimilar questions can be asked about both.Like the basal ganglia, the cerebellum doesnot have efferent pathways of its own tothe body, but rather exerts its influence byprojecting via the thalamus to cortical

areas, one important target area, thoughnot the only one, being the motor cortex(Schmahmann, 1996; Middleton and Strick,2001). Therefore, as in PD, preciselydescribing the motor and nonmotorimpairments of CA patients is still a chal-lenge, because the functions of the cerebel-lum are fervently debated. Different fromthe basal ganglia, the cerebellum is situ-ated close to the scalp, so scalp-recordedERPs might directly pick up cerebellaractivity, but because the cerebellar neuronsare not organized in the parallel columnsformed by the cerebral neurons, they mostprobably do not produce fields that can berecorded from the scalp. In fact, none ofthe studies on CA patients cited hererecorded ERP differences that can bedirectly ascribed to cerebellar activity.However, some data obtained more or lessincidentally in basic research, e.g., byJohnson et al. (1998), raise some doubts asto whether recording cerebellar activity byERPs is really impossible. In any case, thishas not been done in these patient studies.What has been measured were cerebrocor-tical consequences of the cerebellar dys-function.

For simplicity, studies on CA patientswill here be combined with those onpatients whose cerebellar damage wascaused by other diseases, namely, infarc-tions of arteries supplying the cerebellum[the single-case studies reported by Ikedaet al. (1994), and Gerloff et al. (1996); mostpatients in Kitamura et al. (1999); some ofthe patients in Shibasaki et al. (1978) andDaum et al. (1993)] and tumors of the cere-bellum that were resected [all patients inAkshoomoff and Courchesne (1994); onepatient in Daum et al. (1993) and inKitamura et al. (1999)].

Movement-Related Activity

The BP preceding self-initiated fingermovements of CA patients drasticallydiffers from normal by its lower amplitude(Shibasaki et al., 1978; Ikeda et al., 1994;Wessel et al., 1994; Gerloff et al., 1996), by



its earlier onset (i.e., patients need moretime to prepare a movement) (Shibasaki etal., 1978; Wessel et al., 1994), and by itsdiffuse topography, lacking a clear central-midline focus (Tarkka et al., 1993; Wessel etal., 1994; Gerloff et al., 1996). Lesions of thecerebellar dentate nucleus appear to beparticularly harmful (Shibasaki et al., 1978,1986; Kitamura et al., 1999). Thus, differ-ences from what is normal are more consis-tent and more marked than in Parkinson’sdisease.

These is some inconsistency about con-tingent negative variation developingbetween warning and imperative signals inCA. Ikeda et al. (1994) presented a singlecase in which the BP was entirely absent,whereas CNV was not (though amplitudeswere not compared to normal values). Inthe same vein, CNV amplitudes of CApatients in Daum et al. (1993) were notsignificantly smaller than those of healthyparticipants. However, the studies byYamaguchi et al. (1998) and Verleger et al.(1999) demonstrated drastically reducedCNV amplitudes (called late negativedeflection in Yamaguchi et al.), and againlack of a clear central-midline focus inthese patients. Thus, CNV results of CApatients do not principally differ from theirBP results [cf. Verleger et al. (2000a), fordiscussing differences and similarities ofthese two complexes of components]. Thisamplitude reduction was independent ofmovement complexity in Verleger et al.(1999), i.e., although generally smaller inthe patients than in normal subjects, CNVbecame larger by the same amount inpatients and in healthy subjects precedingdifficult bimanual movements, which theCA patients often failed to perform suc-cessfully, compared to simple movements.This led us to suggest that it is the motorcortex, not the cerebellum, that becomesmore active in fine coordination, with thecerebellum being generally involved in anykind of preparatory and executive activity,providing the motor cortex with informa-tion needed for coordinating movements,

being used by the motor cortex forcomplex movements but not necessarilyfor simple ones.

Search for Consequences of CerebellarLesions for Cognition

Akshoomoff and Courchesne (1994) hadparticipants respond to targets in either theauditory and visual modality and, havingdetected the target, shift their attention tothe other modality. Their cerebellar patients(tumor-resected 10-year-old children) haddifficulties in detecting targets when timesince the preceding target in the othermodality was less than 2.5 sec. This wasalso reflected by a lack of differencebetween the patients’ P3 waves evoked bytargets in the to-be-attended and in the to-be-ignored modalities (same modality aspreceding target) when time since the pre-ceding target was less than 2.5 sec. In adifferent paradigm, Yamaguchi et al. (1998)did not find evidence for impaired capacityfor fast shifts of attention, this time ofspatial attention in the visual modality. Inan S1–S2 task, Yamaguchi et al. measuredthe contralateral–ipsilateral differenceevoked by arrow cues or peripheral cueswithin the 0.8 sec S1–S2 interval. Neitherthe parietal difference at about 300 msecafter the cue (cf. van der Lubbe et al., 2000)nor the frontocentral difference at about 400msec after the cue (cf. Verleger et al., 2000b)was reduced or delayed in CA patients.

Tachibana et al. (1995) measured perfor-mance of CA patients in a visual oddballtask (plant names written in Japanese kanasymbols were targets; animal names werethe frequent nontargets) relative to asimple-response task. Patients’ Na compo-nent was delayed (measured as thedifference between nontargets and simpleresponse, peaking at about 200 msec),causing a delay of the ensuing N2 and P3latencies in the target wave shapes.Following the Ritter et al. (1983) interpreta-tion of Na, Tachibana et al. (1995) suggestthat the cerebellar lesion causes animpairment in pattern classification, a skill



that is obviously needed in reading Japanesesymbols. In a subsequent study, Tachibana et al. (1999b) found that those patients who had delayed P3s in this task also had lower frontal blood perfusion, as measuredby single-photon computed tomography(SPECT).

In summary, ERP evidence on impair-ment of cognitive abilities in CA is interest-ing but scarce.

Other Degenerative Diseases

Other degenerative diseases, fortunatelyoccuring relatively infrequently, includeHuntington’s disease (HD), progressivesupranuclear palsy (PSP), and amyotrophiclateral sclerosis (ALS).

Huntington’s Disease

HD is a hereditary degenerative diseaseof the brain, focusing on parts of the basalganglia (nucleus caudatus and putamen)and producing hyperkinesia, akinesia, anddementia. Although excess movements area core symptom of the disease, only onestudy has investigated movement-relatedpotentials in HD. In a pilot study (record-ings at Cz only, without any electro-oculographic recording for artifact control),Johnson et al. (2001) found reduced ampli-tudes in movement preparation, verysimilar to the results obtained for Parkin-son’s disease in the same task studied bythis group of authors (Cunnington et al.,1997). The similarity to PD might seemsurprising, because PD is reflected in hypo-kinesia, HD in hyperkinesia.

Similar to other degenerative disorders(AD and PD), P3 latency in oddball tasks isdelayed in HD (Rosenberg et al., 1985;Goodin and Aminoff, 1986; Filipovic et al.,1990), but the finding that latencies ofearlier components (N1, P2) were speci-fically delayed in HD (Goodin andAminoff, 1986) was not replicated byRosenberg et al. (1995) and Filipovic et al.,1990). In a large study on 30 patients, on 40of the patients’ sons and daughters, and on

60 healthy controls, Hömberg et al. (1986)established that P3 latencies of HDpatients’ (as well as latencies of the earliercomponents N1, P2, N2) were delayed inan auditory oddball task, and that the P3delay occurred not only in patients butalso in their offspring, who are at risk ofdeveloping HD, correlating with deficits inpsychometric tests. Promising as thisfinding was to identify those people whoare at risk for developing HD, in the mean-time such identification has become reli-ably possible by genetic testing, which ismuch more precise than the necessarilyvariable ERP measures.

Münte et al. (1997) used visual searchtasks (modeled after Luck and Hillyard,1990) and a continuous word recognitiontask (cf. Rugg et al., 1997) to characterizemore precisely the cognitive impairment ofHD patients by means of ERPs. Severalremarkable results were obtained. First, inboth tasks the patients’ P1 was markedlyreduced and the following N1 was mas-sively delayed. As Münte et al. note, this isin contrast to earlier studies. [Of thestudies quoted here, Rosenberg et al.(1985), also used visual stimuli and did notobtain an N1 delay.] This might mean thatHD patients have a deficit specifically inperceiving the complex stimuli used inthese tasks [cf. below, the findings in PSPby Johnson (1992)]. Further, in both tasksthe patients’ ERP wave shapes were notmodulated by task demands in the P3latency range: patients displayed neither aP3 in response to targets in the visualsearch task nor a priming-and-recognitionpositivity in the word-recognition task.Although it can be argued that thesemissing modulations are a trivial conse-quence of the distorted input indicated bythe P1–N1 abnormalities, at least the lackof priming positivity is remarkable,because even Alzheimer patients wereshown to have priming positivity (seeabove). Thus, ERP studies might wellcontribute to understanding the specificmechanisms of impairment in HD.



Progressive Supranuclear Palsy

PSP is characterized by palsy of verticalsaccades, loss of voluntary facial move-ments, axial dystonia, gait disturbance, anddementia, due to pathological alterationsin several subcortical brain regions. Aswith other dementing diseases, P3 latencyhas been found to be delayed both invisual (Pierrot-Deseilligny et al., 1989;Johnson et al., 1991) and in auditory(Takeda et al., 1998) oddball tasks, and P3amplitude to be reduced (Johnson et al.,1991; Wang et al., 2000; Pirtosek et al., 2001).In addition, anterior visual P200 compo-nents were much reduced and delayed inthe patients (Johnson et al., 1991) whereasoccipital N1 was not altered. Johnson(1992) describes the results of a compre-hensive series of tasks, including threeother tasks in addition to the oddball tasks:Sternberg’s (1966) memory-scanning task,a final-word verification task (sentenceshad to be read, with the final word beingeither congruent or incongruent to the pre-ceding context) (cf. Kutas and Hillyard,1980), and a mental-rotation task usingpictures of rotated right and left hands as stimuli. Replicating the oddball results,P200 and P3 (as well as response times)were delayed. Moreover, whereas thememory-scanning task resulted in princi-pally similar patterns of patients and con-trols with increasing load, the other twotasks yielded unexpected results. In thefinal-word verification task, no differentia-tion between congruent and incongruentwords was visible in the patients’ ERPs inthe N400 time range, and in the mental-rotation task the hand stimuli elicited verylarge occipital N1 components in thecontrol group but not in the patients. Thedeficient N400 effect is reminiscent of thefinding by Minamoto et al. (2001) in PD,and the lack of N1 enhancement, probablyrelated to lack of a discrimination process(Vogel and Luck, 2000), parallels thefinding by Münte et al. (1997) in HD. Moregenerally, Johnson’s (1992) study is a niceexample of applying a kind of ERP test

battery, probing several diverse but well-investigated ERP effects, from which aneurophysiological–cognitive profile canbe built for the study group.

Amyotrophic Lateral Sclerosis

ALS consists of degeneration of neuronsof the pyramidal tract and the consecutivespinal neurons, progressively reducing thepatient’s ability to move. Questions opento research include (1) how a patient’smotor cortex deals with this efferent block-ade, (2) whether other cognitive functionsare affected by the degenerative process,and (3) whether patients’ ERP responsescan be used as a means for them to com-municate with their environment when thedisease progresses to a point at which theyare no longer able to move.

Movement-related potentials Possiblythe only published study on movement-related potentials in ALS patients is that ofWestphal et al. (1998), who recordedBereitschafts potential before self-pacedmovements. The total group of ALS pa-tients did not differ from healthy controlsbut a subgroup of patients with increasedspasticity had lower amplitudes. In ourCNV study (R. Verleger, unpublished), wedid not find differences between patientsand healthy controls, in contrast to patientswith striatocapsular infarction (see below),who also have to deal with the problem ofefferent blockade. This is in contrast to thePET study by Kew et al. (1993), who foundenlargement of the ALS patients’ handmotor area, very similar to patients withstriatocapsular infarction (Weiller et al.,1993), interpreted in both cases as a meansof compensation. Such compensatory effortmight be expected to be reflected in largermovement-related potentials, which werefound in neither the Westphal et al. study(1998) nor in our unpublished study.

Affection of cognitive functions In theirstudy on ALS patients’ performance in anauditory oddball task, Gil et al. (1995)



found delayed N2 and P3 latencies but noeffects on earlier components. Otherstudies used more complex tasks. Viereggeet al. (1999) presented oddball sequences toboth ears separately, instructing par-ticipants to attend to sounds in one earonly [same design as the Vieregge et al.(1994) study in PD; see above]. Nd wasmuch reduced in ALS patients, both withslow and with fast presentation. Peculiarly,though entirely independent of any overtresponse, the Nd reduction correlated withthe patients’ motor disability but not withneuropsychological tests (in contrast tohealthy subjects, in which Nd reductiondid correlate with tests sensitive to frontallobe function). In addition, the N100amplitudes evoked by all sounds werereduced in ALS patients. [Not reported inVieregge et al. (1999) is that N2 and P3components to targets were unaffected byALS, unlike Gil et al. (1995).] Similarobservations of decreased amplitudes ofearly modality-specific components weremade in the visual modality by Münte andcolleagues, with ALS patients’ temporo-occipital P100 components being virtuallyabsent in response to words or to visualsearch displays (Münte et al., 1998a, 1999),similar to what Münte et al. (1997) hadreported in these tasks for HD, but, unlikeHD, visual N1 was not delayed in ALS. Inaddition, Münte et al. (1998b) reportedrecognition positivity to be absent in ALSpatients, as in their HD study.

To summarize, there is surprisingthough still sparse evidence for abnormali-ties in relatively early sensory ERP compo-nents in ALS. Whether this reflects somedamage of afferent pathways in ALS or thepossibility that efferent pathways must beintact in order to obtain normal sensorycomponents is an open question. In addi-tion, a number of further findings havepointed to abnormalities of higher cogni-tive functions in ALS independent ofmotor dysfunction. How these findingsrelate to the fact that the world’s leadingastrophysicist, Stephen Hawking, has been

suffering from ALS for many years(Hawking, 1989), is certainly an open ques-tion. One might argue (S. Ravizza, per-sonal communication) that ALS patients(as well as PD patients) are permanentlydistracted in such tasks by their greaterdifficulties with the occasional responses,which is why even their potentials to stan-dards might be affected, but it is doubtfulwhether this argument indeed can accountfor the reported abnormalities of earlycomponents.

Communication by locked-in patientsSevere ALS is a model case for the locked-in syndrome, wherein the patients’ ERPscan be used as a means to communicatewith their environment. Use of the P3 com-ponent evoked by target stimuli for thispurpose was suggested and demonstratedin healthy subjects by Farwell andDonchin (1988). Participants might selectletters to spell words and sentences byemitting enhanced P3s to arrays whenthese arrays contain the intended letter.Indeed, a “locked-in” patient (afterischemia of the basilar cerebral artery)studied by Onofrj et al. (1996) emittedN2–P3 complexes to target sounds, thus inprinciple would have been able to use thismeans of communication. In recent years,this line of applied research was pursuedsystematically by Birbaumer’s group(Kübler et al., 1999; Birbaumer et al., 1999,2000) in patients, mostly suffering fromchronic ALS, whose remaining controlover muscles (e.g., for moving the eyes)had become too unreliable to be used forcommunication. In this “thought transla-tion device” patients select letters byenhancing the positive level measuredfrom Cz 1.5 to 2 sec after letter presenta-tion. In doing so, patients indeed becameable to produce messages, though with avery slow speed [e.g., as reported inKübler et al. (1999), 12 sec per letter at best,more than 3 min per letter at worst).

Alternative ways for using the EEG tohelp locked-in patients communicate with



their environment have involved modula-tions of selected frequency spectra of thespontaneous EEG. These approaches (e.g.,Wolpaw et al., 1991; Pfurtscheller et al.,1993) will not be discussed here, becausethey do not involve using ERPs. A compre-hensive review covering all these tech-niques has been published (Kübler et al.,2001).

LESION OF CEREBRAL TISSUE BYINFARCTION OR HEMORRHAGE

Brain tissue can be damaged by disor-ders of blood circulation in two ways: byinfarction, i.e., some vessel is blocked suchthat tissue no longer has adequate bloodsupply, or by hemorrhage, i.e., some vesselis ruptured, causing blood to overflow intobrain tissue.

Before turning to the effects of focallesions induced by sudden ischemic events,the syndrome of “vascular dementia” (VaD)will be discussed, being more akin to thediffuse, gradually increasing effects found indegenerative diseases as discussed so far.Sudden ischemic events will be discussedaccording to main syndromes encountered,which will be hemiparesis, aphasia, andneglect. An alternative distinction, moredown to earth at first sight, would consist indiscussing the main cerebral arteries, oneafter the other. However, as well organizedas these arteries seem at first sight (anterior,media, posterior), their winding courses andtheir distributed branchings blur any simplepicture. Lesions of cerebellar arteries havebeen discussed previously.

Vascular Dementia

VaD refers to reduction of cognitivecapabilities due to widespread cortical orsubcortical pathological changes occurringas consequences of chronically reducedblood perfusion or microlesions broughtabout by hypertension. If these factors of

blood circulation are treated, the progres-sion of VaD may be stopped or delayed atleast. Therefore, distinguishing VaD fromAlzheimer disease has clinical relevance. Areview by Looi and Sachdev (1999) con-cludes that VaD and AD differ in their neu-ropsychological profiles, with VaD patientshaving relatively more impairment infrontal executive functioning and ADpatients in verbal memory.

Most ERP studies on VaD have beenperformed in Japan, VaD being morecommon than AD in East Asia (Looi andSachdev, 1999). Apparently, no diseasespecificity for distinguishing VaD from ADcan be obtained in simple oddball tasks(Neshige et al., 1988), mildly affected VaDpatients also having delayed N2 andnormal P3 latencies (Tachibana et al., 1993),similar to mildly affected AD patients(Verleger et al., 1992). Also, in the moresophisticated repetition-priming paradigm(cf. above, Parkinson’s disease) (Tachibanaet al., 1999a), very mildly affected VaDpatients had normal priming positivity(Tachibana et al., 1999c), similar to thefindings made in AD patients (Rugg et al.,1994; Kazmerski et al., 1995; Kazmerski andFriedman, 1997). However, the vascularpatients’ N400 was generally reduced(Tachibana et al., 1999c), similar toParkinson patients (Tachibana et al., 1999a),possibly likewise indicating pathology offrontal lobe function. In any case, as brieflymentioned above, in an elegant study,Yamaguchi et al. (2000) were able to iden-tify VaD patients by using Knight’s (1984)frontal-lobe-sensitive variant of the oddballtask: in contrast to AD patients, the VaDpatients lacked the anteriorly focused“novelty P3” in response to novel soundsinterspersed in the oddball sequence. Thus,reliable distinction of VaD from AD bymeans of ERPs appears to be possible,although this is certainly of limited rele-vance, because reliable diagnosis of vascu-lar encephalopathy is possible by CT scansand transcranial Doppler measurement ofcerebral blood flow.



Hemiparesis

Hemiparesis, due to infarction (or hem-orrhage) of the middle cerebral artery(MCA), is presumably the most well-known neurological disease, constitutingthe most frequent etiology of movementimpairment, by disabling the contralateralarm and hand. Surprisingly, only very fewpapers have been published on ERPs inhemiparesis. This does not make muchsense because it is easily conceivable (andin fact was expected by the patients whomwe recently investigated) that studyingthese patients’ movement-related poten-tials would, among other things, allowsome prediction about the degree of theirpossible rehabilitation.

Control of hand movements may beimpaired by MCA lesions in two ways—bydamaging either the primary motor cortexor the subcortical course of the pyramidaltract. In the former case, movement-relatedpotentials generated by the motor cortex canbe expected to be severely altered, possiblyreduced, in the latter case, the intact motorcortex has to deal with some efferent block.Three studies measured the Bereitschaftspotential before self-initiated movements inMCA patients. Platz et al. (2000) investigateda relatively homogeneous group of 8 mostlysubcortically affected patients, Kitamura etal. (1996) reported 2 cases of subcorticalinfarction, whereas the 10 patients reportedin Green et al. (1999) were quite heteroge-neous. When the patients of Kitamura et al.(1996) moved their affected arm, BP ampli-tudes were of equal size at the two hemi-spheres. The affected hemisphere did notreach normal, contralateral values (thoughbeing larger than when the other arm wasmoved) and the unaffected hemisphere wassomewhat enhanced. The main finding ofPlatz et al. (2000) was a general reduction ofBP amplitude around 600 msec beforemovement onset, in particular at parietalsites, causing an anterior shift of the BPtopography. Unfortunately, the time courseof the BP was not reported in this work. (Of

further interest, movement-related alphaand beta desynchronizations were mea-sured, which is not within the scope of thisreview.) In our own study on 13 sub-cortically affected MCA patients (Verleger et al., 2002) in an S1–S2 task, we focused oncontralateral–ipsilateral differences (LRP)(Coles, 1989) time-locked to movementonset to show the time course of interplaybetween the contralateral (affected) and theipsilateral (unaffected) motor cortex. Begin-ning 200 msec before finger movements ofthe affected hand, a normal contralateralsurplus of EEG activity was developing.Briefly after response onset, however, theopposite, unaffected cortex, ipsilateral to the responding hand, became additionallyactive, in contrast to movements of theunaffected hand and to healthy participants.This time course precludes any role of ipsi-lateral activity in response initiation of theaffected hand but might indicate prophy-lactic activation of the unaffected motorsystem to compensate for possible failure ofthe affected hand.

Additionally, we found general reductionof the patients’ CNV amplitudes before theimperative signal, in particular at scalp sitesoverlying both motor cortices (C3 and C4)and, similar to Platz et al. (2000), at parietalscalp sites. In summary, much more workshould be done using movement-relatedERPs to explore the impairments and capac-ities of these patients.

Aphasia

Aphasia is usually the consequence ofleft-side MCA infarction (in right-handers).Traditionally, Broca’s area in lateral pre-motor cortex and Wernicke’s area insuperior-posterior temporal cortex havebeen identified as relevant to language pro-cessing. An obvious difficulty in research isthe large heterogeneity between patients,both with regard to focus and extent of thelesion and with regard to language abilities.

In the first study on aphasic patients,Starr and Barrett (1987) focused on the

LESION OF CEREBRAL TISSUE BY INFARCTION OR HEMORRHAGE 323


impairment of auditory verbal short-termmemory, frequently encountered as a resid-ual symptom. Indeed, the four patientsstudied had a specific deficit of the P3 com-ponent in an auditory version ofSternberg’s (1966) memory scanning task,using digits as stimuli, whereas their P3amplitudes were relatively normal in anauditory oddball task with sounds asstimuli and in a visual memory-scanningtask.

Perhaps coming somewhat closer to thedisturbed mechanisms, Aaltonen et al.(1993) reported about a possibility to dis-tinguish electrophysiologically betweenBroca and Wernicke aphasics (n = 2patients each). Mismatch negativities weremissing in the Wernicke aphasics inresponse to deviant synthetic vowels,whereas, as a control, all patients hadMMNs in response to deviant pure tones.Ilvonen et al. (2001) presented evidencethat even MMN amplitudes to pure tonesdeviating by their shorter duration fromstandard tones were smaller in Wernickeaphasics (n = 8) than in healthy controls,possibly reflecting a generalized impair-ment of these patients in discriminatingduration of auditory stimuli. In the mostsystematic approach taken so far, thoughon only four patients with varying lesionssites, Csépe et al. (2001) compared MMNsevoked by deviating pitch, by deviatingvowels, by deviating place of articulation(ga vs. ba), and by deviating voicing (pa vs.ba). MMN became more and more abnor-mal in this order, i.e., it was indistinguish-able from normal with deviating pitch butgrossly reduced with deviating voicingand place of articulation. Unfortunately,Csépe et al. had not included a pure-toneduration deviance as used by Ilvonen et al.(2001). This comparison would be of inter-est because the deviation of voicing wasbased on deviance in voice onset time, thusmight be related to a general problem ofduration discrimination. It should be notedhere that MMN to simple pitch deviancemight be a reliable marker of amusia

(deficit of music perception); according toa study by Kohlmetz et al. (2001), thismight be quite common in unselectedstroke patients.

Only recently has research begun tofocus on the components most frequentlyinvestigated in basic research about lan-guage-related ERPs: the N400, the P600,and left-anterior negativity. A few casestudies have tested the ability of patients todistinguish between semantically appro-priate and inappropriate sentence endingsby measuring the N400 (cf. Kutas andHillyard, 1980). N400 effects were found(Cobianchi and Giaquinto, 2000) even inthe absence of signs of explicit under-standing (Revonsuo and Laine, 1996),encouraging rehabilitative treatment (Con-nolly et al., 1999). In contrast to these opti-mistic reports, on a group level, N400amplitudes to semantically inappropriatewords were found to be attenuated, both inpatients with frontal lesions [Friederici etal. (1999), n = 3] and in those with parietallesions [Reuter et al. (1994), n = 10], and inparticular in patients with more severecomprehension deficits as defined bymedian-splitting the entire group [Hagoortet al. (1996), n = 18 aphasic patients tested;Swaab et al. (1997), n = 14; Swaab et al.(1998), n = 12], with no obvious differencebetween frontoparietally lesioned Brocaaphasics and temporoparietally lesionedWernicke aphasics (Hagoort et al., 1996;Swaab et al., 1997, 1998).

In addition, Friederici et al. (1999) testedsensitivity to syntactic violations (verbrather than noun presented after a preposi-tion), measuring the left anterior negativitypeaking at about 250 msec (cf. Friedericiet al., 1993). Results were, however, notclear-cut, due to the small number ofpatients (n = 3). The P600, a second P3-typecomponent in response to syntactic viola-tions (e.g., Hahne and Friederici, 1999) didnot differ between patients and controls.

Taking a new approach by analyzingpotentials evoked by words read withinstories, ter Keurs et al. (1999) focused on



the difference between open- and closed-class words (i.e., articles, conjunctions,prepositions) in 14 Broca aphasics. Anearly frontal negativity evoked by closed-class words, peaking at 260 msec in healthysubjects, was entirely missing in theaphasic patients, and differences betweenthe two word classes from 400 mseconward were reduced in the patients.

Thomas et al. (1997) recorded slowpotentials during a period of severalseconds during which patients had to men-tally search for synonyms to a word pre-sented at trial onset. The stable left frontalnegativity evoked by this procedure inhealthy subjects (cf. Altenmüller, 1989) wasreplaced by bilateral frontal negativity inaphasic patients tested 2–4 weeks afterstroke (n = 7). When studied at a later timepoint, lateralization in four Broca patientshad become normal, left-lateralized,whereas negativity remained bilateral inthe three Wernicke patients. Cohen et al.(2001) measured the N400 evoked by S1(called “slow wave” in that paper) andslow CNV negativity during the S1–S2interval of a delayed-matching-to-sampletask designed after the “token test” foraphasia, with S1 and S2 consisting of eitherwords or symbols, from 19 aphasic patients(median time after infarction was 1 year)and 18 healthy controls. CNV preceding S2was more clearly left-lateralized in healthycontrols than in the aphasic patients.Different from Thomas et al. (1997), it wasabove all the Broca patients who lacked leftlateralization in this slow CNV negativity,which might, of course, be due to differ-ences in task or patients. The N400 evokedby S1 overlapped the posteriorly focusedP3/slow wave complex and was wellmarked in Broca aphasics only, with anextreme left lateralization. One mightascribe this to some volume defect of thesepatients underlying left frontotemporalscalp sites, but both aphasics and controlshad left-lateralized N400 componentsevoked by words as S2, rather similar tothe Broca aphasics’ N400 evoked by S1,

suggesting some functional meaning ofthis lateralization. On the other hand, thisleft-lateralized N400-type component wasalso invariably evoked in these aphasicpatients by pictures that had to beclassified according to either their gram-matical gender or to their being natural ornot [Dobel et al. (2001); patients largelyidentical to those of Cohen et al. (2001)].Thus, it is still not clear whether this pecu-liar feature of Broca aphasics reflects somefunctional reorganization or some struc-tural alteration.

In summary, research in aphasia hasfocused on several goals: prediction ofrehabilitative success in the N400 single-case studies, finding electrophysiologicalmarkers of subtypes of aphasia in theN400 and MMN group studies, under-standing mechanisms of Broca’s aphasia,and mapping the course of cortical reorga-nization. Undoubtedly, each of these goalsdeserves further study.

Neglect and Extinction

Infarction (or hemorrhage) of posteriorbranches of the MCA, affecting parietaland superior temporal cortex, may cause“basic” disorders of somatosensory per-ception, which is beyond the scope of thisreview [for data on disordered movement-related somatosensory reafference in a fewof these patients see Kopp et al. (1999) andPlatz et al. (2000)]. In addition, lesions onthe right (in right-handers) may result indisorders of spatial cognition and indecreased awareness for the left side ofspace (hemineglect). [For an introductionto and review of this fascinating syn-drome, see Kerkhoff (2001) and Driver andVuilleumier (2001).] MCA infarction is themost frequent etiology, but not the onlyone.

There are at least two published casestudies and three group studies usingendogenous ERPs. In the case study byMarzi et al. (2000) light flashes were pre-sented on the left, right, or both sides, and

LESION OF CEREBRAL TISSUE BY INFARCTION OR HEMORRHAGE 325


data were averaged according to thepatient’s indication of what she had seen.When she reported seeing the right flashonly in the case of actual bilateral stimula-tion, P1 and N1 components were absent atright parietal sites. When she reportedseeing both flashes, or in the case of unilat-eral stimulation, the components were pre-sent. Thus, visual extinction appears toinvolve the early stages of perception.Different results were obtained by the othercase study, by Vuilleumier et al. (2001), whoused the same rationale as Marzi et al. (2000)but obtained N1 components of the sameamplitude to extinguished and perceivedleft stimuli, though P1 was somewhatreduced. Moreover, using faces and shapesas stimuli rather than flashes, these authorsmade use of the fact that faces evoke aspecific ERP signature, consisting of atemporo-occipital N170 and a central P200[see, e.g., Eimer and McCarthy’s (1999)study in a prosopagnosic patient].Extinguished faces not only evoked thesame N1 but also the same N170/P200 asperceived faces. One reason for the strikingdifferences between these two single-casestudies might be variability betweenpatients. Furthermore, neither study used acontrol group [see Verleger (2001), forfurther discussion of Marzi et al.’s study].Relevant to the debate of whether the earlyP1 and N1 components are affected inneglect patients are the studies by Spinelliand colleagues (e.g., Spinelli et al., 1994;Viggiano et al., 1995; Angelelli et al., 1996)measuring steady-state visual evoked poten-tials (i.e., flashes were regularly presentedseveral times per second). Because no taskwas associated with these stimuli, thesestudies are somewhat off the focus of thisreview. However, their results that neglectpatients’ P1 latencies were delayed inresponse to left hemifield stimuli are cer-tainly of interest in the present context [seethe review by Deouell et al. (2000a), for moredetails].

The first group study using endogenousERPs was conducted by Lhermitte et al.

(1985). Light flashes were presented leftand right (not bilaterally) and P3s werefound to be delayed to left stimuli. Thesecond group study, by Verleger et al.(1996), measured potentials evoked by cuesand targets in Posner et al.’s (1984) visualcueing task from 10 patients with lesions ofthe right parietal cortex and from age-matched healthy subjects. In essence, thistask is a computerized version of the visualextinction test as used by Marzi et al.(2000), with the sequence right cue and lefttarget leading to response delays, andtherefore comparable to those bilateralstimuli in Marzi et al. that were extin-guished, and the other sequences of leftand right cues and targets evokingresponses in time, and therefore beingcomparable to those bilateral stimuli inMarzi et al. that were not extinguished. Thepatients’ N1 component evoked by left-side cues was reduced at the right parietalrecording site, suggesting a general impair-ment in processing left-side visual input.Of more importance, the patients’ EEGpotentials evoked by the critical combina-tion of right cue and left target differed intwo features from the other sequences:their mean amplitude of 160–280 msecafter target onset (Nd) was less negativethan with other combinations of cue andtarget, and the following frontal P300 wasenhanced. Thus, the lack of early reduc-tions in response to extinguished stimuliconfirms the results of Vuilleumier et al.(2001), but in addition we were able todescribe a positive ERP sign of extinction,the missing Nd.

The third group study, by Deouell et al.(2000b), changed perspective and investi-gated the mismatch negativity, i.e., the(preattentive) ERP response to somechanged sound in a series of unattendedsounds. The neglect patients had reducedMMN amplitudes when sounds were pre-sented on the left, in particular whenstimuli deviated in location. The N100amplitudes were not reduced to leftstimuli, making a deficit in early percep-



tual processing improbable. Rather, as theauthors suggest, a deficit appears to existin detecting changes in the left side of theenvironment. Clearly, more research ispromising to give further insight into thisdisorder.

INFLAMMATORY DISEASES

In addition to degenerative diseases or disturbances of blood supply, the brain may be damaged by inflammation.Meningoencephalitis is an acute, possiblylife-threatening event caused by bacteria or viruses. ERP studies on meningo-encephalitis have been conducted primarilyon patients with damage to the temporallobe and the hippocampal system; thelesions in these patients have served asmodels to study ERP generators. As statedpreviously, this will not be discussed indetail here. The research discussed here ison the chronic, potentially deterioratingdamage caused by multiple sclerosis andby the human immunodeficiency virus.

Multiple Sclerosis

Multiple sclerosis (MS), viz. encephalo-myelitis disseminata (ED), is character-ized by a diversity of symptoms due toinflammatory-type lesions of neuronalaxons and the insulating myelin. Becausethe optical nerve is frequently affected,measurement of exogenous components ofvisual evoked potentials, not of interesthere, is a standard diagnostic method.Similar to what occurs with degenerativediseases (as discussed previously), for effective therapy the mechanisms ofdestruction in MS have to be understoodon a molecular level. Such information isnot readily accessible via ERP studies,therefore ERPs have not played a sig-nificant role for therapeutic research.Whether there is a role for ERPs in diagno-sis and prognosis and in understandingconsequences of MS on cognition will be

assessed in the following discussion. Anobvious problem for research is the hetero-geneity of the disease, due to the differentlocation of foci and due to individ-ually different courses of progression orremission.

Newton et al. (1989) consequentlyreported a large variability in the ERPresponses of their 20 patients; responseswere obtained for both visual and auditory(three-stimulus) oddball tasks. The sub-group of patients that had delayed orreduced N2, P3, or slow wave componentsalso had more cerebral lesions visible intheir MRI scans. Similar findings weremade in auditory oddball tasks by Honig et al. (1992) (n = 31 patients),Triantafyllou et al. (1992a) (n = 47), and Gil et al. (1993) (n = 101). P3 latency delaystended to be related to physical disabilityin addition to visible brain lesions.Somewhat in contrast to these studies, vanDijk et al. (1992) (n = 30) and Giesser et al.(1992) reported normal ERP components innondemented MS patients, but Giesser et al. (1992) found not only abnormal N2and P3 but also prolonged N100 and P200latencies in those patients diagnosed asdemented. Thus, delayed ERP latenciesappeared to be related to the presence ofdementia only. Only six patients wereincluded in either subgroup of this latterstudy; however, recordings were made ina renowned lab that has contributed muchto basic research, thus it may be arguedthat data quality is better than in the other larger studies. But the bulk of evid-ence indicates that presence of whitematter lesions and physical disability arerelated to ERP abnormalities in MSpatients as measured in the oddball task.Initial studies on the use of oddball task-related ERPs to evaluate drug effects oncognition in MS patients are promising(Filipovic et al., 1997) but not conclusiveyet (Gerschlager et al., 2000).

The oddball task is relatively unspecific.Using tasks that put greater demand onworking memory, two studies have

INFLAMMATORY DISEASES 327


demonstrated ERP effects on mildly affectedMS patients who would presumably havehad normal results in the oddball tasks.Using Sternberg’s memory-scanning task,Pelosi et al. (1997) found that the 8 patients(out of 24) who had somewhat lower scoresin neuropsychological tests of workingmemory had enhanced negativity viz.reduced positivity from about 200 mseconward in their responses to memoryprobes. According to Pelosi et al., this mightreflect the extra resources required to com-pensate for deficits in working memory.Effects were more marked with auditorystimuli, because demands to phonologicalworking memory were probably higher. In atask requiring visual–phonological workingmemory for nonsense words, Ruchkin et al.(1994) found that slow waves during theretention interval were larger at leftfrontaland midposterior sites in patients than incontrol subjects when stimuli were moder-ately difficult to remember (three syllables).The slow waves of healthy subjects reachedthese amplitudes only when stimuli werehard to remember (five syllables), whereasthe increase in load from three to five sylla-bles affected patients’ slow waves much less,possibly because these amplitudes werealready at their ceiling. The effect was taskspecific and were not present in a taskrequiring visual–spatial memory. Theseresults may be interpreted as evidence for aspecific deficit of MS patients in phonologi-cal memory, compensated by the patients inthe case of the moderately difficult task byenhanced effort in verbal (leftfrontal) andvisuospatial (midposterior) domains. Thus,using rather different paradigms andmethods, these two studies lead to conver-gent conclusions about specific problems ofworking memory even in relatively healthyMS patients.

HIV Infection

The human immunodeficiency virus(HIV) may affect the central nervoussystem and may cause cognitive impair-

ment (HIV-associated dementia), as part ofthe acquired immunodeficiency syndrome(AIDS). Because a principal symptom ofthis cognitive impairment is slowing ofcognition, the measurement of ERP latencydelays as an indicator of cognitive slowingmakes much sense.

Goodin et al. (1990) reported that N1,N2, or P3 latencies were delayed in theauditory oddball task in demented AIDSpatients. Delayed latencies in demented orseverely symptomatic patients were gener-ally replicated by later studies, subtle dif-ferences notwithstanding (e.g., Goodwin etal., 1990; Ollo et al., 1991; Messenheimer etal., 1992; Arendt et al., 1993; Baldeweg et al.,1993; though see Connolly et al., 1994), andconsequently could be used as markers ofmedication effects in a longitudinal study(Evers et al., 1998). Yet, of much interest isthe question whether ERPs sensitivelymeasure damage to the central nervoussystem in asymptomatic HIV-infectedpersons. This was indeed reported byGoodin et al. (1990); 12 of 41 asymptomaticmen infected with HIV had prolongedlatencies in the auditory oddball task,making ERPs more sensitive than the stan-dard EEG. Evidence in support of this orig-inal finding is mixed: Schroeder et al. (1994)confirmed the delay of P3 latencies, andMessenheimer et al. (1992) similarly foundincreases of P3 latencies when measure-ments were repeated 6 or 12 months afterinfection. However, no abnormalities werereported by Goodwin et al. (1990), Ollo etal. (1991), and Connolly et al. (1994). Arendtet al. (1993) did not find delayed latenciesbut found reduced N2–P3 amplitudes(whether due to reduced N2 or reduced P3cannot be learned from the publishedpaper). Baldeweg et al. (1993) founddelayed P2 but not delayed P3 latenciesand a general tendency of N2 and P3waves to be less clearly defined. Bungeneret al. (1996) reported delayed N1 latenciesbut not delayed P3 latencies.

To overcome this dissatisfactory state of incongruent findings, other stimuli and



tasks have been used. Ollo et al. (1991)reported delayed and reduced P300 com-ponents with visual stimuli, in contrast totheir auditory data. However, this was notreplicated by Baldeweg et al. (1993), whofound the previously mentioned abnormal-ities with auditory stimuli but not withvisual ones. Presenting unique novel non-targets in addition to standard nontargetsand targets, Fein et al. (1995) measured theanteriorly distributed novelty P3 andfound this to be delayed with auditorystimuli, with the delay increasing whenpatients were symptomatic, possiblyspecifically reflecting delayed function ofthe frontal lobe [cf. Knight (1984), for asso-ciation of novelty P3 to the frontal lobe].Finally, in a pilot study on only 13 patients,Linnville and Elliott (1997) reportedsmaller mismatch negativity in HIV-infected individuals; these results wouldalso be compatible with disturbed frontallobe functioning, in view of the evidencefor the involvement of a frontal lobe com-ponent of MMN (Deouell et al., 1998). Boththis latter result and the findings made byFein et al. (1995) need replication.

In conclusion, it appears as if, by overre-liance on the unspecific oddball task, theERP method has missed the chance ofplaying a more important role in HIVresearch. Had more specific tasks and mea-sures been used, results would perhapshave been more consistent. A complicatingfactor is data quality. Many of the studieswere performed by groups not renownedin ERP research and/or were published injournals whose emphasis is not on qualityof ERP data, such that data quality simplycannot be judged from the publishedreports.

EPILEPSY

Epileptic seizures are unspecific symp-toms of cerebral dysfunction. The seizuremight be focal, or at least start focally,reflecting an circumscribed pathology, or

alternatively might be generalized, i.e.,without an identifiable focus.

ERP research has dealt with three topicsin epilepsy: (1) whether a history of epilep-tic seizures (in combination with anti-epileptic drugs) leads to cognitive slowingand to other deficits of cognitive functions,(2) whether intracranial recordings fromthe hippocampi can predict memory per-formance after surgical removal of theepileptic focus, and (3) whether seizurescan be reduced by biofeedback of event-related slow waves.

Cognitive Slowing and Impairments of Memory

Not surprisingly, the issue of cognitiveslowing has been investigated by measur-ing delays of components evoked in theauditory oddball task. In patients withfocal seizures, delayed P3 latencies werefound by Puce et al. (1989; not explicitlyreported, but evident from comparingtheir Figs. 3a and 4b), by Triantafyllou et al.(1992b), and Fukai et al. (1990) for patientswith temporal lobe seizures; by Drake et al.(1986) for a group of patients with differ-ent epileptic foci, and by Rodin et al. (1989)for patients with different types of epilep-sies. Drake et al. (1986) and Triantafyllou etal. (1992b) also reported delayed N2 laten-cies. Moreover, Puce et al. (1994) reportedincreased variability of the time point ofthe P3 peak measured across single trials.In apparent contrast to these studies,Verleger et al. (1997) did not findsignificant latency delays but did find, inagreement with the general trend of thesestudies, that the duration of epilepsy wasrelated to P3 latency, i.e., the longer apatient suffered from seizures the moredelayed was P3.

In studying patients with idiopathic,generalized epilepsy (IGE) Triantafyllou etal. (1992b) found that these patients dif-fered less than focal-seizure patients fromcontrol subjects, and Fukai et al. (1990)found no difference at all between IGE

EPILEPSY 329


patients and control subjects. In contrastagain, Verleger et al. (1997) found that thesepatients had already slightly delayed N1peaks, and more and more delayed N2 andP3 peaks.

In general, the duration of epilepticsymptoms was reflected in latency delaysof ERP components. Puce et al. (1989) sug-gested that brains suffering from recurringseizures might be prone to advanced elec-trophysiological aging [be it due toepilepsy or to antiepileptic medication(Kubota et al., 1998)] and, indeed, we foundthe patterns of delays strikingly similarbetween IGE patients compared to healthyage-matched controls (Verleger et al., 1997)and healthy elderly compared to youngadults (Verleger et al., 1991).

A serendipitous finding in the visualgo/no-go task used by Verleger et al. (1997)was a massive, isolated delay of the poste-rior N2, by 40 msec, in patients withtemporal foci. Likewise, in patients studiedby Smith and Halgren (1988), a negativecomplex evoked by visual stimuli, seenfrom 180 to 400 msec in healthy subjects at posterior sites, was either reduced orcompletely absent. In these patients, anterior temporal lobes were surgicallyremoved, and were not simply functionallydamaged, as was the case for patients ofVerleger et al., which may account for themore drastic difference from normals inthis negative component. Unfortunately,other studies using visual stimuli inpatients with temporal lobe epilepsy didnot report data from posterior sites(Johnson, 1989; Nelson et al., 1991; Rugg et al., 1991; Scheffers et al., 1991). Thus,there is only limited evidence that an intacttemporal lobe appears to be necessary forthe elicitation of the posterior N2. BecauseN2 is probably generated in the occipitallobe [cf. Luck et al. (1997) on N2pc], this isone of the few neurophysiological resultsin humans to support the notion of “reentrant processing” (Di Lollo et al.,2000), i.e., the notion that visual informa-tion has to pass a second time through

occipital areas in order to be fullyprocessed.

Impaired memory in patients with tem-poral lobe epilepsy was investigated in afew studies by scalp-recorded ERPs. Smithand Halgren (1989) studied patients whoseanterior right or left temporal lobe wasremoved, in a choice–response task inwhich participants had to distinguishbetween words presented previously andnew words. “Recognition positivity”evoked by identified old words wasmissing in the left-resected patients butremained unchanged in the right-resectedpatients. This might be a specific indicatorof the patients’ memory impairment [asinterpreted by Smith and Halgren (1989)]or might reflect the patients’ ambiguitywhen presented with words that theycould not classify with certainty [i.e., itmight be an instance of the general reduc-tion of P3 by “equivocation” (Ruchkin andSutton, 1978)].

Rugg et al. (1991) used a similar thoughnot identical task: the task was continuous,i.e., newly presented words could be pre-sented a second time during later trials. Inthis task, repetition-related positivity wasmissing not only in left-resected patientsbut also in right-resected patients.Furthermore, in groups of presurgicalpatients with left or right foci, positivitywas unilaterally reduced on the side of thefocus. These reductions or completeabsence of positivity must be related toconscious recollection because the same“priming positivity” as in the controlgroup was evoked by repeated words inanother task, where the fact that a wordhad been previously presented was nottask relevant. On the other hand, thereductions were unspecific insofar as theydid not correlate with response times, errorrate, and a verbal memory test.

Measuring potentials evoked by probesin an auditory version of Sternberg’smemory-scanning task, Grippo et al. (1996)found the N1 peak as well as a late positiveshift to be generally reduced in patients



with temporal lobe epilepsy. Specific to asubgroup of patients with worse results inneuropsychological memory tests weredelays of P250 and delays and reductionsof the following N290.

In conclusion, ERPs have proved to besensitive indicators of memory deficits inepileptic patients, but the specificity of thefindings has remained somewhat am-biguous. Good progress in the issue ofmemory specificity has been achieved instudies using intracranial recordings, as isdescribed next.

Intracranial Recordings

In patients with intractable epilepsy, forwhom surgery is chosen as treatment, elec-trodes are routinely inserted within theskull to gain more information about thelocation of the tissue to be removed andabout consequences of the surgery. Thishas proved to be a lucky accident for basicresearch in ERPs and in neuroscience ingeneral because information can be gainedfrom within the skull, beyond the ERPcomponents recorded from the scalp (e.g.,Nobre et al., 1998; Rektor et al., 1998; Clarkeet al., 1999; Yazawa et al., 2000). Of moreinterest in the present context, ERPsrecorded by means of such intracranialelectrodes have been shown to provideimportant information about treatmentoutcomes.

Grunwald et al. (1995) established sensi-tivity of an N400 and an N800, recorded inanterior and posterior hippocampus,respectively, to recognition of repeatedwords in the continuous-recognition taskas used, for example, by Rugg et al. (1991)[cf. preceeding discussion; see also similarmethods and results by Guillem and col-leagues (e.g., Guillem et al., 1999)].

Subsequent work (Elger et al., 1997)made clear that the focus of the N400 wasanterior to the hippocampus, within theanterior medial temporal lobe; ano-ther focus was situated near Wernicke’sarea, in the left middle temporal gyrus.

Amplitudes of this latter left-sided N400 inthe continuous-recognition task correlatedwith the number of words immediatelyrecalled in an auditory verbal learning test,whereas amplitudes of the former, ante-rior-temporal N400 correlated withnumber of words recalled in auditoryverbal learning after a 30-min delay, alsoonly in recordings from the left side, notfrom the right side. (All reported patientshad left-sided language dominance).However, in those patients who under-went left-sided resection either of the ante-rior-temporal lobe or of the hippocampus,their right-sided anterior-temporal N400recorded before surgery was related to thenumber of words recalled after 30 min inpostsurgery verbal learning (Grunwald etal., 1998). That is, N400 recorded from thenondominant hemisphere allowed predic-tion of how much this nondominant hemi-sphere would compensate for the functionof the resected hemisphere. Moreover,reduction of the anterior-temporal N400 onrepetition of words on the side contra-lateral to the hemisphere to be removedpredicted whether the patients wouldindeed become seizure free postsurgery(Grunwald et al., 1999), i.e., according tothe authors, the lack of repetition effect inthe hemisphere assumed to be healthyindicated that this hemisphere was like-wise damaged.

Biofeedback

Event-related potentials have been usedas a therapeutic means in neurologicalpatients. Epileptic patients who are notseizure free even after careful, repeatedvariations of drug treatment may be ableto reduce their seizure rate by learning tocontrol their slow negative potentials.Birbaumer’s group showed this in twoindependent studies [the first one reportedin Birbaumer et al. (1991), and Rockstroh etal. (1993), and the second one reported inKotchoubey et al. (1996, 1997, 1999, 2001)].Specifically, patients had to control the

EPILEPSY 331


movement of a target object on the screenduring a 6-sec period. The object was in factcontrolled by the amount of Cz-recordednegativity (with EOG transmission beingrejected), with more or less negativitymoving the object to one or the other goal.Patients had to learn this game in manytraining sessions and were expected to beable to use this skill to control excess amountof cortical excitation. In fact, patients neededmany more sessions compared to healthypeople to learn this skill, probably related totheir problems in maintaining control of cor-tical excitation but perhaps also related tothe fact that most healthy people investi-gated in this group’s basic research (reviewin Rockstroh et al., 1989) had higher educa-tion and were not distracted by worryingabout their health problems. In favor of thedisease-specific account is the finding thatthe absolute amount of negativity producedby patients during the first phase of biofeed-back training was a negative predictor forthe effectiveness of the training in reducingseizures (Kotchoubey et al., 1999). Onaverage, number of seizures could indeed bereduced by this training. A control conditionin the initial report (Birbaumer et al., 1991)was biofeedback of alpha wave amplitudes,and two control conditions in the mostrecent report (Kotchoubey et al., 2001) weremodifications of the drug regimen during a 6-week hospital stay and learning of arespiration technique to avoid hyper-ventilation in as many training sessions (n =35) as with biofeedback. Biofeedback of slownegativity was shown to be more effectivethan alpha biofeedback and than the res-piration technique and as effective asmodification of the drug regimen. Whetherthis method will find wider applicationremains to be seen.

In summary, as evidenced by theadvanced methods in intracranial record-ings and in the therapeutic applica-tion just described, ERPs certainly con-tinue being a fruitful and relevant method in epilepsy research and treat-ment.

CONCLUDING REMARKS

The discussions here have outlined thestrengths and drawbacks of the use ofevent-related potentials in research on neu-rological patients. Not all areas could becovered, most notably migraine andheadache syndromes (due to this author’slack of knowledge) and those neurologicalsyndromes less often encountered in neu-rology than in neurosurgery (such asclosed-head injury and coma) or in internalmedicine (such as transient global ischemiaand renal encephalopathy). These draw-backs notwithstanding, it is hoped that thepresent chapter serves as a useful guide forfurther research. There is still room forbetter application of ERPs to the three pur-poses emphasized at the beginning of thischapter, i.e., diagnosis and prognosis,understanding consequences of diseaseson cognition, and therapy and rehabilita-tion.

Acknowledgments

I would like to thank my medical col-leagues at our department of neurology;they taught me, a psychologist whorecords potentials from the scalp, to under-stand what the brain actually looks like.Thanks, in particular, to Detlef Kömpf,head of the department, and WolfgangHeide, Andreas Moser, Matthias Nitschke,Peter Vieregge, Clemens Vollmer, BerndWauschkuhn, and Karl Wessel.

ReferencesAaltonen, O., Tuomainen, J., Laine, M., and Niemi, P.

(1993). Cortical differences in tonal versus vowelprocessing as revealed by an ERP componentcalled mismatch negativity (MMN). Brain Lang. 44,139–152.

Akshoomoff, N. A., and Courchesne, E. (1994). ERPevidence for a shifting attention deficit in patientswith damage to the cerebellum. J. Cogn. Neurosci.6, 388–399.

Altenmüller, E. (1989). Cortical DC-potentials aselectrophysiological correlates of hemispheric



dominance of higher cognitive functions. Int. J.Neurosci. 47, 1–15.

Angelelli, P., De Luca, M., and Spinelli, D. (1996).Early visual processing in neglect patients: a studywith steady-state VEPs. Neuropsychologia 34,1151–1157.

Arendt, G., Hefter, H., and Jablonowski, H. (1993).Acoustically evoked event-related potentials inHIV-associated dementia. Electroencephalogr. Clin.Neurophysiol. 86, 152–160.

Baldeweg, T., Gruzelier, J. H., Catalan, J., Pugh, K.,Lovett, E., Riccio, M., Stygall, J., Irving, G., Catt, S.,and Hawkins, D. (1993). Auditory and visualevent-related potentials in a controlled investiga-tion of HIV infection. Electroencephalogr. Clin.Neurophysiol. 88, 356–368.

Barrett, G., Shibasaki, H., and Neshige, R. (1986).Cortical potential shifts preceding voluntarymovement are normal in Parkinsonism.Electroencephalogr. Clin. Neurophysiol. 63, 340–348.

Birbaumer, N., Elbert, T., Rockstroh, B., Daum, I.,Wolf, P., and Canavan, A. (1991). Clinical-psycho-logical treatment of epileptic seizures: a controlledstudy. In “Perspectives and Promises of ClinicalPsychology” (A. Ehlers, ed.), pp. 81–96. PlenumPress, New York.

Birbaumer, N., Ghanayim, N., Hinterberger, T.,Iversen, I., Kotchoubey, B., Kübler, A.,Perelmouter, J., Taub, E., and Flor, H. (1999). Aspelling device for the paralysed. Nature 398,297–298.

Birbaumer, N., Kübler, A., Ghanayim, N.,Hinterberger, T., Perelmouter, J., Kaiser, J., Iversen,I., Kotchoubey, B., Neumann, N. and Flor, H.(2000). The Thought Translation Device (TTD) forcompletely paralyzed patients. IEEE Trans. Rehab.Eng. 8, 190–193.

Bungener, C., Le Houezec, J. L., Pierson, A., andJouvent, R. (1996). Cognitive and emotionaldeficits in early stages of HIV infection: An event-related potentials study. Prog. Neuropsycho-pharmacol. Biol. Psychiatr. 20, 1303–1314.

Castañeda, M., Ostrosky-Solis, F., Pérez, M., Bobes, M.A., and Rangel, L. E. (1997). ERP assessment ofsemantic memory in Alzheimer’s disease. Int. J.Psychophysiol. 27, 201–214.

Clarke, J. M., Halgren, E., and Chauvel, P. (1999).Intracranial ERPs in humans during a lateralizedvisual oddball task: I. Occipital and peri-rolandicrecordings. Clin. Neurophysiol. 110, 1210–1225.

Cobianchi, A., and Giaquinto, S. (2000). Can weexploit event-related potentials for retraininglanguage after stroke? Disability Rehab. 15, 427–434.

Cohen, R., Dobel, C., Berg, P., Koebbel, P., Schönle, P.-W., and Rockstroh, B. (2001) Event-relatedpotential correlates of verbal and pictorial featurecomparison in aphasics and controls. Neuropsycho-logia 39, 489–501.

Coles, M. G. H. (1989). Modern mind-brain reading:psychophysiology, physiology, and cognition.Psychophysiology 26, 251–269.

Connolly, S., Manji, H., McAllister, R. H., Fell, M.,Loveday, C., Kirkis, C., Herns, M., Sweeney, B.,Sartawi, O., Durrance, P., Griffin, G. B., Boland, M.,Fowler, C. J., Newman, S. P., Weller, I. V. D., andHarrison, M. J. G. (1994). Long-latency event-related potentials in asymptomatic humanimmunodeficiency. Ann. Neurol. 35, 189–196.

Connolly, J. F., Mate-Kole, C. C., and Joyce, B. M.(1999). Global aphasia: An innovative assessmentapproach. Arch. Physical Med. Rehab. 80, 1309–1315.

Csépe, V., Osman-Sági, J., Molnár, M., and Gósy, M.(2001). Impaired speech perception in aphasicpatients: Event-related potential and neuropsy-chological assessment. Neuropsychologia 39,1194–1208.

Cunnington, R., Iansek, R., Bradshaw, J. L., andPhillips, J. G. (1995). Movement-related potentialsin Parkinson’s disease: Presence and predictabil-ity of temporal and spatial cues. Brain 118,935–950.

Cunnington, R., Iansek, R., Johnson, K. A., andBradshaw, J. L. (1997). Movement-related poten-tials in Parkinson’s disease: Motor imagery andmovement preparation. Brain 120, 1339–1353.

Cunnington, R., Iansek, R., and Bradshaw, J. L. (1999).Movement-related potentials in Parkinson’sdisease: External cues and attentional strategies.Movement Disorders 14, 63–68.

Cunnington, R., Lalouschek, W., Dirnberger, G.,Walla, P., Lindinger, G., Asenbaum, S., Brücke, T.,and Lang, W. (2001). A medial to lateral shift inpre-movement cortical activity in hemi-Parkinson’s disease. Clin. Neurophysiol. 112,608–618.

Daum, I., Schugens, M. M., Ackermann, H.,Lutzenberger, W., Dichgans, J., and Birbaumer, N.(1993). Classical conditioning after cerebellarlesions in humans. Behav. Neurosci. 107, 748–756.

Deouell, L. Y., Bentin, S., and Giard, M.-H. (1998).Mismatch negativity in dichotic listening:Evidence for interhemispheric differences andmultiple generators. Psychophysiology 35, 355–365.

Deouell, L. Y., Hämäläinen, H., and Bentin, S. (2000a).Unilateral neglect after right hemisphere damage:Contributions from event-related potentials.Audiol. Neuro-Otol. 5, 225–234.

Deouell, L. Y., Bentin, S., and Soroker, N. (2000b).Electrophysiological evidence for an early (pre-attentive) information processing deficit inpatients with right hemisphere damage and uni-lateral neglect. Brain 123, 353–365.

De Toledo-Morrell, L., Evers, S., Hoeppner, T. J.,Morrell, F., Garron, D. C., and Fox, J. H. (1991). A‘stress’ test for memory dysfunction. Electro-physiologic manifestations of early Alzheimer’sdisease. Arch. Neurol. 48, 605–609.

REFERENCES 333


Dick, J. P. R., Rothwell, J. C., Day, B. L., Cantello, R.,Buruma, O., Gioux, M., Benecke, R., Berardelli, A.,Thompson, P. D., and Marsden, C. D. (1989). TheBereitschaftspotential is abnormal in Parkinson’sdisease. Brain 112, 233–244.

Di Lollo, V., Enns, J. T., and Rensink, R. A. (2000).Competition for consciousness among visualevents: The psychophysics of reentrant visualprocesses. J. Exp. Psychol. Gen. 129, 481–507.

Dobel, C., Pulvermüller, F., Härle, M., Cohen, R.,Köbbel, P., Schönle, P. W., and Rockstroh, B. (2001).Syntactic and semantic processing in the healthyand aphasic human brain. Exp. Brain Res. 140,77–85.

Drake, M. E., Jr., Burgess, R. J., Gelety, T. J., Ford, C. E.and Brown, M. E. (1986). Long-latency auditoryevent-related potentials in epilepsy. Clin.Electroencephalogr. 17, 10–13.

Driver, J., and Vuilleumier, P. (2001). Perceptualawareness and its loss in unilateral neglect andextinction. Cognition 79, 39–88.

Ebmeier, K. P. (1992). A quantitative method for theassessment of overall effects from a number ofsimilar electrophysiological studies: Descriptionand application to event-related potentials inParkinson’s disease. Electroencephalogr. Clin. Neuro-physiol. 84, 440–446.

Eichenbaum, H. (2000). A cortical-hippocampalsystem for declarative memory. Nat. Rev. Neurosci.1, 41–50.

Eimer, M., and McCarthy, R. A. (1999). Prosopagnosiaand structural encoding of faces: Evidence fromevent-related potentials. NeuroReport 10, 255–259.

Elger, C. E., Grunwald, T., Lehnertz, K., Kutas, M.,Helmstaedter, C., Brockhaus, A., van Roost, D.,and Heinze, H. J. (1997). Human temporal lobepotentials in verbal learning and memoryprocesses. Neuropsychologia 35, 657–667.

Evers, S., Grotemeyer, K. H., Reichelt, D., Luttmann,S., and Husstedt, I. W. (1998). Impact of antiretro-viral treatment on AIDS dementia: A longitudinalprospective event-related potential study. J.Acquired Immune Def. Syndr. Hum. Retrovirol. 17,143–148.

Falkenstein, M., Hoormann, J., Christ, S., andHohnsbein, J. (2000). ERP components on reactionerrors and their functional significance: A tutorial.Biol. Psychol. 51, 87–107.

Falkenstein, M., Hielscher, H., Dziobek, I.,Schwarzenau, P., Hoormann, J., Sundermann, B.,and Hohnsbein, J. (2001). Action monitoring, errordetection, and the basal ganglia: An ERP study.NeuroReport 12, 157–161.

Farwell, L. A., and Donchin, E. (1988). Talking off thetop of your head: Toward a mental prosthesisutilizing event-related brain potentials. Electro-encephalogr. Clin. Neurophysiol. 70, 510–523.

Fattaposta, F., Pierelli, F., Traversa, G., My, F.,Mostarda, M., D’Alessio, C., Soldati, G., Osborn, J.,

and Amabile, G. (2000). Preprogramming andcontrol activity of bimanual self-paced motor taskin Parkinson’s disease. Clin. Neurophysiol. 111,873–883.

Fein, G., Biggins, C. A., and MacKay, S. (1995).Delayed latency of the event-related brain poten-tial P3A component in HIV disease. Arch. Neurol.52, 1109–1118.

Filipovic, S., Kostic, V. S., Sternic, N., Marinkovic, Z.,and Ocic, G. (1990). Auditory event-related poten-tials in different types of dementia. Eur. Neurol. 30,189–193.

Filipovic, S., Drulovic, J., Stojsavljevic, N., and Levic,Z. (1997). The effects of high-dose intravenousmethylprednisolone on event-related potentials inpatients with multiple sclerosis. J. Neurol. Sci. 152,147–153.

Ford, J. M., Woodward, S. H., Sullivan, E. H., Isaacks,B. G., Tinklenberg, J. R., Yesavage, J. A., and Roth,W. T. (1996). N400 evidence of abnormal responsesto speech in Alzheimer’s disease. Electro-encephalogr. Clin. Neurophysiol. 99, 235–246.

Friederici, A. D., Pfeifer, E., and Hahne, A. (1993).Event-related brain potentials during naturalspeech processing: Effects of semantic, morpholog-ical and syntactic violations. Cogn. Brain Res. 1,183–192.

Friederici, A. D., von Cramon, D. Y., and Kotz, S. A.(1999). Language related brain potentials inpatients with cortical and subcortical left hemi-sphere lesions. Brain 122, 1033–1047.

Frodl, T., Hampel, H., Juckel, G., Bürger, K., Padberg,F., Engel, R. R., Möller, H.-J., and Hegerl, U. (2002).Value of event-related P300 subcomponents in theclinical diagnosis of mild cognitive impairmentand Alzheimer´s disease? Psychophysiology 39,175–181.

Fukai, M., Motomura, N., Kobayashi, S., Asaba, H.,and Sakai, T. (1990). Event-related potential (P300)in epilepsy. Acta Neurol. Scand. 82, 197–202.

Gaeta, H., Friedman, D., Ritter, W., and Cheng, J.(1999). Changes in sensitivity to stimulus deviancein Alzheimer’s disease: An ERP perspective.NeuroReport 10, 281–287.

Gerloff, C., Altenmüller, E., and Dichgans, J. (1996).Disintegration and reorganization of corticalmotor processing in two patients with cerebellarstroke. Electroencephalogr. Clin. Neurophysiol. 98,59–68.

Gerschlager, W., Alesch, F., Cunnington, R., Deecke,L., Dirnberger, G., Endl, W., Lindinger, G., andLang, W. (1999). Bilateral subthalamic nucleusstimulation improves frontal cortex function inParkinson’s disease. An electrophysiological studyof the contingent negative variation. Brain 122,2365–2373.

Gerschlager, W., Beisteiner, R., Deecke, L., Dirnberger,G., Endl, W., Kollegger, H., Lindinger, G., Vass, K.,and Lang, W. (2000). Electrophysiological, neuro-



psychological and clinical findings in multiplesclerosis patients receiving interferon -1b: A 1-year follow-up. Eur. Neurol. 44, 205–209.

Giesser, B. S., Schroeder, M. M., LaRocca, N. G.,Kurtzberg, D., Ritter, W., Vaughan, H. G., andScheinberg, L. C. (1992). Endogenous event-relatedpotentials as indices of dementia in multiple scle-rosis patients. Electroencephalogr. Clin. Neurophysiol.82, 320–329.

Gil, R., Zai, L., Neau, J. P., Jonveaux, T., Agbo, C.,Rosolacci, T., Burbaud, P., and Ingrand, P. (1993).Event-related auditory evoked potentials and mul-tiple sclerosis. Electroencephalogr. Clin. Neurophysiol.88, 182–187.

Gil, R., Neau, J. P., Dary-Auriol, M., Agbo, C., Tantot,A. M., and Ingrand, P. (1995). Event-related audi-tory evoked potentials and amyotrophic lateralsclerosis. Arch. Neurol. 52, 890–896.

Goodin, D. S. (1990). Clinical utility of long latency‘cognitive’ event-related potentials (P3): The pros.Electroencephalogr. Clin. Neurophysiol. 76, 2–5.

Goodin, D. S., and Aminoff, M. J. (1986). Electro-physiological differences between subtypes ofdementia. Brain 109, 1103–1113.

Goodin, D. S., Squires, K. C., and Starr, A. (1978).Long latency event-related components of theauditory evoked potential in dementia. Brain 101,635–648.

Goodin, D. S., Aminoff, M. J., Chernoff, D. N., andHollander, H. (1990). Long latency event-relatedpotentials in patients infected with humanimmunodeficiency virus. Ann. Neurol. 27, 414–419.

Goodwin, G. M., Chiswick, A., Egan, V., St.Clair, D.,and Brettle, R. P. (1990). The Edinburgh cohort ofHIV-positive drug users: Auditory event-relatedpotentials show progressive slowing in patientswith Centers for Disease Control stage IV disease.AIDS 4, 1243–1250.

Gratton, G., Coles, M. G. H., Sirevaag, E. J., Eriksen, C. W., and Donchin, E. (1988). Pre- and post-stimulus activation of response channels: Apsychophysiological analysis. J. Exp. Psychol. Hum.Perception Performance 14, 331–344.

Green, J., and Levey, A. J. (1999). Event-related potentialchanges in groups at increased risk for Alzheimerdisease. Arch. Neurol. 56, 1398–1403.

Green, J. B., Bialy, Y., Sora, E., and Ricamato, A. (1999).High-resolution EEG in poststroke hemiparesiscan identify ipsilateral generators during motortasks. Stroke 30, 2659–2665.

Grippo, A., Pelosi, L., Mehta, V., and Blumhardt, L. D.(1996). Working memory in temporal lobeepilepsy: An event-related potential study. Electro-encephalogr. Clin. Neurophysiol. 99, 200–213.

Grunwald, T., Elger, C. E., Lehnertz, K., van Roost, D.,and Heinze, H.J. (1995). Alterations of intrahip-pocampal cognitive potentials in temporal lobeepilepsy. Electroencephalogr. Clin. Neurophysiol. 95,53–62.

Grunwald, T., Lehnertz, K., Helmstaedter, C., Kutas,M., Pezer, N., Kurthen, M., Van Roost, D., andElger, C. E. (1998). Limbic ERPs predict verbalmemory after left-sided hippocampectomy.NeuroReport 9, 3375–3378.

Grunwald, T., Lehnertz, K., Pezer, N., Kurthen, M.,Van Roost, D., Schramm, J., and Elger, C. E. (1999).Prediction of postoperative seizure control by hip-pocampal event-related potentials. Epilepsia 40,303–306.

Guillem, F., Rougier, A., and Claverie, B. (1999). Short-and long-delay intracranial ERP repetition effectsdissociate memory systems in the human brain.J. Cogn. Neurosci. 11, 437–458.

Hagoort, P., Brown, C. M., and Swaab, T. Y. (1996)Lexical-semantic event-related potential effects inpatients with left hemisphere lesions and aphasia,and patients with right hemisphere lesionswithout aphasia. Brain 119, 627–649.

Hahne, A., and Friederici, A. D. (1999). Electro-physiological evidence for two steps in syntacticanalysis: early automatic and late controlledprocesses. J. Cogn. Neurosci. 11, 194–205.

Halgren, E., Baudena, P., Clarke, J. M., Heit, G.,Marinkovic, K., Devaux, B., Vignal, J.-P., andBiraben, A. (1995). Intracerebral potentials to raretarget and distractor auditory and visual stimuli.II. Medial, lateral and posterior temporal lobe. Electroencephalogr. Clin. Neurophysiol. 94,229–250.

Hansch, E. C., Syndulko, K., Cohen, S. N., Goldberg,Z. I., Potvin, A. R., and Tourtellotte, W.W. (1982).Cognition in Parkinson disease: An event-relatedpotential perspective. Ann. Neurol. 11, 599–607.

Hansen, J. C., and Hillyard, S. A. (1980). Endogenousbrain potentials associated with selective auditoryattention. Electroencephalogr. Clin. Neurophysiol. 49,277–290.

Hawking, S. (1989) “Eine kurze Geschichte der Zeit.”Rowohlt, Hamburg. [Originally published inEnglish (1988). “A Brief History of Time.” BantamBooks, New York.]

Hillyard, S. A., Hink, R. F., Schwent, V. L., and Picton,T. W. (1973). Electrical signs of selective attentionin the human brain. Science 182, 177–180.

Hömberg, V., Hefter, H., Granseyer, G., Strauss, W.,Lange, H., and Hennerici, M. (1986). Event-relatedpotentials in patients with Huntington’s diseaseand relatives at risk in relation to detailed psy-chometry. Electroencephalogr. Clin. Neurophysiol. 63,552–569.

Honig, L. S., Ramsay, R. E., and Sheremata, W. A.(1992). Event-related potential P300 in multiplesclerosis. Relation to magnetic resonance imagingand cognitive impairment. Arch. Neurol. 49, 44–50.

Ikeda, A., Shibasaki, H., Nagamine, T., Terada, K.,Kaji, R., Fukuyama, H., and Kimura, J. (1994).Dissociation between contingent negative varia-tion and Bereitschaftspotential in a patient with

REFERENCES 335


cerebellar efferent lesion. Electroencephalogr. Clin.Neurophysiol. 90, 359–364.

Ikeda, A., Shibasaki, H., Kaji, R., Terada, K.,Nagamine, T., Honda, M., and Kimura, J. (1997).Dissociation between contingent negative varia-tion (CNV) and Bereitschaftspotential (BP) inpatients with parkinsonism. Electroencephalogr.Clin. Neurophysiol. 102, 142–151.

Ilvonen, T.-M., Kujala, T., Tervaniemi, M., Salonen, O.,Näätänen, R., and Pekkonen, E. (2001). The pro-cessing of sound duration after left-hemisphericstroke: Event-related potential and behavioral evi-dence. Psychophysiology 38, 622–628.

Jahanshahi, M., Jenkins, I. H., Brown, R. G., Marsden,C. D., Passingham, R. E., and Brooks, D. J. (1995).Self-initiated versus externally triggered move-ments. I. An investigation using measurement ofregional cerebral blood flow with PET and move-ment-related potentials in normal and Parkinson’sdisease subjects. Brain 118, 913–933.

Johnson, K. A., Cunnington, R., Iansek, R., Bradshaw,J. L., Georgiou, N., and Chiu, E. (2001). Movement-related potentials in Huntington’s disease:Movement preparation and execution. Exp. BrainRes. 138, 492–499.

Johnson, R., Jr. (1989). Auditory and visual P300s in temporal lobectomy patients: Evidence formodality-dependent generators. Psychophysiology26, 633–650.

Johnson, R., Jr. (1992) Event-related potential insightsinto progressive supranuclear palsy. In“Progressive Supranuclear Palsy: Clinical andResearch Approaches” (I. Litran and Y. Agid, eds.).Oxford University Press, New York.

Johnson, R., Jr., Litvan, I., and Grafman, J. (1991)Progressive supranuclear palsy: Altered sensoryprocessing leads to degraded cognition. Neurology41, 1257–1262.

Johnson, R., Jr., Kreiter, K., Zhu, J., and Russo, B.(1998). A spatio-temporal comparison of semanticand episodic cued recall and recognition usingevent-related brain potentials. Cogn. Brain Res. 7,119–136.

Karayanidis, F., Andrews, S., Ward, P. B., and Michie,P.T. (1995). ERP indices of auditory selectiveattention in aging and Parkinson’s disease.Psychophysiology 32, 335–350.

Kazmerski, V. A., and Friedman, D. (1997). Effect ofmultiple presentation of words on event-relatedpotential and reaction time repetition effects inAlzheimer’s patients and young and older con-trols. Neuropsychiatr. Neuropsychol. Behav. Neurol.10, 32–47.

Kazmerski, V. A., Friedman, D., and Hewitt, S. (1995).Event-related potential repetition effect inAlzheimer’s patients: Multiple repetition primingwith pictures. Aging Cogn. 2, 169–191.

Kazmerski, V. A., Friedman, D., and Ritter, W. (1997).Mismatch negativity during attend and ignore

conditions in Alzheimer’s disease. Biol. Psychiatr.42, 382–402.

Kerkhoff, G. (2001). Spatial neglect in humans. Prog.Neurobiol. 63, 1–27.

Kew, J. J. M., Leigh, P. N., Playford, E. D., Passingham,R. E., Goldstein, L. H., Frackowiak, R. S. J., andBrooks, D. J. (1993). Cortical function in amy-otrophic lateral sclerosis: A positron emissiontomography study. Brain 116, 655–680.

Kitamura, J. -I., Shibasaki, H., and Takeuchi, T. (1996).Cortical potentials preceding voluntary elbowmovement in recovered hemiparesis. Electro-encephalogr. Clin. Neurophysiol. 98, 149–156.

Kitamura, J.-I., Shibasaki, H., Terashi, A., andTashima, K. (1999). Cortical potentials precedingvoluntary finger movement in patients with focalcerebellar lesion. Clin. Neurophysiol. 110, 126–132.

Knight, R. T. (1984). Decreased response to novelstimuli after prefrontal lesions in man.Electroencephalogr. Clin. Neurophysiol. 59, 9–20.

Knight, R. T. (1996). Contribution of human hippo-campal region to novelty detection. Nature 383,256–259.

Kohlmetz, C., Altenmüller, E., Schuppert, M.,Wieringa, B. M., and Münte, T. F. (2001). Deficit inautomatic sound-change detection may underliesome music perception deficits after acute hemi-spheric stroke. Neuropsychologia 39, 1121–1124.

Kopp, B., Mattler, U., Goertz, R., and Rist, F. (1996).N2, P3 and the lateralized readiness potential in anogo task involving selective response priming.Electroencephalogr. Clin. Neurophysiol. 99, 19–27.

Kopp, B., Kunkel, A., Mühlnickel, W., Villringer, K.,Taub, E., and Flor, H. (1999). Plasticity in the motorsystem related to therapy-induced improve-ment of movement after stroke. NeuroReport 10,807–810.

Kotchoubey, B., Schneider, D., Schleichert, H., Strehl,U., Uhlmann, C., Blankenhorn, V., Fröscher, W.,and Birbaumer, N. (1996). Self-regulation of slowcortical potentials in epilepsy: A retrial with analy-sis of influencing factors. Epilepsy Res. 25, 269–276.

Kotchoubey, B., Blankenhorn, V., Fröscher, W., Strehl,U., and Birbaumer, N. (1997). Stability of corticalself-regulation in epilepsy patients. NeuroReport 8,1867–1870.

Kotchoubey, B., Strehl, U., Holzapfel, S., Blankenhorn,V., Fröscher, W., and Birbaumer, N. (1999).Negative potential shifts and the prediction of theoutcome of neurofeedback therapy in epilepsy.Clin. Neurophysiol. 110, 683–686.

Kotchoubey, B., Strehl, U., Uhlmann, C., Holzapfel, S.,König, M., Fröscher, W., Blankenhorn, V., andBirbaumer, N. (2001). Modification of slow corticalpotentials in patients with refractory epilepsy: Acontrolled outcome study. Epilepsia 42, 406–416.

Kraiuhin, C., Gordon, E., Coyle, S., Sara, G., Rennie,C., Howson, A., Landau, P., and Meares, R. (1990).Normal latency of the P300 event-related potential



in mild-to-moderate Alzheimer’s disease anddepression. Biol. Psychiatry 28, 372–386.

Kübler, A., Kotchoubey, B., Hinterberger, T.,Ghanayim, N., Perelmouter, J., Schauer, M.,Fritsch, C., Taub, E., and Birbaumer, N. (1999). Thethought translation device: A neurophysiologicalapproach to communication in total motor paraly-sis. Exp. Brain Res. 124, 223–232.

Kübler, A., Kotchoubey, B., Kaiser, J., Wolpaw, J., andBirbaumer, N. (2001). Brain-computer communica-tion: Unlock the locked-in. Psychol. Bull. 127,358–375.

Kubota, F., Kifune, A., Shibata, N., Akata, T., Takeuchi,K., and Takahashi, S. (1998). Study on the P300 ofadult epileptic patients (unmedicated and med-icated patients). J. Epilepsy 11, 325–331.

Kutas, M., and Hillyard, S. A. (1980). Reading sense-less sentences: brain potentials reflect semanticincongruity. Science 207, 203–205.

Lhermitte, F., Turell, E., LeBrigand, D., and Chain, F.(1985). Unilateral visual neglect and Wave P 300.Arch. Neurol. 42, 567–573.

Linden, A., Bracke-Tolkmitt, R., Lutzenberger, W.,Canavan, A. G. M., Scholz, E., Diener, H.-C., andBirbaumer, N. (1990). Slow cortical potentials inParkinsonian patients during the course of anassociative learning task. J. Psychophysiol. 4,145–162.

Linnville, S. E., and Elliott, F. S. (1997). Mismatchnegativity: An index of subclinical neurologicaldifferences in HIV patients during rapid percep-tual processing. J. Neuropsychiatr. Clin. Neurosci. 9,45–54.

Looi, J. C. L., and Sachdev, P. S. (1999). Differentiationof vascular dementia from AD on neuropsycholog-ical tests. Neurology 53, 670–678.

Luck, S. J., and Hillyard, S. A. (1990). Electro-hysiological evidence for parallel and serial pro-cessing during visual search. Percept. Psychophys.48, 603–617.

Luck, S. J., Girelli, M., McDermott, M. T., and Ford, M.A. (1997). Bridging the gap between monkey neu-rophysiology and human perception: An ambigu-ity resolution theory of visual selective attention.Cogn. Psychol. 33, 64–87.

Lutzenberger, W., Elbert, T., and Rockstroh, B. (1987).A brief tutorial on the implications of volumeconduction for the interpretation of the EEG.J. Psychophysiol. 1, 81–89.

Marzi, C. A., Girelli, M., Miniussi, C., Smania, N., andMaravita, A. (2000). Electrophysiological cor-relates of conscious vision: Evidence from unilateral extinction. J. Cogn. Neurosci. 12, 869–877.

Messenheimer, J. A., Robertson, K. R., Wilkins, J. W.,Kalkowski, J. C., and Hall, C. D. (1992). Event-related potentials in human immunodeficiencyvirus infection. A prospective study. Arch. Neurol.49, 396–400.

Middleton, F. A., and Strick, P. (2001). Cerebellarprojections to the prefrontal cortex of the primate.J. Neurosci. 21, 700–712.

Minamoto, H., Tachibana, H., Sugita, M., and Okita,T. (2001). Recognition memory in normal agingand Parkinson’s disease: Behavioral and electro-physiological evidence. Cogn. Brain Res. 11, 23–32.

Münte, T. F., Ridao-Alonso, M. E., Preinfalk, J., Jung,A., Wieringa, B. M., Matzke, M., Dengler, R., andJohannes, S. (1997). An electrophysiological analy-sis of altered cognitive functions in Huntingtondisease. Arch. Neurol. 54, 1089–1098.

Münte, T. F., Tröger, M., Nusser, I., Wieringa, B. M.,Johannes, S., Matzke, M., and Dengler, R. (1998a).Alterations of early components of the visualevoked potential in amyotrophic lateral sclerosis.J. Neurol. 245, 206–210.

Münte, T. F., Tröger, M., Nusser, I., Wieringa, B. M.,Matzke, M., Johannes, S., and Dengler, R. (1998b).Recognition memory deficits in amyotrophiclateral sclerosis assessed with event-related brainpotentials. Acta Neurol. Scand. 98, 110–115.

Münte, T. F., Tröger, M. C., Nusser, I., Wieringa, B. M.,Matzke, M., Johannes, S., and Dengler, R. (1999).Abnormalities of visual search behaviour in ALSpatients detected with event-related brain poten-tials. Amyotroph. Lateral Scler. Other Motor NeuronDisord. 1, 21–27.

Näätänen, R. (1990). The role of attention in auditoryinformation processing as revealed by event-related potentials and other brain measures ofcognitive function. Behav. Brain Sci. 13, 201–288.

Näätänen, R., and Winkler, I. (1999). The concept ofauditory stimulus representation in cognitive neu-roscience. Psychol. Bull. 125, 826–859.

Nelson, C. A., Collins, P. F., and Torres, F. (1991). P300brain activity in seizure patients preceding tempo-ral lobectomy. Arch. Neurol. 48, 141–147.

Neshige, R., Barrett, G., and Shibasaki, H. (1988).Auditory long latency event-related potentials inAlzheimer’s disease and multi-infarct dementia.J. Neurol. Neurosurg. Psychiatr. 51, 1120–1125.

Newton, M. R., Barrett, G., Callanan, M. M., andTowell, A. D. (1989). Cognitive event-relatedpotentials in multiple sclerosis. Brain 112,1637–1660.

Nobre, A. C., Allison, T., and McCarthy, G. (1998).Modulation of human extrastriate visual process-ing by selective attention to colours and words.Brain 121, 1357–1368.

Oishi, M., Mochizuki, Y., Du, C., and Takasu, T.(1995). Contingent negative variation and move-ment-related cortical potentials in parkinson-ism. Electroencephalogr. Clin. Neurophysiol. 95,346–349.

Ollo, C., Johnson, R., Jr., and Grafman, J. (1991). Signsof cognitive change in HIV disease: An event-related brain potential study. Neurology 41,209–215.

REFERENCES 337


Onofrj, M., Fulgente, T., Nobilio, D., Malatesta, G.,Bazzano, S., Colamartino, P., and Gambi, D. (1992).P3 recordings in patients with bilateral temporallobe lesions. Neurology 42, 1762–1767.

Onofrj, M., Melchionda, D., Thomas, A., and Fulgente,T. (1996). Reappearance of event-related P3 poten-tial in locked-in syndrome. Cogn. Brain Res. 4,95–97.

Pekkonen, E., Jousmäki, V., Kononen, M.,Reinikainen, K., and Partanen, J. (1994). Auditorysensory memory impairment in Alzheimer’sdisease: An event-related potential study.NeuroReport 5, 2537–2540.

Pekkonen, E., Jousmäki, V., Reinikainen, K., andPartanen, J. (1995). Automatic auditory discrimina-tion is impaired in Parkinson’s disease.Electroencephalogr. Clin. Neurophysiol. 95, 47–52.

Pekkonen, E., Ahveninen, J., Virtanen, J., andTeräväinen, H. (1998). Parkinson’s disease selec-tively impairs preattentive auditory processing:An MEG study. NeuroReport 9, 2949–2952.

Pelosi, L., Geesken, J. M., Holly, M., Hayward, M., andBlumhardt, L. D. (1997). Working memory impair-ment in multiple sclerosis. Evidence from anevent-related potential study of patients with clini-cally isolated myelopathy. Brain 120, 2039–2058.

Pfefferbaum, A., Ford, J. M., Weller, B. J., and Kopell,B. S. (1985). ERPs to response production and inhi-bition. Electroencephalogr. Clin. Neurophysiol. 60,423–434.

Pfefferbaum, A., Ford, J. M., and Kraemer, H. C.(1990). Clinical utility of long latency ‘cognitive’event-related potentials (P3): The cons.Electroencephalogr. Clin. Neurophysiol. 76, 6–12.

Pfurtscheller, G., Flotzinger, D., and Kalcher, J. (1993).Brain-computer interface—A new communicationdevice for handicapped persons. J. MicrocomputerAppl. 16, 293–299.

Pierrot-Deseilligny, C., Turell, E., Penet, C., Lebrigand,D., Pillon, B., Chain, F., and Agid, Y. (1989).Increased wave P 300 latency in progressivesupranuclear palsy. J. Neurol. Neurosurg. Psychiatr.52, 656–658.

Pirtosek, Z., Jahanshahi, M., Barrett, G., and Lees, A. J.(2001). Attention and cognition in bradykinetic-rigid syndromes: An event-related potential study.Ann. Neurol. 50, 567–573.

Platz, T., Kim, I. H., Pintschovius, H., Winter, T.,Kieselbach, A., Villringer, K., Kurth, R., andMauritz, K.-H. (2000). Multimodal EEG analysis inman suggests impairment-specific changes inmovement-related electric brain activity afterstroke. Brain 123, 2475–2490.

Polich, J. (1991). P300 in the evaluation of aging anddementia. In “Event-Related Brain Research,Electroencephalography and Clinical Neuro-physiology” (C.H.M.Brunia, G.Mulder, andM. N. Verbaten, eds.), Suppl. 42, pp. 304–323.Elsevier, Amsterdam.

Polich, J., and Squire, L. R. (1993). P300 from am-nesic patients with bilateral hippocampal lesions.Electroencephalogr. Clin. Neurophysiol. 86, 408–417.

Polich, J., Ehlers, C. L., Otis, S., Mandell, A. J., andBloom, F. E. (1986). P300 latency reflects the degree of cognitive decline in dementing ill-ness. Electroencephalogr. Clin. Neurophysiol. 63, 138–144.

Posner, M. I., Walker, J. A., Friedrich, F. J., and Rafal,R. D. (1984). Effects of parietal injury on covertorienting of attention. J. Neurosci. 4, 1863–1874.

Praamstra, P., and Plat, F. M. (2001). Failed suppres-sion of direct visuomotor activation in Parkinson’sdisease. J. Cogn. Neurosci. 130, 31–43.

Praamstra, P., Meyer, A. S., Cools, A. R., Horstink, M.W. I. M., and Stegeman, D. F. (1996). Movementpreparation in Parkinson’s disease: Time courseand distribution of movement-related potentials ina movement precueing task. Brain 119, 1689–1704.

Praamstra, P., Stegeman, D.F., Cools, A. R., andHorstink, M. W. I. M. (1998) Reliance on externalcues for movement initiation in Parkinson’sdisease: Evidence from movement-related poten-tials. Brain 121, 167–177.

Praamstra, P., Plat, E. M., Meyer, A. S., and Horstink,M. W. I. M. (1999). Motor cortex activation inParkinson’s disease: Dissociation of electrocorticaland peripheral measures of response generation.Movement Disord. 14, 790–799.

Puce, A., Donnan, G. A., and Bladin, P. F. (1989).Comparative effects of age on limbic and scalp P3. Electroencephalogr. Clin. Neurophysiol. 74,385–393.

Puce, A., Berkovic, S. F., Cadusch, P. J., and Bladin, P.F. (1994). P3 latency jitter assessed using 2 tech-niques. II. Surface and sphenoidal recordings insubjects with focal epilepsy. Electroencephalogr.Clin. Neurophysiol. 92, 555–567.

Pulvermüller, F., Lutzenberger, W., Müller, V., Mohr,B., Dichgans, J., and Birbaumer, N. (1996). P3 andcontingent negative variation in Parkinson’sdisease. Electroencephalogr. Clin. Neurophysiol. 98,456–467.

Ravizza, S. M., and Ivry, R. B. (2001). Comparison ofthe basal ganglia and cerebellum in shifting atten-tion. J. Cogn. Neurosci. 13, 285–297.

Rektor, I., Louvel, J., and Lamarche, M. (1998).Intracerebral recordings of potentials accompany-ing simple limb movements: A SEEG study inepileptic patients. Electroencephalogr. Clin.Neurophysiol. 107, 277–286.

Reuter, B. M., Schönle P. W., and Kurthen, M. (1994).Event-related potentials in aphasic patients—AnN400 study. EEG-EMG, Z. Elektroenzephalogr.Elektromyogr. Verwandte Geb. 25, 180–189.

Revonsuo, A., and Laine, M. (1996). Semantic process-ing without conscious understanding in a globalaphasic: Evidence from auditory event-relatedbrain potentials. Cortex 32, 29–48.



Revonsuo, A., Portin, R., Juottonen, K., and Rinne,J.O. (1998). Semantic processing of spoken wordsin Alzheimer’s disease: An electrophysiologicalstudy. J. Cogn. Neurosci. 10, 408–420.

Ritter, W., Simson, R., Vaughan, H. G., Jr., andFriedman, D. (1979). A brain event related to themaking of a sensory discrimination. Science 203,1358–1361.

Ritter, W., Simson, R., and Vaughan, H. G., Jr. (1983).Event-related potential correlates of two stages ofinformation processing in physical and semanticdiscrimination tasks. Psychophysiol. 20, 168–179.

Rockstroh, B., Elbert, T., Canavan, A., Lutzenberger,W., and Birbaumer, N. (1989). “Slow CorticalPotentials and Behaviour.” Urban andSchwarzenberg, Baltimore, München, Wien.

Rockstroh, B., Elbert, T., Birbaumer, N., Wolf, P.,Düchting-Roth, A., Reker, M., Daum, I.,Lutzenberger, W., and Dichgans, J. (1993). Corticalself-regulation in patients with epilepsies. EpilepsyRes. 14, 63–72 .

Rodin, E., Khabbazeh, Z., Twitty, G., and Schmaltz, S.(1989). The cognitive evoked potential in epilepsypatients. Clin. Electroencephalogr. 20, 176–182.

Rosenberg, C., Nudleman, K., and Starr, A. (1985).Cognitive evoked potentials (P300) in earlyHuntington’s disease. Arch. Neurol. 42, 984–987.

Ruchkin, D. S., and Sutton, S. (1978). Equivocationand P300 amplitude. In “MultidisciplinaryPerspectives in Event-related Brain PotentialResearch” (D. Otto, ed.), pp. 175–177. U.S.Government Printing Office, Washington, D.C.

Ruchkin, D. S., Grafman, J., Krauss, G. L., Johnson, R.,Jr., Canoune, H., and Ritter, W. (1994). Event-related brain potential evidence for a verbalworking memory deficit in multiple sclerosis.Brain 117, 289–305.

Rugg, M. D., Roberts, R. C., Potter, D. D., Pickles, C.D., and Nagy, M. E. (1991). Event-related poten-tials related to recognition memory. Effects of uni-lateral temporal lobectomy and temporal lobeepilepsy. Brain 114, 2313–2332.

Rugg, M. D., Pearl, S., Walker, P., Roberts, R. C., andHoldstock, J. S. (1994). Word repetition effects onevent-related potentials in healthy young and oldsubjects, and in patients with Alzheimer-typedementia. Neuropsychologia 32, 381–398.

Rugg, M. D., Mark, R. E., Gilchrist, J., and Roberts,R. C. (1997) ERP repetition effects in indirect anddirect tasks: Effects of age and interitem lag.Psychophysiology 34, 572–586.

Scheffers, M. K., Johnson, R., Jr., and Ruchkin, D. S.(1991). P300 in patients with unilateral temporallobectomies: The effects of reduced stimulusquality. Psychophysiology 28, 274–284.

Schmahmann, J. D. (1996). From movement tothought: Anatomic substrates of the cerebellar con-tribution to cognitive processing. Hum. BrainMapping 4, 174–198.

Schroeder, M. M., Handelsman, L., Torres, L.,Dorfman, D., Rinaldi, P., Jacobson, J., Wiener, J.,and Ritter, W. (1994). Early and late cognitiveevent-related potentials mark stages of HIV-1infection in the drug-user risk group. Biol.Psychiatr. 53, 54–69.

Schwartz, T. J., Kutas, M., Butters, N., Paulsen, J. S.,and Salmon, D. P. (1996). Electrophysiologicalinsights into the nature of the semantic deficit inAlzheimer’s disease. Neuropsychologia 34, 827–841.

Shibasaki, H., Shima, F., and Kuroiwa, Y. (1978).Clinical studies of the movement-related corticalpotential (MP) and the relationship between thedentatorubrothalamic pathway and readinesspotential (RP). J. Neurol. 219, 15–25.

Shibasaki, H., Barrett, G., Neshige, R., Hirata, I., andTomoda, H. (1986). Volitional movement is notpreceded by cortical slow negativity in cerebellardentate lesion in man. Brain Res. 368, 361–365.

Smith, M. E., and Halgren, E. (1988). Attenuation of asustained visual processing negativity afterlesions that include the inferotemporal cortex.Electroencephalogr. Clin. Neurophysiol. 70, 366–369.

Smith, M. E., and Halgren, E. (1989). Dissociation ofrecognition memory components following tem-poral lobe lesions. J. Exp. Psychol. Learning MemoryCogn. 15, 50–60.

Spinelli, D., Burr, D. C., and Morrone, M.C. (1994).Spatial neglect is associated with increased laten-cies of visual evoked potentials. Vis. Neurosci. 11,909–918.

Stam, C. J., Visser, S. L., Op de Coul, A. A. W., DeSonneville, L. M. J., Schellens, R. L. L., Brunia, C.H. M., de Smet, D. S., and Gielen, G. (1993).Disturbed frontal regulation of attention inParkinson’s disease. Brain 116, 1139–1158.

Starr, A., and Barrett, G. (1987). Disordered auditoryshort-term memory in man and event-relatedpotentials. Brain 110, 935–959.

Sternberg, S. (1966). High-speed scanning in humanmemory. Science 153, 652–654.

Swaab, T., Brown, C., and Hagoort, P. (1997). Spokensentence comprehension in aphasia: Event-relatedpotential evidence for a lexical integration deficit.J. Cogn. Neurosci. 9, 39–66.

Swaab, T. Y., Brown, C., and Hagoort, P. (1998).Understanding ambiguous words in sentence con-texts: Electrophysiological evidence for delayedcontextual selection in Broca’s aphasia.Neuropsychologia 36, 737–761.

Swanwick, G. R. J., Rowan, M. J., Coen, R. F., Lawlor,B. A., Walsh, J. B., and Coakley, D. (1997). Electro-physiologic responses to varying mnemonicdemand: A “stress test” is not diagnostic in verymild Alzheimer’s disease. Biol. Psychiatr. 42,1073–1075.

Tachibana, H., Toda, K., Aragane, K., and Sugita, M.(1993). Chronometrical analysis of event-relatedpotentials and reaction time in patients with

REFERENCES 339


multiple lacunar infarcts. Cogn. Brain Res. 1,193–196.

Tachibana, H., Aragane, K., and Sugita, M. (1995).Event-related potentials in patients with cerebellardegeneration: Electrophysiological evidence forcognitive impairment. Cogn. Brain Res. 2, 173–180.

Tachibana, H., Miyata, Y., Takeda, M., Sugita, M., andOkita, T. (1999a). Event-related potentials revealmemory deficits in Parkinson’s disease. Cogn.Brain Res. 8, 165–172.

Tachibana, H., Kawabata, K., Tomino, Y., and Sugita,M. (1999b). Prolonged P3 latency and decreasedbrain perfusion in cerebellar degeneration. ActaNeurol. Scand. 100, 310–316.

Tachibana, H., Miyata, Y., Takeda, M., Minamoto, H.,Sugita, M., and Okita, T. (1999c). Memory inpatients with subcortical infarction—An auditoryevent-related potential study. Cogn. Brain Res. 8,87–94.

Takeda, M., Tachibana, H., Okuda, B., Kawabata, K.,and Sugita, M. (1998). Electrophysiological com-parison between corticobasal degeneration andprogressive supranuclear palsy. Clin. Neurol.Neurosurg. 100, 94–98.

Tarkka, I. M., Massaquoi, S., and Hallett, M. (1993).Movement-related cortical potentials in patientswith cerebellar degeneration. Acta Neurol. Scand.88, 129–135.

ter Keurs, M., Brown, C. M., Hagoort, P., andStegeman, D. F. (1999). Electrophysiological mani-festations of open- and closed-class words inpatients with Broca’s aphasia with agrammaticcomprehension: An event-related brain potentialstudy. Brain 122, 839–854.

Thomas, C., Altenmüller, E., Marckmann, G., Kahrs,J., and Dichgans, J. (1997). Language processing inaphasia: Changes in lateralization patterns duringrecovery reflect cerebral plasticity in adults.Electroencephalogr. Clin. Neurophysiol. 102, 86–97.

Touge, T., Werhahn, K. J., Rothwell, J. C., andMarsden, C. D. (1995). Movement-related corticalpotentials preceding repetitive and random-choicehand movements in Parkinson’s disease. Ann.Neurol. 37, 791–799.

Triantafyllou, N. I., Voumvourakis, K., Zalonis, I.,Sfagos, K., Mantouvalos, V., Malliara, S., andPapageorgiou, C. (1992a). Cognition in relaps-ing–remitting multiple sclerosis: A multichannelevent-related potential (P300) study. Acta Neurol.Scand. 85, 10–13.

Triantafyllou, N. I., Zalonis, I., Kokotis, P.,Anthracopoulos, M., Siafacas, A., Malliara, S.,Hamburger, H. L. and Papageorgiou, C. (1992b).Cognition in epilepsy: A multichannel event-related potential (P300) study. Acta Neurol. Scand.86, 462–465.

Tsuchiya, H., Yamaguchi, S., and Kobayashi, S. (2000).Impaired novelty detection and frontal lobe dys-

function in Parkinson’s disease. Neuropsychologia38, 645–654.

Van Boxtel, G. (1994). Non-motor components of slowbrain potentials. Dissertation, Katholieke Univer-siteit Brabant, Tilburg, The Netherlands.

van der Lubbe, R. H. J., Wauschkuhn, B., Wascher, E.,Niehoff, T., Kömpf, D., and Verleger, R. (2000).Event-related lateralized components with direc-tion information for the preparation of saccadesand finger movements. Exp. Brain Res. 132,163–178.

van Dijk, J. G., Jennekens-Schinkel, A., Caekebeke, J.F., and Zwinderman, A.H. (1992). Are event-relatedpotentials in multiple sclerosis indicative of cog-nitive impairment? Evoked and event-relatedpotentials, psychometric testing and responsespeed: A controlled study. J. Neurol. Sci. 109, 18–24.

Verleger, R. (1997). On the utility of P3 latency as anindex of mental chronometry. Psychophysiology 34,131–156.

Verleger, R. (2001). Comment on “Electrophysiologicalcorrelates of conscious vision: Evidence from uni-lateral extinction.” J. Cogn. Neurosci. 13, 416–417.

Verleger, R., and Berg, P. (1991). The waltzing oddball.Psychophysiology 28, 468–477.

Verleger, R., Neukäter, W., Kömpf, D., and Vieregge,P. (1991). On the reasons for the delay of P3 latencyin healthy elderly subjects. Electroencephalogr. Clin.Neurophysiol. 79, 488–502.

Verleger, R., Kömpf, D., and Neukäter, W. (1992).Event-related EEG potentials in mild dementia ofthe Alzheimer type. Electroencephalogr. Clin.Neurophysiol. 84, 332–343.

Verleger, R., Heide, W., Butt, C., Wascher, E., andKömpf, D. (1996). On-line brain potential corre-lates of right parietal patients’ attentional deficit.Electroencephalogr. Clin. Neurophysiol. 99, 444–457.

Verleger, R., Lefèbre, C., Wieschemeyer, R., andKömpf, D. (1997). Event-related potentials suggestslowing of brain processes in generalized epilepsyand alterations of visual processing in patientswith partial seizures. Cogn. Brain Res. 5, 205–219.

Verleger, R., Wascher, E., Wauschkuhn, B., Jaskowski,P., Allouni, B., Trillenberg, P., and Wessel, K.(1999). Consequences of altered cerebellar inputfor the cortical regulation of motor coordination,as reflected in EEG potentials. Exp. Brain Res. 127,409–422.

Verleger, R., Wauschkuhn, B., Van der Lubbe, R. H. J.,Jaskowski, P., and Trillenberg, P. (2000a). Posteriorand anterior contributions of hand-movementpreparation to Late CNV. J. Psychophysiol. 14,69–86.

Verleger, R., Vollmer, C., Wauschkuhn, B., Van derLubbe, R. H. J., and Wascher, E. (2000b).Dimensional overlap between arrows as cueingstimuli and responses? Evidence from contra-ipsi-lateral differences in EEG potentials. Cogn. BrainRes. 10, 99–109.



Verleger, R., Adam, S., Rose, M., Vollmer, C.,Wauschkuhn, B., and Kömpf, D. (2002). Synergy ofhemispheres: Timing of ipsilateral activation atmovement onset in patients with subcorticalischemic lesions of the pyramidal tract. Submittedfor publication.

Vidailhet, M., Stocchi, F., Rothwell, J. C., Thompson,P.D., Day, B.L., Brooks, D. J., and Marsden, C. D.(1993). The Bereitschaftspotential precedingsimple foot movement and initiation of gait inParkinson’s disease. Neurology 43, 1784–1788.

Vieregge, P., Verleger, R., Wascher, E., Stüven, F., andKömpf, D. (1994). Selective attention is impairedin Parkinson’s disease—Event-related evid-ence from EEG potentials. Cogn. Brain Res. 2,117–130.

Vieregge, P., Wauschkuhn, B., Heberlein, I., Hagenah,J., and Verleger, R. (1999). Selective attention isimpaired in amyotrophic lateral sclerosis — Astudy of event-related EEG potentials. Cogn. BrainRes. 8, 27–35.

Viggiano, M. P., Spinelli, D., and Mecacci, L. (1995).Phase reversal visual evoked potentials in patientswith visual neglect. Brain Cogn. 27, 17–35.

Vogel, E. K., and Luck, S. J. (2000). The visual N1 com-ponent as an index of a discrimination process.Psychophysiology 37, 190–203.

Vuilleumier, P., Sagiv, N., Hazeltine, E., Poldrack, R.A., Swick, D., Rafal, R. D., and Gabrieli, J. D. E.(2001). Neural fate of seen and unseen faces invisuospatial neglect: a combined event-relatedfunctional MRI and event-related potential study.Proc. Natl. Acad. Sci. U.S.A. 98, 3495–3500.

Wang, L., Kuroiwa, Y., Kamitani, T., Li, M., Takahashi,T., Suzuki, Y., Shimamura, M., and Hasegawa, O.(2000). Visual event-related potentials in pro-gressive supranuclear palsy, corticobasal degener-ation, striatonigral degeneration, and Parkinson’sdisease. J. Neurol. 247, 356–363.

Wascher, E., and Wauschkuhn, B. (1996). The inter-action of stimulus- and response-related processesmeasured by event-related lateralisations of the EEG. Electroencephalogr. Clin. Neurophysiol. 99,149–162.

Wascher, E., Verleger, R., Vieregge, P., Jaskowski, P.,Koch, S., and Kömpf, D. (1997). Responses to cuedsignals in Parkinson’s disease: Distinguishingbetween disorders of cognition and of activation.Brain 120, 1355–1375.

Wascher, E., Reinhard, M., Wauschkuhn, B., andVerleger, R. (1999). Spatial S-R compatibility withcentrally presented stimuli: An event-relatedasymmetry study about dimensional overlap. J. Cogn. Neurosci. 11, 214–229.

Wascher, E., Schatz, U., Kuder, T., and Verleger, R.(2001). Validity and boundary conditions of auto-matic response activation in the Simon task. J. Exp.Psychol. Human Percept. Perform. 27, 731–751.

Weiller, C., Ramsay, S. C., Wise, R. J. S., Friston, K. J.,and Frackowiak, R. S. J. (1993). Individual pat-terns of functional reorganization in the humancerebral cortex after capsular infarction. Ann.Neurol. 33, 181–189.

Wessel, K., Verleger, R., Nazarenus, D., Vieregge, P.,and Kömpf, D. (1994). Movement-related cor-tical potentials preceding sequential and goal-directed finger and arm movements in patientswith cerebellar atrophy. Electroencephalogr. Clin.Neurophysiol. 92, 331–341.

Westphal, K. P., Heinemann, H.A., Grözinger, B.,Kotchoubey, B. J., Diekmann, V., Becker, W., andKornhuber, H. H. (1998). Bereitschaftspotential inamyotrophic lateral sclerosis (ALS): Lower ampli-tudes in patients with hyperreflexia (spasticity).Acta Neurol. Scand. 98, 15–21.

Wolpaw, J. R., McFarland, D. J., Neat, G. W., andForneris, C. A. (1991). An EEG-based brain-computer interface for cursor control. Electro-encephalogr. Clin. Neurophysiol. 78, 252–259.

Wright, M. J., Geffen, G. M., and Geffen, L. B. (1993).Event-related potentials associated with covertorientation of visual attention in Parkinson’sdisease. Neuropsychologia 31, 1283–1297.

Yamaguchi, S., Tsuchiya, H., and Kobayashi, S. (1998).Visuospatial attention shift and motor responsesin cerebellar disorders. J. Cogn. Neurosci. 10,95–107.

Yamaguchi, S., Tsuchiya, H., Yamagata, S., Toyoda, G.,and Kobayashi, S. (2000). Event-related brainpotentials in response to novel sounds in demen-tia. Clin. Neurophysiol. 111, 195–203.

Yazawa, S., Ikeda, A., Kunieda, T., Ohara, S., Mima,T., Nagamine, T., Taki, W., Kimura, J., Hori, T., andShibasaki, H. (2000). Human pre-supplementarymotor area is active before voluntary move-ment, subdural recording of Bereitschaftspotentialfrom medial frontal cortex. Exp. Brain Res. 131,165–177.

REFERENCES 341



C H A P T E R

14

Mismatch Negativity: A Probeto Auditory Perception and

Cognition in Basic andClinical Research

Risto Näätänen, Elvira Brattico, and Mari Tervaniemi

THE MMN: AN AUTOMATICCHANGE-DETECTION RESPONSE

IN AUDITION

The mismatch negativity (MMN)(Näätänen et al., 1978) currently providesthe only valid objective measure of theaccuracy of central auditory processing inthe human brain. The MMN is an electricbrain response, a negative component of the event-related potential (ERP), usu-ally peaking at 100–200 msec from changeonset, to any discriminable change(“deviant”) in some repetitive aspect ofauditory stimulation (“standard”). Thus,an MMN is elicited when, for example, atone changes in frequency, duration, orintensity, or a phoneme is replaced byanother phoneme (Fig. 1). Importantly, theMMN can be elicited even in the absenceof attention, which is of central importancein view of the possible clinical applications.

The MMN depends on the presence of amemory trace formed by the precedingstimuli; that is, the MMN cannot be attrib-uted to “new” or “fresh” afferent elementsactivated by the deviant but not by thestandard stimulus. Tervaniemi et al. (1994)found an MMN to an occasional omission

of the second tone of a tone pair (with veryshort intrapair intervals). MMN data con-sequently suggest that the first standardsin the beginning of a stimulus blockdevelop a memory trace, accurately repre-senting each stimulus feature (includingeven the temporal aspects of stimulation),and, further, that if a deviant stimulusoccurs while this memory trace is stillactive, then the automatic change-detec-tion reaction generating an MMN occurs.

The duration of these traces (as estimatedby the interstimulus interval, with which noMMN can any longer be elicited) is of theorder of 10 sec (Sams et al., 1993; Böttscher-Gandor and Ullsperger, 1992). This is in anagreement with the estimated duration ofthe long store of auditory sensory memory(Cowan, 1984). In addition, several otherstudies, such as those on backward masking(Winkler and Näätänen, 1992), suggest thatthe memory traces of the short store ofauditory sensory memory (Cowan, 1984)can be probed with the MMN (see below).

The MMN has two main generatormechanisms: bilateral auditory cortex gen-erators and a frontal cortex generator[which may also be bilateral, but such thatthe right hemispheric generator is consid-

erably stronger than the left hemisphericone (Giard et al., 1990; Paavilainen et al.,1991; Alho, 1995)]. The activation of thefrontal generator has been associated withinvoluntary attention switch to soundchange preperceptually detected in the audi-tory cortices (Giard et al., 1990). Consistentwith this, the frontal activation is slightlydelayed in time of onset relative to the audi-tory cortex activation (Rinne et al., 2000),supporting the assumption that the change-detection signal generated by the auditorycortex triggers the frontal mechanismsleading to attention switch to sound change(Näätänen, 1990).

Convincing behavioral evidence for theoccurrence of an involuntary attentionswitch shortly following MMN elicitationwas provided by Schröger (1996; see alsoAlho et al., 1997). He showed that the reac-tion time (RT) was prolonged and the hit rate(HR) decreased in the primary task as a con-sequence of the occurrence of even a minorfrequency change in irrelevant auditory stim-ulation eliciting an MMN just before theimperative stimulus (see Figs. 1 and 2).

LINGUISTIC FUNCTIONS ASREVEALED BY THE MMN

Studies have shown that in addition toshort-duration traces developed in thebeginning of a stimulus block, the MMNcan also be used to probe long-term audi-tory memory traces, most typically thoserepresenting the phonemes of one’s mothertongue. Näätänen et al. (1997) found thatthe Estonian vowel /õ/ elicited a muchlarger MMN in Estonian than in Finnishsubjects when this vowel was used as adeviant stimulus (the standard stimulusbeing /e/, which is shared by the two lan-guages), whereas the MMN of the twosubject groups was very similar for the deviants shared by the two lan-guages (/ö/, /o/). Subsequent magneto-encephalographic measurements in theFinnish subjects showed that the mother-tongue-related MMN (MMNm) enhance-ment originated from the left auditorycortex, which thus appeared to be the locusof the language-specific vowel traces of the

344 14. MISMATCH NEGATIVITY


FIGURE 1 Schematic illustration of the mismatch negativity (MMN), showing the event-related potential(ERP) waveforms for the standard and deviant stimuli. On the right, difference waves are obtained by subtract-ing the standard stimulus response from that to the deviant stimulus response recorded at the electrode siteswhere the MMN is maximal.

mother tongue (see Fig. 3). More specifi-cally, these traces seem to be located inWernicke’s area in the posterior temporalcortex (Rinne et al., 1999).

As indexed by the MMN, these lan-guage-specific memory traces for themother tongue emerge during the firstyear of life (Dehaene-Lambertz and Baillet,

LINGUISTIC FUNCTIONS AS REVEALED BY THE MMN 345


FIGURE 2 Grand-averaged (n = 13) functional magnetic resonance imaging activation elicited by a small(10%), medium (30%), and large (100%) increase in frequency deviation superimposed on an individual struc-tural MRI in Talairach space. Images were thresholded at P < 0.01. All deviants induced significant activation inthe superior temporal gyri bilaterally, whereas the opercular part of the right inferior frontal gyrus was signifi-cantly activated only when the large and medium deviants were presented. (See color plates.) Adapted fromOpitz et al. (2002).

FIGURE 3 Language-specific phoneme traces localized in the left temporal lobe reflected by the MMN.Magnetic field-gradient maps of the left- and right-hemisphere MMNs of one typical Finnish subject for Finnishand Estonian deviant vowels. The squares indicate the arrangement of the magnetic sensors. The arrows repre-sent the equivalent current dipoles, indicating activity in the auditory cortex. The Finnish vowel prototype elicitsa much larger MMNm in the left (compared to the right) hemisphere, whereas the nonprototype responses to anEstonian vowel that does not exist in the Finnish language are small in amplitude in both hemispheres. (Seecolor plates.) Adapted from Näätänen et al. (1997).

1998; Cheour et al., 1998a). In addition, thedevelopment of the language-specificmemory traces when adults learn a foreignlanguage can also be monitored by usingthe MMN (Winkler et al., 1999). These lan-guage-specific memory traces function asrecognition patterns sequentially activ-ated by the corresponding phonemes (andlarger units) of a spoken language,enabling one to perceive correctly thespeech sounds uttered (Näätänen, 2001).Consistent with this, the MMNm dipolesfor phoneme changes can be modeled evenwhen these phonemes are uttered by hun-dreds of speakers so that the acousticrepetition within the standard stimuluscategory is minimized (Shestakova et al.,2002).

Findings also indicate that MMNrecordings can serve as an index of thedegree of auditory-processing impairmentin acquired language disorders such as inWernicke’s aphasia. Ilvonen et al. (2001)found that in aphasic stroke patients, theMMN to duration decrement was deterio-rated to right-ear stimulation as long as

approximately 2 years after the stroke,whereas the MMN to left-ear stimulationresembled that of control subjects (see Fig. 4).

Furthermore, the MMN is attenuated indevelopmental language disorders. Indyslexic adults, the MMN did not differfrom that of control subjects when it waselicited by an interval change in a tone pair(Kujala et al., 2000). In contrast, the MMNwas attenuated, relative to that of controls,when this tone pair was preceded and suc-ceeded by an extra tone. The behavioraldiscrimination performance was consist-ent with the MMN data. A subsequentstudy indicated that this difficulty of dys-lexic subjects in discriminating temporalchanges among tone patterns was causedby the masking sound following the soundchange rather than by the one preceding it,suggesting their increased vulnerability tobackward masking (Kujala et al., 2001a).

Most importantly, the MMN alsoindexes the improvement of reading per-formance in dyslexic children receivingaudiovisual training (Fig. 5). The subjectsstudied by Kujala et al. (2001b) werereading-impaired 7-year-old pupils whowere in the first grade. One-half of thesechildren played an audiovisual PC-basedtraining program (Karma, 1999) during 7 weeks, which improved their readingskills considerably more than that of thedyslexic pupils of the control group.Furthermore, the MMN was correspond-ingly enhanced in the training group at theend of the training period, being clearlylarger in amplitude than that of the controlgroup. Importantly, the magnitude of thereading-skills improvement correlatedwith the magnitude of the MMN ampli-tude increase in the training group. It isnoteworthy that the PC game used in train-ing the children did not include linguisticinformation, which suggests that thegeneral deterioration of central auditoryprocessing, rather than that involvingspeech sounds only, is implemented indevelopmental dyslexia.



FIGURE 4 Left-hemisphere auditory dysfunctionin sound-duration discrimination in Wernicke’saphasic patients. In this study, stimuli were monau-rally presented in order to stimulate primarily onehemisphere (contralateral to the stimulation) at atime. Occasional duration decrements in a repetitivesound elicited a similar MMN in control subjects (thinline) when the stimuli were presented to either theleft or the right ear. In contrast, in the patients (thickline), these decrements elicited a small MMN whenthe stimuli were presented to the right ear, and a con-siderably larger MMN when the stimuli were pre-sented to the left ear. Adapted from Ilvonen et al.(2001).

MMN AS AN INDEX OF MUSIC PERCEPTION AND

MUSICAL EXPERTISE

Subjects with superior performance inthe pitch-discrimination task of the tradi-tional Seashore musicality test showed anenhanced pitch MMN when comparedwith subjects with less accurate behavioralpitch discrimination (Lang et al., 1990).Tervaniemi et al. (1997) studied the neuraldeterminants underlying “cognitive musi-cality,” defined as an ability to structurecontinuous sound stream into meaningfulunits (Karma, 1994). To this end, out of the117 subjects tested, the MMN of 14 subjectswith the best test performance (“musical”group) was compared with that recordedfrom 14 subjects with the poorest test per-formance (“nonmusical” group). It wasfound that, although these subjects did notdiffer from each other in the amount ofmusical training they had received, theMMN to sound-order change was consid-erably larger in amplitude in musical thanin nonmusical subjects. This suggests that

the preattentive neural circuits as indexedby the MMN determine performance instructuring musical sounds and detectingregularities in it.

Data (Kohlmetz et al., 2001) from hemi-spheric stroke patients showed that auto-matic pitch classification is selectivelyimpaired only in those patients with musicperception deficits as indexed by a new,specially created test battery. These dataconfirm, then, the previous results (e.g.,Tervaniemi et al., 1997; Koelsch et al., 1999)indicating that music perception is, in part,based on the elementary discriminationprocesses as indexed by the MMN. Theissue of whether the accuracy of automaticpitch-discrimination mechanisms differsbetween musicians and nonmusicians wasalso addressed. The subjects’ pitch MMNto a tiny pitch change among chord stimuliwas compared with their pitch MMN topure tone pitch changes in both “ignore”and “attend” conditions (Koelsch et al.,1999). The results demonstrated that themusically trained subjects (professionalviolin players) automatically detected, asindexed by the MMN elicitation, tiny pitch

MMN AS AN INDEX OF MUSIC PERCEPTION AND MUSICAL EXPERTISE 347


FIGURE 5 Left: MMNs recorded from dyslexic children in response to tone-order reversals. After training, theMMNs were enhanced in the trained group but not in the untrained group. Right: Reading performance [asindexed by the number of correct read words (left) and reading speed (right)]. The reading improvement wasconsiderably greater in the trained group. Adapted from Kujala et al. (2001b).

changes that were undetectable for non-musicians and, further, that these discrimi-native functions were not modified byattentional manipulations. The MMNevoked by a small or large pitch change inpure sinusoidal tones did not differentiateviolinists and nonmusicians, however.Thus, the superior pitch-processing accu-racy of the violinists was manifested onlywhen the pitch change was presentedamong musical sounds.

In addition, musicians showed a signifi-cant MMN when infrequent omissionswere interspersed among tones presentedat both a faster stimulus onset asynchrony(SOA) of 100 msec or a slower SOA of220 msec, whereas nonmusicians’ brainsproduced an MMN only when omissionsoccurred within the fast-paced tone series(Rüsseler et al., 2001). Whereas the above-described pitch–MMN enhancement inmusicians (Koelsch et al., 1999) could beexplained merely by the more accuratetuning of frequency-specific afferentneurons in musicians, this explanationdoes not, of course, apply to the Rüsseler etal. data, for the activation of new afferentneurons cannot explain an MMN when nostimulus was present.

The neural mechanisms behind themusicians’ superior ability to detectinvariances in continuous auditory infor-mation flow was addressed by Tervaniemiet al. (2001). Musicians and nonmusicianswere presented with a short melody-likesound pattern transposed to 12 frequencylevels in passive (video watching) anddiscrimination conditions, administered inan alternating order. An MMN waselicited only in those subjects who in thediscrimination condition were able todetect more than 80% of the contour-changed melodies. Even in them, theMMN was elicited only after the firstdiscrimination condition, suggesting thatconscious attention is a necessary pre-requisite for complex perceptual (andpreperceptual) learning to occur, as shownby Näätänen et al. (1993b).

Magnetoencephalographic (MEG) datashow that the MMNs for sounds differingin their informational content are gener-ated in slightly different parts of the audi-tory cortex. Alho et al. (1996) compared thefrequency-MMN evoked by an identicalfrequency change occurring in single sinu-soidal tones, parallel chords, and serialchords. It was found that the MMN elicitedby chords, both parallel and serial, isgenerated about 1 cm medially to thatelicited by single sinusoidal tones. Further,Tervaniemi et al. (1999a) compared theMMNm generator loci for phonemes andchords matched in spectral complexity andin the magnitude of frequency changeembedded in them. They found that theMMNm to a frequency change of one tonewithin a chord (resulting in a change fromA major to A minor) was generated superi-orly to that generated by a frequencychange of the second formant within aphoneme (resulting in a perceptual changefrom phoneme /e/ to /o/). In addition, inthe right hemisphere, the MMNm waslarger in amplitude for changes amongchords than among phonemes. However,in the left hemisphere, no correspondingdominance for phoneme changes wasfound when compared with chordchanges, which is not consistent with pre-vious results (e.g., Näätänen et al., 1997;Rinne et al., 1999).

The issue of cerebral dominance wasfurther investigated by Tervaniemi et al.(2000a) by using positron emission tomog-raphy (PET) and the phonetic and musicalsounds developed for the MEG studydescribed above. Subjects classified thegender of visually presented words whilethey were presented with sound sequencesconsisting of (1) both deviant and standardsounds or (2) standard sounds only(phonemes and chords in separatesequences). The data showed that thechange from vowel /e/ to /o/ was pro-cessed in the middle and supratemporalgyri in the left hemisphere. In a mirrorlikemanner, the change from A major to A



minor chord was processed in the supra-temporal gyrus of the right auditorycortex. These data thus indicate that thehemispheric specialization for phoneticversus musical processing is present evenwhen these stimuli are to be ignored (and atask not involving these stimuli is per-formed). However, this phenomenon isvery vulnerable to noise (see Shtyrov et al.,1998, 1999) and is subdued to the cortico-cortical connections between the neuralpopulations involved [see Kohlmetz et al.(2001) for hemispheric stroke patient data].

In addition, spectrally rich soundsevoked an MMN that was shorter inlatency and larger in amplitude than thatevoked by pure sinusoidal tones(Tervaniemi et al., 2000b). Complex soundswithout the fundamental frequency do notshow this advantage, however. The longerlatency MMN to missing fundamentalsounds rather implicates an increased diffi-culty in the pitch-extracting process in thepresence of this spectral configuration(Winkler et al., 1997). A study by Brattico etal. (2002) addressed the effects of the soundcontext on pitch discrimination as indexedby the MMN. It was found that the pro-cessing of a pitch change embedded in apattern of familiar sounds (according to theWestern musical tradition) is facilitated, asindexed by the MMN amplitude, whencompared with the same pitch changewhen embedded in an unfamiliar (arith-metically determined) sound pattern.

MMN IN CLINICAL RESEARCH

Because of its several advantages, MMNhas numerous potential clinical applica-tions. One of these advantages is its atten-tion-independent elicitation. This is veryimportant, in particular in studying clinicalgroups as well as infants and newborns.Furthermore, in interpreting the testresults, it is a major advantage that theeffects of attention and motivation areminimal. Data supporting MMN elicitation

even in the absence of attention stem fromdiverse conditions, ranging from selectivedichotic listening [where an MMN hasbeen elicited even by minor changes in theunattended-ear input (Näätänen et al.,1993a; Paavilainen et al., 1993)], to certainsleep stages in adults (Campbell et al.,1991; Sallinen et al., 1994) and in newbornsand infants (Cheour-Luhtanen et al., 1996,1997), and to comatose patients [during thelast days before the recovery of conscious-ness (Kane et al., 1993, 1996)]. Though theresults of some studies (Woldorff et al.,1991, 1998; Trejo et al., 1995; Alho et al.,1992) suggest that the MMN amplitude isattenuated in the unattended channelunder highly focused selective attention,no data suggest that the MMN is totallyabolished by the withdrawal of attention.Moreover, the MMN amplitude is gener-ally not affected by attention in normal(one-channel) “oddball” conditions inwhich the MMN is usually measured forvarious clinical and other applied pur-poses (Näätänen et al., 1982; for a review,see Näätänen, 1992).

In fact, the best way to record the MMNis to use passive conditions, with thesubject or patient’s attention beingdirected elsewhere, such as to a self-selected (silenced) video or visual com-puter game (Kathmann et al., 1999). If thesound sequence used for MMN elicitationis attended, then the MMN is overlappedby other ERP components such as the P165(Goodin et al., 1978) and, most notably, theN2b [described in a visual paradigm byRenault and Lesévre (1978, 1979) and in anauditory paradigm by Näätänen et al.(1982)]. This makes the pure measurementof the MMN component very difficult orimpossible. The MMNm is to a muchlesser extent than the MMN overlapped byattention-related components (seeKaukoranta et al., 1989; Lounasmaa et al.,1989).

As previously mentioned, MMN canalso be measured in comatose patients. AnMMN elicited in an unconscious coma

MMN IN CLINICAL RESEARCH 349


patient appears to provide the single mostreliable predictor of the recovery of con-sciousness (Kane et al., 1993, 1996). As forthe MMN in sleep, in adults it is quite anevasive though yet well-documented effect(e.g., Campbell et al., 1991; Sallinen et al.,1994); it is much easier to obtain an MMNin sleeping infants and newborns, which isvery helpful in evaluating their centralauditory function. Consequently, one of themost promising fields of application of theMMN involves newborns and younginfants. MMN is elicited even in prema-turely born newborns (Cheour-Luhtanen etal., 1996). In fact, the MMN is the earliestcognitive ERP component that can berecorded from the human brain. It appearsthat with the MMN, one could detectdevelopmental problems in the auditorycortex early enough to treat what wouldotherwise prevent the normal perception ofspeech sounds and thus result in delayedspeech development. This problem is par-ticularly common in children with cleftpalate (Cheour et al., 1998b). Ceponiene et al. (2000) indicate that although obliga-tory cortical ERP components do not dif-ferentiate newborns with different types ofclefts, the MMN does, even before the ageof 6 months.

A multitude of studies have shown thatthe MMN (recorded in passive conditions)predicts the behavioral discriminationaccuracy in various perceptual tasksinvolving the different sound types andattributes, such as the frequency of simpletones (Tiitinen et al., 1994; Tremblay et al.,1998; Menning et al., 2000) and complexspectrotemporal patterns (Näätänen et al.,1993b). Even the accuracy of speech processing can be probed with the MMN recordings; a good correspondencebetween MMN parameters and behavioralperformance has been demonstrated forvowels of a foreign language (Winkler et al., 1999), for consonant–vowel (CV) syl-lables of the mother tongue in school chil-dren (Kraus et al., 1996), for differentwithin-category variants of a CV syllable

(Kraus et al., 1995), for toneburst and clicktrains (Ponton and Don, 1995), and for CVsyllables after cochlear-implant installation(Kraus et al., 1993a). These data open newperspectives, for example, in teaching andlearning foreign languages. For instance,progress in the correct perception offoreign speech sounds can, in principle, bemonitored by recording the MMN. Asalready mentioned, the speech-soundmemory traces probed with the MMNprobably serve as recognition patterns nec-essary for the correct perception of unfa-miliar foreign speech sounds and theircombinations. Furthermore, it is quitelikely that the correct pronunciation offoreign speech sounds depends on theaccuracy of sensory information encodedin these memory traces (Näätänen, 2001).

The MMN can also be used to measurethe duration of echoic memory by varyingthe interstimulus interval (ISI). Here theidea is that when the decay of the memorytrace of the standards has reached a certainstage, then deviants can no longer elicit anMMN. Using this logic, Pekkonen et al.(1994) found that Alzheimer patients had anormal MMN when stimuli were pre-sented at a constant ISI of 1 sec but aseverely attenuated one when the ISI wasprolonged to 3 sec. This data patternenabled the authors to conclude that it wassensory memory rather than discrimina-tion in audition that was deteriorated inAlzheimer patients. Analogous resultswere obtained by Cheour and colleagues(1998b) in school-age children with cleftpalate.

It is well-established that our sensorypercepts in audition are not responses onlyto what is acoustically present at thatmoment (corrected with the perceptuallatency) but to what is acoustically presentduring a sliding window of 150–200 msecin duration; this period is called the tempo-ral window of integration (TWI) (seeCowan, 1984; Näätänen, 1990; Yabe et al.,1998). A prime example of the TWI is theloudness summation of very brief sounds,



which are heard louder as their duration isincreased until about 200 msec (Scharf andHoutsma, 1986). Furthermore, backwardmasking is effective when the masker onsetoccurs within a similarly short intervalfrom the start of the test stimulus(Hawkins and Presson, 1986; see alsoWinkler et al., 1993). Winkler and Näätänen(1992) found that the MMN could be usedto determine the duration of the TWI.Placing a masker after each (brief) stimulusof the MMN paradigm, they found that afull-size MMN could be elicited bydeviants only when the silent ISI betweenthe MMN-paradigm stimulus and themasker was 150 msec or longer. The recov-ery of the behavioral discriminationfollowed a very similar time course.

Näätänen (1995) proposed that the TWIdetermines the time needed for the emer-gence of the central sound representationunderlying sound perception.

One form of auditory pathology mightbe related to TWI abnormalities, such thatauditory input is more vulnerable to back-ward-masking effects by subsequent stimuliand stimulus elements. For example, theTWI duration might be prolonged, or thebackward-masking effects within the TWImight be strengthened. MMN data suggestthe presence of TWI-related central audi-tory-processing pathology in school-agedand adult developmental dyslexics (Kujalaet al., 2001a,b; already reviewed above) andin chronic abstinent alcoholics (Ahveninenet al., 1999).

The ERP recording and analyzing facili-ties are very cheap compared with those ofother methodologies and are availablealmost in any hospital. Furthermore, formany applied purposes, only a few EEGchannels are needed. Replicable MMNdata can be obtained rather easily, at leastwhen standardized methods, currentlyunder development, become available (seeLang et al., 1995; Schröger, 1998; Sinkkonenand Tervaniemi, 2000). The test–retestreplicability of the MMN amplitudereaches a satisfactory level in somestimulation paradigms, especially in theduration-deviance paradigm (see Fig. 6)(Escera and Grau, 1996; Kathmann et al.,1999; Tervaniemi et al., 1999b, 2002).Moreover, stable MMNs have beenobtained in the tone-pair paradigm (Kallioet al., 2002), a result that is encouragingespecially with regard to the assessment ofthe neurophysiological determinants ofreading problems in dyslexic subjects (cf.Kujala et al., 2000, 2001a,b). In addition,attempts have already been made to usethe MMN as a tool for the assessment ofthe auditory and cognitive functions at theindividual level (Kraus et al., 1993b;Hämäläinen et al., 2002; for methodologicalissues, see Ponton et al., 1997; McGee et al.,1997).

MMN IN CLINICAL RESEARCH 351


FIGURE 6 Significant test–retest reliabilities of theMMN amplitude and latency for duration, frequency,and intensity increment, or decrement deviants, mea-sured in 15 healthy subjects. The horizontal axis indi-cates the amplitudes recorded in the first session andthe vertical axis indicates values recorded in thesecond session. The 66% duration decrement elicitedan MMN with the most replicable amplitude andlatency. Adapted from Clinical Neurophysiology, 110;M. Tervaniemi, A. Lehtokoski, J. Sinkkonen, J.Virtanen, R.J. Ilmoniemi, and R. Näätänen; Test-retestreliability of mismatch negativity for duration, fre-quency, and intensity changes, pp. 1388–1393.Copyright (1999), with permission of Elsevier Science.

ReferencesAhveninen, J., Jääskeläinen, I. P., Pekkonen, E.,

Hallberg, A., Hietanen, M., Mäkelä, R., Näätänen,R., and Sillanaukee, P. (1999). Brain responsespredict rapid forgetting in alcohol abusers. Alcohol.Clin. Exp. Res. 23, 1507–1515.

Alho, K., Woods, D. L., Algazi, A., and Näätänen, R.(1992). Intermodal selective attention II: Effects ofattentional load on processing auditory and visualstimuli in central space. Electroencephalogr. Clin.Neurophysiol. 82, 356–368.

Alho, K., Tervaniemi, M., Huotilainen, M.,Lavikainen, J., Tiitinen, H., Ilmoniemi, R. J.,Knuutila, J., and Näätänen, R. (1996). Processing ofcomplex sounds in the human auditory cortex asrevealed by magnetic brain responses.Psychophysiology 33, 369–375.

Alho, K., Escera, C., Diaz, R., Yago, E., and Serra J. M.(1997). Effects of involuntary auditory attention onvisual task performance and brain activity.NeuroReport 8, 3233–3237.

Alho, K. (1995). Cerebral generators of mismatch neg-ativity (MMN) and its magnetic counterpart(MMNm) elicited by sound changes. Ear Hear. 16,38–51.

Böttscher-Gandor, C., and Ullsperger, P. (1992).Mismatch negativity in event-related potentials to auditory stimuli as a function of varying interstimulus interval. Psychophysiology 29,546–550.

Brattico, E., Tervaniemi, M., and Näätänen, R. (2002).Context effects on pitch discrimination: Evi-dence from ERP recordings. Music Percept. 19,1–24.

Campbell, K., Bell, I., and Bastien, C. (1991). Evokedpotential measures of information processingduring natural sleep. In “Sleep, Arousal andPerformance” (R. Broughton and R. Ogilvie, eds.),pp. 88–116. Birkhauser, Boston and Cambridge,Massachusetts.

Ceponiene, R., Hukki, J., Cheour, M., Haapanen, M.-L., Koskinen, M., Alho, K., and Näätänen, R.(2000). Development of the auditory cortexpersists in infants with certain cleft types. Dev.Med. Child Neurol. 42, 258–265.

Cheour, M., Ceponiene, R., Lehtokoski, A., Luuk, A.,Allik, J., Alho, K., and Näätänen, R. (1998a).Development of language-specific phoneme repre-sentations in the infant brain. Nat. Neurosci. 1,351–353.

Cheour, M., Haapanen, M.-L., Ceponiene, R., Hukki,J., Ranta, R., and Näätänen, R. (1998b). Mismatchnegativity (MMN) as an index of auditory sensorymemory deficit in cleft-palate and catch-syndromechildren. NeuroReport 9, 2709–2712.

Cheour-Luhtanen, M., Alho, K., Sainio, K., Rinne, T.,Reinikainen, K., Pohjavuori, M., Aaltonen, O.,

Eerola, O., and Näätänen, R. (1996). The onto-genetically earliest discriminative response of the human brain. Psychophysiology 33, 478–481.

Cheour-Luhtanen, M., Alho, K., Sainio, K.,Reinikainen, K., Renlund, M., Aaltonen, O.,Eerola, O., and Näätänen, R. (1997). The mismatchnegativity to changes in speech sounds at the ageof three months. Dev. Neuropsychol. 13, 167–174.

Cowan, N. (1984). On short and long auditory stores.Psychol. Bull. 96, 341–370.

Dehaene-Lambertz, G., and Baillet, S. (1998). A phonological representation in the infant brain.NeuroReport 9, 1885–1888.

Escera, C., and Grau, C. (1996). Short-term replicabil-ity of the mismatch negativity. Electroencephalogr.Clin. Neurophysiol. 100, 1–6.

Giard, M.-H., Perrin, F., Pernier, J., and Bouchet, P.(1990). Brain generators implicated in processingof auditory stimulus deviance: A topographicevent-related potential study. Psychophysiology 27,627–640.

Goodin, D. S., Squires, K. C., Henderson, B. H., andStarr, A. (1978). An early event-related corticalpotential. Psychophysiology 15, 360–365.

Hämäläinen, H., Kujala, T., Kekoni, J., Hurskainen,H., Pirilä, J., Wikström, H., and Huotilainen, M.(2002). Effects of unilateral hippocampus-amygdala resection on arousal, orienting andsensory memory: A case study with EEG andMEG. Submitted.

Hawkins, H. L., and Presson, J. C. (1986). Auditoryinformation processing. In “Handbook ofPerception and Human Performance” (K. R. Boff,L. Kaufman, and J. P. Thomas, eds.), pp. 26.1–26.64. Wiley, New York.

Ilvonen, T. M., Kujala, T., Tervaniemi, M., Salonen, O., Näätänen, R., and Pekkonen, E. (2001). Theprocessing of sound duration after left hemispherestroke: Event-related potential and behavioralevidence. Psychophysiology 38, 622–628.

Kallio, J., Kujala, T., Tervaniemi, M., and Näätänen, R.(2002). The test–retest replicability of the MMN inpaired-tone paradigm. In preparation.

Kane, N. M., Curry, S. H., Butler, S. R., and Gummins,B. H. (1993). Electrophysiological indicator ofawakening from coma. Lancet 341, 688.

Kane, N. M., Curry, S. H., Rowlands, C. A., Manara,A. R., Lewis, T., Moss, T., Cummings, B. H., andButler, S. R. (1996). Event-related potentials-neurophysiological tools for predicting emergenceand early outcome from traumatic coma. IntensiveCare Med. 22, 39–46.

Karma, K. (1994), Auditory and visual temporal struc-turing: How important is sound to musical think-ing? Psychol. Music 22, 20–30.

Karma, K. (1999). Auditory structuring in explainingdyslexia. In “Proceedings of the Eight International



Workshop on the Cognitive Science of NaturalLanguage Processing (CSNLP-8), August 9–11,1999” (P. Mc Kevitt, C. Mulvihill, S. Ó. Nualláin, andC. O’Riordan, eds). Information Technology Centre,National University of Ireland, Galway, Ireland.

Kathmann, N., Frodl-Bauch, T., and Hegerl, U. (1999).Stability of the mismatch negativity under differ-ent stimulus and attention condition. Clin.Neurophysiol. 110, 317–323.

Kaukoranta, E., Sams, M., Hari, R., Hämäläinen, M., and Näätänen, R. (1989). Reactions of humanauditory cortex to changes in tone duration. Hear.Res. 41, 15–22.

Koelsch, S., Schröger, E., and Tervaniemi, M. (1999).Superior attentive and preattentive auditory pro-cessing in musicians. NeuroReport 10, 1309–1313.

Kohlmetz, C., Altenmueller, E., Schuppert, M., Wieringa, B. M., Muente, T. F. (2001). Deficit inautomatic sound-change detection may underliesome music perception deficits after acute hemi-spheric stroke. Neuropsychologia 39, 1121–1124.

Kraus, N., Micco, A. G., Koch, D. B., Mcgee, T., Carrell, T., Sharma, A., Wiet, R. J., andWeingarten, C. Z. (1993a), The mismatch negativ-ity cortical evoked potential elicited by speech incochlear-implant users. Hear. Res. 65, 118–124.

Kraus, N., McGee, T., Ferre, J., Hoeppner, J.-A., Carrel,T., Sharma, A., Nicol, T. (1993b). Mismatch nega-tivity in the neurophysiologic/behavioral evalua-tion of auditory processing deficits: A case study.Ear Hear. 14, 223–234.

Kraus, N., McGee, T., Carrell, T., King, C., Tremblay,K., and Nicol, T. (1995). Central auditory systemplasticity associated with speech discriminationtraining. J. Cogn. Sci. 7, 27–34.

Kraus, N., Mcgee, T., Carrell, T. D., Zecker, S. G.,Nicol, T. G., and Koch, D. B. (1996). Auditory neu-rophysiologic responses and discriminationdeficits in children with learning problems. Science273, 971–973.

Kujala, T., Myllyviita, K., Tervaniemi, M., Alho, K.,Kallio, J., and Näätänen, R. (2000). Basic auditorydysfunction in dyslexia as pinpointed by brain-activity measurements. Psychophysiology 37,262–266.

Kujala, T., Kallio, J., Tervaniemi, M., and Naatanen, R.(2001a). The mismatch negativity as an index oftemporal processing in audition. Clin.Neurophysiol. 112, 1712–1719.

Kujala, T., Karma, K., Ceponiene, R., Belitz, S., Turkkila,P., Tervaniemi, M., and Näätänen, R. (2001b). Plasticneural changes and reading improvement causedby audiovisual training in reading-impaired chil-dren. Proc. Natl. Acad. Sci. U.S.A. 98, 10509–10514.

Lang, H., Nyrke, T., Ek, M., Aaltonen, O., Raimo, I.,and Näätänen, R. (1990). Pitch discrimination per-formance and auditory event-related potentials. In“Psychophysiological Brain Research” (C. H. M.

Brunia, A. W. K. Gaillard, A. Kok, G. Mulder, andM. N. Verbaten, eds.), Vol. 1, pp. 294–298. TilburgUniversity Press, Tilburg, The Netherlands.

Lang, H., Eerola, O., Korpilahti, P., Holopainen, I.,Salo, S., and Aaltonen, O. (1995). Practical issuesin the clinical application of mismatch negativity.Ear Hear. 16, 118–130.

Lounasmaa, O. V., Hari, R., Joutsiniemi, S.-L., andHämäläinen, M. (1989). Multi-SQUID recordingsof human cerebral magnetic fields may give infor-mation about memory processes. Europhys. Lett. 9,603–608.

McGee, T., Kraus, N., and Nicol, T. (1997). Is it really amismatch negativity? An assessment of methodsfor determining response validity in individualsubjects. Electroencephalogr. Clin. Neurophysiol. 104,359–368.

Menning, H., Roberts, L. E., and Pantev, C. (2000).Plastic changes in the auditory cortex induced byintensive frequency discrimination training.NeuroReport 11, 817–822.

Näätänen, R. (1990). The role of attention in auditoryinformation processing as revealed by event-related potentials and other brain measures ofcognitive function. Behav. Brain Sci. 13, 201–288.

Näätänen, R. (1992). “Attention and Brain Function”.Lawrence Erlbaum Assoc., Hillsdale, New Jersey.

Näätänen, R. (1995). The mismatch negativity—Apowerful tool for cognitive neuroscience. Ear Hear.16, 6–18.

Näätänen, R. (2001). The perception of speech soundsby the human brain as reflected by the mismatchnegativity (MMN) and its magnetic equivalent(MMNm). Psychophysiology 38, 1–21.

Näätänen, R., Gaillard, A. W. K., and Mäntysalo, S.(1978). Early selective-attention effect reinter-preted. Acta Psychol. 42, 313–329.

Näätänen, R., Simpson, M., and Loveless, N. E.(1982). Stimulus deviance and evoked potentials.Biol. Psychol. 14, 53–98.

Näätänen, R., Paavilainen, P., Tiitinen, H., Jiang, D.,and Alho, K. (1993a). Attention and mismatchnegativity. Psychophysiology 30, 436–450.

Näätänen, R., Schröger, E., Karakas, S., Tervaniemi,M., and Paavilainen, P. (1993b). Development of amemory trace for a complex sound in the humanbrain. NeuroReport 4, 503–506.

Näätänen, R., Lehtokoski, A., Lennes, M., Cheour, M.,Huotilainen, M., Iivonen, A., Vainio, M., Alku, P.,Ilmoniemi, R. J., Luuk, A., Allik, J., Sinkkonen, J.,and Alho, K. (1997). Language-specific phonemerepresentations revealed by electric and magneticbrain responses. Nature 385, 432–434.

Opitz, B., Rinne, T., Mecklinger, A., von Cramon, D.Y., and Schröger, E. (2002). Differential contribu-tion of frontal and temporal cortices to auditorychange detection: fMRI and ERP results.NeuroImage 15, 167–174.

REFERENCES 353


Paavilainen, P., Alho, K., Reinikainen, K., Sams, M.,and Näätänen, R. (1991). Right-hemisphere domi-nance of different mismatch negativities.Electroencephalogr. Clin. Neurophysiol. 78, 466–479.

Paavilainen, P., Tiitinen, H., Alho, K., and Näätänen,R. (1993). Mismatch negativity to slight pitchchanges outside strong attentional focus. Biol.Psychol. 37, 23–41.

Pekkonen, E., Jousimäki, V., Könönen, M.,Reinikainen, K., and Partanen, J. (1994). Auditorysensory memory impairment in Alzheimer’sdisease: An event-related potential study.NeuroReport 5, 2537–2540.

Ponton, C. W., and Don, M. (1995). The mismatchnegativity in cochlear implant users. Ear Hear. 16,131–146.

Ponton, C. W., Don, M., Eggermont, J. J., and Kwong,B. (1997). Integrated mismatch negativity (MMNI):A noise-free representation of evoked responsesallowing single-point distribution-free statisticaltests. Electroencephalogr. Clin. Neurophysiol. 104,143–150.

Renault, B., and Lesévre, N. (1978). Topographicalstudy of the emitted potential obtained after theomission of an expected visual stimulus. In“Multidisciplinary Perspectives in Event-RelatedBrain Potential Research” (D. Otto, ed.), EPA600/9-77–043I, pp. 202–208. U.S. GovernmentPrinting Office, Washington, D.C.

Renault, B., and Lesévre, N. (1979). A trial-by-trialstudy of the visual omission response in reactiontime situations. In “Human Evoked Potentials” (D.Lehmann and E. Callaway, eds.), pp. 317–329.Plenum Press, New York.

Rinne, T., Alho, K., Alku, P., Holi, M., Sinkkonen, J.,Virtanen, J., Bertrand, O., and Näätänen, R. (1999).Analysis of speech sounds is left-hemispherepredominant at 100–150 ms after sound onset.NeuroReport 10, 1113–1117.

Rinne, T., Ilmoniemi, R. J., Sinkkonen, J., Virtanen, J.,and Näätänen, R. (2000). Separate time behaviorsof the temporal and frontal MMN sources.NeuroImage 12, 14–19.

Rüsseler, J., Altenmueller, E., Nager, W., Kohlmetz, C.,and Muente, T. F. (2001). Event-related potentialsto sound omission differ in musicians and non-musicians. Neurosci. Lett. 308, 33–36.

Sallinen, M., Kaartinen, J., and Lyytinen, H. (1994). Isthe appearance of mismatch negativity duringstage 2 sleep related to the elicitation of K-complex?. Electroencephalogr. Clin. Neurophysiol.91, 140–148.

Sams, M., Hari, R., Rif, J., and Knuutila, J. (1993). Thehuman auditory sensory memory trace persistsabout 10 msec: Neuromagnetic evidence. J. Cogn.Neurosci. 5, 363–370.

Scharf, B., and Houtsma, A. J. (1986). Audition II:Loudness, pitch, localization, aural distortion,pathology. In Handbook of Perception and Human

Performance (K. R. Boff, L. Kaufman, and J. P.Thomas, eds.), pp. 15.1–15.60. Wiley, New York.

Schröger, E. (1996). A neural mechanism for involun-tary attention shifts to changes in auditory stimu-lation. J. Cogn. Neurosci. 8, 527–539.

Schröger, E. (1998). Measurement and interpretationof the mismatch negativity. Behav. Res. MethodsInstrum. Comput. 30, 131–145.

Shestakova, A., Brattico, E., Huotilainen, M., Galunov,V., Soloviev, A., Sams, M., Ilmonienni, R. I., andNäätänen, R. (2002). Abstract phoneme represen-tations in the left auditory cortex: Magnetic mis-match negativity study. NeuroReport (In press.)

Shtyrov, Y., Kujala, T., Ahveninen, J., Tervaniemi, M.,Alku, P., Ilmoniemi, R. J., and Näätänen, R. (1998).Background acoustic noise and the hemisphericlateralization of speech processing in the humanbrain: Magnetic mismatch negativity study.Neurosci. Lett. 251, 141–144.

Shtyrov, Y., Kujala, T., Ilmoniemi, R. J., and Näätänen,R. (1999). Noise affects speech-signal processingdifferently in the cerebral hemispheres.NeuroReport 10, 2189–2192.

Sinkkonen, J., and Tervaniemi, M. (2000). Towardsoptimal recording and analysis of the mismatchnegativity. Audiol. Neuro-Otol. 5, 235–246.

Tervaniemi, M., Saarinen, J., Paavilainen, P., Danilova,N., and Näätänen R. (1994). Temporal integrationof auditory information in sensory memory asreflected by the mismatch negativity. Biol. Psychol.38, 157–167.

Tervaniemi, M., Ilvonen, T., Karma, K., Alho, K., andNäätänen, R. (1997). The musical brain: Brain wavesreveal the neurophysiological basis of musicality inhuman subjects. Neurosci. Lett. 226, 1–4.

Tervaniemi, M., Kujala, A., Alho, K., Virtanen, J.,Ilmoniemi, R. J., and Näätänen, R. (1999a).Functional specialization of the human auditorycortex in processing phonetic and musical sounds:A magnetoencephalographic (MEG) study.NeuroImage 9, 330–336.

Tervaniemi, M., Lehtokoski, A., Sinkkonen, J.,Virtanen, J., Ilmoniemi, R. J., and Näätänen, R.(1999b). Test-retest reliability of mismatch nega-tivity for duration, frequency, and intensitychanges. Clin. Neurophysiol. 110, 1388–1393.

Tervaniemi, M., Medvedev, S. V., Alho, K., Pakhomov,S. V., Roudas, M. S., van Zuijen, T. L., andNäätänen, R. (2000a). Lateralized automatic audi-tory processing of phonetic versus musical infor-mation: A PET study. Hum. Brain Mapping 10,74–79.

Tervaniemi, M., Schröger, E., Saher, M., andNäätänen, R. (2000b). Effects of spectral complex-ity and sound duration in complex-sound pitchprocessing in humans—A mismatch negativitystudy. Neurosci. Lett. 290, 66–70.

Tervaniemi, M., Rytkönen, M., Schröger, E.,Ilmoniemi, R. J., and Näätänen, R. (2001). Superior



formation of cortical memory traces for melodicpatterns in musicians. Learn. Mem. 8, 295–300.

Tervaniemi, M., Kallio, J., Sinkkonen, J., Virtanen, J.,Ilmoniemi, R. J., Salonen, O., and Näätänen, R.(2002). Test-retest stability of the magneticmismatch response (MMNm). Submitted.

Tiitinen, H., May, P., Reinikainen, K., and Näätänen,R. (1994). Attentive novelty detection in humans isgoverned by pre-attentive sensory memory. Nature372, 90–92.

Trejo, L. J., Ryan Jones, D. L., and Kramer, A. F. (1995).Attentional modulation of the mismatch negativ-ity elicited by frequency differences betweenbinaurally presented tone bursts. Psychophysiology32, 319–328.

Tremblay, K., Kraus, N., and McGee, T. (1998). Thetime course of auditory perceptual learning:Neurophysiological changes during speech-soundtraining. NeuroReport 9, 3557–3560.

Winkler, I., and Näätänen, R. (1992). Event-relatedpotentials in auditory backward recognitionmasking: A new way to study the neurophysio-logical basis of sensory memory in humans.Neurosci. Lett. 140, 239–242.

Winkler, I., Reinikainen, K., and Näätänen, R. (1993).Event-related brain potentials reflect echoic

memory in humans. Percept. Psychophys. 53,443–449.

Winkler, I., Tervaniemi, M., and Näätänen, R. (1997).Two separate codes for missing-fundamental pitchin the human auditory cortex. J. Acoustic. Soc. Am.102, 1072–1082.

Winkler, I., Kujala, T., Tiitinen, H., Sivonen, P., Alku,P., Lehtokoski, A., Czigler, I., Csépe, V., Ilmoniemi,R. J., and Näätänen, R. (1999). Brain responsesreveal the learning of foreign language phonemes.Psychophysiology 36, 638–642.

Woldorff, M. G., Hackley, S. A., and Hillyard, S. A.(1991). The effects of channel-selective attentionon the mismatch negativity wave elicited by deviant tones. Psychophysiology 28, 30–42.

Woldorff, M. G., Hillyard, S. A., Gallen, C. C.,Hampson, S. T., and Bloom, F. E. (1998).Magnetoencephalographic recordings demon-strate attentional modulation of mismatch-relatedneural activity in human auditory cortex.Psychophysiology 35, 283–292.

Yabe, H., Tervaniemi, M., Sinkkonen, J., Huotilainen, M.,Ilmoniemi, R. J., and Näätänen, R. (1998). Thetemporal window of integration of auditoryinformation in the human brain. Psychophysiology35, 615–619.

REFERENCES 355


I

l::etoni~n Io

Left Right . . . .

Finnish Subject CHAPTER 14, FIGURE 3 Language-specific phoneme traces localized in the left temporal lobe reflected by the MMN. Magnetic field-gradient maps of the left- and right-hemisphere MMNs of one typical Finnish subject for Finnish and Estonian deviant vowels. The squares indicate the arrangement of the magnetic sensors. The arrows represent the equivalent current dipoles, indicating activity in the auditory cortex. The Finnish vowel prototype elicits a much larger MMN in the left (compared to the right) hemisphere, whereas the nonprototype responses to an Estonian vowel that does not exist in the Finnish language are small in amplitude in both hemispheres. Adapted from N~iatKnen et al. (1997).

APPENDIX D, FIGURE 4 Realistic 3D color maps. Isovoltage color maps viewed from the right frontal profile for two groups of volunteers performing the same visuomotor task have been computed and projected over a 3D realistic template of the head. Before ERPs were recorded, one group received a period of training (experts) and another did not (nonexperts). Note that the rainbow color scale has been used to represent voltage changes over the scalp, and that before being plotted, the voltage levels have been normalized according to the McCarthy and Wood (1985) rescaling method. A point-by-point between-group variance comparison provided a distribution map of F-values, to which a p-values significance map corresponded, yielding a significant focus of voltage points over the right centroparietal scalp.


A P P E N D I X

A

Recording and NeurochemicalMethods: From Molecules

to SystemsGabriele Biella, Alice Mado Proverbio, and Alberto Zani

INTRODUCTION

Most of the present volume is devotedto the analysis of relationships betweenmind and brain as investigated by meansof scalp electrophysiological recordings, inthe context of a cognitive approach. Themethods and theories of cognitive electro-physiology are dealt with as large systemsfrom the viewpoint of the brain.

To enhance comprehension of the back-ground of neurophysiological findings, wepresent here basic descriptions of neuro-anatomical, electrophysiological, and neuro-chemical techniques for investigating brainfunctions.

STEREOTAXIC NEUROSURGERY

Stereotaxic neurosurgery (SN) is aninvasive surgical procedure that is per-formed on patients after a specific centralnervous system region of concern has been identified. Identification of the exactarea is accomplished using brain imagingmethods such as magnetic resonance (MR)or computerized axial tomography (CAT),applying geometric (or stereotactic) coordi-nates referring to points conventionallydenoted as zero points. In its classic

version, SN is performed followingstraight-line surgical invasion trajectoriesthat proceed from the point of access on thebrain surface toward the target area.Images of the path or segment joining thepoint of entry to target point display anumber of inclinations based on spatialordering (or taxis) in the three spatial direc-tions (stereology) of the final point, withreference to the surgical penetration points.Positional identification of the region isaccomplished by spatial superimposition ofthe region using standard regions (SRs). SRposition and size are based on weightedaverages and supplementary measure-ments made using different subjects tomake virtual maps, which have been pub-lished in stereotaxic atlases. The variousatlases available (e.g., Schaltenbrand andWahren, 1977; Talairach and Tournoux,1988; Ono et al., 1990) satisfy different sur-gical or analytical needs. Talairach andTournoux’s atlas is the reference systemused in experimental research. It makes adistinction between direct and indirectlocalization. Direct localization refers toradiographically detectable tissue or cavitystructures. Indirect localization is carriedout using the method introduced byTalairach in the “reference system” basedon a geometric grid made up of virtuallines projected on the anterior and posterior

commissures (AC and PC) and the corre-sponding vertical lines (VAC and VPC) tobe adjusted to suit the specific case.Stereotaxy is thus the result of a positionalcomputation carried out in the space of thetarget zone relative to the reference points.In neurosurgery, the stereotactic procedureis used in a large number of cases: brainbiopsies, destruction of microlesions,insertion of electrical stimulators, and inter-operative guidance in the case of radio-therapy resection and targeting. In themore modern versions, which are stillcomparatively rare, magnetic stereotaxicsystems (MSSs) are used. Using this tech-nique it is possible to position catheters,electrodes, or small microresection instru-ments made of flexible materials that canfollow curved trajectories. This affords fun-damental advantages such as greater preci-sion in tissue targeting and the possibilityof avoiding passing through areas of partic-ular functional interest. For this type of SNinteroperative data from real-time fluoro-scopic imaging are used in combinationwith preoperative MR images. The opera-tion takes place under magnetic guidance,with a continuous computerized compari-son made between the actual trajectory andthe programmed one, and the consequentpossibility of correcting the trajectory in thecase of any undesirable deviation. The forcepushing the magnet through the brain isprovided by rapidly changing magneticfields induced in the brain by super-conductors connected to the machine.

Stereotaxic neurosurgery cannot be dis-tinguished from the set of proceduresknown as functional neurosurgery (FN),which, to be performed properly, require theapplication of stereotaxic neurosurgery. Theterm functional neurosurgery embraces allprocedures that tend to modify pathologicalconditions of functioning by means of elec-trical or neurochemical stimulation or thedestruction of brain regions of varying size.Examples of FN with SN are pallidotomy,thalamotomy, deep brain stimulation, or celltransplants in brain structures. (Pallidotomy

consists of the surgical removal of the inner-most part of the globus pallidus in order toreduce diskinetic disorders related toParkinson’s disease, whereas thalamotomyimplies the general destruction of the inter-mediate ventral nucleus by thermocoagula-tion in order to reduce tremors.) Thefunctional technique involves the applica-tion of cortical electrode matrixes left in situfor 1–2 days on average in order to deter-mine the extent of epileptic firing, and thatof the less active areas, thus reducing thelikelihood of functional damage due to sur-gical aggression.

NEUROSURGICAL METHODS

In addition to the classical procedures ofremoval by cutting or aspiration of brainmatter, it is also possible to use invasiveprocedures such as thermocoagulation andcryocoagulation, and, above all, noninva-sive procedures such as gamma-knife andX-knife.

Invasive Procedures

Thermocoagulation

Used primarily all to block expansiveprocesses by means of high-frequencyelectric currents, the thermocoagulationelectrode is inserted until it reaches thezone to be coagulated, guided by stereo-tactic techniques. The size of the expansiveprocess determines the intensity of thethermocoagulation current used. The thala-mus and the Gasser (trigeminal) ganglionare elective sites for inducing thermo-coagulation. Typical thermocoagulationoperations are percutaneous rhizolysis ormicrovascular decompression.

Cryocoagulation

Cryocoagulation, using stereotactic tech-niques similar to those used in thermo-coagulation, causes degeneration of thebrain tissue by freezing.

360 A. RECORDING/NEUROCHEMICAL METHODS

APPENDIXES

Noninvasive Neurosurgical Techniques

Stereotaxic Neuroradiosurgery

In stereotaxic neuroradiosurgery, stereo-taxic neurosurgical techniques are com-bined with multiple collimated (convergent)rays in the treatment of numerous kinds ofbrain pathologies. Collimating (arranging inparallel beams) or focusing many singleradiation beams on the target point allowshigh doses of radiation to be brought tobear without particularly large loads alongthe trajectories of single rays.

The different methods available forpotential use are linear accelerator irradia-tion or X-knife, gamma-knife, and charged-particle irradiation. Linear accelerator(LINAC) irradiation entails using of a largenumber of rays produced by a particleaccelerator and emitted along stereotacticarcs. The gamma-knife uses 201 fixedcobalt-60 gamma ray sources distributedover a hemispherical helmet collimator.The sources emit gamma rays that are thenfocused on the target.

Charged-particle irradiation uses threeto five rays in a configuration similar tothat employed in classic radiotherapy, butalso exploiting the deep-penetration char-acteristics of charged particles in order toobtain a highly localized dose distribution.A further innovation is represented byusing frameless stereotactic methods forpositioning instead of metal frame stereo-taxy and involves the use of optical posi-tioning methods based on emission diodesor lasers.

NEURONAL RECORDINGS

Multiunit Activities and Local Field Potentials

Extracellular recordings may be per-formed using electrodes made of differentmaterials (e.g., tungsten, or platinum–iridium alloys) and a recording tip of vari-able diameter. When the tip of the probe

exceeds a given diameter (e.g., >50 µm)numerous multiunit activities (MUAs) orlocal field potentials (LFPs) begin to beobserved. MUAs and LFPs representcomplex forms of integration of summa-tion of the different cellular and synapticcomponents of the circuits, or in any caseof the recorded neuron populations. Theirform is represented graphically as slowdeflections that diverge from the recordingline. The deflections are formed by thevarious signals of simultaneously recordedindividual units that are added together toform a common signal. The MUAs areused preferably to record the spontaneousactivity of a neuron population (extendingfor about 200 µm from the electrode tip),whereas the LFPs are due to inputs activat-ing the dendrites and the soma of theneurons within a diameter of several milli-meters. It should be noted that somesuccess has been achieved in the attemptto correlate activation recorded using func-tional magnetic resonance imaging (fMRI)with the various LFPs in different experi-mental situations. In this way it was foundthat fMRI activation is not linked so muchto the neuron activity as to the processes ofsignal input and intracortical processing,and thus not to spontaneous corticalactivity but to that influenced by input.

Extracellular Neuronal Recordings

The extracellular recording of individualunits can be used to detect the activity ofindividual neurons and to provide infor-mation concerning the state of activation ofthe individual cell inside the circuit con-taining it. The activity of the recordedneurons consists of the sum total of electro-chemical events taking place outside theneuronal membrane and that characterizeneuron depolarization. On average, depo-larizations (and repolarizations) of theneuronal membranes take place in lessthan 1 msec, although some neuronsdisplay faster or slower times. The graphicform used to record depolarization is

NEURONAL RECORDINGS 361

APPENDIXES

known as a spike, that is, a rapid bipolardrop and almost equally rapid return to theisopotential base line accompanied by ashort transient period of hyperpolarization(a type of mechanism that completelyrecharges the neuronal membrane, whichmay be likened to a condenser beingcharged and then discharged).

When the electrode tip (which of coursemust be particularly thin in this case) pene-trates the neuronal membrane, intracellularrecordings can be made. Experimental trialshave been conducted involving simul-taneous recording using a large number ofmicroelectrodes (even more than 100) toallow the recording of a large number ofneurons and thus obtain a complexdynamic image made up of the collectiveactivity of individual units during normalbehavior or in response to experimentaltasks in experimental animals. From theconceptual point of view, this allows thegap between behavioral structures and pos-sibly associated areas of the nervous systemto be reduced. The application of sophisti-cated analysis techniques (correlograms,information estimates, graphs, and othergeometric–algebraic methods) makes itpossible to study also the characteristics ofcoding, stabilizing, or weakening of thesynaptic forces inside a nervous circuitunder different behavioral conditions.

Intracellular Neuronal Recordings

Intracellular neuronal recordings aretechnically more complex, compared toextracellular ones, owing to the need tomaintain the electrode inside the neuronand thus guarantee the absolute stability ofthe recording system. To reduce the techni-cal difficulty involved, intracellular record-ings are generally performed on braintissue regions placed in vitro, which canthus be manipulated under conditions ofabsolute mechanical stability and can guar-antee long periods of recordability. The bigadvantage of intracellular recordings stemsfrom the fact that it is possible to read con-

tinuously all the electrochemical eventsoccurring inside a neuron. It should benoted that the membrane potential of aneuron at rest does not remain fixed andstable but continues to fluctuate more orless perceptibly. Each variation in potentialmay be the result of different events, dueboth to the input from other neurons andto spontaneous variations. When thepotential reaches a given threshold theneuron produces a spike. The large quanti-ties of data obtainable using intracellularrecordings (concerning the electrochemicaland electrodynamic aspects of the depolar-ization processes) in vitro are, however,handicapped by the unavoidable problemthat the brain region (whether in the formof tissue “slices” or as neurons dissociatedin a medium) is dissected and separatedfrom the natural connections regulatingmany of its activities. Recent technicalenhancements have made it possible tomake recordings also in experimentalanimals during cognitive-motor tasks.

Patch-Clamp Neuronal Recordings

Patch-clamp recordings are character-ized by different techniques that share thecommon features of using glass electrodes(very fine, hollow tubular devices) with 2– to 4-µm tips filled with solutions mimic-king cytoplasm conditions. The electrode isplaced in the vicinity of a cultivated cell orof in vitro structures and is brought upagainst the membrane surface. Slight suc-tion is used to create complete contactbetween the membrane and the tip. In thisway continuity is ensured between theinterior of the cell and the liquids insidethe electrode. The various techniques usedin any case afford a reasonable means ofstudying the activity of the individual ionchannels and their modulation under dif-ferent experimental conditions. It becomespossible to evaluate the activity ofthousands of channels when, by studyingneurons or small cells, the entire cell isattracted inside the electrode, or else it is


APPENDIXES

possible to study the dynamics of indi-vidual channels when only small areas ofmembrane are observed. It is possible tostudy the effect of variations in electrolytetype and concentration on the conductancebehavior of the ion channels by placing asecond electrode in the vicinity of the first,in order to allow rapid electrolyte renewalaround the ion channel being observed.

NEURONAL RECORDINGS INCLINICAL PRACTICE

Stereotaxic neurosurgery often involvesthe positioning of (stimulus and recording)electrodes in both surface and deep areas.Electrodes positioned in epidural or sub-dural locations, for example, may benecessary in the case of failure to obtain an accurate topographic identification of the trigger areas of a primary epilepticfocus that cannot be detected even using high-resolution electroencephalographicobservation. Recordings using surgicallypositioned electrodes may be epidural,subdural, or intracerebral. Each strategyhas its own specific risks and characteristicbenefits, which also depend on the thera-peutic strategies following identification ofthe electrochemical characteristics of theregions involved. All the invasive record-ing techniques applied to the centralnervous system have the common disad-vantage of providing a representation ofcomparatively small volumes, but on theother hand allow much more specific andanalytical data to be extracted compared tothose obtained using scalp recordings(without any further artifacts produced bymyograms or movement).

Epidural Electrodes

Use of epidural electrodes is veryrestricted. They are generally used for amore accurate localization of an epilepticfocus. They are positioned on the duramater after drilling small holes in the skull.

Through their contact with the dura, theelectrodes produce a large electroencephal-ogram with no interference or muscular ormovement artifacts. They are marked bylow spatial resolution and are feasible onlyin the zones of cerebral convexity.

Subdural Electrodes

Subdural electrodes are made up ofmatrixes (networks) or multielectrodelinear bands. These electrode arrays areplaced in a subdural position on the corti-cal surface. They consist of flat contactpoints mounted on flexible plastic sup-ports. Of course, because of the size ofthese electrodes a craniotomy is requiredto place them, and so their use is plausibleonly in the case that monolateral record-ings prove sufficient. They do not pene-trate the brain tissue and, in view of thesize of the recording network, collect datafrom over a rather wide area. They areeffective in the identification of focal areasand even of cortical regions of high andlow functional significance. This enablesmapping of the patients’ most widely usedcerebral areas, thus reducing the risk ofsignificant functional lesions during resec-tion surgery, especially in the left temporalregions (for instance, by avoiding damageto the speech areas of the brain duringremoval of the damaged portion).

Deep Intracerebral Electrodes

Deep intracerebral electrodes may bepositioned stereotactically in the deepbrain regions in general, with the help ofMR, CAT, and/or angiographic imaging.Their use has gradually declined since theadvent of modern noninvasive recordingtechniques. Recordings are generally asso-ciated with operating techniques involvingthe positioning of stimulus electrodes fortherapeutic purposes.

The electrodes can be inserted into thebrain through small trephinations in theskull. They may be positioned in the globus

NEURONAL RECORDINGS IN CLINICAL PRACTICE 363

APPENDIXES

pallidus, subthalamic nucleus, lateral inter-mediate thalamic nucleus, orbitofrontalregions, cingulate gyrus, amygdala, or hip-pocampus. These recordings may be used inassociation with those of the scalp or evensubdural recordings, which provide a moregeneral picture of the activities recordedanalytically by the deep electrodes.

NEUROCHEMICAL LESIONS

Experimental neurochemical lesions (tobe distinguished from experimental or clini-cal neuroablation lesions or from thermo- orcryolesions) are used according to differentexperimental strategies. For example, it ispossible to produce lesions selectively in oneof the five nuclear groups that are intercon-nected to form the basal ganglia in the basalforebrain and involved in the modulation ofvoluntary movement. For instance, selectivelesion of the substantia nigra (one of thenuclear groups of the basal ganglia) inprimates has allowed partial models ofParkinson’s disease to be advanced.

Drug Protection on Brain Lesioned Areas

Another strategy involving the use ofexperimental targeted lesions of the nervoussystem evaluates the protective effectivenessof certain drugs or natural factors (e.g.,nerve growth factor) or of ganglioside mole-cules on the lesion. Molecular biology tech-niques have also been introduced toestablish the relationship between the devel-opment of the lesion and the associated geneactivation or deactivation, as well as themodulation of cell receptor synthesis. Forexample, injecting anti-D2 antisenseoligonucleotides (anti-RNA of type 2dopamine receptors) into the cerebral ventri-cles in order to study the role of type 2dopaminergic receptors through their selec-tive elimination in the pathological corollar-ies of animal models of Parkinson’s diseaseallows a specific receptor study to be made,

because gene expression of the D-1 (type 1dopaminergic) receptors remains intact.

Neurotoxins

Typical neurotoxic substances include 6-hydroxydopamine (6-OH-DA); ibotenicacid; 1-methyl-4-phenyl-1, 2, 3, 6-tetrahy-dropyridine (MPTP), a neurotoxin that isvery harmful to substantia nigra neurons;and quisqualic acid, which in suitable con-centration induces apoptosis (programmedcell death) of neurons through the phe-nomenon of excitotoxicity (that is, due toan excess of metabolic–ionic phenomenaassociated with supramaximal neuronalexcitation). Quisqualic acid causes neu-ronal cell death but leaves the fiberspassing inside the cerebral nuclei intact.

Strategies of causing lesions to nervoustissue during development have been usedexperimentally. For example, cyitosinearabinoside (Ara-C) has been used inexperiments involving lesions of corticaland hippocampal stages of development.Administration of Ara-C to pregnantfemale mice produced disgenic micro-cephaly and disorder in the cell arrays inthe CA1 layer of the hippocampus. Ara-Cis an antimitotic drug; when used experi-mentally in the conditions described, itinhibits replication of neurons and progen-itor glial cells during development. Lesionsare created that are selective on the celllines, leading to disorder in the constitu-tion of correct nervous circuits.

AUTORADIOGRAPHY WITH RADIOACTIVE

2-DEOXYGLUCOSE LABELED WITH 14C

Autoradiographic analysis can be basedon functional data that emerge after injec-tion of radioactive 2-deoxyglucose labeledwith 14C (2-[14C]DG) and on the fact that 2-[14C]DG is internalized by active neuronsor (more probably) by the glial cells sur-


APPENDIXES

rounding the neurons, which participatetogether in the glial lactic acid cycle thatserves as the energy substrate for neurons.2-[14C]DG is taken up intracellularly but isnot metabolized, and tends to accumulateintracellularly; 2-[14C]DG is thus taken upin a quantity proportional to the state ofactivation of the neurons. The more activea region, the more glucose it absorbs andthe more 2-[14C]DG it accumulates, whichmarks the level of neuronal activation ofthe region.

The brain is extracted and cut into slicesafter the injection of the labeled product.The material is placed in contact with X-ray-sensitive film, which develops inproportion to the intensity of the radio-active product, thus allowing the activationof the brain areas to be mapped.

CEREBRAL MICRODIALYSIS

One of the available cerebral perfusiontechniques, cerebral microdialysis (CM),can be used to perform an in vivo study ofthe brain neurochemical processes. In thissystem a very thin dialysis tube (less than0.3 mm external diameter) is insertedunder stereotactic guidance into the regionof interest; this allows the passive transferof substances flowing inside the capillaryunder the concentration gradient betweenthe perfusion liquid and the extracellularliquid. The liquid collected by the probeelectrode is subjected to standard chemicalanalysis, such as high-pressure liquidchromatography (HPLC), in order to assaythe transmitter substances of specificinterest.

Because it obviously allows specifictransmitters to be investigated, CM canprove useful in the pharmacological studyof conditions requiring a pharmacokineticinvestigation of transmitters and drugmetabolites. Microdialysis has been used inhuman neurosurgery to study intra-operative and posttraumatic ischemicdamage online.

VOLTAMMETRY

In voltammetry a voltage wave isreleased through a carbon fiber electrodepreviously inserted in the brain understereotactic guidance. At a specifiedvoltage each monoamine releases electronsin the direction of the electrode due to anelectrochemical oxidation mechanism. Thereleased electrons induce a current, theintensity of which is proportional to the quantity of electrons released and thusto the quantity of transmitter present in theregion of concern. Because each transmit-ter has its own elective oxidation current,by calibrating the currents it is possible toobtain accurate information about, as wellas to quantify, the types of transmitterpresent in a given area, even without any a priori knowledge of its presence.

Voltammetry is used primarily for mono-aminergic transmitters (dopamine, nora-drenaline, and 5-hydroxytryptamine orserotonin) and their metabolites. Eachvoltammetry cycle (that is, release of currentwave and measure of quantity of electroniccurrent released, with the identification andquantification of transmitter) takes severalseconds to be completed. The result is there-fore not an immediate and temporally selec-tive measure of the actual concentration atany given instant.

The same techniques, of course withoutany need for stereotaxy, can also be usedfor assaying transmitter concentration insolutions, and for in vitro observations oforgans, especially brain slices.

Fast Cyclic Voltammetry

Fast cyclic voltammetry (FCV) allowsthe quantity of neurotransmitters presentto be measured online. The techniqueallows for repeated measurement every20 msec. For example, it is possible torecord the uptake of monoamine, or ratherthe presynaptic internalization of themonoamines, a mechanism of presynaptictransmitter recovery.

VOLTAMMETRY 365

APPENDIXES

Suggested Reading

Stereotactic NeurosurgeryBenabid, A. L. (1999). Histoire de la stereotaxie

[History of stereotaxis]. Rev. Neurol. (Paris) 155(10),869–877.

Ono, M., Kubic, S., and Abernathey, C. D. (1990).“Atlas of the Cerebral Sulci.” georg Thieme Verlag,Stuttgart.

Schaltenbrand, G., and Wahren, H. (1977). “Atlas ofStereotaxy of the Human Brain.” Georg ThiemeVerlag, Stuttgart.

Talairach, J., and Tournoux, P. (1988). “CoplanarStereotaxic Atlas of the Human Brain.” GeorgThieme Verlag, Stuttgart.

Neurosurgical MethodsHussman, K. L., Chaloupka, J. C., Berger, S. B., Chon,

K. S. , and Broderick, M. (1998). Frameless laser-guided stereotaxis: A system for CT-monitoredneurosurgical interventions. Stereotact. Funct.Neurosurg. 71(2), 62–75.

Nutting, C., Dearnaley, D. P., and Webb, S. (2000).Intensity modulated radiation therapy: A clinicalreview. Br. J. Radiol. 73(869), 459–469.

Neuronal RecordingsFox, K., Glazewski, S., and Schulze, S. (2000).

Plasticity and stability of somatosensory maps inthalamus and cortex. Curr. Opin. Neurobiol. 10(4),494–497.

Logothetis, N. K., Pauls, J., Augath, M., Trinath, T.,and Oeltermann, A. (2001). Neurophysiologicalinvestigation of the basis of the fMRI signal.Nature 412(6843),150–157.

O’Donovan, M. J. (1999). The origin of spontaneousactivity in developing networks of the verte-brate nervous system. Curr. Opin. Neurobiol. 9(1),94–104.

Schanze, T., and Eckhorn, R. (1997). Phase correlationamong rhythms present at different frequencies:Spectral methods, application to microelectroderecordings from visual cortex and fun-tional implications. Int. J. Psychophysiol. 26(1–3), 171–89.

Steriade, M., and Amzica, F. (1998). Coalescence ofsleep rhythms and their chronology in corticothal-amic networks. Sleep Res. Online 1(1),1–10.

Neuronal Recordings in Clinical PracticeLozano, A. M., Hutchison, W. D., Tasker, R. R., Lang,

A. E., Junn, F., and Dostrovsky, J. O. (1998).Microelectrode recordings define the ventralposteromedial pallidotomy target. Stereotact.Funct. Neurosurg. 71(4), 153–63.

Tasker, R. R., and Kiss, Z. H. (1995). The role of thethalamus in functional neurosurgery. Neurosurg.Clin. N. Am. 6(1), 73–104.

Neurochemical LesionsOno-Yagi, K., Ohno, M., Iwami, M., Takano, T.,

Yamano, T., and Shimada, M. (2000). Heterotopiain microcephaly induced by cytosine arabinoside:Hippocampus in the neocortex. Acta Neuropathol.100(4), 403–408.

Pollack, A. E. (2001). Anatomy, physiology, and phar-macology of the basal ganglia. Neurol. Clin. 19(3),523–534.

Fast Cyclic VoltammetryMendelowitsch, A. (2001). Microdialysis: Intra-

operative and posttraumatic applications in neuro-surgery. Methods 23(1), 73–81.

Stamford, J. A. (1989). In vivo voltammetry: Prospectsfor the next decade. Trends Neurosci. 12, 407–412.

Stamford, J. A. (1990). Fast cyclic voltammetry:Monitoring transmitter release in real time.J. Neurosci. Methods 34, 67–72.


APPENDIXES


A P P E N D I X

B

A Synopsis of Neurological Diseases

Rolf Verleger

This synopsis is restricted to neurologi-cal diseases affecting the brain. Thisexcludes disturbances affecting peripheralnerves and neural transmission within thespinal cord. Neurological diseases affectingthe brain may be distinguished by theirclass of etiology. The disease may be(1) degenerative, (2) due to failures ofblood supply, (3) due to inflammation andproblems within the immune system,(4) due to tissue growth (tumors), and(5) directly due to external agents.

DEGENERATIVE DISEASES

In degenerative diseases, brain cells aredamaged due to some endogenous reasonaffecting their metabolism or interferingwith signal transmission. Some progresshas been achieved in understanding theinvolved mechanisms, perhaps most of allin Parkinson’s disease, but nevertheless thecausal agents have still remained obscure.A strong hereditary component is involvedin Huntington’s disease and in some typesof cerebellar degeneration. Some heredi-tary factors have also been demonstratedin Alzheimer’s disease and Parkinson’sdisease, but only in infrequent subtypes of

these diseases. The major obvious riskfactor in the latter two diseases is age.

Alzheimer’s Disease

Alzheimer’s disease is the most commoncause of dementia in elderly people.Pathological markers are plaques andtangles within neuronal tissue, above allwithin the temporal and parietal lobes, butthese markers can so far be identified onlyby neuropatholological examination, i.e.,not while the patients are alive. Thus, stan-dard criteria recommend diagnosis of“probable” Alzheimer’s disease by diag-nosing dementia and excluding otherpossible causes of dementia (vascularencephalopathy, lack of certain vitamins orhormones, and other reasons). The mainsymptom is a deficit of working memorybut, in order that Alzheimer’s disease (anddementia in general) be diagnosed, somesecond capacity in addition to memorymust be affected.

Parkinson’s Disease

The cardinal symptoms of Parkinson’sdisease are stiffness, lack of movement,and tremor during rest. There is often a

slight, diffuse impairment of cognitivefunctions, reminiscent of frontal lobepathology. The main pathological mecha-nism is degeneration of dopamine-producing neurons in the brain stem (i.e.,substantia nigra). Because of this, the basalganglia lack the dopamine needed for theiradequate functioning. The basal gangliaproject in different ways via the thalamusto cortical areas. Dopamine replacementtherapy may alleviate the symptoms for anumber of years.

Other Degenerative Diseases

Cerebellar atrophy denotes a class ofdiseases, some idiopathic, some hereditary,that are characterized by impairments ofmovement precision and of balance, obvi-ously related to shrinkage of cerebellarvolume. Often, in the course of the disease,pathology progresses from the cerebellumto neighboring structures (olivo-ponto-cerebellar atrophy, OPCA).

Huntington’s disease is a hereditarydegenerative disease of the brain, focusingon parts of the basal ganglia (nucleus cau-datus and putamen), producing hyper-kinesia, akinesia, and dementia. A coresymptom of the disease is excess move-ments (St. Vitus’ dance).

Progressive supranuclear palsy is char-acterized by palsy of vertical saccades, lossof voluntary facial movements, axial dys-tonia, gait disturbance, and dementia.Pathological alterations in several sub-cortical regions form the basis for theseimpairments.

In amyotrophic lateral sclerosis (ALS)there is degeneration of neurons of thepyramidal tract and of the consecutivespinal neurons, progressively reducing apatient’s ability to move. Different fromother degenerative diseases, ALS is neitherhereditary nor a disease of old age, but hasits peak of incidence during the fifthdecade of age.

LESION OF CEREBRAL TISSUE DUE TO FAILURE

OF BLOOD SUPPLY

Brain tissue can be damaged by disor-ders of blood circulation in two ways: byinfarction, i.e., some vessel is blocked suchthat tissue no longer has blood supply, orby hemorrhage, i.e., some vessel is rup-tured, causing blood to overflow into braintissue.

Infarction or hemorrhage affecting thecortex will cause well-delimited symptomsand syndromes. For example, lesions ofanterior branches of the middle cerebralartery (MCA) may cause palsy of the con-tralateral arm, of the contralateral side ofthe mouth and (if in the left hemisphere inright-handers) Broca-type aphasia. Lesionsof posterior branches of the MCA maycause Wernicke-type aphasia, if on the left,and disorders of spatial abilities, includingneglect, if on the right.

INFLAMMATORY DISEASES

Inflammation of the brain may be acute,possibly life-threatening, in meningoen-cephalitis, and may be caused by bacteriaand viruses (or, hard to distinguish clini-cally, by fungi). There are also chronicinflammatory-type processes caused bymultiple sclerosis, by the human immun-odeficiency virus (HIV), or by prions.

Multiple sclerosis (encephalomyelitisdisseminata) is characterized by a diversityof symptoms, due to inflammatory-typelesions of neuronal axons and the insulat-ing myelin, appearing and disappearing ata variety of cerebral, cerebellar, and spinallocations. The optical nerve is frequentlyaffected as a first symptom.

The human immunodeficiency virus fre-quently affects the central nervous systemand may cause cognitive impairment up toHIV-associated dementia, as part of theacquired immunodeficiency syndrome.

368 B. SYNOPSIS OF NEUROLOGICAL DISEASES

APPENDIXES

TUMORS

Tissue may grow within the head,thereby causing either unspecific symp-toms (nausea, epileptic seizures, etc.) orsymptoms specific to their location, in thiscase similar to consequences of infarction.Depending on the particular tissue, growthmay be benign, i.e., slow and not penetrat-ing other tissue, or malign, i.e., fast andinfiltrating.

EXTERNAL AGENTS

The brain may be damaged by accidentsand poisonous substances. The mostimportant poisonous substance is alcohol,causing cerebellar dysfunction and, medi-ated by lack of vitamin B1, several symp-toms summarized as Wernicke–Korsakowsyndrome. In chronic alcoholics, acutealcohol withdrawal may lead to epilepticseizures.

EXTERNAL AGENTS 369

APPENDIXES


A P P E N D I X

C

State-of-the-Art Equipment for Electroencephalographic

and MagnetoencephalographicInvestigation of the Brain and Cognition

Alberto Zani and Alice Mado Proverbio

INTRODUCTION

This appendix provides basic informa-tion on the equipment necessary in amodern laboratory for recording and ana-lyzing electromagnetic brain signals.

First is a description of a laboratory forrecording electroencephalogram (EEG)and event-related potentials (ERPs) withhigh-density electrode placement. Verylarge laboratories with powerful systemsof calculation can acquire and analyzesignals recorded from more than 100 (e.g.,128) scalp sites. In this case, the largenumber of pieces of equipment are verypowerful in order to manage the largenumber of input channels, but, in prin-ciple, a smaller laboratory would havesimilar equipment with similar features.There is a headbox with input wires arriv-ing from the electrodes on the scalp, thebundles of wires to the analog digital(A/D) converter, and, above all, the ampli-fiers. The storage capacity and the velocityof modern computers allow sophisticatedanalysis algorithms to be applied and

complex calculations to be made onrecorded data, even when the spatialresolution is high (large number of inputchannels) and when the temporal resolu-tion is high (high sampling rate, e.g., 512or even 1024 Hz). Whatever the size of anelectrophysiology laboratory, they all havein common certain principles and stan-dards of construction and function thatensure that recording procedures arecarried out correctly with a high level ofsafety. For example, sound-proofing of therecording cabin, electromagnetic shielding,temperature regulation, and earthing of allequipment, as well as the recording envi-ronment, are all essential and obligatoryprocedures for any EEG laboratory of anylevel to carry out. It is not our intention todiscuss these aspects [for excellent discus-sions of electrophysiology laboratories, seethe book by Regan (1989) and the reviewby Greene et al. (2000)].

Also included here is a consideration ofthe characteristics of a modern magneto-encephalography laboratory.

THE ELECTROPHYSIOLOGYLABORATORY FOR RECORDING

EEG AND ERPs

(Fig. 1A–C) show the typical arrange-ment of a modern, medium-sized, com-puter-based electrophysiology laboratorymainly devoted to ERP recording inhealthy volunteers in order to investigatethe relationships between the mind and thebrain. As shown in Fig. 1A, the volunteeris made to sit in a comfortable chair, prefer-ably one with a high back that supports thetorso and head, in front of a videoscreen(A) connected by a cable to the computerresponsible for managing the stimuli and

experimental tasks. Both the volunteer andthe screen are inside an electricallyshielded cubicle (B), which functions as aFaraday cage (C in Fig. 1B). In order toallow the height of the screen to beadjusted according to the individual’sheight, the screen (A) shown in Fig. 1B issituated on a height-adjustable support (D)fixed to a table (E), the height of this latteralso being adjustable. The table has twoblockable wheels to allow the distancefrom the observer to be altered easily,depending on the requirements of theexperiment. An infrared digital video-camera with adjustable focus (F) is fixed tothe monitor of the stimulator. This video-camera photographs the volunteer and

372 C. EEG AND MEG EQUIPMENT

APPENDIXES

FIGURE 1 Illustration of typical equipment for EEG recording used in a modern ERP laboratory. The actuallaboratory represented (with kind permission from the Director) is in the Institute of Neuroscience andBioimaging of the National Research Council, Milan, Italy. (A) A volunteer sits inside the electrically and magnet-ically shielded cubicle facing a computer monitor and an infrared videocamera. (B) Entrance to the recordingcabin. Part of the Faraday cage is visible on the ceiling; laterally, it is hidden behind the walls. (C) Overall view ofmost of the electrical devices necessary for carrying out ERP recordings and analysis. See text for explanation ofletters.

sends the images to the control screen (P, inFig. 1C) in order to allow the researcher tomonitor any head, eye, or body move-ments throughout the recording session.

The volunteer shown in Fig. 1A wearsan elastic cap (H) on her head. The elec-trodes are fixed to the cap, which is main-tained under tension by a pair of elasticbraces attached to a harness fixed aroundthe volunteer’s chest. The flat wires fromthe cap, with their relative connectors,allow the analog signals recorded by thesingle electrodes to be transmitted to theindividual input channels of the EEGheadbox (I), keeping the transmissionchannel for each electrode completely sep-arate from those of the others. Thisheadbox has the important function,besides merely transmitting the signals tothe amplifiers through a shielded flat wire(J in Fig. 1C), of pre-amplifying the signals.During recording the curtain is kept closedand the area dimly lit. Figure 1A showshow the volunteer rests her arms on thearms of the armchair and places her indexfingers on the response pads, which, ifrequired, will record any response timesduring the experiment. These data can berecorded permanently on the hard disk ofthe stimulator (K in Fig. 1C), or can be keptin working memory (or random accessmemory) and subsequently sent throughthe COM1 or USB serial port into the localnetwork to the so-called master (L) com-puter, which links the behavioral responsesemitted by volunteers to single stimuli tosingle EEG sweeps synchronized withthese latter, before permanently storingboth on the hard disk.

The computer that is dedicated to pre-senting the acoustic, visual, or linguisticstimuli (K) is called the slave computer,because it carries out commands fromanother computer in the local network, themaster computer, shown in the center of thefigure (L). It is the master computer thatstores all of the volunteer’s experimentalacquisition and stimulation data. Thestimulator creates the sensory stimuli. In

order to do this it has a graphic programdesigned to produce visual stimuli, and asound and voice synthesis board, and foronline presentation of these stimuli for theexperiments a device called a video splitter(M) (Fig. 1C, on the left of the slavecomputer), placed below the earphonesand above the auditory interface. Thevideo splitter allows the stimuli stored onthe hard disk of the slave computer to bedisplayed on its screen as well as on aremote screen (A) connected via cable andlocated within the electrically shieldedcubicle. This technical solution allows theresearcher to monitor the correct function-ing of the stimulation procedures duringthe recordings. Using a similar logicalsolution, the researcher can monitor thestream of auditory stimuli through the ear-phones available on the left of the stimula-tor (Fig. 1C). On the right of the slavecomputer the 32-channel amplifiers (N) are receiving EEG signals through theshielded flat cable (J) arriving from theheadbox near the volunteer in the shieldedcubicle. The amplifier shown in the photo-graph is powered by a ± 6-V lead batteryrather than by the main electricity source(which is 220 V in most overseas coun-tries). On the left of the amplifiers there arethe luminance-emitting devices (LEDs) thatsignal any excess impedance of the EEGsignal in one or more channels. On theright of the amplifier, there are controls toregulate high-pass and low-pass filters (seeAppendix D for details on the regulationof these filters). The EEG signals thusamplified and filtered are transmitted tothe interface (O), the “interpreter,” whichallows the amplifiers and A/D conversionboard within the master computer (L) to“talk to each other” efficiently. The A/Dboard converts the bioelectrical analogsignal into numerical values as a functionof time and preselected sampling rate, setvia the master computer’s software. Thevalues thus obtained, and expressed inmicrovolts, are then stored in the bulkmemory of the master computer. At the

THE ELECTROPHYSIOLOGY LABORATORY FOR RECORDING EEG AND ERPs 373

APPENDIXES

end of the recording the raw EEG data,usually a very considerable quantity, canbe transferred directly to another computer(Q), also part of the network, for dataanalysis, or can be downloaded onto a datastorage system, which in the system illus-trated is a CD burner (R).

The EEG signal thus recorded thenundergoes a series of standard analyses byspecifically designed software that allowsthe EEG to be divided and the proceduresof artifact rejection, base line correction,averaging, digital filtering, amplitude andlatency computing of the components ofthe signal, topographic mapping, etc. to becarried out.

Hard copies of the wave forms and thecolor topographic maps calculated on thebasis of the wave forms can be printed bythe printer (S), visible on the table behindthe amplifiers.

THE MEG LABORATORY

A magnetoencephalography (MEG) lab-oratory is very similar to an electro-physiology laboratory but does have some

particular characteristics that distinguish it.Figure 2 offers an example of a possiblesetup of such a laboratory with its differentcomponents. A detail of the magneticallyshielded room (A) can be seen in the upperright. The volunteer and the Dewar probe(C) are inside this room during the experi-ments and recordings. The walls of theroom are made of sheets of aluminium,which predominantly block high-fre-quency fields by the currents induced inthe sheets.

In order to shield out very low-frequency fields as well, several layers of“mu metal” (metal alloys of steel andnickel with an optimal permeability of > 104) are overlapped on the inside wallsof the room in which the recordings fromthe volunteer are made. This metal shieldsthe walls extremely well and can absorbmagnetic field lines. These special alloysare, however, very expensive and as aresult this feature of the MEG laboratory isextremely costly.

Within the shielded room, the volunteeris made comfortable on a transformablehydraulic system (B), which allows posi-tions intermediate between supine and


APPENDIXES

FIGURE 2 An example of a magneto-encephalography laboratory with its variouspieces of equipment.

sitting straight upright. Figure 2 shows thevolunteer in a seated position with hishead in the helmet, which is the basis ofthe Dewar probe (C), in a vertical position.The sensors in the helmet send the mag-netic fields picked up to the superconduct-ing quantum interference device (SQUID)within the probe.

Before the recording, a digitalizationdevice (not shown here) can record thecartesian coordinates in three dimensions(x, y, z) of the positions of markers(vitamin E oil capsules or digital sensors)placed on the volunteer’s nasion and thetwo auricular depressions. Using thesecoordinates it is possible to superimposethe magnetic fields offline with thefindings of the MRI scanning in order tolocate the MEG dipoles of the anatomicalbrain structures.

Inside the shielded room there is aclosed-circuit videocamera trained on thevolunteer so that his behavior can bemonitored directly. Furthermore, there aredevices producing various stimuli: plastictubes with low distortion of the auditorysignal are usually used for acoustic stimuli,a liquid crystal screen may be used forvisual stimuli and commercially marketedvibrotactile stimulators produce somates-thetic stimuli.

The device (D) amplifying the magneticfields measured by the MEG probe islocated outside the shielded room, togetherwith the unit of bulk storage (E) of thesefields. The experimental protocols admin-istered to the volunteers are managed bythe computer (F) at its workstation (in thelower right of Fig. 2). A videocamera filmsthe volunteer, who can be observed on thescreen (G), shown next to the computer. Atwo-way phone (H) allows communicationwith the volunteer.

Once the recording has been made, thefields can be analyzed, the dipoles com-puted, and these latter be recombined with the MRI tomographic images. This iscarried out by a different computer (I),shown in Fig. 2 at another workstation, on

the bottom left. It is essential that this com-puter has a large working memory andbulk memory (J). It must, in fact, performfast mathematical recombination algo-rithms, as well as store and manage thefiles of “voxel” (Vx), which make up theMRI images. The voxel is the smallestthree-dimensional graphic unit of thedigital image of the MRI scanning. It isderived from progressive layers of thetomographic scanning of the MRI. Thisprogression leads to the addition of thethird dimension to the “pixel,” that is, theindividual square-shaped point that,together with innumerable others, willmake up the digital image on a two-dimensional plane. The smaller the dimen-sion of the voxel, the greater the resolution(or sharpness) of the MRI images becausebrain scanning is proceeding in thinnersections. In order to achieve a faithfulreproduction of images made up of anincommensurable number of voxels, it isthus essential that the computer screenalso has a high graphic resolution, orsharpness, and a high cathode scanningvelocity.

Suggested Reading

Safety Standards Greene, W. A., Turetsky, B., and Kohler, C. (2000).

General laboratory safety. In “Handbook ofPhysiology” (J. T. Cacioppo, L. G. Tassinary, andG. G. Bernston, eds.), 2nd Ed., pp. 951–977.Cambridge University Press, Cambridge.

ERP LaboratoryCacioppo, J. T., and Tassinary, L. G. (eds.) (1990).

“Principles of Psychophysiology. Physical, Social,and Inferential Elements.” Cambridge UniversityPress, Cambridge.

Coles, M. G. H., Donchin, E., and Porges, S. W.(1986).” Psychophysiology: Systems, Processesand Applications.” Guilford Press, New York.

Gale, A., and Smith, D. (1980). On setting up apsychophysiological laboratory. In “Techniques inPsychophysiology” (I. Martin, and P. H. Venables, eds.), pp. 565–582. John Wiley & Sons,Chichester.

SUGGESTED READING 375

APPENDIXES

Hugdahl, K. (1995). “Psychophysiology. The Mind-Body Perspective.” Harvard University Press,Cambridge, Massachusetts.

Martin, I., and Venables, P. H. (1980). “Techniques inPsychophysiology.” John Wiley & Sons, Chichester.

Regan, D. (1989). “Human Brain Electrophysiology.Evoked Potentials and Evoked Magnetic Fields inScience and Medicine.” Elsevier, New York.

Stern, R. M., Ray W. J., and Davis C. M. (1980).“Psychophysiological Recording.” Oxford Univer-sity Press, New York and Oxford.

MEG LaboratoryDel Gratta, C., and Romani, G. L. (1999). MEG:

Principles, methods, and applications. Biomed.Technik 44 (Suppl. 2), 11–23.

Hämäläinen, M., Hari, R., Ilmoniemi, R., Knuutila, J.,and Lounasmaa, O. (1993). Magnetoencephal-ography—Theory, instrumentation, and applica-tions to noninvasive studies of the workinghuman brain. Rev. Mod. Phys. 65, 413–497.

Hari, R., and Lounasmaa, O. V. (1989). Recording andinterpretation of cerebral magnetic fields. Science244, 432–436.

Näätänen, R., Ilmoniemi, R. J., and Alho, K. (1994).Magnetoencephalography in studies of humancognitive brain function. Trends Neurosci. 17,389–395.

Regan, D. (1989). “Human Brain Electrophysiology.Evoked Potentials and Evoked Magnetic Fields inScience and Medicine.” Elsevier, New York.

Romani, G. L., Williamson, S. J., and Kaufman, L.(1982). Biomagnetic instrumentation. Rev. Sci.Instruments 53, 1815–1845.

Terbrake, H. J. M., Wieringa, H. J., and Rogalia, H.(1991). Improvement of the performance of a u-metal magnetically shielded room by means ofactive compensation. Measures Sci. Technol. 2,596–601.

Wieringa, H. J. (1993). “MEG, EEG and the Integrationwith Magnetic Resonance Images.” DoctoralThesis, CIP-Gegevens Koninklijke Bibliotheek,Den Haag.


APPENDIXES


A P P E N D I X

D

Recording and Analysis ofHigh-Density Electromagnetic

Signals of the BrainAlberto Zani and Alice Mado Proverbio

INTRODUCTION

An electroencephalogram (EEG) consistsof a surface recording of the difference inelectrical potential between two active sitesof the scalp, or between an active scalp siteand a neutral one. The difference is gen-erated by changes in membrane potentialsof large cell assemblies in the under-lying brain. Excitatory and inhibitory post-synaptic potentials (EPSPs and IPSPs) ofthese assemblies “inwardly” (sink) and“outwardly” (source) flow through thebrain, passing through the skull and thescalp. These potentials may be recordedthrough electrolytic contacts using externalsensors, called electrodes, placed at variouslocations on the surface of the scalp. Due totheir low magnitude, these potentials mustbe run through preamplifiers and ampli-fiers in order to increase their outputmagnitude. These signals are actuallyrepresented by variations of varying rapid-ity in the potential fields, characterized byan amplitude expressed in microvolts(1 µV = 1/1,000,000 V = 1/1000 mV) andby a frequency expressed in cycles/secondor hertz (Hz). After passing through theamplifiers and being converted into aninfinite series of digits, the EEG appears asa continuous, rhythmically “waxing andwaning” wave form. By averaging a

number of EEG sweeps related to an eventrepeatedly occurring over time (bothexogenous—i.e., a stimulus from theoutside world—and endogenous—i.e., anestimation of the time elapsing betweenone stimulus and another), an averageevent-related potential (ERP) is obtained.This is because the theoretical principleholds for the latter technique that, by aver-aging, consistent signals are enhanced andinconsistent signals are cancelled out. Thefollowing discussions provide a primer ofmethods and materials for ERP recordingand analysis.

ELECTRODES

Electrodes normally consist of sensorsmade of metal, owing to the high conduc-tion properties possessed by metals ingeneral. However, because ferrous metalstend to develop spontaneous polarizations,the most suitable metals are platinum,gold, silver, or silver chloride (Ag/AgCl).It is essential that the construction metal bethe same for all electrodes used for EEGrecording and that different metals are notmixed (Davidson et al., 2000). The best elec-trodes for ERP recording are the nonpolar-izable Ag/AgCl type, which can accuratelyrecord very slow changes in potential.

Electrode Types

Electrodes may be found in differentforms. They may consist of hypodermicneedles, hollow or flat disks and cups, orflat disks of fine silver chloride fixed in arigid plastic cup. For the reference elec-trode attached to the ear lobe, so-calledclip-leads are a very practical solutionbecause they can be clipped on like ear-rings. Autoadhesive electrodes prove morepractical for use as ocular electrodes. Withthe exception of hypodermic electrodes, allelectrodes have a hole in the top throughwhich they may be filled with an elec-trolytic jelly, usually composed of water,talc, and salt. The electrode is filled withthe jelly after being attached to the scalp.The jelly provides a “flexible” electrolyticbridge contact with the scalp tissue so thatmovements of scalp or jelly do not disturbthe contact with the electrode metal.

Electrode Placement

Up to only a few years ago, electrodes forresearch purposes were affixed to the scalpby means of an adhesive. The most com-monly used adhesive was collodion, whichis celluloid dissolved in ether (removableonly with acetone or ethyl alcohol).Collodion was available in various commer-cial formats, typically in small tubes or aslarge jars of liquid. It was enough to place alittle of the tube collodion around thecontour of the electrodes and to dry it usingcold or warm air jets (for instance, usingblasts of compressed air, or a hair dryer).With liquid collodion, a small quantity waspoured into a small beaker or dish, andsmall gauze patches were soaked in it beforebeing placed over the electrode. After dryingby one of the methods mentioned above, thegauze patches firmly secured the electrodesin place.

Regardless of the electrode placementmethod used, in order to enhance elec-trolytic contact and conductance level,before positioning the electrode on thechosen point on the scalp it was necessary

to abrade the scalp slightly. This wasaccomplished by rolling a blunted obsidiantip over the scalp site (so-called skindrilling), or by rubbing the scalp with aspecial abrasive paper or cotton flocksoaked in abrasive paste. In general, theabrasive paste was made of water, pumice,and salt. Because of the greater sensitivityof skin, especially that of the face, com-pared with the scalp, electrolytic contactbetween electrode and skin required lightabrasion of the skin using a cotton flock or,better, special commercially availablepreprepared abrasive wipes, soaked in analcohol solution.

These procedures were long andtedious. Luckily, only a few electrodes,mostly affixed to few sites of the ante-rior–posterior midline, were used by mostresearch groups engaged in the investiga-tion of brain and cognition.

Electrode Caps

The modern application of recordingelectrodes has been greatly facilitated bythe recent introduction of special elasticcaps; these are kept in place by strapsattached to a chest harness, on which arefixed a number of Ag/AgCl electrodes,covered by perforated plastic cups, whichcan accommodate a minimum of 16 or 32,and a maximum of 128, electrodes. High-density electrode arrays obviously guaran-tee a better spatial resolution of the EEGsampling, which can be very useful, espe-cially when the objects are source analysis(the localization of the underlying braingenerators) or the combining of ERPsignals with hemodynamic or functionalneuroimaging data. There is evidence thatusing a large number of electrodesimproves spatial resolution in identifyinghighly localized patterns of EEG activity(Davidson et al., 2000). For example,Srinivasan et al. (1998) showed how focalhot spots could be washed out by 19- and32-channel recording montages as a resultof spatial aliasing.

378 D. RECORDING AND ANALYSIS

APPENDIXES

Regardless of the number of electrodesfixed to the cap, it is important for the capto adhere tightly to the head of the individ-ual subjected to recording, so as to ensurebetter contact between the electrodes andthe scalp. Furthermore, it is important toensure that the cap fits the head properly.In other words, it must not be too tight ortoo large as a result of excessive stretchingof the elastic tissue of the cap or loosesagging over the head, because this couldproduce a progressive drift of the elec-trodes away from the initially electrolyti-cally active scalp locations, or inadequatecontact of the electrodes. In both cases, thisleads to recording difficulties involving rel-atively noisy EEG signals, not to mentionmisleading localizing data when sourceanalysis is concerned. All this may beavoided by using electrode caps of differ-ent sizes for individuals of different ages,height, or gender. Head size may actuallyvary considerably even among adults. Forexample, average head sizes are usuallylarger for men than for women.

After fitting the cap to the head, theelectrodes are filled with electrolytic jellyusing a blunted hypodermic syringeneedle, which is used also for gently abrad-ing the skin beneath the electrode bymeans of small rotational movementsduring the insertion of the jelly.

Electrode Impedance

After completing the procedure ofaffixing the electrodes, the effectiveness ofelectrode contact is assessed by measuringthe resistance, defined as R, and measuredin ohms ( Ω ), of the skin (or scalp) to thepassage of the current; the impedance ofthe electrodes is measured in thousands ofohms, using special external or software-controlled impedance meters. It is commonpractice to keep electrode impedance wellbelow 5000 Ω (e.g., Eimer, 1998). Indeed,the guidelines for the use of ERPs inresearch on cognition, prepared by thecommittee appointed by the Society for

Psychophysiological Research (see Pictonet al., 2000), recommend that impedanceshould be reduced to less than 2000–10,000Ω . The inverse of the resistance is conduc-tance (defined as G). Conductance denotesthe facility with which current flows, andis measured in siemens (S) (or mho—i.e.,the inverse of ohm). As already mentionedbriefly, unlike plastics, gums, and clothing,metals and salt solutions display a highconductance.

Electrode Sites

Whatever technique is used for affixingelectrodes to the scalp, they are positionedfollowing the standard coordinates intro-duced in the “ten–twenty” internationalsystem (IS) by the International Federation ofElectroencephalography (cf. Jasper, 1958).These coordinates more or less guaranteeelectrode positioning in certain corticalareas regardless of the shape and size ofthe head of the patient for which the EEGis to be recorded. The coordinates are com-puted taking into account the distancebetween the nasion, which is the nasaldepression at the superior end of the nosebeneath the forehead, and the inion, theposterior apophysis at the bottom of theskull. This distance, which in an adult isusually about 34–36 cm, represents 100%of the total distance between the INz sitelocated on the inion, and the Nz sitelocated on the nasion (see Fig. 1).

By computing percentages varyingbetween 10 and 20% of this distance, it ispossible to position all the electrodes onthe medial line of the scalp passingthrough these sites or lateral to it. Theintersection between the nasion–inion lineand the one joining the two auriculardepressions, which are also positionedtransversely at a distance of 100% (see Fig.1 again), is indicated as the vertex (or Cz).

To give a concrete example, in a headwith a nasion–inion distance of 35 cm, 10%will amount to about 3.5 cm and the occip-ital sites O1 and O2 will lie at this distance

ELECTRODES 379

APPENDIXES

to the left and right of Oz, which is itselfpositioned 3.5 cm above the inion.Traditionally, a full 10–20 montageinvolved 19 active scalp sites, which wereusually supplemented with two recordingleads for the recording of the electrooculo-gram (EOG). However, in modern ERP lab-oratories recordings with as many as 32,64, or 128 channels have become common.To deal with higher density electrode mon-tages, an extended system, defined as the10% system, in which electrode sites areplaced halfway between each of the main

10–20 placements, was advanced byChatrian et al. (1985), and its use was laterchampioned (see Nuwer, 1987).

In order to allow systematic treatment ofthe introduction of standard arrays com-prising a much larger number of electrodesthan those of the traditional 10–20 IS, in1990 the American Electroencephalo-graphic Society commissioned an overhaulof the system described above (seeMyslobodsky et al., 1990). This overhaulentailed partly reinstating the 10% method.The electrodes are indicated by an initial


APPENDIXES

FIGURE 1 The “ten–twenty” electrode system, updated. Electrode nomenclature and positions on the scalp asviewed from the right and left sides and from the front and back. Note how the number of electrodes for topo-graphic studies is very much increased with respect to the traditional 10–20 system originally proposed by Jasper(1958). The electrode sites reported in the traditional system are gray; all of the other electrodes are black. Becauseof the large increment in the electrode number, in the up-to-date system electrodes are affixed equally spacedfrom each other at 10% of the inion–nasion distance.

letter denoting the topographical area inwhich they are placed (for example, C,central; F, frontal; P, parietal; T, temporal; O,occipital) or by means of two or more lettersdenoting intermediate areas (e.g., FC, fron-tocentral; PO, parietooccipital). These lettersare followed by a progressive number thatdepends on the distance of the electrode sitefrom the medial sagittal line, indicated withthe ending z, which denotes the value 0 (thezenith). Odd numbers refer to locations onthe left side of the scalp; even numbers referto locations on the right side of the scalp. Forexample, PO3 is located half way betweenPOz and PO7 on the left of the medial line.

In this revision a number of modificationswere made to the original nomenclatureused for several electrodes. For example,electrodes originally indicated with T3 andT5 on the left hemisphere, and T4 and T6 onthe right, in this new system are denoted asT7 and P7, and T8 and P8, respectively. Moreinterestingly, a number of anatomotomo-graphical research lines based on both post-mortem and computer axial tomographyanalytic studies support the cortical localiza-tion of electrodes indicated by the aforemen-tioned nomenclature. For an exhaustivereview of these research lines, see Homan etal. (1987). Whenever nonstandard sites areused it is important to specify as fully aspossible the position of the electrode sitesused for recording so as to allow interlabora-tory comparison. It is recommended (Pictonet al., 2000) that standard electrode positionsand nomenclature be used whenever possi-ble, and that ERPs be recorded simultane-ously from regularly spaced multiple scalpelectrodes. Also, the way in which the elec-trodes are affixed to the scalp and the type ofreference should always be specified inscientific reports.

BIPOLAR AND MONOPOLAR RECORDINGS

The electrodes may be connectedtogether into several different kinds of

bipolar and monopolar montages. Inbipolar montages all electrodes are con-nected together in chains, with the secondinput to one channel becoming the firstinput to the next channel. The mostcommon use of a bipolar montage is torecord the electrooculogram. In this casehorizontal eye movements are recorded bymeans of two electrodes placed at theexternal canthus of the left and right eyes,and blinks and vertical eye movements arerecorded by means of two electrodesplaced below and above the right eye, witha bipolar montage. In a monopolarmontage, also called a common referencemethod, one electrode is active and theother one (or two electrodes linkedtogether) acts as a reference electrode.Typical arrangements for the referenceelectrode(s) are the linked ears and thelinked mastoids configurations. The refer-ence lead must be as electrically neutral aspossible with regard to brain activity,while it nevertheless records the basicactivity underlying the various physiologi-cal functions of the body in spite of anypossible external noise (electromagneticwaves). What is actually recorded is thedifference that exists between the potentialof the active site and that of the referencesite. The most commonly used referenceleads are the left or right ear (or ear lobe),or the linked or balanced ears (or earlobes), the left or right mastoid, or thelinked or balanced mastoids, the tip of thenose, the chest, and the balanced non-cephalic sternovertebral lead [see Tyner et al. (1983), p. 166, for a buildup scheme ofthis reference lead]. None of these is totallyfree of problems or is the best in anabsolute sense. What is, however, impor-tant is the distance of these sites from theactive electrode. Neighboring sites willcancel out electrical activation sites similarto the two electrodes. Conversely, distantsites will tend to capture more artifactsfrom many different sources. When usinglinked references, such as the two ear lobes(i.e., the so-called A1 and A2) or mastoids

BIPOLAR AND MONOPOLAR RECORDINGS 381

APPENDIXES

(i. e., the so-called M1 and M2), it is advis-able to use a balanced strategy to removepossible variations in impedance betweenthe two linked reference sites. This allows aso-called balanced linked reference to beobtained. The balancing may be achievedin several different ways. For example, thesame weighting (intensity) may beassigned to the two combined signals byconnecting together two low-value (e.g.,5000 Ω ) resistances (R1 and R2), which are

coupled in series to the electrode con-nectors. Alternatively, it is possible to usetwo variable resistances in series and tochange their value as a function of possiblevariations in impedance during recording.Both procedures offer advantages anddisadvantages.

The most important reason for making acareful choice of reference is the stronginfluence it has on the surface topographicdistribution of the bioelectric signal (that


APPENDIXES

FIGURE 2 Examples of spline or isoline maps viewed from the back. The maps have been computed at P1peak latency (120 msec) on a difference wave form obtained by subtracting ERPs to gratings relevant in locationbut irrelevant in spatial frequency, from ERPs to gratings relevant in both features and presented in the rightvisual field. Left: Changes in voltage (microvolts) isolines and, as a consequence, in scalp topography with“offline” rereferencing (linked Fp1–2) with respect to referencing during “online” ERP recording (linked ears).Note that unlike the latter, the linked Fp1–2 reference determines a decrease in positivity at occipitotemporalsites, and an increase in negativity at right temporoparietal sites. Middle: Regardless of the reference, the compu-tation of the Laplacian operator provides the same scalp current density (SCD) (in V/m2) spline map. Here alarger neat positive focus, centered on O1, O2, and POz mesial-occipital scalp sites, and a smaller one, focused atthe posterior-temporal (T5) site, can be observed. (For electrode locations over the posterior scalp maps, see theelectrode montage insert at the bottom.) Right: Examples of misleading representations of SCD maps. Top: A toolarge scale has been chosen, and too few isolines have been drawn in the map to represent the currents’ topogra-phy over the scalp. In this way, not much topographic information is provided. Bottom: Unlike above, here a toosmall scale has been chosen, and too many isolines have been drawn in the map to represent the current sourcetopography over the scalp, providing an overload of information to the observer. It follows that an extra effort isneeded to figure out the topographic information.

is, on its geographic distribution over thescalp). As we have seen, the reference mustbe electrically “silent” (or neutral), butunfortunately there are no absolutely silentpoints in a living biological system.Consequently, because the recorded signalis the result of computing the differencebetween the signal recorded at the activeelectrode and that recorded at the earthedreference, the topographic distribution ofthe EEG signal on the scalp will vary as thereference value varies. This is of more thantrivial importance. The topography of thebiolectric signal from the same individualtaken as the active agent in the same psy-chomotor task will be found to vary as thereference varies (see Fig. 2). This raisesconsiderable problems as regards theidentification of the cerebral areas acti-vated during the performance of the task.To get around them several reference-freesignal transformation solutions have beendeveloped. The first of these consists ofusing the so-called average reference,which, as its name suggests, is based ontaking the average value of all the activeelectrodes as reference (for a detailed treat-ment of the advantages and disadvantagesof this method see Appendix E in thepresent volume). An alternative to this con-sists of using a method that enhances thelocal sources (i.e., radial currents, that is,currents perpendicular to the head), whichminimizes the voltage gradients (or cur-rents tangential to the head) due to spuri-ous correlations among the variouselectrodes. This is done by transformingthe scalp voltage values into scalp currentdensity (SCD) by means of a Laplaciananalysis. It involves solving the Laplaceequation or second derivative of the inter-polated voltage area. By acting as a spatialfilter, this method eliminates the distantsources contribution (or remote potentialfields). In this way it is possible to obtain areference-free topographic representationof brain activation. So no matter what ref-erence is used during the recording, SCDmapping provides a unique topographical

solution (see Fig. 2). This procedure entailsusing a large number of electrodes becauseit is based on computing the differencebetween one specific electrode and manyothers that surround it. For a recordingthat takes in the entire head surface, aminimum of 32 electrodes is thus required(for further details on this method, seelater, the section on “ERP TopographicMapping”).

To prevent charge accumulation duringEEG recordings, participants have to beconnected to charge dispersion devices bymeans of a ground lead linked to the secu-rity plant of the building where the lab islocated, or, much better, to a small pit dugin the earth and constructed in accordancewith the relevant norms. Grounded activeleads, (e.g., Fz) located on the scalp arecommonly used.

Amplifiers

The bioelectric signals detected by theelectrodes are conveyed separately to elec-tronic amplifying devices—one channel foreach electrode site. The amplifiers aremade up of a series of resistors and capa-citors that have the function of filteringand amplifying the signal. Biological poten-tials actually have only tiny voltages—millionths of a volt—which must bestepped up to the level of several tens ofvolts in order to be recorded. The purposeof the amplifiers is to supply energy to tinypotential differences so that they are multi-plied tens of thousands of times. Forinstance, with a gain of 40 the signal ismagnified 40,000 times. Unlike the EEG,the EOG is given a lower amplification.Because the EOG signals are actually gen-erated by the electrophysiological signalsof the eye muscles developed during eyemovement, they have a relatively highamplitude and do not require muchamplification.

To distinguish physiological potentialsfrom potential differences with reference tothe ground, amplifiers amplify the poten-

BIPOLAR AND MONOPOLAR RECORDINGS 383

APPENDIXES

tial difference between their input termi-nals (i.e., electrodes) and are relativelyinsensitive to a potential between these ter-minals and the ground. For this reasonthey are known as differential or balancedamplifiers. The voltage with respect toground common to two electrodes is calledan in-phase or common mode signal. Thepotential difference between the electrodesis called the antiphase or differential signal.The specific property of a balancedamplifier is to record and amplify theantiphase signals while eliminating the in-phase signals. This property is referred toas the high common mode rejection ratio.

“Online” Analog Filters

The electric circuits in the amplifier areequipped with filters. The main compo-nents of a filter usually consist of a resistorand a capacitor (R–C circuit). This circuitallows selective elimination of electric fre-quencies causing disturbance, or in anycase that are extraneous to the brain’s bio-electric activity, such as those due tomuscle movement or to the alternatingcurrent circulating in the electrical equip-ment. Or else, depending on the aim of theresearch, filters allow the elimination ofrecorded physiological signals above orbelow a specified frequency, called theturnover or cutoff frequency (fc). High-pass, low-pass, or band-pass filters cancommonly be found on commerciallyavailable amplifiers. For any of thesefilters, several frequency settings can bemade. With high-pass filters, the settingregulator shows the cutoff frequency abovewhich higher frequencies are passed, andlower frequencies are attenuated. For thisreason they are also called low-frequency,or “L.F. cut,” filters. Conversely, for low-pass filters the cutoff frequency settingindicates the frequency above whichhigher frequencies are attenuated, whilelower frequencies are allowed to pass. Forthis reason they are called high-frequency,or “H.F. cut,” filters. The band-pass filters

delimit a “window” of frequenciesaccepted by the amplifier within twoupper and lower extremes. For example, aband-pass filter suitable for recordingevoked potentials could be 0.01–100 Hz forEOG and 0.1–100 Hz for EEG. However,with studies aimed at investigating cogni-tive processes, which develop more slowlythan sensory processes (for instance, longerlatency linguistic processes), a band of0.01–100 Hz for EEG filtering is certainlymore suitable. Notch filters to exclude theline frequency range (60 Hz for the UnitedStates and 50 Hz for most overseas coun-tries) are not recommended in that theymay significantly distort the recordedsignal (Picton et al., 2000).

Common settings for high-pass filtersindicate, rather than a cutoff frequency,how long it takes for a signal to return tothe base line following an exponentialcurve. This is a curve that approaches itsfinal value at a decreasing rate, with aslope of attenuation that is also called therolloff. This is characterized by the time—usually measured in seconds—withinwhich the signal amplitude falls to 37% ofits initial value before the filter action takesplace. This value, which is independent ofthe magnitude of the initial step, is calledthe time constant (TC). For these filters, then,the cutoff frequency has to be estimatedthrough the following expression:

fc = 1/2TC,

where TC is the time constant and 1/2 is aconstant equal to ~0.159, obtained bydividing 1 by the double of = 3.14. Thus,fc = 0.159/TC may also be found. Forexample, with TC = 3 sec, fc = 0.053 Hz; withTC = 10 sec, fc = 0.0159 Hz. Clearly, in thoserare cases in which fc, rather than TC, isknown, the latter may be obtained by sim-ple transposition of the above-mentionedexpression: namely, TC= 1/2/fc. Forinstance with fc = 0.053, TC = 0.159/0.053 =3 sec. Much care has to be taken over theseinversions in order to communicate accu-rately the characteristics of filters used


APPENDIXES

during experimental recordings. Indeed,some risks are run, not only of com-municating incorrect values of TC becauseof inconsistency in filter nomenclaturesamong amplifier manufacturers, but also ofobtaining misleading data regardingsignals of interest. For instance, whenstudies are carried out aimed at investigat-ing late latency slow processes of the brain,these processes could be cut off by a mis-leading setting of the turnover frequency.

Different definitions are, in fact, reportedfor fc by different manufacturers, and confu-sion may arise. Mostly, fc is defined as thehalf-power frequency, that is, the frequency atwhich the “output power-to-input powerratio” (i.e., also called gain or sensitivity) is0.5. The fc setting, by allowing a gain inpower of 0.5, actually determines a gain inamplitude of 0.707, the latter simply beingthe square root of 0.5 (i.e., power = ampli-tude2). In other words, this means that thesensitivity amounts to 70.7% of itsmaximum value. Indeed, this is thedefinition of fc that is implicitly contained inthe expression used above to compute TC.Because amplifier gain was often tra-ditionally expressed in decibels (dB)—thelogarithmic expression of frequency—thisreduction in gain is the same as a “3-dBdown” of the maximum value.

For some manufacturers, however, fc

indicates the half-amplitude frequency,which is different from the half-power,because it actually marks the frequency atwhich a 50% decrease in sensitivity occurs.In decibels, this entails a further halving ofthe power with respect to 3 dB, that is, a“6-dB down.” Obviously, at this point,when, unknown to the experimenter, insetting up a filter, the half-power is con-fused with the half-amplitude, forexample, at fc = 0.053, the late-latency com-ponents—such as P300—might be partlywashed out because of too fast a TC. Withhalf-amplitude-regulated amplifiers, inorder to record ERPs with, for instance, aTC = 3 sec, the fc value should be lower,namely, 0.031.

EEG Signal Digitation Rate

Using an interface set up between theamplifiers and the computer, the EEGsignals are fed into the analog/digital(A/D) converter in the computer. Thecomputer converts the analogic variationsin potential into a series of discrete digitswith a given digitation rate as a function oftime. What characterizes the strength ofERPs is their high temporal resolution,which allows the faster sensory processingof the brain to be investigated without anytime lag. For example, a digitation rate of512 Hz indicates that over a time span of 1sec the potential value is sampled aboutonce every 2 msec. In some cases, such asin recording auditory brain stem poten-tials, which reach the primary projectioncortex in 10 msec, even a sampling rate of1 or 0.5 msec may be utilized. The set ofsampled points may thus be stored in themass memory or on the hard disk (HD) ofthe computer and subsequently displayedand analyzed in the form of continuousoscillating signals (EEG) using dedicatedgraphics display software. A/D conver-sion should be carried out at a rate that issufficiently rapid to allow those frequen-cies that reflect the EEG and ERP signals ofinterest to be properly recorded. The A/Dconversion rate, as well as the amplifierfilter set up in terms of low and high cutofffrequencies, should always be specified inscientific reports (Picton et al., 2000).

ARTIFACTS IN EEG RECORDING

Electrophysiological artifacts, electricsignals extraneous to the actual brainactivity, overlap the brain activity signals,making them impossible to detect and ren-dering their recorded values unreliable[see Barlow (1986), and Pivik et al. (1993)for a discussion of artifact processing andminimization in EEG recordings]. For thisreason artifact signals are very undesirableand should be identified at the outset in

ARTIFACTS IN EEG RECORDING 385

APPENDIXES

order to be remedied immediately. In orderto eliminate them effectively andefficiently, it must be borne in mind thatthey originate from different sources insideand outside the EEG volunteer’s body andthat the methodological solutions adoptedto avoid them differ according to theirorigin. The following artifacts of externalorigin are among the most important:

1. Alternating current: activity having thesame frequency as the main voltage supply(50 or 60 Hz); to avoid induced alternatingcurrent it is mandatory for all metal struc-tures and equipment in the lab be appro-priately grounded.

2. Switching artifacts: rapid dischargespikes caused by sudden voltage oscilla-tions associated with switching electricalequipment on and off.

3. Radio frequencies: preamplifiers andamplifiers can synchronize with oscillatingfrequencies of radio signals in the ether.

In order to avoid these artifacts therecording area must be shielded. Theshielding may be provided by a Faradaycage, but can also be ensured by a simplegrid made of metal or other conductingmaterial if shielding is not already pro-vided within the silent cabin or cubicle (seeAppendix C, this volume), where data col-lection generally takes place. In this regardit is absolutely essential for the EEG volun-teer to be properly grounded.

Internal or physiological artifacts aredealt with in a different way. The mostimportant and easiest to detect are thoseoriginating from the following sources:

1. Body movements: irregular high-voltage activity due to variations in thedistribution of static electrical charges canoccur when the EEG volunteers rock intheir chair, cough, chew, yawn, or swingtheir legs, or when they wear tennis shoesor when plastic surfaces are not grounded.

2. Electrocardiograms: slow spiked wavessynchronized with the electrocardiogramare detected when a balanced sternoverte-

bral lead is used [for further details seeTyner et al. (1983)].

3. Eye blinks: V-shaped waves, stronglyvisible in the prefrontal area yield a deflec-tion that is positive or positive/negative;during the experiment they should bereduced to less than 10/minute.

4. Horizontal eye movements: slow wavesof variable polarity; in bipolar montagesthe electrode that becomes positive indi-cates the direction of the movement.

5. Muscle activity: isolated spikes causedby contraction of the neck muscles or scalp;it is a good idea to place a cylindricalcushion behind the volunteer’s neck; inpatients suffering from headache due totension a continuous electromyographicactivity is observed in the frontal or occipi-tal regions.

6. Sweating: very slow waves due to thebioelectric activity of the sweat glands areobserved when room temperature is highor when the volunteer is anxious.

In order to avoid artifactual signalsspecial care should also be taken to avoiderrors or methodological oversights duringelectrode assembly and during recording.Artifacts may in fact be the result ofcommon errors such as (1) placementasymmetries (nonsymmetrical amplitude inderivations from homologous regions dueto incorrect electrode positioning or uni-lateral references) and (2) electrode move-ments (irregular spike-shaped or squaredischarges due to momentary interruptionof the electrolytic contact between the elec-trode and the skin).

ERP AVERAGING

The electroencephalogram recordedusing the described technique is subjectedto automated standard procedures, includ-ing breaking down the EEG signals intodiscrete epochs time-locked to stimulus ormental events, of variable duration (forexample, 1 sec), depending on research


APPENDIXES

goals, base line correction, and artifactrejection. When artifacts are intermittentand infrequent (for example, blinks or sac-cades), artifacts rejection is accomplishedby removing contaminated trials from theaveraging process. The rejection may beperformed automatically and/or by visualinspection. Any trial showing electricalactivity greater than a criterion level (e.g., ±50 µV) in any recording channel should berejected from averaging. All proceduresused to eliminate artifacts (such as com-pensation or correction procedures) shouldbe adequately documented.

The averaging technique represents thevery essence of evoked or event-relatedpotentials techniques. By means of averag-ing it is possible to obtain the brain’samplified response to a stimulus or eventsumming numerous EEG sweeps synchro-nized with the stimulus. In general, aver-aging should be sufficient to make thesignal distinguishable from any noise. Themore sums used the better the signal-to-noise ratio. Indeed, this ratio increases as afunction of the square root of the numberof EEG sweeps summed and averaged.Averaging may be computed only if theEEG is time-locked to a series of externalimpulses, called triggers, which temporallycode the simultaneous onset of the stimu-lus and the brain response. In this case,stimulus-locked average wave forms areobtained. Conversely, in the case ofresponse-locked ERPs, the wave forms aretime-locked to mechanical signals, such as,the pressing of a button or an electromyo-gram (EMG) measurement. [For furtherdetails concerning averaging methods seethe authoritative treatise by Regan (1989).]

“Offline” Digital Filtering

Even under ideal conditions and with asufficient number of trials in each ERPaverage, EEG averaging may often leaveresidual noise, including high-frequencynoise contaminating the unfiltered ERPrecord, or any EEG rhythmical activity not

time-locked to any stimulus events.Appropriate digital filtering can effectivelyabolish these residual frequencies, whichcomplicate component identification andmeasurement. This kind of filteringinvolves a wide range of techniques thatshare the fact of being grounded on mathe-matical algorithms applied to discretenumeric representations of continuouswave forms to selectively smooth outcertain frequencies. These algorithms areof two principal types: those operating “inthe time domain,” and those representingsignals “in the frequency domain.” Withthe former, a time series of digital valuesfor voltage or some other parameter as afunction of time is represented. In filteringout noise, these algorithms base their com-putation on a template-matching proce-dure, where the template is determinedfrom “past” data. Decisions concerningnoisy “present” data are then made withrespect to this template, which is cus-tomized for each individual. Seriouscaveats regarding these filters are the lossof data at the beginning and at the end ofthe ERP trace, as well as the introductionof phase errors distorting the final waveform.

Unlike time-based filters, frequency-based digital filters produce filtered waveforms with zero phase shift. This is accom-plished by representing the data using aFourier transform. The principle underly-ing this method is that any stationarywave form may be represented as the sumof a set of sinusoidal wave forms, each of adifferent frequency, amplitude, and phaseangle. As a consequence, three successivesteps must be performed in order to filterundesired frequencies.

As a first step, a direct Fourier transfor-mation of the original time series to thefrequencies that are present in the ERPtrace has to be performed. Second, the fre-quencies of the transform to be abolishedare set to zero. As a last step, an inverseFourier transform reconverts the frequencydomain to the time domain, leaving out

ERP AVERAGING 387

APPENDIXES

the band of frequencies that are set to zero.In this way a filtered version of the ERPwave form is obtained.

SCALP TOPOGRAPHIC MAPPING

The average wave forms obtained sepa-rately for the different stimulus categoriesby means of averaging display a series ofdeflections (or peaks). It may seem obviousthat, regardless of the possible differencesamong stimulus categories, the instant (i.e.,latency) at which the deflection reaches itspeak, and the height reached (i.e., ampli-tude) by the latter, will vary from electrodeto electrode. In other words, in each stimu-lus category the parameters of amplitudeand latency of these peaks (or components)will vary according to both the scalp area(i.e., space) and the poststimulus instant(i.e., time) considered. This is all the moretrue, the greater the number of electrodesused for the electrophysiological recording.It will thus not be easy to perceive thetopographies of the underlying compo-nents with the naked eye.

In order to get some idea of the data it isthus very useful to make a visual displayof the distribution of scalp potential ampli-tude—in other words, to map it. Thismapping may be focused on a singlepotential value, for example, the instant inwhich the component reaches itsmaximum peak at a given cluster of elec-trodes, thus providing a map of the signalamplitude distribution (in microvolts) overthe scalp (or spatial distribution) in thatgiven moment (stationary mapping). Due tothe aforementioned variations in time andspace of the component latency and ampli-tude, however, this kind of map is oftennot a satisfactory substitute for the display-ing of wave forms recorded at multipleelectrodes.

Conversely, the mapping may involve asuccessive voltage time series (dynamicmapping) within a latency range of concern(e.g., 80–140 msec for P1 in spatial atten-

tion studies), thus allowing the computa-tion of multiple maps indicating how thetopography of brain electrofunctional acti-vation varies as a function of time andspace (space–time distribution).

There are different ways of graphicallydisplaying or mapping voltage or currentdensity distributions over the scalp. Thesemay consist of spline or isoline maps repre-senting the aforementioned measures.These types of maps are made using thesame kind of technique used to markcontour lines on geographic maps. Thesignal amplitude value decreases from acentral area of maximum amplitude (thefocus) toward the periphery in a graduallydecreasing number of concentric linearlevels (see Fig. 2 for an example of thesemaps). The number of levels (and, there-fore, of lines) depends partly on signalamplitude and partly on the scale selectedfor representing the amplitude. In addi-tion, frequent use is also made of displaysin which the scalp potential values are rep-resented by different colors or by changesin color saturation of one or two hues, eachcorresponding to a specific range of values(see Fig. 3).

Whatever type of mapping is used, it ismandatory that, before it is carried out, acareful examination is made and a clearidea gained of the wave forms obtained.Mapping cannot be used as a tool for mea-suring the waves. Therefore, if we wish tomake a topographic map of a maximumpeak, we must know precisely at whatlatency the peak reaches its maximum andhave some idea of which electrode orgroup of electrodes displays its maximumamplitude.

With all types of maps, topographicanalysis is based on the mathematicalinterpolation of the potential or of thecurrent density among numerous elec-trodes. The starting point for the mappingis the voltage measured at one instant intime at a certain number of electrodes.However numerous, the electrodes arealways some distance apart and certainly



SCALP TOPOGRAPHIC MAPPING 389


FIGURE 3 Back view of isocontour color saturation maps plotted depicting increases in color saturation levelsof black and white hues. Stimulus and task conditions as well as latency of computation are the same as in Fig. 2.(A) As for spline maps, voltage-related saturation levels of the two hues change as a function of reference lead.The scalp current density (SCD) map has the advantage of providing a reference-free representation of currentsource density over the scalp. (B) Examples of misleading two-hue saturation maps. Top: Too few saturationlevels of the black and white hues provide insufficient information on SCD topography. Bottom: The map showstoo many hue saturation levels and a too small scale was used. This makes the reading of topographic informa-tion far from being straightforward. Note, for example, how the highest saturation level for the white hue coversa large cluster of electrodes without any change in saturation.

do not cover the whole scalp area. To plot adistributed color map, the gap among theelectrodes must be “filled in.” This meansthat probabilistic estimates of values“falling” between the various electrodeshave to be computed, interpolating thetrue values measured at these sites. Theinterpolation may be carried out both two-dimensionally (2D) and three-dimension-ally (3D). Regardless of its mathematicalbasis, it may be explained, with a degree ofsimplification, as a series of successivesteps of reelaboration of the ERP data pro-viding the above-mentioned approximateestimated values. First, the space coordi-nates of the electrodes attached to the scalpare reported in (or rather projected into)the circle—in the case of 2D maps—or the sphere—in the case of 3D maps—thatbest fits the head surface and scalp.Subsequently, over the surface of the circleor the sphere, interpolations are performedamong the values obtained at the variouselectrodes. In interpolating a specifiedpoint (in other words, the voltage orcurrent density to estimate), all the elec-trodes are taken into account. However,electrodes closer to the interpolated point

have a higher “weight” (i.e., influence)than do those more distant [a principle firstadvanced and applied to ERP mapping byBuchsbaum et al. (1982)].

Of interest, in this context, is a specialinterpolation method known as triangula-tion. This is performed by taking points onvertically adjacent top and bottom con-tours, and connecting them anticlockwiseby the shortest lines between the points toform triangles. In this way, signal mappingis possible although the circumference ofthe head is, from top to bottom (or viceversa), smaller in some parts than inothers, thus producing a different numberof points per contour.

Spherical spline interpolation is verygradual and there are no abrupt variationsin the interpolated values. As far as thecolor coding is concerned, the algorithmused is fairly simple. The minimum (Vmin)and maximum (Vmax) voltage values aresingled out and made to correspond to theminimum and maximum indexes on acolor scale. Then, for each interpolatedpoint (Vi), a color index in the scale is com-puted. In practical terms, having a numeri-cally coded scale of N colors (0, 1, 2, 3, ….,


APPENDIXES

FIGURE 4 Realistic 3D color maps. Isovoltage color maps viewed from the right frontal profile for two groupsof volunteers performing the same visuomotor task have been computed and projected over a 3D realistic tem-plate of the head. Before ERPs were recorded, one group received a period of training (experts) and another didnot (nonexperts). Note that the rainbow color scale has been used to represent voltage changes over the scalp,and that before being plotted the voltage levels have been normalized according to the McCarthy and Wood(1985) rescaling method. A point-by-point between-group variance comparison provided a distribution map of F-values, to which a p-values significance map corresponded, yielding a significant focus of voltage points over theright centroparietal scalp. (See color plates.)

N, where 0 corresponds to the minimumand N to the maximum value), such anindex may be obtained by solving the fol-lowing formula:

Index = N (Vi – Vmin) / (Vmax –Vmin).

Whenever Vi is equal to Vmin, the index willbe 0, and so the first color in the scale willbe plotted on the map. Conversely, when-ever Vi is equal to Vmax, corresponding toN, the final color on the scale is plotted onthe map. Obviously, with intermediatevalues of Vi, intermediate colors of thescale relatively close to the first or last colorare used, depending on the index valueobtained. An example of color-coded three-dimensional mapping can be seen in Fig. 4.

As we have already seen, color indexesused for plotting signal topographies arecomputed on the basis of interpolated mea-sures, i.e., Vi, thus among many electrodes,the reliability of these plots depends on thedegree of rigor with which other electrodeshave been “weighted” in determining theindex value at electrode En, and the imme-diate surroundings. This criterion must bedecided before implementing the mappingprocedure by choosing a value for the vari-able . The more stringent the criterion set,the lower the weights of the other elec-trodes and the more realistic the estimatedindex, because, in this way, only currentsradial to the interpolated point over thesphere will be taken into account. This willincrease the spatial resolution with whichthe map reflects on the skull the electro-functional activation of the brain as theflow of mental processes progresses.

The use of stringent values to useradial currents only for signal mapping isspecific to SCD mapping. The method rep-resenting SCDs derived from the spatialderivatives (cf. Perrin et al., 1987, 1989) ofinterpolated values is based on computingthe Laplacian operator. Through this math-ematical procedure the potential at eachelectrode is converted into an estimate ofthe current density entering or exiting thehead at that site. In this way, data distor-

tion due to the spatial smearing of thesurface potential dependent on the volumeconductive properties of the brain and sur-rounding tissues in general is reduced.This has the further advantage that, unlikeisocontour voltage maps, SCD maps areindependent of the location of the refer-ence—that is, they are reference free. One ofthe interesting properties of this method isthat it does not create those possibly mis-leading polarity reversals sometimesreported when using the “average” refer-ence method.

It must be remembered, however, thatprefixed proper values of do not exist.They must change dynamically as a func-tion of electrode montage density. In practi-cal terms, if = 0.0008 represents a propercriterion for mapping ERPs recorded witha 32-electrode montage, this same value isnot proper when 64 or more electrodes areused. In the latter case, in fact, the surfacepotential would be somewhat spatiallysmeared because of the use of aninsufficiently stringent criterion.

Obtaining a focus giving information onthe maximal location of potentialsrecorded or SCD does not, however, meanlocalizing the intracranial source of electri-cal activity. To gain knowledge of thesource, or dipole, far more complex mathe-matical algorithms are required. The basicprinciples of these algorithms may befound in Chapter 2 and Appendix E of thisvolume.

PROBLEMS AND LIMITS OFTOPOGRAPHIC MAPPING

We have seen that topographic mappinghas the advantage of combining into asingle image an overall view of the spatialrelationships existing between the elec-trodes. This information is essential whenERP recordings have been made with sucha large number of electrodes that it issomewhat difficult to grasp how thespatial data evolve. Thanks to technologi-

PROBLEMS AND LIMITS OF TOPOGRAPHIC MAPPING 391

APPENDIXES

cal advances in the field of electronics andabove all the reduction in the cost of elec-trophysiological equipment, this is nowconsolidated practice. The use of topo-graphical maps to sum the results reportedin the literature has thus become increas-ingly widespread.

However, the types of maps used havevaried widely, not only in terms of spheri-cal splines or color isocontour voltagemaps, but also in terms of which and howmany colors are used to represent varia-tions in scalp voltage. This is why Picton et al. (2000) deemed it important to tacklethis matter, providing advice and sugges-tions to be followed in topographicmapping.

In general, problems of interpretingboth spline and isocontour color mapshave proved similar and arise from a lackof standards. For instance, one importantquestion regarding spline maps is thenumber of isolines necessary to displaypotential levels or, in other words, theappropriate distance between the variousisocontours. This problem arises out of thefact that, in order to highlight a voltagefocus, it is possible, for graphic purposes,to modify the above-mentioned number oflines at will during mapping. Clearly, thenumber of lines should be sufficient todisplay the potential difference among thevarious scalp regions. However, using toomany lines could be confusing (see Fig. 2for an example). Guidelines by Picton et al.(2000) indicate that a resolution of 10 levelsshould generally be used.

As far as color maps are concerned, oneadditional problem is that of the choice ofcolor spectrum. It would be possible, forexample, to use simple gray-scale maps inwhich, instead of colors, there are gradualchanges in the shades of gray from black towhite, passing through all the intermediatelevels (see Fig. 3). Again, several authorsprefer to use only two colors to denotezones of positive or negative polarity. Forexample, the colors may vary from a brightred (representing maximum positive

voltage) that gradually changes until itbecomes dark blue at the other end (repre-senting minimum negative voltage),passing through intermediate gradations.However, any other color pair can be used,for instance, blackblue or blackgreen. Withthis two-color representation, however,there is a risk that in the areas where thevoltage turns from positive to negative, thevery light gradations of the two colors mayappear as white, making it difficult to dis-tinguish between the two polarities (see,for example, Fig. 3A).

In most cases, multiple color scales havebeen reported. Rather than a single huegradually changing in saturation, differenthues have been used to represent consecu-tive levels of the scale. With this solution,grasping the various levels of the measurescale proves to be straightforward. How-ever, it also entails the substantial risk ofgiving the observer the impression thatsignificant voltage variations have occurredwhen this is sometimes not true at all (seeFig. 4).

Because several research journals do notaccept color maps, or simply wish to avoidthe heavy printing costs involved, manyelectrophysiologists prefer to use a grayscale. As we saw, however, this has the dis-advantage of not allowing any easy imme-diate identification of positive and negativezones, because it practically corresponds toa coding scheme based on a single color.The maps should thus be used for nonin-ferential but exclusively descriptive pur-poses, and only after having carried outstatistical analyses of the data measured inelectrode subarrays. Or else considerablehelp may be obtained from any availablealgorithms for statistically analyzing thesignificance of possible topographic dif-ferences between two maps, thus allow-ing the t-Student or Fisher’s F-value to bemapped, together with the relative statis-tical probability (or p-value), as shown byway of example in Fig. 4.

The greatest drawback of maps,however, is that they may give erroneous


APPENDIXES

information concerning anatomical local-ization. This is for two main reasons. First,because of volume conduction, source cur-rents flowing in a cluster of cells give riseto extracellular currents that spreadthroughout the conducting volume of thehead and reach the scalp surface throughthe skull. Because the skull varies in thick-ness, depending on skull region, and hasseveral apertures, it is not a homogeneousconductor. This may cause distortion in thetranslation of current onto the scalp, withconsequent possible distortions in scalptopography. Second, with respect to tomo-graphic techniques, for which anatomicalresolution is solidly grounded on eachpixel of the tomographic image, the spatialresolution of ERP maps depends on thenumber of electrodes used, which,however large, is always limited withrespect to the total surface of the scalp. Asa result, the resolution is also limited, espe-cially when it is considered that a veryhigh percentage (nearly 99%) of the pixelsused in the reconstruction of an ERP mapderives from an interpolation of the datafrom a small number of closely spacedelectrodes. In this regard, the larger thenumber of electrodes used, the greater theresolution obtained. In any case, a largenumber of electrodes means high costs interms, for example, of time of application,of the management of the quantity of datarecorded, and of the interpretation of thedata. In spite of this, a number of laborato-ries today use more than 128 electrodes.

Several algorithms have been developedfor the recombination of ERP voltagevalues with 3D anatomical MRI in order tosolve the problem of the localization of this voltage in the cortical brain areas.These algorithms require the 3D digit-ization of the spatial coordinates of theelectrode positioning on the scalp of eachindividual subjected to recording, whichwill subsequently be used for the recombi-nation of voltage pixels interpolated withpixels of the brain computed tomographicimage.

RECORDING OF MAGNETIC FIELDS

(MAGNETOENCEPHALOGRAPHY)

Flowing from their source along theneural pathways of the brain, potentialfields give rise “online” to magnetic fields,orthogonal to the potential fields. The elec-trofunctional activation of the brain is thenactually reflected by its electromagneticcounterparts. The magnetic fields areextremely weak—in the order of 0.4 gauss,amounting to 10–15 T—very much smallerthan the potential fields, the latter being inthe order of microvolts to 10–6 V. For thisreason, only highly sensitive electronicdevices are able to detect and record thesemagnetic signals in the form of a magne-toencephalogram (MEG).

This type of device is based on the useof highly sensitive magnetic sensors calledsuperconducting quantum interference devices(SQUIDs), characterized by so-calledJosephson’s contacts and operating at verylow temperatures, in the order of –273 C°(–460 F°). These temperatures are reachedby maintaining the SQUID continuouslydipped in liquid helium or nitrogen.Indeed, in order to achieve superconduc-tion, the whole device is housed inside asuperinsulated cryogenic columnar struc-ture (see Fig. 5A) mounted on a nonmetal-lic supporting structure. The base of thecryogenic structure is shaped like a helmetthat, in more sophisticated systems, mayhost up to 155 ultrasensitive sensors suit-able for whole-head recordings. As shownin Fig. 5, the columnar structure housingthe MEG probe consists of a superinsu-lated double-walled dewar. The spacebetween the walls is maintained undervacuum to prevent heat diffusion and thesurfaces facing toward the interior aremade of heat-insulating material to main-tain the liquid gases at low temperatures.

Unlike what has been described forERPs, for the detection of event-related fields(ERFs) of the brain the sensors mounted on

RECORDING OF MAGNETIC FIELDS (MAGNETOENCEPHALOGRAPHY) 393

APPENDIXES

the helmet at the bottom of the dewar donot have to be affixed directly to the scalp.In addition, the scalp does not have to beabraded to reduce skin resistance belowoptimal values. Instead, the magneticsensors are simply maintained contiguouswith the head, and located about 20 mm

away from the head, thus making prepara-tion and recording procedures very fast.

The SQUID detects the weak magneticfields irradiating from the brain and con-verts them into electrical signals, which aremuch easier to measure. The electronicprinciple on which this conversion is based


APPENDIXES

FIGURE 5 (A) MEG cryostat and gradiometers. On the left,the superconducting quantum interfering device (SQUID) isshown dipped in liquid helium to guarantee the very low tem-peratures that allow superconductivity of the Josephson’s con-tacts. The SQUID device is hosted within a superinsulatedcryogenic columnar structure consisting of a superinsulateddouble-walled dewar, the base of which is configured into ahelmet for whole-head recordings. On the right, a schematicview shows the features of the helmet, which has a shapebased on head anatomy. Examples of some first-order gra-diometers mounted on the helmet and connected to the SQUIDare also portrayed. Below the helmet, flux transformers, ordetection coils, are configured as a simple magnetometer or asprogressively increasing nth-order gradiometers. Note how thegradiometers are constructed as increasingly complex coils andhow the flux of the bias current fed through them reverts as it passes through the separate loops of the coils. (B) Output ofMEG sensors. Progressive steps of the process of filtering andextraction for auditory ERFs. Conveying the signal through theprogressive orders of software or hardware gradiometersallows the brain’s event-related magnetic response to beenhanced.

is that the current that mechanically trans-ports the “quantum” through a weak bond(i.e., Josephson’s contacts) in a small super-conductor circuit is a function of the mag-netic flux circulating through the circuit.The flow sensitivity of a modern SQUIDcan be as high as 20–30 Tm2. To furtherincrease its sensitivity, the SQUID iscoupled to specially designed external cir-cuits operating as flux transformers,known as detection coils. The simplestconfiguration for such transformers is amagnetometer (see Fig. 5A). This consists ofa single loop of wire inductively connectedto the SQUID, which detects the projec-tions of the magnetic field along the loop.

Bearing in mind that brain magneticfields are weak, and that many nonbiologi-cal sources of disturbance exist in the envi-ronment spatially uniform with respect tothe magnetometer, it is possible to cancelout this environmental noise by buildinginput coils with more complex configura-tions that respond only to the field spatialgradients. This configuration is called agradiometer. Essentially, it consists of a fluxtransformer comprising a system of two ormore subtractive loops. For hardware gra-diometers, the coils may be connected to asingle SQUID sensor, whereas softwaregradiometers are combined with signaldigital filtering software routines. Thelinking of two magnetometers havingopposite flow directions, and thus oppositepolarities, goes to make up a “first-ordergradiometer.” Likewise, the linking of twofirst-order gradiometers of opposite polar-ity makes up a second-order gradiometer,and again, the linking of two second-ordergradiometers of opposite polarity makes athird-order gradiometer. The latter isobtained by altering the distance betweenthe coils in individual gradiometers (seeFig. 5A).

The passage of the field flux through thecoils in the opposite direction cancels outmore uniform fields generated by distantsources. The result is an absence of currentin the gradiometer and of flow inside the

SQUID. Instead, a magnetic field gradientvarying in space will produce a current inthe gradiometer that is equal to the differ-ence across the opposite polarity fluxes inthe various loops of the latter.

Conveying the signal through thevarious different progressive orders ofsoftware or hardware gradiometers allowsthe brain’s spontaneous or event-relatedresponse to be extracted. This not onlycancels out the environmental noise, butalso rejects spurious signals in the order ofpicoteslas (1 pT = 10–12 T), stemming fromthe internal “milieu” (e.g., magnetic fieldsderiving from the myocardiac activity) andoverlapping the aforementioned response.The steps of this progressive process offiltering and extraction for auditory event-related fields are shown in the example inFig. 5B.

The pioneering investigations of MEGwere carried out using a single gradiome-ter. It was thus extremely laborious toobtain a spatial map of the magnetic field.This was because the experimental para-digm had to be repeated many times whilethe gradiometer was moved over the dif-ferent areas of the volunteer’s head.Current commercial multichannel equip-ment has arrays consisting of tens of inputcoils, allowing whole-head recordings tobe made during a single session (seeFig. 5A), even though an optimal signal-to-noise ratio is always obtained by averag-ing a large number of responses (forfurther information on modern MEG labo-ratory equipment, see Appendix C).

Once average ERFs have been obtainedtheir sources may be estimated by thensuperimposing them on a topographicalmap of the field, as is done for ERPs.However, it is more satisfactory to usesophisticated computer algorithms to“merge” the estimated generators of theMEG signal with the 3D MRI imagesobtained from the same volunteer sub-jected to tomographic scanning. In order toachieve this merging, three or more land-mark points (commonly, nasion and preau-

RECORDING OF MAGNETIC FIELDS (MAGNETOENCEPHALOGRAPHY) 395

APPENDIXES

ricular depressions) marked on the volun-teer’s head are used. Vitamin E capsulesare placed on these landmark sites duringMRI scans. Later on, during MEG record-ing, small coils are placed on the landmarksites as where the capsules were. Thepoints thus marked provide a coordinatesystem centered on the volunteer’s head inorder to localize the position of the dewarprobe and the gradiometers. Using compu-tational transformations of coordinatesprovided by vitamin E capsules, MRI scansof the brain are first situated inside thissystem. Thanks to the coil-based coordi-nates, the MEG sources, or dipoles, canthen be laid over the MRI scans, as can beobserved in the image shown in Fig. 6.Further examples of this superimpositioncan be found in this volume in the out-standing reviews of MEG findings onvisual (Chapter 5) and auditory (Chapter14) cognition.

Before concluding this overview of MEGrecording methods it is important to again

stress that, as partly anticipated in Chapter2 and at the beginning of this section, ERPand MEG techniques are complementaryand both present advantages and disad-vantages. The maximum progress in ourknowledge of the brain’s functional activ-ity can thus certainly be obtained only byusing a combination of both recordingtechniques. Although still very expensive,this combination is currently made easierby the fact than many commercial MEGsystems now allow the simultaneousrecording of both MEG fields and ERPs,thus providing integrated amplificationsystems for the recording of both potentialstogether with only MEG equipment.

RECRUITMENT OF VOLUNTEERS

The recruitment of volunteers to partici-pate in brain wave recording studies mustbe monitored very carefully in order toavoid the likelihood of interindividual


APPENDIXES

FIGURE 6 Successive steps of the superimposition of MEG dipoles over an axial MRI image of the brain for asingle volunteer. Left: Examples of averaged event-related fields to standard and deviant tones in the varying-feature condition (top) and subtraction wave forms in the varying- and constant-feature conditions (bottom).Middle: isocontour field maps for N1m and MMMm, recorded in both conditions and viewed from the right-hemisphere, with superimposed equivalent current dipoles (represented by the arrows). Right: The equivalentdipoles are localized on the right hemisphere of the MRI image. Note that they are reported on the left of thebrain image because by convention this is viewed from below. Reprinted from NeuroReport 4; M. Huotilainen, R. J. Ilmoniemi, J.Lavikainen, H. Tititen, K. Alho, J. Sinkkonen, J. Knuutila, and R. Näätänen; Interaction betweenrepresentations of different features of auditory sensory memory, pp. 1279–1281. Copyright (1993) with kindpermission from Oxford Ltd. and Dr. Huotilainen.

variability due to differences in anatomicaland functional brain organization ofvarious individuals; these differences mayhave a greater effect on ERPs than do theexperimental factors of interest.

Before dealing with factors related to therecruitment of participants, a few wordsmust be said about how to refer to volun-teers. The term “experimental subject” isconsidered disrespectful of the human per-sonality (see the guidelines of the Societyfor Neuroscience on the bioethics of humanexperimentation). The term “participants”or “volunteers” is preferred as being morerespectful and more correct. [Oddly, in theguidelines of Picton et al. (2000) for usinghuman event-related potentials to studycognition, the term “subjects” was exten-sively used.] Similarly, the use of pronouns(he, him) or masculine nouns (e.g., “inman”) to indicate individuals of bothgenders should be avoided. The awkwardeffect due to the repeated use of the she/heformula can be avoided, for example, byusing pronouns in their plural form (they).

It is fundamental to gather a homo-geneous sample of participants whenhemispheric asymmetries and handednessare concerned, in order to avoid statisticalfalse negatives due to a misleading mixingof individuals with left- and right-sidedbrain asymmetries, causing a “washingout” effect. To keep this independent vari-able under control, prior to recording, thelateral dominance of the eye, the foot, andthe hand must be measured by means ofsuitable motor tasks or written self-questionnaires [for example, the well-known laterality questionnaire by Oldfield(1971)]. Additionally, control should beexerted over volunteers’ morningness–eveningness preference by means of self-questionnaires, such as the one by Horneand Ostberg (1976). These traits are perva-sively influential, with different diurnalphases over the day, both on overtperformance and psychophysiologicalfunctions (e.g., Mecacci et al., 1984; Zani,1986). Furthermore, somehow control

should be exerted over volunteers’ longerspan biological rhythms (e.g., Zani, 1989)as well as cognitive skills (Zani and Rossi,1990, 1991).

In general, it is advisable to drawsamples from a population as homogenousas possible in terms of age, handedness,educational level, and gender. Becausegender affects many electrophysiologicalmeasurements, experimenters should gen-erally use an equal number of female andmale subjects, or subjects of one genderonly.

Unless investigating brain aging orontogenetic development, when the aim ofthe investigation is the study of normalcognitive brain functions in average youngadults, it is advisable to test individuals inthe age range of 18–40 years. In fact, beforethe age of 18 the frontal executive controlfunction is still somewhat immature,whereas after the age of 40 the beginningof a faint slowing down of brain responsesmay be observed.

Naturally, it is crucial for participants tobe able to perceive the stimuli properly.The experimenter should make sure thatthey have normal hearing (at 20 dB HL),normal or glasses-corrected vision, normalacuity, and color vision. Again, theyshould not have suffered from cranialtraumas followed by coma, and should notbe affected by any psychiatric syndrome(unless the study is aimed directly at indi-viduals suffering from such syndromes, orspecified groups of people with psy-chomotor handicaps). The investigatorshould also make sure that the participantsare not taking prescription medicationsthat may affect cognitive processes, andare not under the influence of alcohol orrecreational drugs.

In general, it is recommended that thetotal number of participants recruited,which should be large enough to representa given population, along with the meanand range of the ages of the participants,be provided in scientific reports. It is alsoessential to obtain the informed consent of

RECRUITMENT OF VOLUNTEERS 397

APPENDIXES

human participants involved in any kindof experimentation (Picton et al., 2000).

References and Suggested Reading

General Normative Guidelines for Using Human EEG–ERP Recordings in ResearchPicton, T. W., Bentin, S., Berg, P., Donchin, E.,

Hillyard, S. A., Johnson, R., Miller, G. A., Ritter, W.,Ruchkin, D. S., Rugg, M. D., and Taylor, M. J.(2000). Guidelines for using human event-relatedpotentials to study cognition: Recording standardsand publication criteria. Psychophysiology 37,127–52.

Pivick, R. T., Broughton, R. J., Coppola, R., Davidson,R. J., Fox, N., and Nuwer, M. R. (1993). Guidelinesfor recording and quantitative analysis of elec-troencephalographic activity in research contexts.Psychophysiology 30, 547–558.

Basic General Techniques in PsychophysiologyCacioppo, J. T., and Tassinary, L. G., and Bernston,

G. G. (eds.) (2000). “Handbook of Psycho-physiology,” 2nd Ed. Cambridge University Press,Cambridge.

Coles, M. G. H., Donchin, E., and Porges, S. W. (1986).“Psychophysiology: Systems, Processes andApplications.” Guilford Press, New York.

Hugdahl, K. (1995). “Psychophysiology. TheMind–Body Perspective.” Harvard UniversityPress, Cambridge, Massachusetts.

Martin, I., and Venables, P. H. (1980). “Techniques inPsychophysiology.” John Wiley & Sons,Chichester.

Stern, R. M., Ray, W. J., and Davis, C. M. (1980).“Psychophysiological Recording.” OxfordUniversity Press, New York and Oxford.

Fundamentals of EEG and ERP TechnologyBarrett, G. (1986). Analytic techniques in the estima-

tion of evoked potentials. In “Handbook of EEGand Clinical Neurophysiology”, Vol.2, ClinicalApplications of Computer Analysis of EEG andOther Neurophysiological Signals” (F. H. Lopes daSilva, W. Storm Van Leeuwen, and A. Rémond,eds.), pp. 311–333. Elsevier, Amsterdam.

Davidson, R. J., Jackson, D., and Larson, C. L. (2000).Human electroencephalography. In “Handbook ofPsychophysiology” (J. T. Cacioppo, L. G. Tassinary,and G. Berntson, eds.), pp. 27–52. CambridgeUniversity Press, Cambridge.

Eimer, M. (1998). Methodological issues in event-related brain potential research. Behav. Res.Methods Instrum. Comput. 30, 3–7.

International Federation of Societies for Electro-encephalography and Clinical Neuropsychology(1974). A glossary of terms commonly used by clinical electroencephalographers. Electro-encephalog. Clin. Neurophysiol. 37, 538–548.

Nuwer, M. R. (1988). Quantitative EEG: I. Techniquesand problems of frequency analysis and topo-graphic mapping. J. Clin. Neurophysiol. 4, 321–326.

Srinivasan, R., Tucker, D. M., and Murias, M. (1998).Estimating the spatial Nyquist of the human EEG.Beh. Res. Methods Instrum. Comput. 30, 8–19.

Tyner, F. S., Knott, J. R., Mayer, W. B. (1983).“Fundamentals of EEG Technology. Vol. 1, BasicConcepts and Methods.” Raven Press, New York.

Zappulla, R. A. (1991). Fundamentals and applica-tions of quantified electrophysiology. Ann. N.Y.Acad. Sci. 620, 1–21.

Electrode Placement Systems forTopographic InvestigationAmerican Electroencephalographic Society (1991).

Guidelines for standard electrode position nomen-clature. J. Clin. Neurophysiol. 8, 200–201.

Chatrian, G. E., Lettich, E., and Nelson, P. L. (1985).Ten percent electrode system for topographicstudies of spontaneous and evoked EEG activities.Am. J. Electroencephalogr. Technol. 25, 83–92.

Jasper, H. H. (1958). The ten–twenty electrode systemof the International Federation. EEG Clin.Neurophysiol. 10, 371–375.

Myslobodsky, M. S., Coppola, R., Bar-Ziv, J., andWeinberger, D. R. (1990). Adeguacy of the inter-national 10–20 electrode system for computedneurophysiologic topography. J. Clin. Neurophysiol.7, 507–518.

Nuwer, M. R. (1987). Recording electrode nomencla-ture. J. Clin. Neurophysiol. 4, 121–133.

Anatomical Location of EEG ElectrodesHoman, R. W., Herman, J., and Purdy, P. (1987).

Cerebral location of international 10–20 systemelectrode placement. EEG Clin. Neurophysiol. 66,376–382.

Artifact Rejection and Minimization MethodsBarlow, J. S. (1986). Artifact processing (rejection and

minimization) in EEG data processing. In“Handbook of EEG and Clinical Neurophysiology,Vol. 2, Clinical Applications of Computer Analysisof EEG and Other Neurophysiological Signals” (F. H. Lopes da Silva, W. Storm Van Leeuwen, andA. Rémond, eds.), pp. 15–62. Elsevier, Amsterdam.

EEG AveragingDawson, G. D. (1951). A summation technique for

detecting small signals in a large irregular back-ground. J. Physiol. (Lond.) 115, 2–3.


APPENDIXES

Regan, D. (1989). “Human Brain Electrophysiology.Evoked Potentials and Evoked Magnetic Fields inScience and Medicine.” Elsevier, Amsterdam.

Basic Principles of Digital FilteringCook III, E. W., and Miller, G. A. (1992). Digital

filtering: Background and tutorial for psychophys-iologists. Psychophysiology 29, 350–367.

Farwell, L. A., Martinerie, J. M., Bashore, T. R., Rapp,P. E., and Goddard, P. H. (1993). Optimal digitalfilters for long-latency components of the event-related potentials. Psychophysiology 30, 306–315.

Ruchkin, D. S. (1988). Measurement of event-relatedpotentials: Signal extraction. In “Handbook ofElectroencephalography and Clinical Neuro-physiology” (D. Otto, ed.), Vol. 3, pp. 7–43.Elsevier, Amsterdam.

MappingCoppola, R. (1990). Topographic mapping of multi-

lead data. In “Event-Related Brain Potentials” (J.W. Rohrbaugh, R. Johnson, and R. Parasuraman,eds.), pp. 37–43. Oxford University Press, Oxford.

Duffy, F. H. (1989). Topographic mapping of brainelectrical activity: Clinical applications and issues.In “Topographic Brain Mapping of EEG andEvoked Potentials” (K. Maurer, ed.), pp. 19–52.Springer-Verlag, New York.

Hassainia, F., Medina, V., Donadey, A., and Langevin,F. (1994). Scalp potential and current densitymapping with an enhanced spherical spline inter-polation. Med. Prog. Technol. 20, 23–30.

Kahn, E. M., Weiner, R. D., Brenner, R. P., andCoppola, R. (1988). Topographic maps of brainelectrical activity—Pitfalls and precautions. Biol.Psychiatr. 23, 628–636.

Perrin, F., Bertrand, O., and Pernier, J. (1987). Scalpcurrent density mapping: Value and estimationfrom potential data. Biomed. Eng. 34, 283–288.

Perrin F., Pernier J., Bertrand O., and Echallier J. F.(1989). Spherical splines for scalp potential andcurrent density mapping. Electroncephalogr. Clin.Neurophysiol. 72, 184–187.

Best EEG Reference for MappingMacGillivray, B. B., and Sawyers, F. J. P. (1988). A com-

parison of common reference, average and sourcederivations in mapping. In “Statistics andTopography in Quantitative EEG” (D. Samson-Dollfus, J. D. Guieu, J. Gotman, and P. Etevenon,eds.), pp. 72–87. Elsevier, Amsterdam.

Pioneering Interpolation Method for Signal MappingBuchsbaum, M. S., Rigal, F., Coppola, R., Cappelletti,

J., King, C., and Johnson, J. (1982). A new systemfor gray-level surface distribution maps of electri-

cal activity. Electroencephalogr. Clin. Neurophysiol.53, 237–242.

Pitfalls and Problems with MappingLopes Da Silva, F. H. (1990). A critical review of clini-

cal applications of topographic mapping of brainpotentials. J. Clin. Neurophysiol. 7 (4), 535–551.

Perrin, F., Bertrand, O., Giard, M. H., and Pernier, J.(1990). Precautions in topographic mapping andin evoked potential map reading. J. Clin. Neuro-physiol. 7 (4), 498–506.

Dipoles and Brain SourcesDe Munck, J. C., Van Dijk, B. W., and Speckreijse, H.

(1988). Mathematical dipoles are adequate todescribe realistic generators of human brain activ-ity. IEEE Trans. Biomed. Eng. BME 35, 950–966.

Scherg, M., and Ebersole, J. S. (1993). Models of brainsources. Brain Topogr. 5, 419–423.

Scherg, M., Vajsar, J., and Picton, T. W. (1989). A source analysis of the late human auditoryevoked potentials. J. Cogn. Neurosci. 4, 336–355.

Combining Electrophysiological andHemodynamic SignalsGevins, A., Brickett, P., Costales, B., Le, J., and

Reutter, B. (1990). Beyond topographic mapping:Towards functional-anatomical imaging with 124-channel EEG and 3-D MRIs. Brain Topogr. 3, pp. 53–64.

Gevins, A., Le, J., Brickett, P., Reutter, B., andDesmond, J. (1991). Seeing through the skull:Advanced EEGs use MRIs to accurately measurecortical activity from the scalp. Brain Topogr. 4, 125–131.

McCarthy, G. (1999). Event-related potentials andfunctional MRI: A comparison of localization insensory, perceptual and cognitive tasks. In“Functional Neuroscience: Evoked Potentials andMagnetic Fields. The 6th International EvokedPotential Symposium” (C. Barber, G. G. Celesia, I. Hashimoto, and R. Kakigi, eds.), pp. 3–12.Elsevier, Amsterdam.

Scherg, M. (1992). Functional imaging and localiza-tion of electromagnetic brain activity. Brain Topogr.5, 103–111.

Use of Statistics in Electrophysiology andMap ComparisonsHassainia, F., Petit, D., and Montplaisir, J. (1994).

Significance probability mapping: The final touchin t-statistics mapping. Brain Mapping 7, 3–8.

John, E. R., Harmony, T., and Valdes-Sosa, P. (1987).The use of statistics in electrophysiology. In“Handbook of EEG and Clinical Neurophysiology,Vol. 1, Methods of Analysis of Brain Electrical and

REFERENCES AND SUGGESTED READING 399

APPENDIXES

Magnetic Signals” (A. S. Gevins and A. Rémond,eds.), pp. 497–540. Elsevier, Amsterdam.

McCarthy, G., and Wood, C. (1985). Scalp distributionof event-related potentials: An ambiguity associ-ated with analysis of variance models.Electroencephalogr. Clin. Neurophysiol. 62, 203–208.

MEG Recording: Principles and MethodsDel Gratta, C., and Romani, G. L. (1999). MEG:



Huotilainen, M., Ilmoniemi, R. J., Lavikainen, J.,Tititen, H., Alho, K., Sinkkonen, J., Knuutila, J.,and Näätänen, R. (1993). Interaction betweenrepresentations of different features of auditorysensory memory. NeuroReport 4, 1279–1281.

Josephson, B. D. (1962). Possible new effects in super-conductive tunneling. Phys. Rev. Lett. 1, 251.

Näätänen, R., Ilmoniemi, R. J., and Alho K. (1994).Magnetoencephalography in studies of humancognitive brain function. Trends Neurosci. 17,389–395.

Romani, G. L., and Rossini, P. (1988). Neuromagneticfunctional localization: Principles, state of the art,and perspectives. Brain topogr. 1, 5–21.

Vrba, J. (1996). SQUID gradiometers in real environ-ments. In “SQUID Sensors: Fundamentals,Fabrication and Application” (H. Weinstock, ed.),pp. 117–178. Kluwer Academic Publisher, NewYork.

Wieringa, H. J. (1993). “MEG, EEG and the Integrationwith Magnetic Resonance Images.” Doctoral

Thesis, CIP-Gegevens Koninklijke Bibliotheek,Den Haag.

Volunteer RecruitmentHorne, J. A., and Ostberg, O. (1976). A self-assessment

questionnaire to determine morningness–evening-ness in human circadian rhythms. Int. J. Chronobiol.4, 97–110.

Mecacci, L., Misiti, R., and Zani, A. (1984). The rele-vance of morningness–eveningness typology inhuman factor research: A review. In “HumanFactors in Organizational Designs and Manage-ment” (H. W. Hendrick, and O. Brown, Jr., eds.),pp. 503–509. Elsevier, Amsterdam.

Oldfield, R. C. (1971). Assessment and analysis ofhandedness: The Edinburgh inventory. Neuro-psychologia 9, 97–113.

Zani, A. (1986). Time of day preference, patternevoked potentials and hemispheric asymmetries:A preliminary statement. Percept. Motor Skills 63,413–414.

Zani, A. (1989). Brain evoked responses reflect infor-mation processing changes with the menstrualcycle in young female athletes. J. Sport Med.Physical Fitness 29, 113–121.

Zani, A., and Rossi, B. (1990). Differences in atten-tional style in skeet and trap shooters: An Event-related brain potential study. In “AIESEP’88 World Congress on Humanism and NewTechnology in Physical Education in Sport”(J. Duran, J. L. Hernandez, and L. M. Ruiz, eds.),pp. 325–329. INEF, Madrid.

Zani, A., and Rossi, B. (1991). Cognitive psycho-physiology as an interface between cognitive andsport psychology. Int. J. Sport Psychol. 22, 376–398.


APPENDIXES

I

l::etoni~n Io

Left Right . . . .

Finnish Subject CHAPTER 14, FIGURE 3 Language-specific phoneme traces localized in the left temporal lobe reflected by the MMN. Magnetic field-gradient maps of the left- and right-hemisphere MMNs of one typical Finnish subject for Finnish and Estonian deviant vowels. The squares indicate the arrangement of the magnetic sensors. The arrows represent the equivalent current dipoles, indicating activity in the auditory cortex. The Finnish vowel prototype elicits a much larger MMN in the left (compared to the right) hemisphere, whereas the nonprototype responses to an Estonian vowel that does not exist in the Finnish language are small in amplitude in both hemispheres. Adapted from N~iatKnen et al. (1997).

APPENDIX D, FIGURE 4 Realistic 3D color maps. Isovoltage color maps viewed from the right frontal profile for two groups of volunteers performing the same visuomotor task have been computed and projected over a 3D realistic template of the head. Before ERPs were recorded, one group received a period of training (experts) and another did not (nonexperts). Note that the rainbow color scale has been used to represent voltage changes over the scalp, and that before being plotted, the voltage levels have been normalized according to the McCarthy and Wood (1985) rescaling method. A point-by-point between-group variance comparison provided a distribution map of F-values, to which a p-values significance map corresponded, yielding a significant focus of voltage points over the right centroparietal scalp.


A P P E N D I X

E

Topographical Analysis ofElectrical Brain Activity:Methodological Aspects

Wolfgang Skrandies

INTRODUCTION

In general, human electrophysiologicalstudies have to rely on noninvasive mea-surements of mass activity originatingfrom many neurons simultaneously (theonly exception being intracranial record-ings in patients before surgery for epilepsyor during tumor removal). The basis forscalp recordings is the propagation of fieldpotentials, originating in large neuronalpopulations, through volume conduction.For brain activity to be detected at somedistance by large electrodes, many neuronsmust be activated synchronously, and onlyactivity originating from “open” intra-cranial electrical fields can be assessed byscalp recordings. In contrast to this,“closed” electrical fields are formed bypopulations of neurons arranged so thatelectrical activity cancels. This constellationis found in many subcortical structuresthat consequently are inaccessible todistant mass recordings. The major neu-ronal sources for electrical activity on thescalp are the pyramidal cells located in thecortical layers, arranged in parallel perpen-dicular to the cortical surface, which,however, is folded in intricate ways so thatthe generators cannot be assumed to beperpendicular to the outer surface of the

brain. The study of evoked (or so-calledevent-related) activity aims at elucidatingdifferent brain mechanisms, and it allowsassessment of sensory or cognitive pro-cessing while the participant or patient isinvolved in perceptual or cognitive tasks.

As in most neurophysiological experi-ments, topographical analysis of electricalbrain activity is used in order to detectcovariations between experimental condi-tions manipulated by the investigator andfeatures of the recorded brain activity.Evoked scalp potential fields yield infor-mation on a number of partly independentneurophysiological parameters such ascomponent latency, which indicates neuralprocessing time, or field strength, whichindexes the amount of synchronous activa-tion of a neuronal population engaged instimulus processing or during the execu-tion of cognitive tasks.

Measures derived from such data areused as unambiguous descriptors of theelectrical brain activity, and they have been employed successfully to study visual information processing in humans(Skrandies, 1987, 1995, 2002). The aim ofevoked potential (or event-related poten-tial) studies is to identify subsets or so-called components of electrical brainactivity that are defined in terms of latencywith respect to some external or internal

event and in terms of topographical scalpdistribution patterns. Irrespective ofwhether the exact intracranial generatorpopulations can be determined, the inter-pretation of scalp potential data combinedwith knowledge of the anatomy and physi-ology of the human central nervous systemmay allow drawing useful physiologicalinterpretations (Skrandies, 2002; see alsoChapter 3, this volume). Scalp topographyis taken as a means to characterize electri-cal brain activity objectively in terms oflatency, neural response strength, and scalplocation. Comparison of scalp potentialfields obtained in different experimentalconditions (e.g., different physical stimulusparameters, different subjective or psycho-logical states, or normal vs. pathologicalneurophysiological traits) may be used totest hypotheses on the identity or non-identity of the neuronal populations acti-vated in these conditions. Identical scalppotential fields may or may not be gener-ated by identical neuronal populations,whereas nonidentical potential fields mustbe caused by different intracranial genera-tor mechanisms. Thus, we can study non-invasively systematic variations of theelectrical brain activity, and we are inter-ested in variations of scalp potential fieldscaused by the manipulation of indepen-dent experimental parameters.

The basic ideas on topographical map-ping are similar irrespective of whetherspontaneous electroencephalogram (EEG)or event-related brain activity is consid-ered. Here we are concerned only with theanalysis of the scalp distribution of evokedelectrical brain activity, with only little con-sideration of the underlying neural struc-tures. [For a review on the localization ofintracranial processes and human electro-physiological signals (EEG and event-related brain activity), the reader is referredto Skrandies (2001).]

With modern imaging techniques [com-puted tomography (CT), structural or func-tional magnetic resonance imaging (fMRI),or positron emission tomography (PET)],

the determination of anatomical brainstructures or of hemodynamical responsesto different processing demands is avail-able at high spatial resolution, but typicallyhas to rely on longer integration times inorder to derive significant signals thatreflect changes in metabolic responses.Different from brain imaging methodssuch as fMRI or PET, noninvasive electro-physiological measurements of sponta-neous EEG and evoked potential fields [ormeasurements of the accompanying mag-netic fields by magnetoencephalography(MEG)] possess high temporal resolution,in the order of milliseconds, and tech-niques to quantify electric brain topogra-phy are unsurpassed when functionalvalidity is required in order to characterizecentral nervous processing in humans. Inaddition, electrical measurements are rela-tively easy and inexpensive to perform,and offer the possibility to assess brainfunction directly in real-life situations,without referring to indirect comparisonsbetween experimental states and neutralbase line conditions or between differenttask demands.

REASONS FOR TOPOGRAPHICBRAIN MAPPING AND

BASIC IDEAS

Electrophysiological data are recordedfrom discrete points on the scalp againstsome reference point, and conventionallyhave been analyzed as time series of poten-tial differences between pairs of recordingpoints. Multichannel recordings allowassessment of the topographical distribu-tion of brain electrical activity. For imag-ing, wave form patterns are transformed todisplays reflecting the electrical landscapeof brain activity at discrete time points orfor different frequency content of therecorded EEG signals. The results of atopographical transformation are mapsthat show the scalp distribution of brainactivity at discrete time points or for

402 E. TOPOGRAPHICAL ANALYSIS

APPENDIXES

selected frequencies of the spontaneousEEG. Such functional imaging (1) possessesthe high degree of time resolution neededto study brain processes, (2) allows charac-

terization of sequences of activation pat-terns, and (3) is very sensitive to statechanges and processing demands of theorganism.

REASONS FOR TOPOGRAPHIC BRAIN MAPPING AND BASIC IDEAS 403

APPENDIXES

FIGURE 1 Potentials evoked by a checkerboard reversal stimulus (60¢ checks, 14 ¥ 18° test field, 95% contrast)recorded from 30 electrodes over the occipital and parietal areas (see electrode scheme in Fig. 2). The same dataset was computed offline for different reference electrodes. A frontal electrode in the midline (A), linked mastoids(B), or the average reference (C) yields small but significant changes in the pattern of wave forms at all recordingelectrodes. Note how amplitudes and latencies of all peaks differ with different reference electrodes. The calibra-tion bars correspond to 256 msec (horizontally) and 5 V (vertically); positive is up. Data from a healthy volun-teer with normal vision.

Conventionally, electrophysiologicaldata were displayed and analyzed as manyunivariate time series. Only the technicalpossibility of simultaneous acquisition ofdata in multichannel recordings allowedfor the treatment of EEG data as potentialdistributions (Lehmanan and Skrandies,1984; Skrandies, 1983, 1986, 1987). Thestrength of mapping of brain activity liesnot only in the display of brain activationin a topographical form, but mapping is aprerequisite of an adequate analysis ofbrain activity patterns. Because EEG andevoked potentials are recorded as potentialdifferences between recording sites, thelocation of a reference point will drasticallyinfluence the shape of activity recordedover time. The basic properties of scalppotential maps and adequate topographi-cal analysis avoid the fruitless discussionabout an electrically neutral referencepoint.

When data are recorded in many chan-nels, at each electrode an evoked potentialwave form is obtained. Figure 1 illustratesconventional checkerboard reversal visualevoked potentials (VEPs) recorded from 30electrodes overlying the occipital and pari-etal brain regions. Over the occipital areas,the wave forms clearly display a negativecomponent at about 70 msec followed bythe classical P100 component occurring at alatency of about 100 msec. It is importantto note that the form of the recordedactivity is directly determined by thechoice of the recording reference: in Fig. 1A

a frontal reference was used—a choice thatis common in many laboratories whenvisual evoked brain activity is recorded. Ofcourse, linked mastoids may also beemployed, yielding a pattern of activityshown in Fig. 1B. Note that these are thesame data, because only the reference pointhas been changed. Again the VEPs aredominated by two components; however,comparison to Fig. 1A reveals subtle butsignificant differences. Another widelyused recording method is the averagereference, whereby the mean of all elec-trodes is computed as a reference value ateach time point. The resulting 30 waveforms are illustrated in Fig. 1C. As men-tioned above, in this figure we are dealingwith identical data seen from differentview points—the evoked brain response, ofcourse, is independent of the choice of therecording reference. The search for a so-called inactive reference point has no prac-tical solution because evoked potentialwave forms always constitute measure-ments of the continuously fluctuatingpotential gradients between two points,whether both on the scalp or not (see alsoNunez, 1981).

It is evident that from identical data, dif-ferent conclusions can be derived. Inevoked potential research, conventionalamplitude and latency measures aretreated as independent variables, but from comparison of the wave forms inFig. 1A–C, it is obvious that all amplitudeschange, depending on the recording refer-


APPENDIXES

TABLE 1 P100 Latencies Measured at Five Different Electrodesa

Recording channel

Reference electrode 12 17 21 27 29

Midfrontal (Fig. 1A) 104 112 110 112 112

Linked mastoids (Fig. 1B) 102 100 108 112 102

Average reference (Fig. 1C) 80 96 110 120 102

aIdentical data are analyzed with different reference electrodes (evoked potential data illustrated in Fig. 1). The values are peaklatencies (in milliseconds); the electrode recording channel numbers refer to the electrode scheme shown in Fig. 2.

ence. In addition, the latency of a givencomponent appears to be different. As apractical example, analysis of the ampli-tude peaks of channels 12, 17, 21, 27, and29 results in significantly different P100peak latencies, as illustrated in Table 1.

In this restricted set of electrodes,component latencies range between 80 and120 msec. Of course, reality is even morecomplex because the data have beenrecorded in many channels. When 30 scalpelectrodes are used for recording, 870different wave forms may be obtained (435of these are sign inverted), and many moreapparently different components emerge.We also note that none of the electrodelocations can be privileged as referenceelectrode, because there is no inactive pointavailable (Nunez, 1981), and each of the 30electrodes may be determined to serve asrecording reference. It is impossible toevaluate such extensive data sets thatcontain redundant information by simplevisual inspection or by wave shape com-parisons, and thus the analysis of electricalactivity must not deal with reference-dependent potential wave forms. Theexample in Fig. 1 demonstrates that con-ventional analysis of evoked brain activityrecorded simultaneously in many channelswill always result in a confusing number ofindividual component latencies that cannotbe interpreted in a physiologically mean-ingful way [see also Figs. 3, 4, and 5 inSkrandies (1987), for further illustration].Of course, all data sets illustrated in Fig. 1carry identical information; changes of thereference does not change the underlyingneural process but only the appearance ofthe curves. This is not obvious from Fig. 1because the polarity and the latencies of allindividual peaks may change drastically. Itis also virtually impossible to see that thedata of Fig. 1 represent an identical dataset: with linked mastoids as reference anoccipital positivity is seen around 100 msec(P100 component), which however, showsa latency range between 98 and 122 msecamong the 15 most posterior channels.

With a frontal reference, all peak latenciesare different, ranging from 106 to 130msec. In a similar line, the N70 componenthas peak latencies between 70 and 90 msecfor the linked mastoid reference, whereasthe frontal reference yields latency valuesbetween 66 and 72 msec. Some of thechanges may appear small. In clinicalapplications, however, latency differencesof only a few milliseconds may determinea given patient’s data as pathological. Thussmall alterations may attain great impor-tance; also, for physiological questions,time differences in the order of milli-seconds are important (Skrandies, 1987).

For the analysis of the topographicalaspects of electroencephalographic activityit is important to keep in mind that we aredealing with electrical fields originating inbrain structures whose characteristics varywith recording time and space: the posi-tion of the electrodes on the scalp deter-mines the pattern of activity recorded, andmultichannel EEG and evoked potentialdata enable us to analyze topographicallythe electrical fields that are reconstructedfrom many spatial sampling points. Froma neurophysiological point of view, evokedcomponents are generated by the activa-tion of neural assemblies located in certainbrain regions with certain geometric con-figurations. Landscapes of electrical brainactivity may give much more informationthan conventional wave form analysis thatstresses only restricted aspects of theavailable electrical data (Lehmann, 1987;Skrandies, 1986, 1987).

As a response to visual stimulation thebrain generates an electrical field thatoriginates from neuronal assemblies in thevisual cortex. This electrical field changesin strength and topography over time, andthe data of Fig. 1 are displayed as series ofpotential distributions in Fig. 2. Thesemaps simply show the potential distribu-tion within the recording array at varioustime points after stimulation (between 40and 190 msec after the presentation of avisual stimulus). At each latency time a

REASONS FOR TOPOGRAPHIC BRAIN MAPPING AND BASIC IDEAS 405

APPENDIXES

characteristic topography is elicited. Notethat the structure of each map does notchange when a different recording refer-ence is employed: all topographical fea-tures remain identical when the electricallandscape is viewed from different points,similar to the constant relief of a geogra-phical map, where the sea level is arbi-trarily defined as zero level. The locationsof maxima and minima, as well as the loca-tions and strengths of potential gradientsin a given map, are independent of thereference point that defines zero. It isobvious that the topography as well as the strength of the electrical field change asa function of time. Around 70 msec, orbetween 100 and 120 msec, the field relief is pronounced: there are high voltagepeaks and troughs, associated with steeppotential gradients. Obviously these timeinstances indicate strong synchronous acti-vation of neurons in the visual cortex.

The maps in Fig. 2 are potential mapsthat were reconstructed from data recordedfrom the electrode array shown in theinset. The 30 electrodes are distributed as aregular grid over the brain regions understudy. Because only potential differenceswithin the scalp field are of interest, alldata are referred to the computed averagereference. This procedure results in aspatial high-pass filter, eliminating thedirect current offset potential introducedby the necessary choice of some point asrecording reference. In order to interpretthe topographic patterns, more detailedanalysis is needed. It is important to keep

in mind that the absolute locations of thepotential maxima or potential minima inthe field do not necessarily reflect thelocations of the underlying generators.This fact has led to confusion in the inter-pretation of EEG data. A striking examplefor visual evoked activity is the phenome-non known as paradoxical lateralization(see below). Rather than potential maxima,the location of steep potential gradientsappears to be a more adequate parameter,reflecting intracranial source locations(Skrandies, 2002).

MAPPING OF BRAIN ACTIVITYAND DEFINITION OF

COMPONENTS

Topographical analyses should not berestricted to the qualitative graphicaldisplay of maps at many time points,instead of conventional time series at many recording points. It is mandatorythat quantitative methods are applied tomultichannel EEG and evoked potentialdata in order to extract relevant informa-tion from such maps series. In the follow-ing discussion, methods for quantitativetopographical data analysis are illustrated,and it is shown how field strength, compo-nent latency, and topography can be usedto quantify electrical brain activity.

Mapping of electrical brain activity initself does not constitute an analysis of therecorded data, but it is a prerequisite toextract unambiguously quantitative fea-


APPENDIXES

FIGURE 2 The data of Fig. 1 illustrated as maps series between 40 and 190 msec after stimulus reversal.Recordings were obtained from 30 electrodes over the occipital areas (see head inset). Lines are in steps of 1.0 V;hatched areas are negative with respect to the average reference.

maximal number of intracranial neuronalelements. In order to quantify the amountof activity in a given scalp potential fieldwe have proposed a measure of “globalfield power” (GFP), which is computed asthe mean of all possible potential differ-ences in the field corresponding to thestandard deviation of all recording elec-trodes with respect to the average refer-ence (Lehmann and Skrandies, 1984). Scalppotential fields with steep gradients andpronounced peaks and troughs will resultin high global field power, whereas globalfield power is low in electrical fields withonly shallow gradients that have a flatappearance. Thus, the maximum in a plotof global field power over time determinescomponent latency. In a second step thefeatures of the scalp potential field areanalyzed at these component latencies.Derived measures, such as location ofpotential maxima and minima, and steep-ness and orientation of gradients in thefield are by definition independent of thereference electrode, and they will give anadequate description of the electrical brainactivity.

Global field power is computed as themean potential deviation of all electrodesin the recording array (see Fig. 3). With anarray of equidistant electrodes on the scalpsurface, the potentials ei, i = 1, …, n, yieldthe measured voltages Ui = ei – ecommon refer-

ence. From this potential distribution at onetime instant, a reference-independentmeasure of GFP is computed as the meanof all potential differences within the field

(1)

GFP corresponds to the root-mean-square amplitude deviations between allelectrodes of a given array recording theelectrical field. Note that this measure is not influenced by the choice of the referenceelectrode, and allows for a reference-

GFPn

U Ui

n

ij

n

j= Â Â= =

12 1 1

2( – )

tures of the scalp recorded electrical activ-ity. In a second step of data analysis thederived topographical measures must beemployed to test statistically differencesbetween experimental conditions orbetween groups of subjects.

For the analysis of evoked brain activityone of the main goals is the identificationof so-called components that are com-monly interpreted as steps of informationprocessing. Multichannel recordings ofelectrical activity result in a large amountof data: mapping of potentials evoked by asimple checkerboard reversal stimulussampled at 500 Hz in 30 channels over anepoch of 256 msec results in 3840 indi-vidual amplitude values. Of course, noteach of these amplitude measures attainsphysiological meaningfulness. As is evi-dent from the maps shown in Fig. 2, thereoccur potential field distributions withonly very little activity (few field lines,shallow gradients between 40 and 60 msec,or at 140 msec), whereas at other latencytimes maps display high peaks and deeptroughs with large potential gradients inthe potential field distributions (at 70 or at100 msec in Fig. 2).

It appears reasonable to define com-ponent latency as the occurrence time ofmaximal activity in the electrical fieldreflecting synchronous activation of a

MAPPING OF BRAIN ACTIVITY AND DEFINITION OF COMPONENTS 407

APPENDIXES

FIGURE 3 Global field power (GFP) as a functionof poststimulus time computed on the data shown inFig. 2. Field strength displays maxima at 76 and 108msec, defining component latencies.

independent treatment of electrophysio-logical data.

In a similar way, GFP may be computedfor EEG data referred to the averagereference:

(2)

This is mathematically equivalent to theformula shown in Eq. (1).

Equations (1) and (2) illustrate that GFPreflects the spatial standard deviationwithin each map (i.e., at a given instant):pronounced potential fields with highpeaks and troughs and steep gradients areassociated with high GFP values whereaswith flat fields GFP is small. Field strengthas determined by GFP results in a singlenumber at each latency point, and it maybe plotted as a function of time. This illus-trates how field strength varies over time,and the occurrence of its maximum valuein a predetermined time window can beused in order to determine componentlatency. It is important to note that allrecording electrodes contribute equally tothe computation of GFP, and that at eachtime point it results in one value that isindependent of the reference electrode thatwas used for recording. In this way theproblems of conventional wave shapeanalysis are avoided.

High global field power also coincideswith periods of stable potential field con-figurations when the spatial characteristicsof the fields remain unchanged. Figure 3illustrates GFP as a function of time, and itis evident that two components occur at 76and at 108 msec. The brain’s response tothe visual stimulus resulted in electricalfield distributions with maximal activity atthese time points. In this way, componentlatency is defined topographically andindependent of the reference electrode. Forfurther topographical analysis, latency,field strength, or the complete potentialfields at component latency may be com-

GFP Un

Ui jj

n

i

n= Â

ÊËÁ

ˆ¯

Â==

–1

11

2

pared directly between experimental con-ditions or between groups of subjects.

All of these derived measures can besubmitted to statistical analysis and can beinterpreted in a physiological context:component latency may be equated withneural processing time, whereas fieldstrength is an index of the amount of syn-chronous activation or the spatial extent ofa neuronal population engaged in stimulusprocessing. In addition, derived measurescan be used to quantify the topography ofpotential fields. One useful parameter forthe definition of topographical characteris-tics is the location of the centers of gravity(centroids) for the positive and negativeareas within each potential field. Theselocations consider the information of allrecording points in a given map, and theyconstitute a meaningful data reduction thatcan then be treated with statisticalmethods. An example is given in Fig. 4.Visual brain activity elicited by checker-board reversal stimulation with stimulipresented to the fovea or in the left or rightvisual field was recorded in 30 channelsfrom 22 healthy volunteers. Three differentcomponents were identified by the occur-rence of maximal field strength (GFP) at78.2, 118.2, and 186.7 msec latency, and themean locations of the positive and negativecenters of gravity are displayed. Becauselateral visual stimuli were also employed,this figure concentrates on the effects seenin the left–right direction, giving com-ponent locations over the left or righthemisphere. With all three components(Fig. 4A–C), central stimulation results inmaximal activity in the midline whereaslateral visual input yields a complexpattern of lateralization of activity over theleft and right hemispheres. As is evidentfrom Fig. 4, there is no lateralization withcentral stimuli, and centroids are shiftedtoward the left or right hemisphere whenstimuli occur in the visual half-fields. Suchlocation data can also be treated statisti-cally, and significant effects of stimuluslocation on the topographic distribution


APPENDIXES

are derived from further analysis, enablingthe comparison of experimental conditions.

Such differences in the topographicallocalization characterize the spatiotemporaldistribution over longer time epochs.Skrandies (1988) illustrates how differencesin visual information processing is reflectedby sustained differences in topographical

features. Visual stimuli of different spatialfrequency yield significant differences inthe geometry of underlying neural assem-blies selectively sensitive to different stimu-lus characteristics. This is seen not only atthe occurrence time of evoked componentsbut may persist up to 27 msec (Skrandies,1988).

Statistical analysis may also be per-formed on complete scalp distributionmaps when data obtained in differentexperimental conditions or in differentgroups of subjects are obtained. The proce-dure is straightforward: a statistical mea-sure (such as a t-value) is computed ateach electrode site, and its significance isthen plotted as topographical distribution.Such a comparison of complete mapsresults in so-called significance probabilitymaps (see Chapter 3, this volume).

STATISTICALLY DEFINEDCOMPONENTS: PRINCIPAL

COMPONENT ANALYSIS

The Extraction of Components

Electroencephalographic data recordedfrom many electrodes constitute multi-dimensional observations, and waveshapes of evoked potentials have beenanalyzed by multivariate statisticalmethods (Chapman and McCrary, 1995;Donchin, 1966; Skrandies, 1981). Evokedbrain potentials may be considered as thesum of independent components, andprincipal component analysis (PCA) hasbeen used to extract such underlyingcomponents. The amplitudes of evokedpotentials show some correlation betweensuccessive time points as well as betweenneighboring electrode sites. In addition tothe autocorrelation inherent in evokedpotential data, variation of the indepen-dent experimental variables introducessystematic variation in the data set yield-ing patterns of correlation caused bystimulus or subject conditions.

STATISTICALLY DEFINED COMPONENTS: PRINCIPAL COMPONENT ANALYSIS 409

APPENDIXES

FIGURE 4 Mean centroid locations in the left–-right direction on the scalp at different componentlatencies. (A) 78.2 msec; (B) 118.2 msec; (C) 186.7msec. A checkerboard reversal stimulus was pre-sented in the center or in the left or right visual half-field. Note that there is no lateralization with centralstimuli, and centroids are shifted toward the left orright hemisphere when stimuli occur in the visualhalf-fields. Numbers give the distance from themidline in percentages of the nasion–inion distance.Positive centers of gravity are indicated by closedsymbols; negative centroids are indicated by opensymbols. Mean data of 22 healthy adults.

The use of PCA is first to find a reducedset of nonredundant wave form descrip-tors that explain most of the variance in theoriginal data, and PCA has been applied toevoked potential wave forms by a numberof researchers. Principal components areby definition orthogonal, and their correla-tion is zero. After component extraction theloading pattern may be rotated to a simplestructure, maintaining orthogonal relation-ships between components (varimax rota-tion) (Harman, 1967; Glaser and Ruchkin,1976). This procedure is used to maximizehigh component loading values and mini-mize low loading values.

In a second step the contribution of eachof the derived components to the originaldata is determined by examining thecomponent scores associated with eachstimulus or subject condition. These scoresare dependent measures like the conven-tional amplitude values, and experimentaleffects may be revealed by treating compo-nent scores with conventional statisticalmethods (Dillon and Goldstein, 1984).

Different from factor analysis (Harman,1967), PCA does not consider unique fac-tors that show high loadings only on indi-vidual variables. This appears appropriate,because, due to the autocorrelation inevoked potential data mentioned above,cortical activity modulated by experimen-tal conditions never influences only iso-lated time points or electrode positions. Inaddition, it has been shown that a highpercentage of the variance in a given dataset is accounted for by only a few principalcomponents, indicating that most of thevariance of the data is related to fewcommon factors and that unique factorscan be neglected.

With evoked potential wave forms,amplitudes measured at successive timepoints are entered as variables, whereasdifferent electrode positions, subjects, andexperimental conditions are the observa-tions in the input data matrix. Topogra-phical effects may then be analyzed bytesting the statistical significance of the

scalp distribution of component scoresobtained in different experimental or sub-ject conditions. The topographical analysisof principal component scores has beensuccessfully performed on both one-dimensional potential profiles and two-dimensional potential fields (Skrandies,1981, 1983; Skrandies et al., 1998).

For direct topographical analysis aspatial PCA may be used in order to reducethe dimensionality of the original datamatrices, where the amplitudes at eachelectrode location are the variables, andtime points, different subjects, or differentexperimental conditions serve as the obser-vations. Then the matrix of correlations (orcovariances) between electrodes is decom-posed by PCA. When wave forms are used,component loadings form basic waveforms whereas with topographical mapsthe spatial PCA results in basic maps (i.e.,scalp distributions of component loadings).The statistical method of PCA computationextracts components that are orthogonal,with the first principal component repre-senting the maximum variation in the data;the second principal component is orthog-onal to the first and represents the maxi-mum variation of the residual data. Thisprocess is repeated several times, andbecause the original variables are cor-related only a small number of principalcomponents accounts for a large propor-tion of the variance in the original data.When PCA is performed on multichannelevoked potential wave shapes, between 6and 10 principal components are found(Skrandies, 1981, 1983), whereas a spatialPCA results in about three to four basicfield distributions that account for morethan 90% of the variance (Skrandies, 1989;Skrandies and Lehmann, 1982; Skrandieset al., 1998). An example is given in Fig. 5.

The general linear model of evokedpotential or EEG maps reads as

M(i) = S1C1(i) + S2C2(i) + … + SnCn(i) + X, (3)

or in matrix notation


APPENDIXES

M = SC’ + X, (4)

where M(i) is the potential map defined byi electrodes, Cn are fundamental compo-nents (maps of component loadings), Sn arecomponent scores (gain factors) associatedwith subject or stimulus conditions, and Xis the grand mean of the original dataentered in the PCA. In this model thegrand mean is added for visualizationbecause it is removed from the originaldata when the covariance or correlationmatrix is computed. In a similar way thematrix M of Eq. (4) contains the originalamplitude values, C’ represents the compo-nent loadings, S the component scores, and X contains the grand mean values.Components are extracted from the covari-ance or correlation matrix, thus the contri-bution of each component is relative to thegrand mean of the original data. When the basic components derived from the cor-relation matrix are illustrated in the formof scalp distribution maps, the metric ofeach component can be restored to micro-volts by multiplying component loadingsby their respective standard deviations.

As a result of the spatial PCA, largenumbers of electrodes are reduced to a fewunderlying components, each of which isweighted by a component score that indi-cates the contribution of each component toa given experimental condition [see Eq. (3)].For a detailed description of the mathe-matical and statistical backgrounds offactor analysis, see the statistical literatureby Harman (1967) or Dillon and Goldstein(1984), for example. The application of PCAto biomedical signals has been reviewed byGlaser and Ruchkin (1976), John et al.(1978), and Chapman and McCrary (1995).

Examples of Spatial PrincipalComponents Analysis

After extracting principal components,their relation to stimulus conditions is ofmajor interest, and conventional statisticssuch as analysis of variance (ANOVA) on

the component scores may reveal significantexperimental effects. An example is given inFig. 7, where the results of an experiment onthe scalp distribution of electrical brainactivity elicited by visual motion stimuli in14 healthy adults are displayed. Stimuliwere square wave gratings of high or lowcontrast moving with a velocity of 4.9° persecond on a computer monitor. Adaptationto motion was varied by changing the so-called duty cycle of stimulus presentation(i.e., the relation of motion to nonmotiontime) in order to enhance motion-relatedactivity. Data obtained with motion stimuliwere compared to checkerboard patternreversal evoked activity. Spatial principalcomponents analysis on the data of all sub-jects and experimental conditions revealsfour latent topographical components thataccounted for 92.05% of the variance (Fig. 6).

Two components showed occipital ex-treme values surrounded by steep potentialgradients (components 1 and 2 in Fig. 6),whereas there occurred another two statisti-cally independent components displayinglateralized activity (components 3 and 4 inFig. 6). From the plot of cumulative vari-ance, it is obvious that more than 50% of thevariance can be explained by the first com-ponent. It is important to note, however,that components associated with less vari-ance also carry useful information. Analysisof the contribution of these spatial compo-nents to the observed potential fieldsrevealed significant differences betweenpattern reversal- and motion-evoked activ-ity seen with components 3 and 4 (fordetails see Skrandies et al., 1998). Significantdifferences in scalp topography evoked bycheckerboard reversal and by motionstimuli are revealed by the analysis of com-ponent 1 occurring at 158 msec latency. It isimportant to note that identical basic fieldconfigurations contribute to different experi-mental conditions, where, however, the rela-tive weighting is significantly different. Thisfurther stresses that with PCA not only theextraction of components is critical but themain aim is the analysis of component


APPENDIXES

scores that allows for comparisons of experi-mental conditions.

Results of a spatial PCA on evoked poten-tial recordings in 45 channels obtained fromsix participants are shown in Fig. 5. Visualpotentials were evoked by a contrast revers-ing checkerboard pattern in five stimulusconditions (two stimulus sizes in two lateralretinal locations and one upper hemiretinalstimulus). Further details on the experimen-tal conditions and the VEP recordings isgiven by Skrandies and Lehmann (1982).The computation of GFP identified twocomponents with mean latencies of 105.8and 145.1 msec in all subjects and condi-tions. The original data consisted of 45 vari-ables (amplitude values) by 60 cases(experimental stimulus conditions) thatwere entered in a spatial PCA using the cor-relation matrix. The amplitudes at the 45electrodes were reduced to only three com-ponents with eigenvalues greater than 1.0,

which accounted for 93.4% of the variance.As is evident from Fig. 5, component 1shows anterior–posterior distribution dif-ferences with an extreme value in themidline over occipital areas; component 2mainly displays lateral differences, and com-ponent 3 has a concentric distribution with amaximum at parietal sites. This appears as ameaningful numerical decomposition of thescalp field distributions. As a result of thespatial PCA the number of variables isreduced from 45 amplitude values at allelectrodes to three component scores foreach experimental condition. The topo-graphic pattern of principal componentsillustrated in Fig. 5 is very similar to the dis-tribution of factor scores derived from fre-quency analyses of spontaneous EEGs, andJohn et al. (1993) have illustrated that thetopographical distribution of scores on alimited number of factors may be used suc-cessfully to quantify the abnormalities of


APPENDIXES

FIGURE 5 Results of a spatial PCA. Sixty visually evoked scalp distribution maps at GFP component latency(schema on the left) obtained from six participants in 45 channels were entered in a PCA. This procedureextracted three varimax-rotated components with eigenvalues greater than 1.0, which accounted for 93.4% of thevariance. Thus, for each experimental condition 45 amplitude values are reduced to three components plus thegrand mean field, which is computed as the mean over all subjects and stimulus conditions. Components areshown with a score (scaling factor) of 1.0, and the metric was restored to microvolts by multiplying componentloadings by their respective standard deviations. Field lines are in steps of 0.3 V. From Skrandies (1989), withpermission.

patients with a wide variety of psychiatricdisorders.

A Note on the PhysiologicalInterpretation of Principal Components

The general problem of the referencelocation that applies to the analysis ofevoked potential wave forms (i.e., thechange of wave forms with a change of thereference location as described above) alsoapplies to principal components. Changingthe reference means to subtract a vectorfrom the original data, and this procedurewill cause some changes in the covarianceand correlation matrix computed from theoriginal amplitude measures. Because PCAreproduces the input matrix as a set oflinear combinations, any change of theoriginal data caused by different referenceelectrodes must also influence the derivedsolution, and PCA may not overcomeproblems with the original data.

In addition, it is important to keep inmind that PCA results in statisticallydefined components. Thus, the interpreta-tion of components derived from PCAcomputations must always consider that aspecific principal component reflectsnothing more but a source of variance inthe original data set. The pattern of com-ponent loadings depends on the covaria-tion of potential amplitudes at variousscalp locations over experimental condi-tions or recording time points. As shownabove, different experimental conditionsproduce different principal components(see topographic component patterns inFigs. 5 and 6). In this example the differ-ences in scalp distributions of componentloadings are caused by the fact that differ-ent patterns of correlation between therecording electrodes on the scalp are pro-duced when central or left and righthemiretinal areas are stimulated. Principalcomponents are always directly influenced


APPENDIXES

FIGURE 6 The maps illustrate the spatial distribution of principal components. These are basic field shapeswith a component score of 1.0. The histograms show cumulatively the amount of variance explained.Component 1 explains more than 50%, and nearly 100% is explained by all four components. All componentloadings were converted to microvolt values, and lines are in steps of 0.5 V for components 1 and 2, and 0.25 V for components 3 and 4. Hatched areas are negative, white areas are positive. Data from Skrandies et al.(1998).

by the variation in the data set caused bydifferent experimental conditions. Thus,the temporal or spatial pattern of compo-nent loadings must not necessarily repre-sent physiological components, and therelation between principal componentsand physiological mechanisms cannot bewarranted by the statistical method.

It is well accepted that PCA is able toreproduce underlying components with ahigh degree of accuracy (Wood andMcCarthy, 1984), but there may occur prob-lems when component scores are analyzedin ANOVAs. The “misallocation” of vari-ance across components described in asimulation study by Wood and McCarthy(1984) may be caused by the rotation of thecomponent loadings. Möcks and Verleger(1986) and Rösler and Manzey (1981) havealso drawn attention to possible problemsof orthogonal rotations of principal compo-nents. We also note that the problem ofmisallocation of variance of componentscores that occurs with overlapping com-

ponents in PCA (Wood and McCarthy,1984) is certainly not restricted to PCAresults. When analyzing amplitude mea-sures of evoked potentials, componentoverlap certainly introduces similar prob-lems. This is due to the fact that we aredealing with scalp-recorded data that areproduced by the simultaneous activationof many intracranial neuronal generatingprocesses.

On the other hand, PCA is a powerfultool when used as a means of data reduc-tion. As obvious from Figs. 6 and 7, manyamplitude measures can be reduced to avery small set of components that aremeaningfully and strongly related toexperimental parameters. This has beenrepeatedly reported by a number ofresearchers (Chapman and McCrary, 1995;Donchin, 1966; Skrandies, 1983; Skrandieset al., 1998). Along a similar line, the findingthat only three components (basic field dis-tributions) may explain more than 95% ofthe variance when data are analyzed overtime is in good agreement with the resultsof segmentation studies that showed thereoccurrence of similar spatial patterns overtime when evoked potential maps serieswere segmented by topographical criteria(Lehmann and Skrandies, 1986; Skrandies,1988).

OUTLOOK AND POSSIBLEAPPLICATIONS OFTOPOGRAPHICAL BRAIN MAPPING

It is evident from many results and datathat topographic mapping of brain electri-cal activity constitutes a means for thevisualization of electric field distributionson the scalp, and for the adequate statisti-cal and quantitative analysis of multi-channel electrophysiological recordings.These methods may be applied for brainactivity that occurs spontaneously or iselicited by sensory stimulation or psycho-


APPENDIXES

FIGURE 7 Scores on component 1 (shown inFig. 6) elicited by motion (left) and pattern reversalstimuli (right) occurring 158 msec after stimulation.Note that an identical component contributes to theevoked potential field with different weighting scoresand different polarity when experimental conditionsare compared. Inset maps display the mean topo-graphical distribution of the component appropri-ately scaled by the scores. Lines are in steps of 0.5 V;hatched areas are negative, white areas are positive.Mean values computed on the data of 14 volunteers.Data from Skrandies et al. (1998).

logical events. Electrical brain activity canbe characterized in terms of latency (i.e.,processing times), synchronous involve-ment and extent of neuronal populations(i.e., field strength), and topographicaldistribution of potential components.

Practical applications are twofold: muchis to be learned from studies on functionalstates of the human brain, information pro-cessing, and motor planning in healthyvolunteers, and clinical questions may beanswered on the functionality and intact-ness of the central nervous system ofpatients suspected of central nervoussystem or psychiatric disease. Noninvasiveexperimental investigations are part ofaddressing contemporary neurophysio-logical questions on how global statesaffect brain functions such as processing ofsensory or psychological information,movement planning and execution, orinternal states related to cognition andemotion. In healthy people as well as inpatients, such processes can be studied andcharacterized by spatiotemporal patternsof electrical brain activity.

In clinical settings, sensory evoked brainactivity is recorded in order to test theintactness of afferent pathways and centralprocessing areas of various sensory modal-ities in neurological, ophthalmological, oraudiological patients. Event-related brainactivity elicited during cognitive tasks hasits main application in the fields of psychi-atry and psychology, where perception,cognition, attention, learning, or emotionalprocesses are under study. These fieldsprofit from the application of topographicmapping and analysis of brain electricalactivity in real-time. Future applications oftopographic mapping of electrophysiologi-cal activity will include the coregistrationof the high time-resolution EEG recordingswith brain-imaging methods such as func-tional MRI. One may expect that the collab-oration of the fields will lead to functionalimaging of brain activity with high tempo-ral and high spatial resolution.

Acknowledgment

Supported in part by DeutscheForschungsgemeinschaft, DFG Sk 26/5–3and DFG Sk 26/8–3.

ReferencesChapman, R. M., and McCrary, J. W. (1995). EP

component identification and measurement byprincipal components analysis. Brain Cogn. 27,288–310.

Dillon, W. R., and Goldstein, M. (1984). “MultivariateAnalysis Methods and Applications.” John Wiley& Sons, New York.

Donchin, E. (1966). A multivariate approach to theanalysis of average evoked potentials. IEEE Trans.Biomed. Eng. 13, 131–139.

Glaser, E. M., and Ruchkin, D. S. (1976). “Principlesof Neurobiological Signal Analysis.” AcademicPress, New York.

Harman, H. H. (1967). “Modern Factor Analysis,” 2ndEd. The University of Chicago Press, Chicago.

John, E. R., Ruchkin, D. S., and Vidal, J. J. (1978).Measurement of event- related potentials. In“Event-related Brain Potentials in Man”(E. Callaway, R. Tueting, and S. H. Koslow, eds.),pp. 93–138. Academic Press, New York.

John, E. R., Easton, P., Prichep, L. S., and Friedman, J.(1993). Standardized varimax descriptors of eventrelated potentials: Basic considerations. BrainTopogr. 6, 143–162.

Lehmann, D. (1987). Principles of spatial analysis. In“Handbook of Electroencephalography andClinical Neurophysiology. Rev. Series, Vol. 1:Analysis of Electrical and Magnetic Signals”(A. Gevins, and A. Rémond, eds.), pp. 309–354.Elsevier, Amsterdam.

Lehmann, D., and Skrandies, W. (1980). Reference-freeidentification of components of checkerboard-evoked multichannel potential fields. Electro-encephalogr. Clin. Neurophysiol. 48, 609–621.

Lehmann, D., and Skrandies, W. (1984). Spatial analy-sis of evoked potentials in man: A review. Prog.Neurobiol. 23, 227–250.

Lehmann, D., and Skrandies, W. (1986). Time segmen-tation of evoked potentials (EPs) based on spatialscalp field configuration in multichannel record-ings. Electroencephalogr. Clin. Neurophysiol. (Suppl.38), 27–29.

Möcks, J., and Verleger, R. (1986). Principal componentanalysis of event-related potentials: A note on mis-allocation of variance. Electroencephalogr. Clin.Neurophysiol. 65, 393–398.

Nunez, P. L. (1981). “Electric Fields of the Brain.”Oxford University Press, New York.

REFERENCES 415

APPENDIXES

Rösler, F., and Manzey, D. (1981). Principal compo-nents and varimax-rotated components in event-related potential research: Some remarks on theirinterpretation. Biol. Psychol. 13, 3–26.

Skrandies, W. (1981). Latent components of potentialsevoked by visual stimuli in different retinal loca-tions. Int. J. Neurosci. 14, 77–84.

Skrandies, W. (1983). Information processing andevoked potentials: Topography of early and latecomponents. Adv. Biol. Psychiatr. 13, 1–12.

Skrandies, W. (1986). Visual evoked potential topogra-phy: Methods and results. In “TopographicMapping of Brain Electrical Activity” (F. H. Duffy,ed.), pp. 7–28. Butterworths, Boston.

Skrandies, W. (1987). The upper and lower visual fieldof man: Electrophysiological and functional differ-ences. Prog. Sensory Physiol. 8, 1–93.

Skrandies, W. (1988). Time range analysis of evokedpotential fields. Brain Topogr. 1, 107–116.

Skrandies, W. (1989). Data reduction of multichannelfields: Global field power and principal compo-

nents. Brain Topogr. 2, 73–80.Skrandies, W. (1995). Visual information processing:

topography of brain electrical activity. BiologicalPsychol. 40, 1–15.

Skrandies, W. (2001). Electroencephalogram topogra-phy. In “The Encyclopedia of Imaging Science andTechnology” (J. P. Hornak, ed.), Vol. 1, pp. 198–210.John Wiley & Sons, New York.

Skrandies, W., and Lehmann, D. (1982). Spatial princi-pal components of multichannel maps evoked bylateral visual half-field stimuli. Electroencephalogr.Clin. Neurophysiol. 54, 662–667.

Skrandies, W., Jedynak, A., and Kleiser, R. (1998). Scalpdistribution components of brain activity evokedby visual motion stimuli. Exp. Brain Res. 122, 62–70.

Wood, C. C., and McCarthy, G. (1984). Principal com-ponent analysis of event-related potentials:Simulation studies demonstrate misallocation ofvariance across components. Electroencephalogr.Clin. Neurophysiol. 59, 249–260.


APPENDIXES


A P P E N D I X

F

Brain Imaging Techniques:Invasiveness and Spatial and

Temporal ResolutionAlberto Zani, Gabriele Biella, and Alice Mado Proverbio

INTRODUCTION

In this appendix we provide some esti-mates of the spatial and temporal resolu-tion, as well as of the invasiveness, of themost frequently used neuroimaging tech-niques. Before illustrating these estimates,it is important to remember that neuro-imaging techniques may be subdividedinto two broad categories according totheir different aims: imaging of brainanatomy (structural imaging), or of brainfunction (functional imaging). Structuralimaging is used to examine the static out-lines of brain structures in both physiologi-cal and pathological situations. Functionalimaging, on the other hand, is used to gainknowledge on (1) which structures areactivated during a specific cognitive task,at sensory and/or cognitive levels, (2) theinteractions between the structures that areactivated, and (3) the way the functionalactivation of the brain is reorganized inindividuals affected by neurological dis-eases, strokes, or head injuries.

Generally speaking, the structural cate-gory of neuroimaging techniques includestwo well-known neuroradiological tech-niques: computerized axial tomography(CAT) and magnetic resonance imaging(MRI). As far as brain imaging capacity is

concerned, both are far superior to theirforebear, the X-ray technique.

Functional imaging includes a widerange of techniques, which are listed herein increasing order of spatial and temporalresolution: (1) 2-deoxyglucose cerebralblood flow (2-deoxyglucose CBF), a fore-runner of hemodynamic techniques rarelyused nowadays; (2) single-photon emissioncomputed tomography (SPECT); (3)positron emission tomography (PET); (4)functional magnetic resonance imaging(fMRI); (4) electroencephalography andevent-related potentials (EEG–ERP); (5)magnetoencephalography (MEG); and last,but not the least, (6) microelectrode single-unit recording, which involves recordingelectrophysiological signals from inside oroutside the membrane of a single neuronbody by means of microelectrodes.

Although the categorization of neuro-imaging techniques into the two aforemen-tioned approaches is generally sound, itshould be mentioned that all these tech-niques have some limitations and attemptsare being made to overcome these. Somevery recently devised techniques combineboth structural and functional imaginginformation and cannot be classified intoeither category. A rather interestingexample of these combined techniques isthe so-called CAT–PET.

SPATIAL AND TEMPORAL RESOLUTION

The accuracy with which the imagingtechniques are able to provide definiteimages of the anatomy of the centers of thecentral nervous system (CNS), and/or theactivation of these centers, in order to beable to localize them reliably, is defined asspatial resolution. Conversely, the speedwith which the techniques can keep onscanning the CNS anatomy and physiol-ogy, taking into account all intrinsic limita-tions, i.e., the minimum time that mustnecessarily pass between the collection of ameasure of one CNS activation and thesuccessive one, is described as temporalresolution.

Figure 1 is a graphical representation ofan estimate of the normal spatial and tem-poral resolution for each of the imaging

techniques mentioned above. The spatialresolution, expressed in millimeters, isreported on the ordinate axis; the temporalresolution––here indicated in seconds on alogarithmic scale—is depicted on theabscissa. The height and width of theforms with which the different techniquesare represented in the figure indicate theknown range of spatial and temporal reso-lutions, respectively, for each of the tech-niques. The increasing saturation of thegray hue of the different shapes representsthe increasing degree of invasiveness of thetechniques.

Structural techniques currently have themost accurate spatial resolution or, in otherwords, localization capacity. Both CAT andPET techniques have a spatial resolutionthat is vastly superior to that of previoustechniques. Indeed, their spatial resolutionhas become so good that it is now in theorder of millimeters.

418 F. IMAGING TECHNIQUES

APPENDIXES

FIGURE 1 Invasiveness, spatial resolution, and temporal resolution of the main imaging techniques used inhumans to investigate function and structure of the brain. The different sizes of the shapes representing the dif-ferent techniques vary as a function of the level of both spatial resolution (in milimeters) and temporal resolution(in seconds). Note that the increasing level of saturation of the gray color represents the increasing levels of inva-siveness of the techniques.

In PET, for example, the activated parts ofthe brain selectively take up a radioactivetracer previously administered intraven-ously to a patient or healthy volunteer. Thegamma rays deriving from the emission ofpositrons by these activated structures allowa functional map of cerebral activation to bebuilt; as illustrated in Fig. 1, such a map hasa precision ranging between about 2 and 5mm. fMRI, on the other hand, can reflectstructural variations caused by increasedlocal blood flow and dilatation of cerebraltissues with a mean precision of 3 mm,although the range is from 2 to 4 mm.

Notwithstanding their high spatial reso-lution, none of the functional imagingtechniques, with the exception of MEG, canprovide functional images that are alsoaccurate in temporal terms. In fact, thetemporal resolution with which they canprovide accurate images of ongoing func-tional activation of the brain is rather poor.This resolution can reach the order of atenth of second (~100–150 msec) with themost technologically advanced type offMRI––that is, 3 or 4 tesla echo-planarfMRI (or event-related fMRI)—but stillremains in the order of seconds with lesspowerful equipment. The temporal resolu-tion of PET is tens of seconds or evenminutes. The significance of this technicallimitation to research can readily beappreciated by considering that an actionpotential originating in the pyramidalmotor neurones of the premotor cortexpropagating along the efferent pathwaystakes about 150 msec to reach the musclebundles of the forearm, causing flexion ofthe terminal phalanx of the index finger, inorder, for example, to press a button formeasuring reaction times. Or consider thatwe can identify an object that enters ourvisual field within a few hundreds of milli-seconds (~180–220 msec). It is clear that thevelocity with which the above-mentionedneural processes occur means that theirsubprocesses escape measurement tech-niques because of the interval between suc-cessive sampling.

Given that the final aim of research onthe mind and brain should be to constructa model of functional relations betweenthe pathways and centers of the brain fromwhich mental life comes, besides simplelocalization of these to particular areas of the brain, it is important to have atemporal resolution of milliseconds for theprocesses involved. The only imagingtechniques that have such a good temporalresolution are the techniques used sys-tematically or on single cells that measurethe electromagnetic activity of the braindirectly. As illustrated by Fig. 1, the maxi-mal temporal resolution, as well as spatialresolution, is provided by single-unitrecordings. Thanks to these it is possible tocarry out neurofunctional investigationswith a temporal resolution below the orderof milliseconds (<10–3), and with a spatialresolution under 1 mm. It is, however,unthinkable to use this technique for func-tional imaging of the human brain becauseof its invasiveness; it would require neuro-surgery to implant the microelectrodes.

Unlike single-unit recordings, scalprecordings of voltages (EEGs and ERPs)that mirror the intracranial currents origi-nating from neuronal sources in the braincortex, and spreading by volume conduc-tion throughout the brain and the scalp,can be used as tools for human research.Indeed, while having the advantage of 1-msec temporal resolution, or quite closeto this level, the recording method is com-pletely noninvasive. However, because ofits irregularities, the skull is not a homoge-neous conductor. The volume currents thatcome in contact with the electrodes overthe scalp are distorted by irregularitiessuch that the technique does not havesufficient spatial resolution to be ablelocate the real intracranial sources of thecurrents.

This difficulty translates into a spatialresolution that cannot be relied on forlocalization purposes. In the best cases,localization of the electric dipole rangesbetween a minimum of 6 mm and a maxi-

SPATIAL AND TEMPORAL RESOLUTION 419

APPENDIXES

mum of 11 mm or even beyond, dependingon a whole series of recording conditionsand modeling parameters (see Fig. 1). Eventhen, there are some extreme cases, such asthe so-called far-field potentials, in which thescalp site at which the largest signal ismeasured is actually far away from thesource area; for example, sensory-evokedresponses of the auditory cortex to uni-lateral stimuli, despite originating on thedorsolateral side of the contralateral brainhemisphere close to the ear, produce theirlargest amplitudes at the top of the scalp.

Although having the same temporal res-olution as the EEG, magnetoencephal-ography is only minimally influenced bythe nonuniform conductivity of the brain,skull, and scalp, and if the head is modeledusing spherical geometry the recordedmagnetic field can be considered com-pletely independent of the conductivity ofthe head. Referring once again to Fig. 1 itcan be seen that only MEG has high levelsof both spatial and temporal resolution. Indetail, the spatial resolution of this tech-nique ranges from a minimum of 1.5 mmto a maximum of 4 mm for the corticalareas of the brain. Unfortunately, this reso-lution decreases dramatically to some cen-timeters for subcortical regions. Thistechnique could, therefore, be the tech-nique of choice for localizing activity in thesuperficial areas of the brain, i.e., the cere-bral cortex, which is responsible for mentalprocesses in general. Unfortunately, thistechnique is still too expensive to becomeas widely used as the ERPs.

In order to overcome the various limita-tions of each of the techniques presentedhere, strategies are being developed to usecombined methods; for example, the imagesobtained by MRI, fMRI, or PET can be com-bined with those from microelectrodes, orthose from MEG and ERPs. Combinationmethods can be based on a direct or anindirect approach. The former uses hemo-dynamic images to obtain a real structuralbasis of the estimates of functional activa-tion acquired separately, whereas the latter

involves parallel recording of these para-meters in a single experimental paradigm.Although it is certainly true that the differ-ent approaches are very useful individually,only integration of the techniques with dif-ferent spatial and temporal resolutions canprovide truly valuable information on theneurofunctional mechanisms of mentalprocesses, and on their temporal course ofactivation.

INVASIVENESS

Techniques to investigate the functionand structure of the nervous system can beclassified as invasive, semiinvasive, andnoninvasive. The difference between theinvasive and semiinvasive techniques isthat the former implies a surgical lesion(mechanical) and the latter involves aphysicochemical stress (with radioopaqueor radioactive substances).

Techiques that can be defined as nonin-vasive are MEG and EEG, both of whichinvolve simple recordings of electricalpotentials or natural magnetic fields pro-duced by brain activity without experi-mentally introduced chemicostructuralperturbations of the cerebral regions.

Semiinvasive techniques provide ameans to observe dynamic and structuralproperties through images that are recon-structed on the basis of a principle of bothactive and passive detectability. In the firstcase, the products of rapidly metabolizedcompounds or radioactive substances witha short half-life are detected. The com-pounds are injected intravenously and,being distributed preferentially to certainregions of the brain, show in the variousareas of the brain a differential distributionthat is proportional to the state of activa-tion of the area (PET, SPECT, and, whentracers are used, also fMRI and MRI). Inthe case of passive detection (e.g., CAT),the X radiations are delivered by themachine and hit the cerebral tissues ab externo (with the usual radiographic


APPENDIXES

principle). Even MRI and fMRI, withoutinjection of markers, induce changes incerebral or spinal structures. In fact, apply-ing a magnetic field causes a change, albeittransitory, in the characteristics of thespatiotemporal molecular organization,with realignment of the electromagneticdipoles of charged molecules.

Mechanically invasive techniques belongto the realm of functional neurosurgery;this involves extra- or intradural record-ings, or even recordings from deep struc-tures (in the case of therapeutic placementof stimulatory electrodes). These techniquesobviously involve surgery on specific nerv-ous tissue and perforation or opening of theskull.

Mechanical or physicochemical invasive-ness implies a different degree of dangerfor the patient. Naturally this creates aboundary; the economic costs and benefitsof the results of the diagnostic or surgicalintervention must be compared to not car-rying out the procedure. On average, givenprecise conditions and the noninstrumentaldiagnosis, the indication for an invasiveintervention is highly controlled andjustifiable.

Suggested Reading

Single-Unit RecordingsHuguenard, J., and McCormick, D. (1994). “Electro-

physiology of the Neuron.” Oxford UniversityPress, New York.

Llinás, R. (1988). The intrinsic electrophysiologicalproperties of mammalian neurons: Insights intocentral nervous system function. Science 242,1654–1664.

Nicholls, J., Nartin, A., and Wallace, B. (1993). “FromNeuron to Brain.” 3rd Ed. Sinauer, Sunderland,Massachusetts.

Hemodynamic Functional ImagingBandettini, P. A., Rasmus M. B., and Donahue, K. M.

(2000). Functional MRI. Background, method-ology, limits, and interpretation. In “Handbook ofPsychophysiology” (J. T. Cacioppo, L. G. Tassinary,and G. G. Berntson, eds.), 2nd Ed., pp. 978–1014.Cambridge University Press, Cambridge,Massachusetts.

Binder, J. R., and Rao, S. M. (1994). Human brainmapping with functional magnetic resonanceimaging. In “Localization and Neuroimaging inNeuropsychology” (A. Kertesz, ed.), pp. 185–212.Academic Press, Orlando.

De Yoe, E. A., Bandettini, P., Neitz, J., Miller, D., andWinans, P. (1994). Functional magnetic resonanceimaging (fMRI) of the human brain. J. NeurosciMeth 54, 171–187.

Frith, C. D., and Friston, K. J. (1997). Studying brainfunction with neuroimaging. In “CognitiveNeuroscience” (M. D. Rugg, ed.), pp. 169–195.Psychology Press, Taylor & Francis Group, Hove,East Sussex, UK.

Josephs, O., Turner, R., and Friston, K.J. (1997). Event-related fMRI. Hum. Brain Mapping 5, 243–248.

Perani, D., and Cappa, S. (1999). Neuroimagingmethods in neuropsychology. In “Handbook ofClinical and Experimental Neuropsychology” (G.Denes, and L. Pizzamiglio, eds.), pp. 69–94.Psychology Press, Taylor & Francis Group, HoveEast Sussex, UK.

Posner, M. I., and Raichle, M. E. (1994). “Images ofMind.” W. H. Freeman, New York.

Reiman, E. M., Lane, R. D., Van Petten, C., andBandettini, P. A. (2000). Positron emission tomog-raphy and functional magnetic resonance imaging.In “Handbook of Psychophysiology” (J. T.Cacioppo, L. G. Tassinary, and G. G. Berntson,eds.), 2nd Ed., pp. 85–118. Cambridge UniversityPress, Cambridge, Massachusetts.

Rosen, B. R., Buckner, R. L., and Dale, A. M. (1998).Event-related MRI: Past, present, and future. Proc.Natl Acad. Sci USA 95, 773–780.

Rugg, M. D. (1999). Functional neuroimaging incognitive neuroscience. In “The Neurocognition ofLanguage” (C. M. Brown and P. Hagoort, eds.),pp. 15–36. Oxford University Press, Oxford andNew York.

Electromagnetic ImagingDel Gratta, C., and Romani, G. L. (1999). MEG:



Hillyard, S. A., and Picton, T. W. (1987). Electro-physiology of cognition. In “Handbook ofPhysiology, Sect. 1, Vol. 5, Higher Functions of theBrain” (F. Plum, ed.), pp. 519–584. AmericanPhysiological Society, Bethesda, MD.

Hillyard, S. A. (1993). Electrical and magnetic brainrecordings: Contributions to cognitive neuro-science. Curr. Opin. Neurobiol. 3, 217–224.

Kutas, M., and Dale, A. (1997). Electrical and mag-netic reading of mental functions. In “CognitiveNeuroscience” (M. D. Rugg, ed.), pp. 197–242.

SUGGESTED READING 421

APPENDIXES

Psychology Press, Taylor & Francis Group, HoveEast Sussex, UK.

Kutas, M., Federmeier, K. D., and Sereno, M. I. (1999).Current approaches to mapping language inelectromagnetic space. In “The Neurocognition ofLanguage” (C. M. Brown and P. Hagoort, eds.),pp. 317–392. Oxford University Press, Oxford andNew York.

Näätänen, R., Ilmoniemi, J., and Alho, K. (1994).Magnetoencephalography in studies of cognitivebrain function. Trends Neurosci. 17, 389–395.

Regan, D. (1989). “Human Brain Electrophysiology:Evoked Potentials and Evoked Magnetic Fields inScience and Medicine.” Elsevier, Amsterdam.

Scherg, M. (1992). Functional imaging and localizationof electromagnetic brain activity. Brain Topogr. 5,103–111.

Scherg, M., and Ebersole, J. S. (1993). Models of brainsources. Brain Topogr. 5, 419–423.

Swick, D., Kutas, M., and Neville, H. J. (1994).Localizing the neural generators of event-relatedbrain potentials. In “Localization and Neuro-imaging in Neuropsychology” (A. Kertesz, ed.),pp. 73–121. Academic Press, Orlando.

Vaughan, H. G. (1988). Topographic analysis of brainelectrical activity. In “The London Symposia (EEGSuppl. 39)” (R. J. Ellingson, N. M. F. Murray, andA. M. Halliday, eds.), pp. 137–142. Elsevier,Amsterdam.

Combining TechniquesDale, A. M., and Sereno, M. I. (1993). Improved local-

ization of cortical activity by combining EEG andMEG with MRI cortical surface reconstruction: Alinear approach. J. Cogn. Neurosci. 5, 162–176.

Halgren, E., and Dale, A. M. (1999). Combining ofelectromagnetic and hemodynamic signals toderive spatiotemporal brain activation patterns:Theory and results. Biomed. Technik 44 (Suppl. 2),53–60.

Luck, S. J. (1999). Direct and indirect integration ofevent-related potentials, functional magneticresonance images, and single-unit recordings.Hum. Brain Mapping 8, 115–120.

Wieringa, H. J. (1993). “MEG, EEG and the Integrationwith Magnetic Resonance Images”. DoctoralThesis.” CIP-Gegevens Koninklijke Bibliotheek,Den Haag, Nederlands.


APPENDIXES

Index

AAaltonen, O., 324Activation stimulus

cross studies compared, 44–53frequency influence, 78–79interstimulus interval

in Alzheimer disease, 311in reflexive attentional

orienting, 249–251periodic stimulation, 114–116spatial frequency of a visual

stimulus, 286–288, 292–298Adey, W. R., 210Age, neurocognitive development

effects, 223–239face perception development,

230–233face processing

across early school years,231–233

in Williams syndrome, 233infant face recognition, 231

language functionneuroplasticity, 234–239

American Sign Languagestudies, 235–236

bilingual adult studies, 234cerebral organization, 236–238deaf adult studies, 234–235influencing factors, 238–239primary language acquisition

effects, 236–238overview, 223–224, 238–239spatial attention, 228–230

auditory deprivation effects,228

visual deprivation effects,228–230

visual stream development,224–228

Ahlfors, S. P., 96Aine, C. J., 93–130, 284Akshoomoff, N. A., 318Alcohol, brain damage

relationship, 369Alho, K., 348Allen, J. J. B., 202Allport, A., 189Alternating current, artifacts in

EEG recordings, 388Alvarez, T. D., 233Alzheimer disease

event-related potential studiesdiagnostic problem, 309–310episodic retrieval studies, 181hippocampal activity

measurement, 310–311memory deficit probing,

311–312neurological symptoms, 367American Sign Language,

neuroplasticity developmentstudies, 235–236

Amidzic, O., 117Amplifiers

analog filters, 386–387high-density electromagnetic

signal detection, 379,385–386

Amyotrophic lateral sclerosisevent-related potential studies,

320–322affection of cognitive

functions, 320–321locked-in patient

communications, 321–322

movement-related potentials,320

neurological symptoms, 368Analog filters, high-density

electromagnetic signaldetection, 386–387

Anderson, S. J., 111Anllo-Vento, L., 299Anterior attentional system, visual

selective attention to objectfeatures, 273–279

Anterior cingulate cortex,executive functionrelationship

action monitoring, 199–202action regulation model,

207–208adaptation, 206–207dopamine effects, 205–206electrophysiology, 213–216executive control, 199–202overview, 198–199, 216–217Papez circuit, 206–207, 209theta dynamics, 202–205

Aphasia, event-related potentialstudies, 323–325

Arendt, G., 328Armstrong, R. A., 101Artifacts, in EEG recordings,

387–388Asada, H., 203, 204Attention, see also Perception

activation stimulus, 44–53attentional visual processing

attention to object features,273–301

anterior attentional system,273–279


Attention, see also Perception(continued)

color, 298–301feature-directed attention,

285–291features conjunction,

292–298frequency-based attentional

selection, 291–292neural systems, 274–280object perception, 292–298orientation-directed

attention, 288–291overview, 273–274, 301posterior attentional system,

273–274, 279–280primary visual area

modulation, 280–285space-based attentional

selection, 291–292spatial frequency-directed

attention, 286–288,292–298

covert visual attention, 245spatial attention

electrophysiologicalmeasures, 247–248

neurocognitivedevelopment, 228–230

steady-state visual evokedpotential relationship,258–262


steady-state visual evokedpotentials, 257–271

cognitive processrelationship, 258

contrast response, 268–270magno cellular pathways,

265–268overview, 257–258, 270–271parvocellular pathways,

265–268phase effects, 262–265

auditory deprivation effects, 228central fixation, 128–129cognitive regions, 63magnetoencephalographic

studies, 118–120, 128–129neural mechanisms, 245–255

electrophysiological measures,247–248

event-related potentialgenerator modeling, 251

functional MRI, 254–255localization using combined

ERP and neuroimaging,251–254

locus of selection, 248–249overview, 245–246, 255reflexive attentional orienting,

249–251voluntary attention

localization, 251Attention-related negativity, in

cortical auditory potentials, 23Auditory system

deprivation effects on spatialattention development, 228

evoked potentialsevent-related fields, 35event-related potentials

brain stem potentials, 22–23cortical auditory potentials,

23–24middle latency potentials,

22–23mismatch negativity, see

Mismatch negativitymusic perception, 347–349

Automated processes, 24–25Autoradiography, 2-deoxyglucose

labeled with 14C, 364–365Axford, J. G., 95

BBaldeweg, T., 328, 329Barcelo, F., 276Barrett, G., 313, 323Bauer, H. J., 88Bergua, A., 83Bernstein, P. S., 200Biella, Gabriele, 359–366, 421–425Biofeedback, epilepsy event-

related potential studies,331–332

Bipolar recordings, 383–387Birbaumer, N., 331Blocked studies, cross-functional

study comparisons ofcognitive functions, 55, 57–59

Bokura, H., 213Braeutigam, S., 117Brain

alcohol effects, 369cerebral tissue lesions

event-related potentials,322–327

aphasia, 323–325extinction, 325–327hemiparesis, 323neglect, 325–327vascular dementia, 322

neurological symptoms, 368

cognitive electrophysiology, seeCognitiveelectrophysiology

cognitive functions, seeCognitive functions

event-related potentialsrelationship, 6–9

evoked electrical activity, 72–75,88–89

language structurecomprehension, 144–146

neurochemical lesionsdrug protection, 364neurotoxins, 364

neurocognitive development, seeNeurocognitivedevelopment

neurological disease, see specificdiseases

primary language acquisitioneffects on cerebralorganization, 236–238

Brain stem potentials, 22–23Brattico, Elvira, 343–352Braver, T. S., 57, 58, 60, 64Brazhnik, E. S., 210, 215Breitmeyer, B., 99Broca, 117Broca’s aphasia, event-related

potential studies, 323–325Brodmann areas, 44–45Buchsbaum, M. S., 284Bungener, C., 328Buño, W. J., 210Bush, G., 199Buzsáki, G., 13, 208, 210

CCabeza, Roberto, 7, 41–65Campbell, F. W., 288Cancer, neurological symptoms,

368–369Caplan, D., 157Carter, C. S., 201Castañeda, M., 312Caton, R., 72Central fixation, in

magnetoencephalographicstudies, 128–129

Ceponiene, R., 350Cerebellar atrophy

event-related potential studies,317–319

cognition, 318–319movement-related activity,

317–319neurological symptoms, 368

424 INDEX

Cerebral microdialysis, 365Chapman, R. M., 410, 414Chatrian, G. E., 382Cheour, M., 350Chomsky, Noam, 145Cloze probability, 31Cognitive electrophysiology

description, 4–5self-regulation,

electrophysiological signs,213–217

action regulation, 213–214context updating, 215–216distraction, 214–215novelty, 214–215

Cognitive functions, see also specificfunctions

controlled processes, 24–25control measures in Parkinson’s

disease, 316cross study comparisons, 43–62

functions, 53–62blocked studies, 55, 57–59description, 53–57

event-related fMRI studies,59–62

methods, 43–46overview, 43, 52–53results, 46–52

medial temporal lobes,50–53

midline regions, 48–49, 53parietal regions, 49–50, 53prefrontal regions, 46–48,

52–53temporal regions, 50, 53

evoked visual informationprocessing study, see Visualinformation processing

self-regulation, 197–217action regulation mechanisms,

205–209affective modulation,

212–213amplitude modulation,

211–212corticolimbic integration,

208–209dopamine effects, 205–206electrophysiology, 213–214executive control, 209limbic theta, 209models, 207–208motivational control,

209–213Papez circuit, 206–207, 209prediction errors, 205–206theta rhythm phase reset,

208–211

anterior cingulate cortex,198–209


207–208adaptation, 206–207dopamine effects, 205–206electrophysiology, 213–214executive control, 199–202Papez circuit, 206–207, 209theta dynamics, 202–205



overview, 197–198Cognitive slowing, epilepsy

studies, 329–331Cognitive theory, event-related

potentials relationship, 3–6Cohen, D., 93, 325Coles, M. G. H., 200, 205, 207, 209,

216Color vision

attentional visual processing,298–301


Common reference method, 19–20,383–387

Comparison studies, see Cross-functional approach

Connolly, J. F., 32Connolly, S., 328Context updating, executive

function regulationelectrophysiology, 215–216

Contingent negative variationstudies

amyotrophic lateral sclerosis,320

aphasia, 324–325cerebellar atrophy, 318motor potentials, 26Parkinson’s disease, 313–315

Contrast response, attentioneffects, 268–270

Contrast threshold,magnetoencephalographicstudies, 98–101

Controlled processes, see alsoExecutive functions; specificprocesses

description, 24–25Cooper, L. A., 121Corbetta, M., 7, 120, 128, 279Correct-related negativity, in

executive control study,200–201

Cortical auditory potentials, 23–24Corticolimbic integration, theta

rhythm relationship, 208–209Coulson, S., 153Courchesne, E., 318Covert visual attention, 245Cross-functional approach

overview, 41–43, 62–65study comparisons, 43–62

cognitive functions, 53–62blocked studies, 55, 57–59event-related fMRI studies,

59–62overview, 53–57





Cryocoagulation, 360–361Csépe, V., 324Cue invariance,

magnetoencephalographicstudies, 109

Cunnington, R., 313, 314Curran, T., 180

DDale, A., 15Damasio, A. R., 4, 106, 198Daum, I., 318Degenerative disease, see specific

diseasesde Haan, M., 231Dehaene, S., 199Demiralp, T., 214, 216De Monasterio, F. M., 102Deouell, L. Y., 3262-deoxyglucose, in

autoradiography, 364–365Desmedt, J. E., 33D’Esposito, M., 59, 60Dick, J. P. R., 313Dickinson, A., 211Digital manipulations, of

high-density electromagneticsignals

offline digital filtering, 389–390signal digitation rate, 387

Dikman, Z. V., 202

INDEX 425

Dillon, W. R., 413Dipoles, see Electromagnetic

dipolesDirect problem, in

electroencephalogram, 16–19Di Russo, Francesco, 257–271Distraction, executive function

regulation electrophysiology,214–215

Dolan, R. J., 204Donchin, E., 21, 216, 321Dopamine

executive function actionregulation mechanisms,205–206

Parkinson’s disease study, 314Dorsal processing stream,

magnetoencephalographicstudies, 102–104, 109–113

Drake, M. E. Jr., 329

EEarly left anterior negativity

in language comprehensionstudies, 153–155

in linguistic potentials, 31Eason, R. G., 119, 285Ebmeier, K. P., 316Electrocardiogram, artifacts in EEG

recordings, 388Electrodes

deep intracerebral electrodes,363–364

epidural electrodes, 363high-density electromagnetic

signal recording, 379–383caps, 380–381impedance, 381placement, 380sites, 381–383types, 380

reference electrode, 19–20,383–387

subdural electrodes, 363Electroencephalogram, see also

Event-related potentialsanterior cingulate cortex study,

205brain activity neurophysiology,

72–75, 88–89component analysis, 19dipole localization, 19–20direct problem, 16–19electromagnetic dipoles, 15–16electromagnetic signals, 13–15evoked potentials, see Evoked

potentials

high-density electromagneticsignals, 379–400

amplifiers, 379, 385–386artifacts, 387–388bipolar recordings, 383–387electrodes, 379–383

caps, 380–381impedance, 381placement, 380reference electrode, 19–20,

383–387sites, 381–383types, 380

monopolar recordings,383–387

offline digital filtering,389–390

online analog filters, 386–387overview, 379scalp topographic mapping,

see Topographicmapping

signal averaging, 388–390signal digitation rate, 387volunteer recruitment,

398–400inverse problem, 19laboratory set up, 374–376language comprehension study,

147overview, 13, 373

Electromagnetic dipolesdirect problem, 16–19electroencephalogram

relationship, 15–16inverse problem, 19localization, 19–20

Electromagnetic signalselectroionic origins, 13–15high-density signals, 379–400

amplifiers, 379, 385–386artifacts, 387–388bipolar recordings, 383–387electrodes, 379–383

caps, 380–381impedance, 381placement, 380sites, 381–383types, 380

event-related potentialaveraging, 388–390




see Topographic

mappingsignal digitation rate, 387volunteer recruitment,

398–400Electrooculogram

bipolar recordings, 383–387electrode placement, 382

Electrophysiological indicesof implicit memory, 182–183of retrieval attempts, 186–191

Electroretinogram, evoked visualinformation processing, 75

Elliot, R., 204Elliott, F. S., 329Encephalomyelitis disseminata,

event-related potentialstudies, 327–328

Engel, S. A., 291Epilepsy, event-related potential

studies, 329–332biofeedback, 331–332cognitive slowing, 329–331intracranial recordings, 331memory impairments, 329–331

Episodic memory, 169–191, see alsoLong-term memory; Workingmemory

encodingactivation stimulus, 44–53cognitive regions, 63description, 170–175

episodic retrievalactivation stimulus, 44–53cognitive regions, 63electrophysiological indices

implicit memory, 182–183retrieval attempts, 186–191

familiarity effects, 179–182old/new effects

description, 175–176at frontal scalp sites,

183–186left-parietal event-related

potentials, 176–179putative electrophysiological

correlates, 179–182overview, 169–170, 191

Error-related negativityelectrophysiology, 213–216overview, 199–202, 216–217theta dynamics, 202–205,

212–213Eulitz, C., 116Evaluative negativity, in anterior

cingulate cortex study, 201Event-related fields

auditory fields, 35description, 13, 34–35somatosensory fields, 36

426 INDEX

superconducting quantuminterference device use,395–398

visual fields, 35–36Event-related fMRI studies, see

Functional magneticresonance imaging

Event-related potentials, see alsoElectroencephalogram

attention localizationERP generator modeling, 251neuroimaging combined,

251–254to object features, 273–301



285–291features conjunction,

292–298frequency-based attentional

selection, 291–292neural systems, 274–280object perception, 292–298orientation-directed

attention, 288–291overview, 273–274, 301posterior attentional system,




attention, 286–288,292–298

brain relationship, 6–9cognitive theory relationship,

3–6description, 3, 13, 20–22,

223–224episodic encoding, 170–175episodic retrieval, 175–191error-related negativity

electrophysiology, 213–216overview, 199–202, 216–217theta dynamics, 202–205,

212–213familiarity effects, 179–182

high-density electromagneticsignal recording andanalysis, 379–400

amplifiers, 379, 385–386artifacts, 387–388averaging, 388–390bipolar recordings, 383–387electrodes, 379–383

caps, 380–381

impedance, 381placement, 380sites, 381–383types, 380



online analog filters, 386–387overview, 379scalp topographic mapping

description, 390–393limits, 393–395

signal digitation rate, 387volunteer recruitment,

398–400laboratory set up, 374–376language comprehension study,

149–155, 163late components, 24–26linguistic potentials, 30–33mismatch negativity, see

Mismatch negativitymotor potentials, see Motor

potentialsneurocognitive development

study, 223–224, 228–230neurological disease study,

309–332cerebral tissue lesions, 322–327

aphasia, 323–325extinction, 325–327hemiparesis, 323neglect, 325–327vascular dementia, 322

degenerative diseases,309–332

Alzheimer disease, 181,309–312

amyotrophic lateralsclerosis, 320–322

cerebellar atrophy, 317–319Huntington’s disease, 319,

368Parkinson’s disease,

312–317progressive supranuclear

palsy, 320epilepsy, 329–332

biofeedback, 331–332cognitive slowing, 329–331intracranial recordings, 331memory impairments,

329–331inflammatory diseases,

327–329HIV infection, 328–329multiple sclerosis, 327–328

overview, 309, 332

old/new effectsdescription, 175–176at frontal scalp sites, 183–186left-parietal event-related

potentials, 176–179P300, 21, 24–26primary language acquisition

effects on cerebralorganization, 236–238

subsequent memory effects,170–172

topographic mapping, seeTopographic mapping

Evoked potentialsauditory evoked potentials

event-related fields, 35event-related potentials

brain stem potentials, 22–23cortical auditory potentials,

23–24middle latency potentials,

22–23mismatch negativity, see

Mismatch negativitysomatosensory evoked

potentials, 33–34visual information processing

studieshigher cognitive processes,

79–83multichannel recording, 75–78neural plasticity, 79–83neurology applications, 88neurophysiological bases,

72–75, 88ophthalmology applications,

88overview, 27–30, 71–72,

88–89perceptual learning, 79–83steady-state visual evoked

potentials, 78–79, 257–271cognitive process

relationship, 258contrast response, 268–270magno cellular pathways,

265–268overview, 257–258,

270–271parvocellular pathways,

265–268phase effects, 262–265spatial attention, 258–262

stereoscopic perception, 83–88stimulation frequency

influence, 78–79topographic mapping, 75–78

Excitatory postsynaptic potentials,13–14, 379

INDEX 427

Executive functions, see also specificfunctions

controlled processes, 24–25control measures in Parkinson’s

disease, 316cross study comparisons, 43–62

functions, 53–62blocked studies, 55, 57–59event-related fMRI studies,

59–62overview, 53–57





evoked visual informationprocessing study, see Visualinformation processing

self-regulation, 197–217action regulation mechanisms,

205–209affective modulation,

212–213amplitude modulation,


208–209dopamine effects, 205–206electrophysiology, 213–214executive control, 209limbic theta, 209models, 207–208motivational control,

209–213Papez circuit, 206–207,

209prediction errors, 205–206theta rhythm phase reset,

208–211anterior cingulate cortex,

198–209action monitoring, 199–202action regulation model,

207–208adaptation, 206–207dopamine effects, 205–206electrophysiology, 213–214executive control, 199–202Papez circuit, 206–207,

209theta dynamics, 202–205



overview, 197–198Eye movement

artifacts in EEG recordings, 388

central fixation, 128–129Eyes, see Visual information

processing

FFace perception


neurocognitive development,age and experience effects,230–233

face processingacross early school years,

231–233in Williams syndrome, 233

infant face recognition, 231Fahle, M., 81Fahy, F. L., 128Falkenstein, M., 201, 213, 316Familiarity, in episodic retrieval,

179–182Farwell, L. A., 321Fast cyclic voltammetry, 365–366Fattaposta, F., 313Federmeier, Kara D., 143–163Fein, G., 329Felleman, D. J., 94, 112Fensik, D. E., 201Ffytche, D. H., 110, 111Field, D., 298Filipovic, S., 319Filters, of high-density

electromagnetic signalsoffline digital filtering,

389–390online analog filters, 386–387

Fletcher, P. C., 185Foder, J. A., 146Folded feedback, 298Ford, J. M., 200, 311, 312Frequency

attentional selection, 291–293spatial frequency


of a visual stimulus,286–288, 292–298

stimulus frequency, influence onsteady-state visual evokedpotentials, 78–79

Freud, S., 4Friederici, A. D., 324Friedman, D., 171, 174, 311Friedman-Hill, S. R., 296Fries, P., 127Friston, K. J., 7Frith, C. D., 7Fukai, M., 329Functional magnetic resonance

imaginganterior cingulate cortex study,

201attention localization, 254–255cross-functional study

comparisons of cognitivefunctions, 54–56, 59–62

episodic encoding study, 175invasiveness, 424–425language comprehension study,

147–149, 157–158, 161overview, 6, 41, 44, 71spatial attention modulation

study, 284spatial resolution, 421–424temporal resolution, 421–424

Functional neurosurgery, 360Furmanski, C. S., 291Fuster, J. M., 8Fylan, F., 110

GGabriel, M., 206–209, 215, 216Gaeta, H., 311Gaetz, M., 122Gardiner, J. M., 174Gauthier, I., 108, 109Gazzaniga, M. S., 4Gehring, W. J., 201, 202Gemba, H., 200Gerschlager, W., 314Giesser, B. S., 327Gil, R., 320, 327Gilbert, C. D., 284Givens, B., 210Glaser, E. M., 413Global field power, 409–411Goldstein, M., 413Goodin, D. S., 310, 328Goodwin, G. M., 328Gouras, P., 102Green, J. B., 323Grill-Spector, K., 109Grippo, A., 330Grossberg, S., 298Grunwald, T., 331Guido, W., 289Gurney, K., 206

428 INDEX

HHalgren, E., 330Hansch, E. C., 315Hari, R., 97, 98, 113, 114, 125Harman, H. H., 413Harter, M. R., 287, 289, 290, 292Hashimoto, I., 101Hawking, Stephen, 321Haxby, J. V., 102Helmholz, Herman Von, 245Hemiparesis, event-related

potential studies, 323Hemodynamic functional

anatomy, visual selectiveattention to object features,280

Henson, R. N. A., 175, 185Hess, R., 298High-density electromagnetic

signals, seeElectroencephalogram;Magnetoencephalography;Neuronal recordings

Hillyard, Steven A., 257–271, 299

HIV/AIDSevent-related potential studies,

328–329neurological symptoms, 368

Hockett, Charles, 146Holliday, I. E., 111Holroyd, C. B., 205, 207, 209Homan, R. W., 383Hömberg, V., 319Honig, L. S., 327Horne, J. A., 399Hubel, D. H., 102Huntington’s disease

event-related potential studies,319

neurological symptoms, 368

IIkeda, A., 318Ilvonen, T.-M., 324, 346Implicit memory,

electrophysiological indices,182–183

Inflammatory diseases, see alsospecific diseases

event-related potentials, 327–329

neurological symptoms, 368Information processing, see

Attention; Cognitivefunctions; Visual informationprocessing

Inhibitory postsynaptic potentials,13, 379

Interstimulus intervalin Alzheimer disease, 311in reflexive attentional orienting,

249–251Intracranial recordings, epilepsy

event-related potentialstudies, 331

Invasiveness, 424–425Inverse problem, in

electroencephalogram, 19Ito, M., 284Iwaki, S., 122

JJahanshahi, M., 313James, William, 246Jedynak, A., 83Jeffreys, D. A., 95Johannes, S., 202John, E. R., 414, 415Johnson, R. Jr., 171, 184, 186, 187,

317, 319, 320Just, M. A., 157

KKahana, S., 209Kajola, M. J., 114Kaneoke, Y., 112Kanizsa, G., 297Kanwisher, N., 107Karayanidis, F., 316Karhu, J., 118Katayama, J., 214, 215Katznelson, R. D., 15Kaufman, L., 93, 99, 121Kawamichi, H., 122Kawashima, R., 290Kazmerski, V. A., 311Kew, J. J., 320King, J. W., 150, 151, 156Kitamura, J.-I., 323Kleins, M., 26Kluender, Robert., 143–163Knight, R. T., 26, 216, 322, 329Know response

episodic encoding, 174–175familiarity effects, 179–180

Köhler, S., 7Kohlmetz, C., 324Kopp, B., 213Kosslyn, S. M., 121Kujala, T., 346Kuriki, S., 32Kutas, Marta, 15, 31, 143–163

L

LaBar, K. S., 57, 60LaBerge, D., 284Lam, K., 111Language

comprehension, 143–163brain function examination

methods, 146–148comprehension, 148perception, 148–151processing patterns, 151–155

language structure, 144–146long-term memory, 158–162overview, 143, 162–163working memory, 155–158mismatch negativity studies,

32–33, 345–347neuroplasticity development,

234–239American Sign Language

studies, 235–236bilingual adult studies, 234cerebral organization, 236–238deaf adult studies, 234–235influencing factors, 238–239primary language acquisition

effects, 236–238semantic differential technique,

80semantic retrieval, 44–53syntactic positive shift, 31–32

Late centroparietal positivity,31–32

Late components, 21, 24–26Late-positive complex, in anterior

cingulate cortex study, 201Lateral geniculate nucleus, visual

stream development, 224–225Leads, see ElectrodesLearning, evoked visual

information processing study,79–83

Least squares criterion, 19Le Doux, J. E., 4Left anterior negativity

in language comprehensionstudies, 153–155

in linguistic potentials, 31Lehmann, D., 415Leinonen, L., 128Leipert, K. P., 75, 88Lesions

cerebral tissue lesionsevent-related potential

studies, 322–327aphasia, 323–325extinction, 325–327hemiparesis, 323

INDEX 429

Lesions (continued)neglect, 325–327vascular dementia, 322

neurological symptoms, 368neurochemical lesions

drug protection, 364neurotoxins, 364

Lhermitte, F., 326Limbic lobe

executive function relationshipaction regulation mechanisms

corticolimbic integration,208–209

limbic theta, 209anterior cingulate cortex,

198–209action monitoring, 199–202action regulation model,


Nauta’s limbic set points, 197Linden, A., 314Linear accelerator irradiation, 361Linguistic potentials, see also

Languagedescription, 30–33language comprehension study,

149–155, 163primary language acquisition


Linkenkaer-Hansen, K., 108Linnville, S. E., 329Lisman, J. E., 210Liu, J., 108Liu, L., 107Livingstone, M. S., 102Local field potentials, recording

methods, 361Logothetis, N. K., 106, 108Long-term memory, see also

Episodic memory; Workingmemory

epilepsy event-related potentialstudies, 329–331

language comprehensionstudies, 158–162

recollection, 169, 177, 183Looi, J. C. L., 322Lu, S. T., 107Luber, B., 119Luck, S. J., 284Lurija, A., 8Luu, Phan, 197–217

MMaclin, E., 96Maffei, L., 288Magnetic resonance imaging, see

Functional magneticresonance imaging

Magnetoencephalographyanterior cingulate cortex study,

204–205laboratory set up, 376–377magnetoencephalogram

component analysis, 19dipole localization, 19–20direct problem, 16–19electromagnetic dipoles,

15–16electromagnetic signals, 13–15inverse problem, 16–19magnetic field recording,

395–398overview, 13, 395–398reference electrode, 19–20,

383–387overview, 373superconducting quantum

interference devices,395–398

visual information processingstudies

basic visual functions, 98–113color vision, 104–109contrast threshold, 98–101cue invariance, 109dorsal processing stream,

102–104, 109–113motion vision, 109–113spatial frequency, 98–101spatial vision, 102–104,

109–113temporal frequency, 98–101ventral processing stream,

102–109central fixation issues,

128–129higher order processes,

118–127mental imagery, 120–122selective attention, 118–120,

128–129working memory, 122–127

multimodality imaging,129–130

oscillatory behavior, 113–118induced activity, 116–118periodic stimulation,

114–116overview, 93–95retinotopy identification,

95–98

source modeling issues,127–128

spatial attention modulationstudy, 284

visual area identification,95–98

Magno cellular visual pathways,attention effects, 265–268

Makeig, S., 202, 204Malach, R., 107Mangels, J. A., 174, 175Mangun, George R., 7, 245–255,

291Manzey, D., 417Marzi, C. A., 325, 326McCarthy, G. A., 284, 417McCrary, J. W., 410, 414McKhann, G. M., 88Mecklinger, A., 181Medial temporal lobes, cross-

function studies comparedblocked paradigms, 55, 57–59event-related paradigms, 56,

59–62results, 50–53

Mellet, E., 122Memory, see Episodic memory;

Long-term memory; Workingmemory

Mental imagery,magnetoencephalographicstudies, 120–122

Messenheimer, J. A., 328Miceli, G., 300Michel, C. M., 121Microdialysis, cerebral

microdialysis, 365Middle latency potentials, 22–23Middle occipital gyrus, attention

localization, 252–253Middle temporal gyrus, visual

stream development, 224–225Midline regions, cross-function

studies comparedblocked paradigms, 55, 57–59event-related paradigms, 56,

59–62results, 48–49, 53

Miller, R., 209Mills, D. L., 233Minamoto, H., 320Mishkin, M., 102, 279Mismatch negativity

in Alzheimer disease, 311clinical research, 349–352in cortical auditory potentials,

24linguistic function study, 32–33,

345–347

430 INDEX

music perception study, 347–349overview, 343–345in Parkinson’s disease, 316–317in Wernicke’s aphasia, 324–325

Mitchell, Teresa V., 223–239Miura, Y., 211Möcks, J., 417Monopolar recordings, 383–387Monsell, S., 189Morgan, S. T., 259, 260, 262Morphemes, 144Moscovitch, M., 7, 61, 183, 184Motion vision,


Motor potentialsAlzheimer disease study, 310amyotrophic lateral sclerosis

study, 321attentional visual processing to

object features, 293, 295, 299cerebellar atrophy study, 318description, 26–27epilepsy study, 329–330executive control regulation

electrophysiology, 213–217HIV/AIDS study, 328multiple sclerosis study, 327negative deflection effects, 30Parkinson’s disease study, 312,

316Movement-related activity, event-

related potential studiescerebellar atrophy, 317–319Parkinson’s disease, 312–313,

315Müller, M. M., 261, 262Multichannel recording, visual

information processingevoked potentials, 75–78magnetoencephalographic

studies, 129–130Multiple sclerosis

event-related potential studies,327–328

neurological symptoms, 368Multisphere model, 18Münte, T. F., 162, 319–321Muscles, artifacts in EEG

recordings, 388Music perception, mismatch

negativity studies, 347–349

NN1

color attention study, 299epilepsy study, 329–330HIV/AIDS study, 328

Huntington’s disease study, 319progressive supranuclear palsy

study, 320spatial attention study, 247–248visual stream development

study, 226–230N2

Alzheimer disease study, 310amyotrophic lateral sclerosis

study, 321attentional visual processing to

object features, 293, 295, 299cerebellar atrophy study, 318description, 26–27epilepsy study, 329–330executive control regulation

electrophysiology, 213–217HIV/AIDS study, 328multiple sclerosis study, 327negative deflection effects, 30Parkinson’s disease study, 312,

316N60, steady-state visual evoked

potentials for attention, 264N100

in language comprehensionstudies, 149

multiple sclerosis study, 327N100m, 35N140, 33

description, 33steady-state visual evoked

potentials for attention, 264N170, face perception

development study, 230–233N200, face perception

development study, 230N278

cortical auditory potentials, 23in language comprehension,

151–155visual evoked potentials, 30, 289,

294, 300N320, face perception

development study, 233N400

Alzheimer disease study, 311aphasia study, 324–325description, 26epilepsy study, 331in language comprehension

studies, 151, 153, 159–161in linguistic potentials, 32Parkinson’s disease study, 317

N800, epilepsy study, 331Näätänen, Risto, 24, 343–352Naito, T., 112Nakamura, M., 101Narici, L., 114, 115

Nauta, W. J. H., 197Nauta’s limbic set points, 197Necker cube, 122Negative deflection, motor

potentials, 30Negative difference

in cortical auditory potentials, 24in Parkinson’s disease, 316

Network view of neuroimaging, 43

Neural specificity theory, ofattentional selection, 289

Neurochemical lesionsdrug protection, 364neurotoxins, 364

Neurocognitive development, ageand experience effects,223–239

face perception development,230–233

face processingacross early school years,

231–233in Williams syndrome, 233

infant face recognition, 231language function

neuroplasticity, 234–239American Sign Language

studies, 235–236bilingual adult studies, 234cerebral organization, 236–238deaf adult studies, 234–235influencing factors, 238–239primary language acquisition

effects, 236–238overview, 223–224, 238–239spatial attention, 228–230



visual stream development,224–228

Neuroimaging, seeElectroencephalogram; Event-related potentials;Magnetoencephalography;Neuronal recordings

Neurological disease, see specificdiseases

Neuromodulators, theta rhythmmodulation, 211–213

Neuronal recordings, see alsoElectroencephalogram; Event-related potentials;Magnetoencephalography

clinical applications, 363–364deep intracerebral electrodes,

363–364

INDEX 431

Neuronal recordings, see alsoElectroencephalogram; Event-related potentials;Magnetoencephalography(continued)epidural electrodes, 363subdural electrodes, 363

extracellular recordings, 361–362high-density electromagnetic

signals, 379–400amplifiers, 379, 385–386artifacts, 387–388bipolar recordings, 383–387electrodes, 379–383


event-related potentialaveraging, 388–390




see Topographic mappingsignal digitation rate, 387volunteer recruitment,

398–400intracellular recordings, 362intracranial recordings, epilepsy

study, 331invasiveness, 424–425laboratory set up, 374–376local field potentials, 361multichannel recording

multiunit activities, 361visual information processing

studies, 75–78overview, 42–43, 374patch-clamp recordings, 362–363reference electrodes

common reference method,383–387

dipole localization, 19–20spatial resolution, 421–424temporal resolution, 421–424

Neurophysiologycognitive electrophysiology

description, 4–5self-regulation, 213–217

evoked electrical brain activity,72–75, 88–89

Neurosurgeryinvasive procedures, 360–361noninvasive procedures, 361

stereotaxic neurosurgery,359–361

Neurotoxins, lesion relationship,364

Neurotransmitters, voltammetricmeasurement, 365–366

Neville, Helen J., 223–239Newton, M. R., 327Nielsen-Bohlman, L., 26Nishitani, N., 112–113Nobre, A. C., 188, 189Novelty, executive function

regulation electrophysiology,214–215

Nunez, P. L., 13Nyberg, Lars, 7, 41–65Nyman, G., 128

OOakley, M. T., 285Object features, attentional visual

processing, 273–301anterior attentional system,

273–279color, 298–301feature-directed attention,

285–291features conjunction, 292–298frequency-based attentional

selection, 291–292neural systems, 274–280object perception, 292–298orientation-directed attention,

288–291overview, 273–274, 301posterior attentional system,

273–274, 279–280primary visual area modulation,

280–285space-based attentional


attention, 286–288, 292–298Obsessive-compulsive disorder,

error-related negativityrelationship, 202

O’Donnell, R. D., 258Oishi, M., 314Ojemann, G., 128Okada, Y., 99, 130Okusa, T., 109Old/new effects, in episodic

retrievaldescription, 175–176at frontal scalp sites, 183–186left-parietal event-related

potentials, 176–179Ollo, C., 328, 329

Onofrj, M., 321Ophthalmology, see Visual

information processingOpiates, theta rhythm modulation,

211–213Orban, G. A., 290Orientation

reflexive attentional orienting,249–251

visual attention to objectfeatures, 288–291

Oscillatory behaviors, in visualsystem,magnetoencephalographicstudies, 113–118

induced activity, 116–118periodic stimulation, 114–116

Ostberg, O., 399Osterhout, L., 155Otmakhova, N. A., 210Otten, L. J., 172, 175

PP0z, steady-state visual evoked

potentials for attention,268–269

P04, steady-state visual evokedpotentials for attention,268–269

P1in visual evoked potentials, 30visual stream development

study, 225–226P2

description, 27Huntington’s disease study, 319Parkinson’s disease study, 316

P3Alzheimer disease study,

310–312amyotrophic lateral sclerosis

study, 321cerebellar atrophy study,

318–319epilepsy study, 329–330HIV/AIDS study, 328–329Huntington’s disease study, 319multiple sclerosis study, 327Parkinson’s disease study, 316progressive supranuclear palsy

study, 320P3a, executive control regulation

electrophysiology, 213–217P3b

executive control regulationelectrophysiology, 213–217

visual selective attention toobject features, 276–277

432 INDEX

P50m, 35P100

latencies, 406–407steady-state visual evoked

potentials for attention, 264P150, in language comprehension

studies, 149P190

description, 33visual selective attention to

object features, 279P200, multiple sclerosis study, 327P250, epilepsy study, 331P290, epilepsy study, 331P300

attentional visual processing toobject features, 293, 295, 299

in event-related potentials, 21,24–26

P400, infant face recognitiondevelopment study, 231

P600description, 31–32in language comprehension

studies, 151–155, 162Paller, K. A., 187, 188Papez circuit, executive function

regulation relationship,206–207, 209

Paradoxical laterlization, 78Parietal regions

cross-function studies comparedblocked paradigms, 55, 57–59event-related paradigms, 56,

59–62results, 49–50, 53

late centroparietal positivity,31–32

left-parietal event-relatedpotentials, old/new effects,176–179

posterior parietal cortex, visualstream development, 225

Parkinson’s diseaseevent-related potential studies,

312–317executive control measures,

316memory, 316–317

movement preparationafter ambiguous imperative

signals, 315before imperative signals,

313–315oddball tasks, 315–316self-initiated movements, 313

neurological symptoms, 367–368Parra, J., 114Parvocellular visual pathways,

attention effects, 265–268Patch-clamp recordings, 362–363Patterns

pattern onset modality in visualevoked potentials, 29

processing patterns in languagecomprehension, 151–155

Patzwahl, D. R., 111Pekkonen, E., 311, 350Pelosi, L., 328Perception, see also Attention

activation stimulus, 44–53evoked visual information

processing studyperceptual learning, 79–83stereoscopic perception, 83–88

face perceptionmagnetoencephalographic

studies, 106–108neurocognitive development

age and experience effects,230–233

face processing across earlyschool years, 231–233

face processing in Williamssyndrome, 233

infant face recognition, 231language comprehension,

148–151music perception, mismatch

negativity studies, 347–349object perception, 292–298

Petersen, S. E., 32Petit, L., 128Phillips, N. A., 32Picton, T. W., 394Pilgreen, K. L., 15Plat, F. M., 315Platz, T., 323Polich, J., 214, 215, 216Portin, K., 98, 103, 104Positron emission tomography

Alzheimer disease study, 310anterior cingulate cortex study,

204attention localization

event-related potentialscombined, 251–254

visual attention to objectfeatures, 290

invasiveness, 424–425language comprehension study,

147, 157–158language localization, 32overview, 6–7, 41, 44, 54–56, 71spatial resolution, 421–424temporal resolution, 421–424

Posner, M. I., 246–248, 326Posterior attentional system, visual

information processing toobject features, 273–274,279–280

Posterior cingulate cortex, see alsoAnterior cingulate cortex

function, 207Posterior parietal cortex, visual

stream development, 225Praamstra, P., 314, 315Prefrontal regions

cross-function studies comparedblocked paradigms, 55, 57–59event-related paradigms, 56,

59–62results, 46–48, 52–53

visual selective attention toobject features, 275–279

Premotor positivein visual evoked potentials, 30visual stream development

study, 225–226Previc, F. H., 287, 289Principal component analysis

component extraction, 412–414description, 19, 412–418examples, 414–415physiological interpretation,

416–417Probe technique, from event-

related potentials, 21Processing negativity

in cortical auditory potentials, 23in language comprehension,

151–155in visual evoked potentials, 30,

289, 294, 300Progressive supranuclear palsy

event-related potential studies,320

neurological symptoms, 368Proverbio, Alice M., 3–11, 13–36,

80, 273–301, 359–366, 373–377,379–400, 421–425

Puce, A., 329, 330Pulvermüller, F., 314Putative index of familiarity, in

episodic retrieval, 179–182

RRadiation

autoradiography, 364–365linear accelerator irradiation,

361Radio frequencies, artifacts in EEG

recordings, 388Raij, T., 122Raile, A., 79Raizada, R. D. S., 298

INDEX 433

Random-dot stereograms, evokedvisual information processingstudy, 82–88

Ranganath, C., 59, 60, 64, 187, 188Readiness potential, in motor

potentials, 27Recollection, 169, 177, 183Recording methods, see Neuronal

recordings; specific methodsRecruitment, for brain wave

recording studies, 398–400Redgrave, P., 205, 206, 210Reference electrode

common reference method,383–387

dipole localization, 19–20Reference potential

description, 27Huntington’s disease study, 319Parkinson’s disease study, 316

Reflexive attentional orienting,249–251

Remember responseepisodic encoding, 174–175familiarity effects, 179–180

Retinotopy, visual areaidentification usingmagnetoencephalography,95–98

Revonsuo, A., 311, 312Rif, J., 35Ritter, W., 318Robb, W. G. K., 190Robertson, D., 33Rodin, E., 329Rogers, R. D., 189Rosenberg, C., 319Rösler, F., 417Ruchkin, D. S., 328, 413Rugg, M. D., 172, 175, 180, 182–184,

186, 188, 189, 190, 311, 330, 331Rüsseler, J., 348

SSachdev, P. S., 322Salenius, S., 113, 114Salmelin, R., 113, 114Sams, M., 107Sanquist, T. F., 176, 177Sato, N., 103Scabini, D., 216, 284Scalp current density mapping, see

also Electroencephalogram;Event-related potentials;Neuronal recordings; specificcognitive functions

analytical methods, 403–418applications, 417–418

components, 408–418extraction, 412–414physiological

interpretation, 416–417spatial components

analysis, 414–415overview, 403–408rational, 404–408

description, 29, 390–393electrodes, 379–383


evoked visual informationprocessing study, 75–78,81–82, 290

isoline maps, 384, 390–392limitations, 393–395primary language acquisition


stationary maps, 390visual attention to object

features, 290Scheffers, M. K., 200Schendan, H. E., 149Schroeder, C. E., 290, 328Schröger, E., 344Seki, K., 100Selection negativity

cortical auditory potentials, 23in language comprehension,

151–155visual evoked potentials, 30,

289, 294, 300Selective attention, see also

Perceptionmagnetoencephalographic

studies, 118–120, 128–129to object features, 276–277top-down selection, 280

Self-regulation, of executivefunctions, 197–217

action regulation mechanisms,205–209

affective modulation, 212–213amplitude modulation,


208–209dopamine effects, 205–206electrophysiology, 213–214executive control, 209limbic theta, 209models, 207–208motivational control, 209–213Papez circuit, 206–207, 209

prediction errors, 205–206theta rhythm, 208–211

anterior cingulate cortex,198–209





overview, 197–198Semantic differential technique,

evoked visual informationprocessing study, 80

Semantic retrieval, activationstimulus, 44–53

Sensory processes, see Perception;specific senses

Sergent, J., 106Sharing view of neuroimaging,

42Sharpe, Helen, 169–191Sheinberg, D. L., 106, 108Shepard, R. N., 121Shulman, G. L., 279, 284Shultz, W., 205, 211Signal digitation rate, for

electroencephalograms, 387Silberstein, R. B., 13, 258, 260Singer, W., 127Skin drilling, 380Skrandies, Wolfgang, 71–89,

403–418Smith, M. E., 174, 177, 330Sokolov, A., 117Somatosensory event-related

fields, 36Somatosensory evoked potentials,

33–34Somers, D. C., 284Sommer, W., 171, 172, 175Source modeling, in

magnetoencephalography,127–128

Spatial attentionattention to object features, 284electrophysiological measures,

247–248neurocognitive development,

228–230

434 INDEX



steady-state visual evokedpotential relationship,258–262

Spatial frequencymagnetoencephalographic

studies, 98–101of a visual stimulus, 286–288,

292–298Spatial resolution, overview,

421–424Spatial vision,

magnetoencephalographicstudies, 102–104, 109–113

Speech, see LanguageSpinelli, D., 267, 326Squire, L. R., 184Srinivasan, R., 115, 380Stam, C. J., 316Starr, A., 323Steady-state visual evoked

potentials, attentional visualprocessing, 257–271

cognitive process relationship,258

contrast response, 268–270magno cellular pathways,

265–268overview, 257–258, 270–271parvocellular pathways, 265–268phase effects, 262–265spatial attention, 258–262

Stephen, J. M., 93–130Stereoscopic perception, evoked

visual information processingstudy, 83–88

Stereotaxic neurosurgery, 359–361Sternberg, S., 312, 320, 324Stromswold, K., 157Subdivision view of neuroimaging,

42–43Subsequent memory, episodic

encoding, 170–171Supek, S., 97, 98Superconducting quantum

interference devices, inmagnetoencephalography,395–398

Surgery, see NeurosurgerySweating, artifacts in EEG

recordings, 388Swithenby, S. J., 107Syntactic positive shift

description, 31–32in language comprehension

studies, 151–155, 162

TTachibana, H., 317, 318, 319Talairach, J., 359Tallon-Baudry, C., 116Tanaka, K., 290Taylor, C., 215Taylor, M. J., 231, 232Teder-Sälejärvi, Wolfgang A.,

257–271Temporal frequency,


Temporal regions, cross-functionstudies compared

blocked paradigms, 55, 57–59event-related paradigms, 56,

59–62results, 50, 53

Temporal resolution, overview,421–424

Tendolkar, I., 181Ten-twenty electrode system,

electrode sites, 381–383ter Keurs, M., 324Tervaniemi, Mari, 343–352Tesche, C. D., 114, 118Thermocoagulation, 360Theta dynamics, executive

function action regulationmechanisms

for error-related negativity,202–205

limbic theta, 209theta rhythm

amplitude modulation,211–213

corticolimbic integration,208–209

phase reset, 209–211Thomas, C., 325Three-spheres model, 18Tononi, G., 115Tootell, R. B., 254Top-down selection, visual

selective attentionmodulation, 280

Topographic mapping, see alsoElectroencephalogram; Event-related potentials;Neuronal recordings; specificcognitive functions

analytical methods, 403–418applications, 417–418components, 408–418

extraction, 412–414physiological interpretation,

416–417spatial components

analysis, 414–415

overview, 403–408rational, 404–408

description, 29, 390–393electrodes, 379–383


evoked visual informationprocessing study, 75–78,81–82, 290

isoline maps, 384, 390–392limitations, 393–395primary language acquisition


stationary maps, 390visual attention to object

features, 290Touge, 313Tournoux, P., 359Toxins, lesion relationship, 364Triantafyllou, N. I., 327, 329Trott, C., 174Tsivilis, D., 181, 182Tsuchiya, H., 316Tucker, Don M., 197–217Tulving, E., 174Tumors, neurological symptoms,

368–369

UUngerleider, L. G., 102, 279Uusitalo, M. A., 112, 125Uutela, K., 120, 129

VVanderwolf, C. H., 211van Dijk, J. G., 327Van Essen, D. C., 94, 112Vanni, S., 98, 112, 114, 120,

129Vascular dementia, event-related

potential studies, 322Ventral processing stream,


Verleger, Rolf, 309–332, 367–369,417

Vertex potentialsdescription, 33steady-state visual evoked

potentials for attention, 264Vidal, F., 200Vieregge, P., 316, 321

INDEX 435

Visual information processingattention, see Attentioncolor vision


processing, 298–301deprivation effects on

development, 228–230event-related fields, 35–36evoked potential studies

higher cognitive processes,79–83

multichannel recording, 75–78neural plasticity, 79–83neurology applications, 88neurophysiological bases,

72–75, 88ophthalmology applications,

88overview, 71–72, 88–89perceptual learning, 79–83selection negativity, 30, 289,

294, 300steady-state visual evoked

potentials, 78–79, 257–271cognitive process

relationship, 258contrast response, 268–270magno cellular pathways,

265–268overview, 257–258, 270–271parvocellular pathways,

265–268phase effects, 262–265spatial attention, 258–262

stereoscopic perception, 83–88stimulation frequency

influence, 78–79topographic mapping, 75–78,

81–82, 290magnetoencephalographic

studies, 93–130basic visual functions, 98–113

color vision, 104–109contrast threshold, 98–101cue invariance, 109dorsal processing stream,

102–104, 109–113motion vision, 109–113spatial frequency, 98–101spatial vision, 102–104,

109–113temporal frequency, 98–101ventral processing stream,

102–109central fixation issues, 128–129

higher order processesmental imagery, 120–122selective attention, 118–120,

128–129working memory, 122–127

multimodality imaging,129–130

oscillatory behavior, 113–118induced activity, 116–118periodic stimulation,

114–116overview, 93–95retinotopy identification,

95–98source modeling issues,

127–128visual area identification,

95–98object feature selection, 273–301



285–291features conjunction, 292–298frequency-based attentional

selection, 291–292neural systems, 274–280object perception, 292–298orientation-directed attention,

288–291overview, 273–274, 301posterior attentional system,




attention, 286–288,292–298

oscillatory behaviors,magnetoencephalographicstudies, 113–118

induced activity, 116–118periodic stimulation, 114–116

spatial frequencymagnetoencephalographic

studies, 98–101of a visual stimulus, 286–288,

292–298visual stream development

age effects, 226–228atypical early experience

effects, 225–226word formation system, 32

Vogels, R., 290Voltage fluctuations, see Event-

related potentialsVoltammetry

fast cyclic voltammetry, 365–366overview, 365

Volume conductor, 17Volunteer recruitment, for brain

wave recording studies,398–400

Vomberg, H. E., 87Von Helmholtz, Herman, 245Vuilleumier, P., 326

WWagner, A. D., 171Wang, L., 122, 123Ward, A. A., 198Wascher, E., 314Wernicke, 117Wernicke’s aphasia, event-related

potential studies, 323–325Westphal, K. P., 320Wilding, Edward L., 169–191Williamson, S. J., 93, 99, 114, 125Williams syndrome, face

processing effects, 233Wilson, G. F., 258Wilson Card Sorting Task, 122–123Winkler, I., 351Woldorff, M. G., 35, 280Wood, C. C., 417Working memory, see also Episodic

memory; Long-term memoryactivation stimulus, 44–53cognitive regions, 63language comprehension

studies, 155–158magnetoencephalographic

studies, 122–127Wright, M. J., 314Wylie, G., 189

YYamaguchi, S., 318, 322

ZZani, Alberto, 3–11, 13–36, 80,

273–301, 359–366, 373–377,379–400, 421–425

Zeki, S. M., 104

436 INDEX

neurociencia the cognitive electrophysiology of mind and brain - a. zani, a. proverbio (ap, 2002) ww

Documents