read it interesting

IEEE REVIEWS IN BIOMEDICAL ENGINEERING, VOL. 1, 2008 23

Multimodal Functional Neuroimaging: IntegratingFunctional MRI and EEG/MEGBin He, Fellow, IEEE, and Zhongming Liu, Student Member, IEEE

Methodological Review

Abstract—Noninvasive functional neuroimaging, as an impor-tant tool for basic neuroscience research and clinical diagnosis,continues to face the need of improving the spatial and tem-poral resolution. While existing neuroimaging modalities mightapproach their limits in imaging capability mostly due to fun-damental as well as technical reasons, it becomes increasinglyattractive to integrate multiple complementary modalities in anattempt to significantly enhance the spatiotemporal resolutionthat cannot be achieved by any modality individually. Electro-physiological and hemodynamic/metabolic signals reflect distinctbut closely coupled aspects of the underlying neural activity.Combining fMRI and EEG/MEG data allows us to study brainfunction from different perspectives. In this review, we start withan overview of the physiological origins of EEG/MEG and fMRI,as well as their fundamental biophysics and imaging principles, weproceed with a review of the major advances in the understandingand modeling of neurovascular coupling and in the methodologiesfor the fMRI-EEG/MEG simultaneous recording. Finally, we sum-marize important remaining issues and perspectives concerningmultimodal functional neuroimaging, including brain connectivityimaging.

Index Terms—EEG, fMRI, human brain mapping, MEG, mul-timodal neuroimaging, neurovascular coupling.

I. INTRODUCTION

N EUROIMAGING plays a critically important role inneuroscience research and management of neurological

and mental disorders. Modern neuroimaging techniques relyon various “source” signals that change across different spatialand temporal scales in accompany with neuronal activity.For instance, neuronal activity intensifies electrophysiologicalsignals, such as action potentials and post-synaptic potentials,which serve as the primary messengers for communicationamong neurons. It is generally agreed that neural activity iswell-characterized by electrophysiological processes whichtightly correspond with neuronal dynamics. In addition, neuralactivity is also coupled with metabolic and hemodynamicprocesses. As brain function requires sustained blood flow tosupply oxygen to compensate for cerebral metabolic energy

Manuscript received September 30, 2008; revised October 03, 2008. Firstpublished November 05, 2008; current version published December 10, 2008.This work was supported in part by the National Institutes of Health under GrantRO1EB007920 and Grant RO1EB00178, and by the National Science Founda-tion under Grant BES-0411898.

The authors are with the Department of Biomedical Engineering, Universityof Minnesota, Minneapolis, MN 55455 USA (e-mail: [email protected]).

Digital Object Identifier 10.1109/RBME.2008.2008233

consumption, changes in neural activity often induce cascadedchanges in cerebral metabolic rate of oxygen ( ),cerebral blood flow (CBF), oxygen extraction fraction (OEF),cerebral blood volume (CBV), etc. In contrast to electrophys-iological signals, metabolic and hemodynamic responses aremuch slower and reflect the indirect and secondary effects ofneural activity.

All noninvasive neuroimaging modalities are based on bio-physical signals related to either brain electrophysiology orhemodynamics/metabolism. Electroencephalography (EEG)[1] and magnetoencephalography (MEG) [2] are based onelectrophysiological principles. Functional magnetic reso-nance imaging (fMRI) [3]–[5], positron emission tomography(PET) [6], [7], single-photon emission computed tomography(SPECT) and near-infrared spectroscopy (NIRS) are basedon hemodynamic and/or metabolic principles. Strengths andlimitations of these modalities depend largely upon the spa-tiotemporal characteristics of the measured “source” signals inrelation to neuronal activity, as well as many diverse sensingand imaging methods applied to individual modalities.

EEG and MEG measure external electrical potentials andmagnetic fluxes, respectively, which arise collectively frommass neuronal responses within the brain. Such electromagneticsignals propagate (virtually) instantaneously from the activatedneuronal tissues via volume conduction to the recording siteson/above the scalp surface [8]–[11]. The instantaneous natureof EEG/MEG indicates an intrinsically high temporal resolu-tion and precision, which make them well suited for studyingbrain functions on the neuronal time scale. The collectivenature suggests low spatial resolution and specificity, whichimpede mapping brain functions in great regional details [12].This is regardless of recent advancements in electromagneticsource imaging (ESI) [13]–[15], which has led to great stridesin improving the EEG/MEG spatial resolution to a centimeterscale or even smaller.

The strength and limitation of fMRI are almost precisely theobverse of those of EEG and MEG. As neural activity elevates,the concomitant alternation of local oxyhemoglobin versus de-oxyhemoglobin content gives rise to a so-called blood oxygenlevel dependent (BOLD) magnetic resonance (MR) signal [16].Since its advent in the early 1990s [3]–[5], the BOLD-contrastfMRI has rapidly gained a dominant position in neuroscienceresearch [17]. The merits of fMRI include its whole brain cov-erage, relatively uniform sensitivity, high spatial resolution andspecificity. These advantages are primarily attributable to well-established MR imaging techniques (e.g., echo-planar imaging)

1937-3333/$25.00 © 2008 IEEE

24 IEEE REVIEWS IN BIOMEDICAL ENGINEERING, VOL. 1, 2008

Fig. 1. Schematic illustration of the ranges of spatial and temporal resolutionof various noninvasive (in blue) imaging techniques and invasive (in red) exper-imental techniques.

that allow for the frequency and phase encoding of spatially dis-tributed MR signals. However, fMRI is also limited by its poortemporal resolution as well as its indirect nature with respect toneuronal activity. These limitations often pose concerns to anysimple interpretation of the BOLD signal as a surrogate indexof neural activity.

For an illustrative overview, we graph the existing noninva-sive neuroimaging methodologies, in comparison with relevantinvasive recording/imaging techniques, with respect to theirspatial and temporal resolution, as shown in Fig. 1. Typicalinvasive techniques include intracranial electrophysiologicalrecordings of the single-unit activity (SUA), multi-unit activity(MUA) and local field potential (LFP). In addition, intrinsiclight optical imaging, which can be applied to exposed brainsurface, is also capable of imaging the hemodynamic signals[18], [19]. Noticeably, none of existing noninvasive modalitiescan alone achieve high resolution in both spatial and temporaldomains. One technique may have merits in the temporalaspect but limitations in the spatial aspect (such as EEG/MEG),or vice versa (such as fMRI). These complementary featureshave motivated recent developments in integrating multipleneuroimaging modalities, particularly EEG/MEG and fMRI[20]–[24]. As illustrated in Fig. 2, such a multimodal functionalneuroimaging approach is based on the fundamental relation-ships between electromagnetic signals generated by neuronalactivity via volume conduction, and BOLD-contrast fMRIsignals related to the hemodynamic responses. A number ofefforts in recording technology, multimodal data fusion and theneurovascular modeling, have collectively led to a promisingfMRI-EEG/MEG integrated functional neuroimaging approach,which holds the potential to reach millimeter spatial resolutionand millisecond temporal resolution, thereby opening a uniqueand noninvasive window to investigate dynamic brain activityand connectivity.

However, the integration of fMRI and EEG/MEG also facesgreat challenges, which arguably compromise its intuitivepromise and benefits to neuroimaging and neuroscience fields.To list a few most critical ones, what is the physiological

Fig. 2. Illustration of the fMRI-EEG/MEG integrated multimodal neu-roimaging (part of figure adapted from [46, Fig. 1] with permission).

and physical relationship between hemodynamic and electro-physiological responses? How can EEG be reliably recordedduring concurrent fMRI scans without affecting the signalquality for both modalities? How should we fuse the recordedfMRI-EEG/MEG data in a principled way for integrativeimaging with enhance spatiotemporal resolution?

The past decade has witnessed significant progresses inboth fundamental and technical aspects related to these ques-tions. Most of them are still under active research or remainpoorly understood. In this review, we start with an overviewof the physiological origins of EEG/MEG and fMRI, as wellas their fundamental biophysics and imaging principles; weproceed with a review of major progresses in understandingand modeling the neurovascular coupling, methodologies forthe fMRI-EEG/MEG integration and EEG-fMRI simultaneousrecording; finally, important remaining issues and perspectivesare summarized. Although some of the points we make in thisreview might be speculative or even controversial, we hopethey serve to identify important scientific elements that wouldhelp to stimulate future discussion and research.

II. OVERVIEW OF ELECTROMAGNETIC SOURCE IMAGING AND

FMRI

A. Origin of EEG/MEG

As the mass responses from all activated neural tissues,EEG/MEG signals can be regarded as a weighted sum ofinstantaneous neuronal currents throughout the entire brainvolume. In other words, EEG/MEG reflects the synchronizedelectrical behavior of an assembly of neurons within a certainbrain region or even across regions. Asynchronized electricalactivities have little contribution to EEG/MEG, because theirrandom consequences are virtually cancelled out when summedover locations. The large-scale synchrony serves as the majorfactor to identify which specific aspects of brain electrophysi-ology contribute to EEG/MEG.

Brain electrophysiological processes take place in a varietyof spatial and temporal scales. In a microscopic scale down toa single neuron, the electrophysiological activity manifests it-self as the neuronal spiking activity and the post-synaptic po-tential (PSP). The neuronal spiking activity refers to the ac-tion potentials that propogate along the axon. The spiking ac-tivity reaches and accumulates at the synapse through which

HE AND LIU: MULTIMODAL FUNCTIONAL NEUROIMAGING: INTEGRATING FUNCTIONAL MRI AND EEG/MEG 25

one neuron connects to another. When the accumulated spikingactivity exceeds a certain threshold, it controls the amount ofchemical transmitters released from the synapse. The releasedneurotransmitters further control the gating of the ion channelsat the post-synaptic neuron and hence modulates the PSP. Forthe spiking activity above the threshold, the synapse effectivelybehaves as a low-pass filter, owing to its accumulative effect andthe delay due to the electrochemical conversion. Under such cir-cumstances, the pre-synaptic spiking activity may be temporallycorrelated with the PSP. However, for the subthreshold spikingactivity, such correlation does not necessarily hold true.

EEG/MEG measurements in a macroscopic level are insensi-tive to action potential sources with short-duration bidirectionalcurrent flows. As action potentials propagate along an axon, theassociated electrical currents flow in opposite directions (i.e.,along and opposite to the direction of action potential propaga-tion) in both intracellular and extracellular spaces. Signals gen-erated by such action potential sources are only recordable froma sensor at a close vicinity to the neuron. At a relatively remotesensor (such as on the scalp surface), the electromagnetic sig-nals generated by opposing currents are essentially cancelledout. Moreover, the brief duration of the action potential also re-quires a very strong degree of synchrony with great temporalprecision in order to be observable from the scalp surface. How-ever, high-frequency ( ) components associated withneuronal spiking activity are significantly underrepresented inEEG/MEG.

In fact, the primary sources of EEG/MEG are the synapticcurrents flowing through the apical dendrites of large pyramidalneurons within the cortical gray matter [8], [13], [25], [26]. Theapical dendric trees extend toward the pial surface and are or-ganized into columns – a unique structure facilitating the for-mation of regionally synchronized synaptic currents [12]. It hasbeen approximated that the simultaneous activation of as few as0.1% synapses within a cortical area of around 40 wouldsuffice to produce a detectable EEG/MEG signal [9]. Moreover,the view that the primary EEG/MEG sources are synaptic cur-rents is also supported by a general similarity between the fre-quency spectra of scalp potentials and post-synaptic potentials[27]. This is regardless of the fact that extracranial potentialsmeasured with EEG have much smaller amplitudes and lowerfrequency than intracranial potentials, such as local field poten-tials (LFP) and electrocorticograms (ECoG), owing to the dif-ferent spatial scale to which each recording technique is applied.In general, low-frequency signals tend to be synchronized ina larger spatial scale, whereas synchronous high-frequency ac-tivity is often confined to a smaller scale. The same relationshipalso holds true for MEG.

B. Origin of Bold Signals

Most fMRI applications utilize a BOLD contrast, whosemechanism is attributable to some basic biophysical propertiesof hemoglobin and its susceptibility effects in nuclear magneticresonance (NMR). Deoxyhemoglobin is paramagnetic andits concentration inversely depends on the blood oxygenationlevel [28]. The paramagnetic property of the blood causes thebulk susceptibility difference between a blood vessel and its

surrounding neural tissue. This effect, in turn, gives rise tothe inhomogeneity of the local magnetic field and introducesadditional resonance frequency shifts and phase dispersion forextra-vessel molecules. The BOLD contrast is pronounced ingradient echo (i.e., -weighted) MR images where higheroxygenation level leads to larger MR signals [16]. In addition,the echo-planar imaging (EPI) technique further providesremarkable temporal resolution for recording dynamic BOLDsignals in realtime [29].

In the first three pioneer studies, the BOLD-contrast fMRI hasbeen independently demonstrated in noninvasively mapping invivo neurophysiologic changes due to either visual stimulation[3], [4] or motor tasks [5]. In all these three experiments as wellas numerous fMRI studies that followed, the brain regions en-gaged in performing a task or processing a stimulus demonstrateincreased BOLD signals relative to the resting state, namelythe positive BOLD response (PBR). The PBR is widely be-lieved to result from the over-compensation of oxygen usageby the cerebrovascular system, which causes a regional hemo-dynamic influx in excess of the oxygen consumption of the ac-tivated neuronal tissue [6], [7]. In fact, the BOLD signal reflectsthe combined effect of CBF, CBV and [30]. Severalmodels have been proposed to describe the BOLD signal as afunction of the neural activity-induced changes in CBF, CBVand [30]–[35]. Qualitatively, the CBF contributes pos-itively to the BOLD signal while the CBV and havenegative contributions; yet such relationships are not necessarilylinear.

Another fundamental question concerning the interpretationof BOLD-fMRI data lies in the neurophysiological origin ofBOLD signals. There have been debates regarding whetherthe BOLD signal arises from neuronal inputs or outputs (i.e.,synaptic or spiking activities, respectively). Early studies [36],[37] suggest the dependence of the BOLD responses uponneuronal spiking activity. This conclusion was drawn from theobservations that the peak magnitudes of the BOLD signalsmeasured in human V5 [36] and V1 [37] were linearly cor-related with the averaged spiking rates invasively recorded inanalogous areas in primates. More recently, Mukamel et al.also found a close correlation between the BOLD signals mea-sured from healthy subjects and the neuronal spiking activityrecorded from epilepsy patients when both groups underwentidentical stimuli [38]. However, questions may be raised, sincethese studies compared the BOLD and neuronal spiking signalsacquired from different species or subjects.

In contrast, more studies have demonstrated that BOLD-fMRI reflects synaptic activity rather than spiking activity[39]–[42]. In an influential and technically demanding ex-periment, Logothetis and his colleagues recorded the spikingactivity (i.e., single-unit activity (SUA) and multi-unit activity(MUA)) and the synaptic activity (i.e., local field potential(LFP)) simultaneously with the BOLD-fMRI signals frommonkeys [39]. Through linear transformation systems, the LFPwas found to yield better estimates of BOLD responses thanthe MUA. Hence, they conclude that BOLD signals primarilyreflect synaptic activities [39], [43], [44]. Following this study,similar findings were also reported in several independentstudies [40]–[42].


Between two competing views, we and many others supportthe synaptic origin of BOLD signals [17]. Hemodynamic re-sponses are driven by metabolic energy demands, nearly all ofwhich are imposed by synaptic activity instead of action po-tential firing [45], [46]. The observed correlation between theBOLD signal and the neuronal spiking rate [36], [37] might beowing to the fact that the pre-synaptic neuronal spiking activityis sometimes correlated with the synaptic current, as previouslydiscussed in this review and elsewhere [46], [47]. However,when LFP and MUA are disassociated, the BOLD response isprimarily correlated with LFP [41], [42].

C. Electromagnetic Source Imaging

Electromagnetic source imaging (ESI) is a model-based neu-roimaging technique. It entails: 1) modeling the brain electricalactivity; 2) modeling the head volume conduction process soas to link the modeled electrical activity to EEG/MEG; and3) reconstructing the brain electrical activity from recordedEEG/MEG data. The first two modeling steps serve to solvethe EEG/MEG forward problem; the third step is the inverse ofthe second step, thereby commonly known as the EEG/MEGinverse problem.

A model of brain electrical activity (in short “source model”)is composed of bioelectric units distributed within the brainvolume or over the brain surface, or confined to few locationsin the brain. A single source unit is often defined as a cur-rent dipole, which well-approximates the synchronized synapticcurrents at a columnar level [9], [12]. Any brain electrical ac-tivity can be modeled by a distribution of current dipoles withinthe entire brain volume. One may further choose to confinethe dipole locations to the cortical gray matter and constrainthe dipole orientations to be perpendicular to the cortical sur-face [26]. Alternatively but less frequently, a bioelectric sourceunit can also be defined as a current monopole [48], currentmultipole [49], or extracellular potential [50]–[58]. Moreover,when brain electrical activity is confined to a few focal regions,one may prefer to use a source model consisting of only a fewdiscrete dipoles, namely the equivalent current dipole (ECD)model [59]–[62]. As opposed to the general applicability of thedistributed source models, the ECD model assumes small ex-tents of the neural responses, which may or may not be validdepending on the particular circumstances. Even if the assump-tions are valid, the appropriate number of dipoles is often dif-ficult to be determined a priori or solely from the EEG/MEGdata [63].

A linkage from the source model to the electromagneticsignals at the sensor locations on the scalp can be establishedby constructing a volume conductor model that resembles thehuman head in both geometry and conductivity. Historically,the head volume conductor is modeled as concentric three-shell[57], [64], [65] or four-shell [66] spherical models. Moreaccurate forward solutions become possible by using numericalalgorithms, such as the boundary element method (BEM) [61],[67], [68], finite-element method (FEM) [69], [70], [211], andfinite difference method (FDM) [71]. Such numeric modelsallow us to incorporate the realistic geometries of the head com-partments segmented from anatomical images such as MRI.

The conductivities of different head tissues, especially for theskull in EEG, may be specified according to empirical values[64], in vitro measurements [72], in vivo estimates [73], [74],or potentially the results of electrical impedance tomography(EIT) [75] or MR electrical impedance tomography (MREIT)[76]. In addition, diffusion tensor magnetic resonance imaging(DT-MRI) [77] provides a new means to obtain the anisotropicconductivity of the cerebral white matter [78]–[80].

Given a source model and a head volume conductor model,the EEG/MEG can be linked with the brain electric sources by

(1)

where is the vector of EEG or MEG signals, is thesource vector, is the transfer matrix, and is the noisevector, respectively.

As opposed to the well-posed forward modeling and com-putation, the EEG/MEG inverse problem is more challenging.Such a problem is to estimate the source signals from EEGor MEG measurements . As the number of EEG/MEG sen-sors is in general smaller than the number of sources withinthe brain, the EEG/MEG inverse solution is non-unique if noconstraints are given. Over the past two decades, a number ofefforts have been made to tackle this challenge. Evidence hasincreasingly indicated that reasonable EEG/MEG inverse solu-tions can be obtained if appropriate constraints, as derived frombrain anatomy and physiology, are placed on the source distri-bution and volume conductor. For instance, one may specify theoptimal solution as the most energy efficient one among thosefitting equally well with the data. In a noise-free condition, thisconstraint leads to the linear least-squares source estimate. Innoisy conditions, this constraint can be incorporated as a min-imum norm side constraint, giving rise to the minimum normestimate (MNE) [81]. Other variations of the MNE include thelead-field normalized weighted minimum norm (WMN) [82],low-resolution brain electromagnetic tomography (LORETA)[83], and their extensions to statistical mapping [21], [84], etc.The common feature shared by these algorithms is the linearityof the inverse solution, meaning that the inverse solution canbe obtained through transforming the measurements through alinear system (or inverse matrix). The linear inverse solutionsreturn low-resolution images of current density or its statistics.Iterative algorithms have been developed to enhance the focalsources [85]. These iterative methods have been suggested tobe equivalent to a nonlinear inverse solver that minimizes acost function formulated with L-p norms for both the data fit-ting term and the side constraint [86]. This is in line with othernonlinear inverse algorithms that specifically utilize the solutionconstraint other than the L-2 norm, such as the minimum cur-rent estimation (MCE) [87], L1 norm [88]–[90] and L-p norm[91].

In parallel to current density imaging, dipole source localiza-tion can be approached using nonlinear optimization [59]–[62],[92], [93] or sub-space scanning procedures [94]–[99]. More-over, the beamforming techniques based on linear spatial fil-tering can be used to estimate the source activity at a region ofinterest (ROI) or every individual location in the source space


Fig. 3. Schematical illustration of BOLD-contrast fMRI. The regional neuronal activity alters the local CBF, CBV and �� , which collectively leads tochanges in the blood oxygen level. The increase of oxygen level, meaning the decrease of local field inhomogeniety, produces a longer �� or �� and thereforelarger MR signals. The frequency-and-phase encoding technique (e.g., echo-planar imaging) allows for the fast acquisition of a so-called �-space data, which canbe transformed to the original image space through Fourier transformation. The signal is most frequently analyzed voxel by voxel, yielding statistic maps indicatingregions with significant hemodynamic effects related to external stimuli/tasks or internal events. These regions arguably define the activated neuronal populations.

[100]–[103]. The beamformer for a specific ROI or source loca-tion is derived in an attempt to minimize the interference fromother locations.

The imaging performance of all linear and nonlinearEEG/MEG inverse algorithms varies from case to case, de-pending on the nature of the brain electrical activity. Themethodological choice is up to the users’ own judgment asto whether the activity is sparse or extended, stationary ornonstationary, etc. Although the evaluation of some typical al-gorithms have been done in a few experimental and simulationsettings, a general conclusion or agreement regarding a singleoptimal source imaging approach, if possible at all, has notbeen reached.

D. fMRI Statistic Parametric Mapping

The vast majority of fMRI applications use fMRI for func-tional localization of the brain regions engaged in specific sen-sory processing or cognitive functions [107]. FMRI as a brainmapping tool depends on experimental designs, data analysismethods and an explicitly or implicitly assumed relationship be-tween BOLD signals and neural responses. Fig. 3 summarizesthe principle of fMRI.

FMRI is a comparative technique in nature. An fMRI experi-ment must include at least two conditions in contrast. A task (orstimulus) condition places specific demands to the brain, while auser-defined control condition may involve a “baseline” task ora resting state. The signal difference between the task and con-trol conditions is evaluated, typically voxel by voxel, to identifythe regions engaged in the task execution. Popular fMRI exper-imental designs define the alternation of conditions in either ablock-design or event-related manner. In line with earlier PETparadigms, the block-design involves prolonged task and con-trol periods, so as to observe sustained BOLD signal changeswith a high contrast-to-noise ratio (CNR). The event-relateddesign focuses on the averaged single-trial BOLD response in

ways analogous to event-related potentials/fields (ERP/ERF) inEEG/MEG [108], [109].

For both types of study designs, the BOLD signal at an “acti-vated” voxel is expected to change in a way related to the tran-sitions from one condition to another. This rationale allows usto make inference about regionally specific effects in responseto the task/stimulus. For experiments with only two conditions,voxel-wise statistical inference may be simply based on a Stu-dent’s t-test or period cross correlation [5]. A more general andflexible approach is based on the general linear model (GLM)[110], [111]. Stimulus functions encoding the occurrence of aparticular event or experimental state (e.g., boxcar-functions)are convolved with a hemodynamic response function (HRF) toform regressors in the GLM. Fitting the GLM to the data al-lows for the estimation of model parameters and the statisticinference against a null hypothesis (i.e., the voxel is not acti-vated). Such inference is classic in terms of statistics, as op-posed to more recent methods based on the Bayesian inferencewhich provides the posterior probability that the voxel is acti-vated given the data [112]. All of these methods discussed sofar are model-driven in a sense that they require specific as-sumptions about the time courses of the processes contributingto the measured signals and/or a priori statistical distributionsof the signal and the noise. To remove such model dependence,data-driven methods have been implemented using independentcomponent analysis (ICA) [113], [114].

III. NEUROVASCULAR COUPLING

Statistical inference based on fMRI data alone can only drawconclusions about regional hemodynamic effects. However, it isthe neuronal effect that is of ultimate interest to neuroimaging.Interpretations of BOLD fMRI data in terms of neural responsesare crucially dependent upon the neurovascular coupling, whichis commonly referred to as the relationship between electro-physiological responses and BOLD signals. This relationshipshould also serve as the most fundamental ground for all fMRI-


EEG/MEG integrated neuroimaging methods (to be discussedlater).

In order to infer neural activity from fMRI, the neurovascularrelationship is most frequently modeled as a linear convolutionsystem. It means that the BOLD signal surrounding the acti-vated neural tissue results from the local neural activity convo-luted with a time-invariant HRF, which describes the hemody-namic consequence of an impulse neural response. The linearHRF serves as an approximation for the complex interactionsbetween neuronal activity, metabolic demand and blood flowand oxygenation [47]. With approximations made at variousdegrees, the HRF may be simply a Gauss, gamma [115] ordouble-gamma function [116]. While all these functions cap-ture the key features of hemodynamics (e.g., the sluggishnessand long duration), the double-gamma HRF entails the most de-tailed features, including a delayed onset time, initial dip, over-shoot, undershoot, etc.

A linear HRF largely simplifies the analysis and interpreta-tion of BOLD fMRI data. It serves as the central assumption ofthe classic GLM analysis [110], [111] and the rapid event-re-lated fMRI design [109]. Note that, in such fMRI analyses anddesigns, the linear system is specified to the relationship be-tween BOLD responses and external stimuli (or tasks), whileimplicitly assuming linear, yet unknown, neural responses to thestimuli. However, a nonlinear stimulus-to-BOLD relationshiphas been found in a number of studies [117]–[123]. The ob-served nonlinearity may originate from neural and/or vascularsources (e.g., neuronal and vascular refractory effects), sincethe BOLD response is the ultimate consequence of cascadedprocesses involving the neural response, the neurovascular cou-pling and the vascular response [119], [123].

To address the linearity or nonlinearity of the neurovascularcoupling, investigators need to monitor both neural activityand hemodynamic response through invasive [39], [121],[122], [124] or noninvasive measurements [125]–[127]. Themeasured electrophysiological (e.g., MUA, LFP, EEG andMEG) and hemodynamic (e.g., BOLD and CBF) signals arequantified individually before being compared against a linearfunction. In this regard, a critical concern lies in the varietyof methods for quantifying the multimodal signals. Hemody-namic signals have been quantified as its peak height [124],steady-state height [128] or integral over time [129]. Waysfor quantifying electrophysiological signals are even morediverse. No consensus has been reached so far with regardto an appropriate pair of quantitative measures for assessingthe cross-modal relationship. It is likely that some reportednonlinear neurovascular coupling may be simply due to the useof mismatched quantitative measures. Additional challengesmay be further appreciated by considering the highly differenttemporal scales of the hemodynamic and electrophysiologicalresponses. Table I lists typical values related to the temporaland frequency features of the single-trial ERP/ERF and BOLDsignals. A quantitative cross-modal relationship is even moreobscured when the electrophysiological signal is analyzed in anevent-related manner while the hemodynamic signal changesacross prolonged periods as in block-design experiments.

For the purpose of integrating ERP/ERF and fMRI or investi-gating the relationship between them, it is particularly important

TABLE IFOR EVENT-RELATED POTENTIAL OR FIELD AND EVENT-RELATED FMRI, THE

RESPONSE DELAY, DURATION, SIGNAL SAMPLING RATE AND FREQUENCY

to develop a theoretically-driven approach to interpret the fMRIsignals as certain meaningful indices in terms of ERP/ERF. Inthis regard, we have developed a mathematical model [24] todescribe the interactions between stimuli (or tasks), neuronalsynaptic currents and BOLD responses. This model allows us toderive the relationship between event-related synaptic responsesand BOLD fMRI signals.

This model is composed of two cascaded linear systems. Thefirst system is characterized by the neuronal impulse responsefunction (NRF). The NRF varies across locations and repre-sents the power of regional synaptic currents , with avery short duration up to several hundred milliseconds. Thetask-induced or stimulus-evoked electrophysiological response,in terms of the biophysical energy, is modeled as the convolu-tion of the tasks or stimuli with the NRF. As the second system,the neurovascular coupling is simply modeled as a location-independent and time-invariant linear system, represented bythe hemodynamic impulse response function (HRF), denoted as

. Therefore, the BOLD fMRI signal , in response toa block of sustained stimuli, can be written as (2).

(2)

where is a delta function, is the inter-stimulus-interval(ISI), is the noise, and stands for convolution. Equa-tion (2) is valid when the ISI is longer than the neuronal refrac-tory period and the linear neurovascular coupling holds true.

Since the HRF evolves much slower and lasts much longerthan the NRF, (2) can be re-written as (3).

(3)

where . As graphicallyillustrated in Fig. 4, (3) suggests that the BOLD signal at anactivated voxel can be modeled as the product of a predictorfunction and a parameter proportional to the time integral of thepower of the local event-related synaptic currents plus the noise.This model resembles the GLM used in the conventional fMRIanalysis, except that the predictor function is defined as the con-volution of the HRF with a train of delta functions (instead ofa box-car function) which represent the occurence of discretestimuli. The model parameter (or regression coefficient) can beestimated by fitting the model to voxel-specific BOLD fMRItime series. The estimated parameter, known as the BOLD ef-fect size, represents the relative strength of the regional BOLDresponse, since it reflects the ratio between the measured andpredicted BOLD time courses (i.e., ). More im-portantly, the model shown in Fig. 4 gives the BOLD effect


Fig. 4. The BOLD fMRI signal at an “activated” voxel can be modeled using a linear system. The signal is the scaled version of a location-independent predictorsignal, derived from the stimulus function and the hemodynamic response function (HRF). The scaling factor (called the BOLD effect size) is proportional to thetime integral of the event-related synaptic power.

size a clear biophysical interpretation meaningful to EEG/MEG.That is, the BOLD effect size provides an fMRI-based estimatefor the time integral of the power of the regional EEG/MEGsource activity during an effective duration of the ERP/ERF.As discussed later in this review, such a biophysical implica-tion serves as the theoretical basis for important developmentsin fMRI-EEG/MEG integrated neuroimaging [24].

It is worth mentioning that the above theoretical model as-sumes the baseline activity remains unchanged from the resting-state or control condition to the stimulus condition. However, itis possible that the baseline activity changes by itself or in re-sponse to certain stimuli. This depends on the brain location ofinterest as well as the stimulus type and property. For instance,there are a number of default-mode regions that are functionallyactive during the resting state but are likely suppressed givencertain stimuli or tasks that request attentional resources [130].It is also possible that the baseline activity at some brain re-gions spontaneously modulates over time, with or without anyrelationship to the external stimulus. As a result, if the stimulusis given when the baseline activity modulation reaches a lower(or higher) level than during the resting-state control state, thecorresponding BOLD signal change reflects not only the stim-ulus evoked neuronal response but also the modulation of thespontaneous baseline activity.

IV. FMRI-EEG/MEG INTEGRATED NEUROIMAGING

Methods for the fMRI-EEG/MEG integrated neuroimagingare categorized into two types: the fMRI-constrained electro-magnetic source imaging or localization and the EEG/MEG-in-formed fMRI analysis. As the names suggest, the former restson the EEG/MEG source imaging while incorporating spatialinformation from fMRI; the latter utilizes the time and/or fre-quency specific electrophysiological information to derive re-gressors in the fMRI analysis.

A. fMRI-Constrained Electromagnetic SourceImaging/Localization

The earliest efforts in the fMRI-constrained EEG/MEGsource imaging utilize fMRI statistical parametric maps toobtain a priori information on where EEG/MEG sourcesare likely located. Depending on different source models,

the spatial information from fMRI is typically used to con-strain the locations of multiple current dipoles, namely thefMRI-constrained dipole fitting [131]–[133], or to constrain thedistributed source distribution, namely the fMRI-constrainedcurrent density imaging [20]–[22], [24], [134]–[139].

When neural activity is confined to a few regions with smallextents, the fMRI analysis should yield several fMRI foci (orhotspots). It is reasonable to model the electrical activity ateach fMRI hotspot as an equivalent regional current dipole. Thedipole locations are fixed to the fMRI foci, or initially seededto the fMRI foci, yet are adjustable to best fit the EEG/MEGdata. Given the dipole locations are assigned or estimated, thedipole moments can be uniquely estimated by fitting the ECDmodel to the EEG/MEG data. From the estimated dipole timecourse, one can tell the temporal dynamics of the regional neuralactivity. Note that the primary goal of the fMRI-seeded dipolefitting technique is not to image brain activity; rather, it aimsat retrieving the time course of the brain activity at identifiedfMRI activation foci. However, this technique is questionablewhen applied to imaging extended neural responses.

In contrast, the fMRI-constrained distributed source imagingis generally applicable regardless of whether the source ac-tivity is focal or extended. Perhaps, the most straightforwardapproach is the fMRI-weighted current density imaging, whichuses the fMRI activation map to derive weighting factors forthe inverse solutions compatible with the EEG/MEG signals.This approach can be implemented in a Wiener filter [20],[21], [26] and weighted minimum norm frameworks [22],[134]–[136]. Although seemingly different, these implementa-tions are fundamentally equivalent [23]. Both of them are basedupon the identical core assumption that locations inside thefMRI activations are more favored than those outside in termsof the likelihood of being active current sources. The degree ofpreference to fMRI hightlighted areas is controlled by an fMRIweighting factor.

These methods have limitations in both technical and funda-mental aspects. Technically, the fMRI weighting factor is mostfrequently up to users’ subjective choice, although several em-pirical values have been suggested to be 10 [20], 3 [135] or 1.4[134] based on results obtained in several simulation studies.This technical limitation may be solvable by using data-drivenmethods for choosing the fMRI weighting factor, such as the


expectation maximization (EM) algorithm [137], [138], [140].Moreover, we may bypass this uncertainty by using an alterna-tive two-step estimation algorithm which avoids using the fMRIweighting factor [139]. In the first step, the EEG/MEG sourcespace is strictly confined to the regions highlighted in fMRI,giving rise to an inverse solution firmly constrained by the fMRI.In the second step, the fMRI constraint is removed, and the solu-tion obtained in the first step is re-entered as the initial solutionto fit the EEG/MEG data again employing a so-called Twomeyregularization [141].

These technical limitations are primarily due to the funda-mental mismatches between fMRI and EEG/MEG, owingto highly different temporal scales in which fMRI andERP/ERF data are generated and collected (see Table I). ThefMRI-EEG/MEG mismatches may be further categorized intothree types, namely fMRI extra sources, fMRI invisible sourcesand the fMRI displacement [20], [134], [139]. The fMRI extrasources represent the source regions that are deemed as activein fMRI but do not contain the sources for the EEG/MEG at acertain time instant. During a short period of interest followingthe event onset, the fMRI activations are thought of as “static”(or time-invariant) while the EEG/MEG signals are variableand the source imaging is carried out instant by instant. Theelectrical source activity in a millisecond time scale may onlyinvolve a subset of the activated fMRI areas, whereas otherareas may appear as false positives if all of the “activated” fMRIvoxels are included in the prior spatial constraint [24], [139].The fMRI invisible sources are the real EEG/MEG sources butnot deemed as active by fMRI. A transient current source maygenerate observable EEG/MEG signals whereas it may last toobriefly to induce a sustained BOLD response. In this condition,the fMRI-derived time-invariant spatial constraint includesfalse negatives, which often result in the underestimation offMRI invisible sources as reported in several independentstudies [20], [24], [139]. The fMRI displacement refers to thespatial difference between the vascular and electrophysiolog-ical sources [134], [139]. Because of these fMRI-EEG/MEGmismatches, it is problematic to constrain the temporally vari-able current source estimates to “time-invariant” fMRI spatialpriors, which may entail both fMRI false positives and falsenegatives.

To tackle these technical and fundamental issues, we haveproposed a new framework for the fMRI-EEG/MEG integratedneuroimaging. As illustrated in Fig. 5, this approach rests ona unified system underlying the signal generation and analysisfor both modalities. The system assumes a common neuronalsource (i.e., synaptic activity), from which fMRI and EEG/MEGsignals are generated via a temporal low-pass filter and a spatiallow-pass filter, respectively. The EEG/MEG inverse problemsessentially deal with the spatial deconvolution – the process ofreversing the head volume conduction. The inverse solutions re-tain the temporal source evolution even though it may fail to re-construct the spatial source distribution. In other words, at everysource location, the souce waveform estimated from EEG/MEGis much less distorted (in terms of its normalized “shape”) thanits absolute magnitude, since the filtering applies to the spatialdomain instead of the time domain. This feature is as opposed tothe temporal regression of fMRI data (e.g., the GLM analysis),

Fig. 5. Flow-chart of the fMRI-EEG/MEG co-registration.

which theoretically ends up with high-resolution spatial mapsof brain activations but with little or no temporal information.

Such complementary multimodal results can be simplymerged by utilizing the cross-modal relationship, derived anddiscussed in Section III. That is, the BOLD effect size estimatedfrom the fMRI signal in each voxel is proportional to the timeintegral of the local source power underlying the ERP/ERFsignals.

(4)

where is the source power at location and time ,is the quantified BOLD effect size at location . is the dura-tion of the electrophysiological response induced by a transientstimulus or task, which is referred to as the “event-related pe-riod.”

This relationship allows us to constrain, voxel by voxel, theintegral of the power of the EEG/MEG source estimates in ac-cordance with the BOLD effect size. Specifically, we can scalethe source time course estimated from the ERP/ERF, so that thetime integral of the power of the scaled source estimates equalsthe BOLD effect size [24] as illustrated in Fig. 5. Namely, thisprocess is referred to as the fMRI-EEG/MEG co-registration.


Furthermore, we can re-fit the source estimates to theEEG/MEG data while taking the fMRI-EEG/MEG co-reg-istered source power estimates as time-variant prior spatialconstraints. This additional estimation step can be imple-mented by means of an adaptive Wiener filter (AWF) [24]. Itensures that the spatiotemporal source reconstruction dependsprimarily upon EEG/MEG measurements, and thus providesrobustness against inaccurate fMRI constraints. Relative tothe fMRI-EEG/MEG co-registration, the AWF represents anintegrative multimodal imaging superior to simply puttingtogether the results separately obtained from fMRI and EEG.

The performance of the AWF was assessed by computersimulations, wherein encouraging results were obtained inimaging multiple focal sources from simulated fMRI and EEGdata [24]. Fig. 6 shows the data obtained from a pilot experi-ment exploring the cortical pathway specialized in processingunilateral visual stimuli [24]. The experiment included two sep-arate sessions with the identical visual stimuli for the EEG andfMRI data collection. The visual stimulation was a rectangularcheckerboard within the lower left quadrant of the visual field;the checkerboard pattern was reversed at 2 Hz. With signifi-cantly enhanced spatial resolution, the AWF algorithm revealeda pathway sequentially activating V1/V2, V3/V3a, V5/V7, andintraparietal sulcus, in general agreement with the hierarchicalorganization of the visual system [142]. This pathway was alsoobserved in the low-resolution images reconstructed from theVEP alone. In contrast, the fMRI-weighted source imagingalgorithm [20], [21] showed a false positive source region inand around V1/V2 at the latency of 212 ms, whereas a morelikely high-tier EEG source around V5, as observed from theEEG data, was missed. This experimental result indicates thepromises in dynamic neuroimaging by integrating fMRI withEEG using the model-based adaptive Wiener filter.

B. EEG-Informed fMRI Analysis

The EEG/MEG-informed fMRI analysis rests on the fMRIanalysis frameworks (e.g., the GLM analysis) while incorpo-rating temporal- or frequency-specific information availablein EEG/MEG. Such a multimodal strategy differs from theconventional fMRI analysis in its unique ability to selec-tively localize the fMRI correlates to specific neuronal eventsor rhythms. This ability directly benefits from the EEG/MEGmeasurements which reflect the mass neural responses, whereasthe analysis based on fMRI alone relies on the timing of thestimuli or tasks. Note that the approaches in this category usu-ally prefer EEG to MEG due to the highly desired simultaneousrecordings of fMRI and electrophysiological signals.

Techniques employing this strategy are particularly useful instimulus- or task-free experimental conditions. Perhaps the mosttypical example is the interictal spike-related fMRI mapping, inwhich the interictal epilepiform events manifest themselves asspike-like discharges in EEG [143], [144]. Some other exam-ples have been witnessed in studies exploring the neural sub-strates underlying the rhythmic modulations in the resting orpathological brain [145]–[149]. In these studies, predictors forthe fMRI regression analysis are derived from frequency-spe-cific EEG modulations. By assessing the correlation between

Fig. 6. (A) The pattern-reversal checkerboard visual stimulation, (B) fMRI ac-tivation map with a corrected threshold � � ��, (C) the global field powerof VEP and the dynamic cortical source distribution at three VEP latencies(76, 112, 212 ms after the visual onset) imaged from EEG alone (first row), orfMRI-EEG integration using our proposed adaptive wiener filter (second row)and the conventional 90% fMRI weighted algorithm (third row). Both the sourceimages and the fMRI activation map are visualized on an inflated representationof cortical surface. (From [24] with permission).

the fMRI signals and the EEG-defined predictors, one may lo-calize the neural regions responsible for the generation of therhythmic modulations of interest.

A more sophisticated approach is based on parametric taskmanipulations and single-trial EEG-fMRI covariation [150],[151]. A range of parametrically graded experimental condi-tions are employed to identify cortical regions for which theBOLD response shows the same modulation across conditionsas a specific single-trial ERP component. More specifically, ex-ternal stimuli are designed to induce the variation of single-trialEEG responses. This single-trial variability is specific to eachtime point within a trial. For a specific time point, the signalvarying over trials can serve as the predictor in the fMRI re-gression analysis, yielding a map of fMRI correlates. Repeatingthe analysis at multiple or potentially all latencies, one canobtain a series of fMRI maps, each of which is associated witha specific instant within the time scale of a single trial. Thisapproach, although interesting, is theoretically demanding.The single-trial variability can result from spontaneous activityand noise that may or may not be differentiable from thatdriven by the external stimuli. Moreover, it requires a carefuldesign of stimulus properties to allow for an effective controlof single-trial amplitudes.

Compared to the fMRI-constrained EEG/MEG sourceimaging, the EEG-informed fMRI analysis appears to bewholly different. In fact, both types of methods share an impor-tant commonality in their fundamental assumption—that is, theneuronal electrophysiological response is linearly correlatedwith the BOLD fMRI signal. To date, it is difficult to judge


which method is superior to the other, although the EEG-in-formed fMRI analysis is formulated in a better posed mannerthan the fMRI-constrained source imaging.

V. SIMULTANEOUS EEG-FMRI MEASUREMENT

In the context of multimodal neuroimaging, one uniquefeature of EEG, as opposed to MEG, lies in the feasibility ofrecording fMRI and EEG simultaneously. This feature is oftendesirable in studies that integrate both modalities or investi-gate the cross-modal relationship. For instance, simultaneousfMRI-EEG recordings are indispensable for using fMRI tolocalize the neural substrates underlying interictal EEG spikes.For this purpose, the fMRI response is not manipulated byuser-defined external stimuli or tasks; but rather, it is drivenby spontaneous biological events. The onsets of these eventscan be determined by detecting the interitcal spikes from theEEG recorded during concurrent fMRI scans, serving as thetemporal markers necessary for carrying out the event-relatedfMRI analysis [143], [144]. For similar reasons, simultanouesfMRI-EEG recordings are desirable for mapping the fMRIcorrelates to continuous rhythmic EEG modulations during theresting state [145]–[147] or sleep [148], [149].

However, strong cross-modal interferences pose greatchallenges to simultaneous fMRI-EEG recordings. Bothtime-varying and static magnetic fields in the MRI/fMRI en-vironment can introduce large artifacts to EEG recordings.During the fMRI scans, the radio-frequency (RF) excitationpulses and rapidly switching magnetic gradients result in largedB/dt, which in turn induce gradient artifacts (GA) about 1000times larger than the normal EEG magnitude [152], [153].The pulsatile motion of EEG leads associated with heart beatscauses cardiac ballistic artifacts (CBA) [154]. In addition,subjects’ safety is worth special attention, since the tissuessurrounding the electrodes may be heated or damaged by largeartificial currents [155]. The quality of MRI/fMRI images mayalso be affected by the EEG electrodes as well as other devicesplaced inside the scanner [156].

Active research has been conducted to resolve these practicalissues (for reviews, see [157], [158]). Special efforts havebeen put forth upon devising MR-compatible EEG caps andamplifiers [152], [153], synchronizing the EEG sampling withfMRI pulses [159], EEG lead design and placement [155],[160], [161], cable wiring [152], signal transmitting [153],[162], subject comfort and safety [155], [163], etc. Signalprocessing algorithms have been developed to remove the GA[153], [164]–[168] and CBA [152], [154], [168]–[171] fromthe EEG simultaneously recorded with fMRI. Most of thesemethods are based on template matching/subtraction algorithms[153], [154], [164] or the blind source separation with principalcomponent analysis [165]–[168] or independent componentanalysis [168], [170], [171]. Interleaved strategies are occa-sionally adopted to acquire fMRI data during the delayed timewindows of little or no interest to EEG recordings [172], [173].

Fig. 7 illustrates a typical experimental setting for si-multaneous fMRI-EEG recordings. Carbon electrodes andcurrent-limiting resistors are often used to minimize the MRsusceptibility effect and the safety concerns [155]. Short copper

Fig. 7. Typical experimental setting for simultaneous fMRI-EEG recordings.

wires connecting electrodes should be twisted, bundled and sta-bilized using taps and sandbags. Local amplifiers are preferredfor signal amplification and digitization. Optical fibers are usedfor transmitting signals to the recording devices outside theMR environment. An ECG lead is placed at the subject’s backto record a temporal marker for every heart beat. The onsets offMRI volume acquisitions are recorded by connecting the TTLoutput from the scanner to the EEG recording device. Both theECG and the scanner TTL signals are useful for the artifactremoval during the post-processing. It may also be desirable tosynchronize the EEG sampling with the clock driving the fMRIpulse sequence to avoid any phase-asynchrony of the GA.

Artifact reduction algorithms are often designed according tothe fact that the GA and CBA remain relatively stable and inde-pendent of the EEG signals. The most widely used (and perhapsstill the most effective) algorithms for removing the GA andCBA are based on average template subtraction [153], [154],[164]. The central strategy can be briefly described as follows. Asignal-free artifact template is obtained by averaging the raw (orpreprocessed) data phase-locked to the fMRI volume onsets orthe ECG-derived heart-beat markers. Assuming the signal andthe artifacts are additive and mutually independent, the EEGsignal should be retained by subtracting the averaged artifacttemplate from the original data. Although these procedures endup with a significant artifact reduction, some artificial residualsoften remain present owing to possibly non-phased-locked ordrifted GA or instable CBA. These residual artifacts can be fur-ther removed by use of independent component analysis (ICA),which allows us to identify and remove the noisy and artificialcomponents, whose spatial, temporal and frequency patterns arein accordance with those of the artifact templates [127].

As the above techniques appear promising, simultaneousfMRI-EEG recordings have proven to provide reasonable dataquality for event-related potentials, frequency-specific oscilla-tory activities and rhythmic modulations. To attest this point,three control experiments have been conducted with the datashown in Fig. 8. In an experiment using 2-Hz full-screen visual


Fig. 8. Examples of simultaneous fMRI-EEG data in three control experi-ments. (A) VEP waveforms obtained inside the fMRI scanner with (blue) orwithout (dashed red) fMRI scans, or outside the scanner (black). The scalppotential map corresponds to the 120-ms peak latency in the VEP acquiredduring fMRI scans. (B) SSVEP power spectrum and the spatial distributionof the power at the stimulus frequency. (C) Time-frequency representation ofthe alpha modulation induced by self-paced eye-open and eye-close. (Panel Afrom [174] with permission).

stimuli [174], similar visual evoked potentials (VEP) were ob-tained from the EEG data recorded with or without concurrentfMRI scans [Fig. 8(a)]. No obvious distortions were found ineither MRI or fMRI [174]. In another experiment with 9-Hzvisual stimuli, sustained oscillatory EEG signals at the stimulusfrequency, known as the steady-state visual evoked pontetials(SSVEP), were found at occipital electrodes [Fig. 8(b)] [127].The last example involved a self-paced eye-open-and-closetask, which is known to change the occipital rhythmic EEGin the alpha band ( ). Such an expected rhythmicmodulation was clearly observed from the time-frequencyrepresentation of the EEG data at the occipital electrodes[Fig. 8(c)] [127].

Simultaneous recording is challenging but sometimes desir-able in fMRI-EEG integrated neuroimaging and neuroscienceresearch. Advances, mainly in signal processing techniques,have allowed us to remove various MR-related artifacts fromthe EEG data recorded during concurrent fMRI scans. Future

development of hardware and software will no doubt furtherimprove the quality of EEG and fMRI from simultaneousrecordings. Although the simultaneous fMRI-EEG recordingis desirable for a variety of purposes, it is not always neces-sary when studying some passive sensory evoked responses.In this regard, one has to base his/her own choice on thereproducibility of the task (or stimulus) of interest versus thepossible risk of dealing with largely contaminated EEG dataif recorded simultaneously with fMRI. Regardless of the the-oretical efficacy of artifact reduction algorithms, the outcomeof these algorithms is after all partly artificial and inevitably“worse” than artifact-free signals. For some simple paradigmsonly involving passive stimulus-evoked responses, it might notalways be worthwhile to obtain these simultaneously collectedsignals and, rather, ensure clear signals by conducting separatebut highly reproducible experiments.

Efforts have also been made to develop SQUID-based instru-mentation for ultralow-field MRI, which aims at acquiring MEGwith MRI [175]. However, achieving high quality data in an ex-perimental setting remains elusive.

VI. CONCLUSION AND PERSPECTIVES

Noninvasive functional neuroimaging, as an important toolfor basic neuroscience research and clinical diagnosis, continuesto have room for improvement of spatial and temporal resolu-tion. While existing neuroimaging modalities might approachtheir limits in imaging capability mostly due to theoretical aswell as technical reasons, it becomes increasingly attractive tointegrate multiple complementary modalities in an attempt tosignificantly enhance the spatiotemporal resolution that cannotbe achieved by any modality individually.

In this regard, the integration of fMRI and EEG/MEG hasreceived the most interest. Electrophysiological and hemody-namic/metabolic signals reflect distinct but closely coupledaspects of the underlying neural activity. Combining fMRI andEEG/MEG data allows us to study brain function from differentperspectives. Convergent evidence, on one hand, definitelyleads to much more confident conclusions on understanding theneural mechanisms. Contradictory observations, on the otherhand, also pose new hypotheses and challenges necessary toguide further investigations of the human neural system.

Our understanding of the coupling between fMRI andEEG/MEG has been substantially improved in the past decade,providing a great opportunity to combine these modalities ina more fundamental and principled way. Agreements havebeen reached in general regarding the neurovascular and neu-rometabolic coupling. Quantitative models of the cross-modalrelationship, like those discussed in this review, representimportant progress along this line. Although these modelscan at best approximate the complex interactions betweenhemodynamics and electrophysiology while subjected to ex-perimental conditions and fundamental assumptions, they doserve as a sound basis for developing multimodal neuroimagingtechniques, which promise to enhance the existing imagingcapability, at least relative to fMRI and EEG/MEG alone.

However, this is not meant to guarantee the success ofmultimodal neuroimaging. Existing theories fail to explainevery aspect of the explosively expanding imaging datasets


documented in thousands of research articles. To date, theprimary bottleneck is still more fundamental than technical.Methodologies that rest on an assumed or modeled physiolog-ical linkage between fMRI and EEG/MEG, almost certainly failunder particular circumstances when the linkage is invalid. Onone hand, theoretical modeling and experimental investigationneed to be performed across microscopic, mesoscopic andmacroscopic scales, and proceed in parallel to further solidifythe physiological and physical basis for the multimodal integra-tion. On the other hand, careful considerations should be takenin experimental designs to justify the rationale of combiningdifferent modalities, regardless of whatever algorithms are usedto fuse the multimodal datasets. Cautions have to be taken aswell in the interpretation of the imaging results.

In what follows, we will discuss a few important remainingissues that deserve systematic investigations and may representcritical challenges as well as opportunities to future develop-ments in multimodal neuroimaging.

A. Negative Bold

Most fMRI studies rely on the task-induced positive BOLDresponse (PBR) as the hemodynamic index of increased neu-ronal activity. The negative BOLD response (NBR), commonlyobserved as the sustained BOLD signal decrease relative to theresting-state baseline, is occasionally observed in both humansand animals undergoing sensory [176]–[183] and cognitivetasks [184].

Regardless of the increasing observations of the negativeBOLD, its origin and relationship to neuronal activity remainspoorly understood and controversial. The NBR may originatefrom reduced neuronal activity [176], [180], [183], [185],[186], or vascular blood stealing [187], [188], or perhaps both[181], [182]. In addition, the NBR can be either dependentor independent upon the task or stimulus content [17], [185].For example, in response to a particular stimulus, the NBRmay be found at non-stimulated regions within the relevantsensory system [177]–[183]. Such a task-dependent NBR isoften hypothesized to reflect neuronal suppression at regionscomplementary to those with increased neural activity [182].On the other hand, decreases in BOLD (or CBF) signals arealso observable at locations that change little across a widevariety of tasks [130], [176]. These task-independent NBRregions are believed to be functionally active [130], [185] andconnected [189] in the resting state, collectively responsiblefor a default-mode brain function [130], [190]. At these re-gions, the NBR may be induced as a consequence of decreaseddefault-mode activity when the resting-state brain function isinterrupted by the execution of attention demanding tasks [17].

Nevertheless, how the NBR should be interpreted in the con-text of fMRI-EEG/MEG integration remains to be investigated.

B. Frequency-Dependent Neurovascular Coupling

As discussed in this review, the vast majority of methodolo-gies integrating fMRI and EEG/MEG are based upon a linearneurovascular coupling. However, recent studies have proposedalternative hypotheses [38], [191]. That is, a negative correlationbetween low-frequency (e.g., alpha-band) electrophysiologicalsignals and BOLD fMRI signals [38], [145]–[147], whereas

high-frequency components (e.g., gamma-band) contribute pos-itively to the BOLD signal [38], [192]. Although this relation-ship remains highly controversial and often varies considerablyacross subjects [193], it does point to some cross-modal rela-tionship that has been ignored or unexplained by conventionalneurovascular coupling models. This alternative view, althoughbeing speculative so far, may help in revising the neurovascularmodel by extending the model from the spatiotemporal domainto the spatial-frequency or spatial-temporal-frequency domain.Along this line, an interesting paper deserves attentions [191]. Inthis analytical work, Kilner et al. propose a heuristic model sug-gesting that an increase in hemodynamic signals is associatedwith a shift of electrophysiological power spectrum from low tohigh frequencies (i.e., a loss in low-frequency power relative to again in high-frequency power). As the authors admit, the modelmay be oversimplified. Its value to neuroimaging also remainsto be demonstrated. Moreover, the validity of this model is ob-viously challenged by the fact that the stimulus-driven low-fre-quency power increase is associated with an increase in theBOLD response. For instance, visual stimuli presented with lowtemporal frequency (e.g., ) induce a noticeable in-crease in the occipital EEG/MEG power at the stimulus fre-quency to accompany a positive increase of the BOLD fMRIsignals in the primary visual cortex.

In short, the frequency-dependent neurovascular coupling re-mains to be verified. Even if true, its impact on interpretationof fMRI, EEG/MEG and their combined imaging remains to beinvestigated.

C. Imaging Brain Functional Connectivity

One of the important applications of the multimodal fMRI-EEG/MEG imaging lies in imaging brain functional connec-tivity. Static images indicating brain regions responsible for theexecution of particular tasks do not convey sufficient informa-tion with respect to how these regions communicate with eachother. The concept of brain connectivity now plays an importantrole in neuroscience, as a way to understand the organized be-havior of brain regions or to reveal the functional brain circuitry[194], [195].

Investigators have computed cortical connectivity patternsbased on hemodynamic or metabolic measurements such asfMRI [196]–[199], whereas the indirect nature of fMRI signalsconfounds the interpretation of fMRI-derived connectivity interms of neuronal interaction [200]. The cortical networks areformed and characterized by organized neuronal oscillationsthat span several orders of magnitude in frequency [201]. Asdiscussed in this review, the neurovascular coupling behaves asa temporal low-pass filter with a cut-off frequency at around 0.4Hz. High-frequency oscillations may be effectively excludedfrom the connectivity patterns estimated from fMRI. Eventhough the fMRI connectivity may reveal coherent low-fre-quency modulations of high-frequency oscillatary neuralactivities, the sluggishness of BOLD fMRI signals representsa significant challenge, if all possible, to imaging the fullspectrum of brain functional connectivity.

EEG/MEG holds the promise to reveal dynamic connectivitysince it is sensitive to transient neural activities occurring onthe order of milliseconds [202]–[204]. A variety of techniques


have been used, most of which have amounted to evaluating thecross-correlation or phase synchronization of signals betweenpairs of scalp electrodes or sensors [205]. Graph-theory-basedtools from the study of complex network have also been devel-oped to describe the connectivity of large-scale networks [206].However, the relationship between the observed connectivitypattern in the sensor space and that in the source space is com-plicated by the dispersion of electromagnetic signals from thecortex to the sensors.

Multimodal fMRI-EEG/MEG integrated neuroimaging ap-proaches hold the potential to greatly enhance our ability toreveal the brain functional connectivity, due to the combinedhigh spatial and temporal resolution. The more precise is ourability to image the brain functional anatomy, the better wewould be able to pinpoint neural connectivity among specificbrain regions. The higher temporal resolution we can achieve,the better chance we would be able to view a wide range ofdynamics in functional interactions within a local or large scale.Efforts have been made in integrating fMRI with electromag-netic source imaging [22] using the directed transfer function(DTF) approach. Other existing methods, such as structuralequation modeling (SEM) [207], and Partial Directed Coher-ence (PDC) [208], are also applicable to fMRI-EEG/MEGimaging data [209].

Also of importance is the investigation of the relationshipsbetween the functional connectivity as derived from fMRI dataand from EEG/MEG data. Due to the different time scales andspatial resolutions of these two modalities there has not beena clearly established consistency between the results obtainedfrom fMRI or EEG/MEG alone. Perhaps, this is simply owingto the fact that these modalities reflect the different and comple-mentary consequences of the same neurophysiological origin.Combining both modalities promises to offer a more completeconclusion on the connectivity among assemblies of neurons.

Retrieving connectivity patterns from functional neu-roimaging data essentially deals with a problem of data-drivensystem identification. That is to find a directional or nondi-rectional functional network that can explain or predict therelationship among the data at various brain locations. Whileneuronal interactions are physically enabled by anatomicalconnections, which may be reconstructed from diffusion tensorMRI [77], the consistency between functional and anatomicalconnectivity remains to be investigated. Moreover, neuronalcircuitry at microscopic scales entails inhibitory and excitatoryconnections. The dynamic yet balanced behaviors of neuronalinhibition and excitation fulfill the regional computation, aswell as the large-scale synchrony and interaction among re-gions. The linkage between the large-scale functional networkand neuronal microcircuits remains unclear. Interpretations offunctional connectivity in terms of excitation and inhibition arecurrently missing, but potentially important.

REFERENCES

[1] H. Berger, “Über das elektroenkephalogram des menschen,” Arch. f.Psychiat., vol. 87, pp. 527–70, 1929.

[2] D. Cohen, “Magnetoencephalography: Evidence of magnetic fieldsproduced by alpha-rhythm currents,” in Science, Aug. 23, 1968, vol.161, pp. 784–786.

[3] S. Ogawa, D. W. Tank, R. Menon, J. M. Ellermann, S. G. Kim, H.Merkle, and K. Ugurbil, “Intrinsic signal changes accompanying sen-sory stimulation: Functional brain mapping with magnetic resonanceimaging,” in Proc. Nat. Acad. Sci. U.S.A., Jul. 1, 1992, vol. 89, pp.5951–5955.

[4] K. K. Kwong, J. W. Belliveau, D. A. Chesler, I. E. Goldberg, R. M.Weisskoff, B. P. Poncelet, D. N. Kennedy, B. E. Hoppel, M. S. Cohen,and R. Turner, “Dynamic magnetic resonance imaging of human brainactivity during primary sensory stimulation,” in Proc. Nat. Acad. Sci.U.S.A., Jun. 15, 1992, vol. 89, pp. 5675–5679.

[5] P. A. Bandettini, E. C. Wong, R. S. Hinks, R. S. Tikofsky, and J. S.Hyde, “Time course EPI of human brain function during task activa-tion,” Magn. Reson. Med., vol. 25, pp. 390–397, Jun. 1992.

[6] P. T. Fox and M. E. Raichle, “Focal physiological uncoupling of cere-bral blood flow and oxidative metabolism during somatosensory stim-ulation in human subjects,” Proc. Nat. Acad. Sci. U.S.A., vol. 83, pp.1140–1144, Feb. 1986.

[7] P. T. Fox, M. E. Raichle, M. A. Mintun, and C. Dence, “Nonoxidativeglucose consumption during focal physiologic neural activity,” in Sci-ence, Jul. 22, 1988, vol. 241, pp. 462–464.

[8] P. L. Nunez, Electric Fields of the Brain: The Neurophysics of EEG.New York: Oxford Univ. Press, 1981.

[9] M. Hamalainen, R. Hari, R. J. Ilmoniemi, J. Knuutila, and O. V.Lounasmaa, “Magnetoencephalography—Theory, instrumentation,and applications to noninvasive studies of the working human brain,”Rev. Mod. Phys., vol. 65, pp. 413–497, 1993.

[10] B. He, Ed., Modeling and Imaing of Bioelectrical Activity Principlesand Applications Norwell, MA, Kluwer , 2004.

[11] B. He, Ed., Neural Engineering Norwell, MA, Kluwer, 2005.[12] P. L. Nunez and R. Srinivasan, Electrical Fields of the Brain: The Neu-

rophysics of EEG. New York: Oxford Univ. Press, 2005.[13] S. Baillet, J. C. Mosher, and R. M. Leahy, “Electromagnetic brain map-

ping,” IEEE Signal Processing Mag., vol. 18, pp. 14–30, 2001.[14] B. He and J. Lian, “High-resolution spatio-temporal functional neu-

roimaging of brain activity,” Crit. Rev. Biomed. Eng., vol. 30, pp.283–306, 2002.

[15] C. M. Michel, M. M. Murray, G. Lantz, S. Gonzalez, L. Spinelli, andR. G. de Peralta, “EEG source imaging,” Clin. Neurophysiol., vol. 115,pp. 2195–2222, Oct. 2004.

[16] S. Ogawa, T. M. Lee, A. R. Kay, and D. W. Tank, “Brain magnetic res-onance imaging with contrast dependent on blood oxygenation,” Proc.Nat. Acad. Sci. U.S.A., vol. 87, pp. 9868–9872, Dec. 1990.

[17] M. E. Raichle and M. A. Mintun, “Brain work and brain imaging,”Annu. Rev. Neurosci., vol. 29, pp. 449–476, 2006.

[18] R. D. Frostig, E. E. Lieke, D. Y. Ts’o, and A. Grinvald, “Cortical func-tional architecture and local coupling between neuronal activity and themicrocirculation revealed by in vivo high-resolution optical imaging ofintrinsic signals,” Proc. Nat. Acad. Sci. U.S.A., vol. 87, pp. 6082–6086,Aug. 1990.

[19] M. M. Haglund, G. A. Ojemann, and D. W. Hochman, “Opticalimaging of epileptiform and functional activity in human cerebralcortex,” in Nature, Aug. 20, 1992, vol. 358, pp. 668–671.

[20] A. K. Liu, J. W. Belliveau, and A. M. Dale, “Spatiotemporal imagingof human brain activity using functional MRI constrained magnetoen-cephalography data: Monte Carlo simulations,” in Proc. Nat. Acad. Sci.U.S.A., Jul. 21, 1998, vol. 95, pp. 8945–8950.

[21] A. M. Dale, A. K. Liu, B. R. Fischl, R. L. Buckner, J. W. Belliveau, J.D. Lewine, and E. Halgren, “Dynamic statistical parametric mapping:Combining fMRI and MEG for high-resolution imaging of cortical ac-tivity,” Neuron, vol. 26, pp. 55–67, Apr. 2000.

[22] F. Babiloni, F. Cincotti, C. Babiloni, F. Carducci, D. Mattia, L. Astolfi,A. Basilisco, P. M. Rossini, L. Ding, Y. Ni, J. Cheng, K. Christine, J.Sweeney, and B. He, “Estimation of the cortical functional connectivitywith the multimodal integration of high-resolution EEG and fMRI databy directed transfer function,” in Neuroimage, Jan. 1, 2005, vol. 24, pp.118–131.

[23] Z. Liu, L. Ding, and B. He, “Integration of EEG/MEG with MRI andfMRI in functional neuroimaging,” IEEE Eng. Med. Biol. Mag., vol.25, pp. 46–53, 2006.

[24] Z. Liu and B. He, “fMRI-EEG integrated cortical source imaging byuse of time-variant spatial constraints,” in Neuroimage, Feb. 1, 2008,vol. 39, pp. 1198–1214.

[25] J. C. Eiccles, “Interpretation of action potentials evoked in the cerebralcortex,” Electroencephlogr., vol. 3, pp. 449–464, 1951.


[26] A. M. Dale and M. I. Sereno, “Improved localization of cortical ac-tivity by combining EEG and MEG with MRI cortical surface recon-struction,” J. Cognitive Neurosci., vol. 5, pp. 162–176, 1993.

[27] M. R. Klee, K. Offenloch, and J. Tigges, “Cross-correlation analysis ofelectroencephalographic potentials and slow membrane transients,” inScience, Jan. 29, 1965, vol. 147, pp. 519–521.

[28] L. Pauling and C. D. Coryell, “The magnetic properties and structureof hemoglobin, oxyhemoglobin and carbonmonoxyhemoglobin,” Proc.Nat. Acad. Sci. U.S.A., vol. 22, pp. 210–216, Apr. 1936.

[29] R. Turner, D. Le Bihan, C. T. Moonen, D. Despres, and J. Frank,“Echo-planar time course MRI of cat brain oxygenation changes,”Magn. Reson. Med., vol. 22, pp. 159–166, Nov. 1991.

[30] S. Ogawa, R. S. Menon, D. W. Tank, S. G. Kim, H. Merkle, J. M. Eller-mann, and K. Ugurbil, “Functional brain mapping by blood oxygena-tion level-dependent contrast magnetic resonance imaging. A compar-ison of signal characteristics with a biophysical model,” Biophys. J.,vol. 64, pp. 803–812, Mar. 1993.

[31] R. B. Buxton, E. C. Wong, and L. R. Frank, “Dynamics of blood flowand oxygenation changes during brain activation: The balloon model,”Magn. Reson. Med., vol. 39, pp. 855–864, Jun. 1998.

[32] R. B. Buxton, K. Uludag, D. J. Dubowitz, and T. T. Liu, “Modeling thehemodynamic response to brain activation,” Neuroimage, vol. 23, no.Suppl 1, pp. S220–33, 2004.

[33] J. B. Mandeville, J. J. Marota, C. Ayata, G. Zaharchuk, M. A.Moskowitz, B. R. Rosen, and R. M. Weisskoff, “Evidence of a cere-brovascular postarteriole windkessel with delayed compliance,” J.Cereb. Blood Flow Metab., vol. 19, pp. 679–689, Jun. 1999.

[34] K. J. Friston, A. Mechelli, R. Turner, and C. J. Price, “Nonlinear re-sponses in fMRI: The Balloon model, Volterra kernels, and other hemo-dynamics,” Neuroimage, vol. 12, pp. 466–477, Oct. 2000.

[35] A. Aubert and R. Costalat, “A model of the coupling between brainelectrical activity, metabolism, and hemodynamics: Application to theinterpretation of functional neuroimaging,” Neuroimage, vol. 17, pp.1162–1181, Nov. 2002.

[36] G. Rees, K. Friston, and C. Koch, “A direct quantitative relationshipbetween the functional properties of human and macaque V5,” Nat.Neurosci., vol. 3, pp. 716–723, Jul. 2000.

[37] D. J. Heeger, A. C. Huk, W. S. Geisler, and D. G. Albrecht, “Spikesversus BOLD: What does neuroimaging tell us about neuronal ac-tivity?,” Nat. Neurosci., vol. 3, pp. 631–633, Jul. 2000.

[38] R. Mukamel, H. Gelbard, A. Arieli, U. Hasson, I. Fried, and R.Malach, “Coupling between neuronal firing, field potentials, andFMRI in human auditory cortex,” in Science, Aug. 5, 2005, vol. 309,pp. 951–954.

[39] N. K. Logothetis, J. Pauls, M. Augath, T. Trinath, and A. Oeltermann,“Neurophysiological investigation of the basis of the fMRI signal,” inNature, Jul. 12, 2001, vol. 412, pp. 150–157.

[40] M. Lauritzen, “Relationship of spikes, synaptic activity, and localchanges of cerebral blood flow,” J. Cereb. Blood Flow Metab., vol. 21,pp. 1367–1383, Dec. 2001.

[41] K. Thomsen, N. Offenhauser, and M. Lauritzen, “Principal neuronspiking: Neither necessary nor sufficient for cerebral blood flow in ratcerebellum,” in J. Physiol., Oct. 1, 2004, vol. 560, pp. 181–189.

[42] A. Viswanathan and R. D. Freeman, “Neurometabolic coupling in cere-bral cortex reflects synaptic more than spiking activity,” Nat. Neurosci.,vol. 10, pp. 1308–1312, Oct. 2007.

[43] N. K. Logothetis, “The underpinnings of the BOLD functional mag-netic resonance imaging signal,” in J. Neurosci., May 15, 2003, vol.23, pp. 3963–3971.

[44] N. K. Logothetis and B. A. Wandell, “Interpreting the BOLD signal,”Annu. Rev. Physiol., vol. 66, pp. 735–769, 2004.

[45] C. Mathiesen, K. Caesar, N. Akgoren, and M. Lauritzen, “Modificationof activity-dependent increases of cerebral blood flow by excitatorysynaptic activity and spikes in rat cerebellar cortex,” J. Physiol., vol.512, pt. 2, pp. 555–566, Oct. 1998.

[46] O. J. Arthurs and S. Boniface, “How well do we understand the neuralorigins of the fMRI BOLD signal?,” Trends Neurosci., vol. 25, pp.27–31, Jan. 2002.

[47] D. J. Heeger and D. Ress, “What does fMRI tell us about neuronalactivity?,” Nat. Rev. Neurosci., vol. 3, pp. 142–151, Feb. 2002.

[48] B. He, D. Yao, J. Lian, and D. Wu, “An equivalent current source modeland laplacian weighted minimum norm current estimates of brain elec-trical activity,” IEEE Trans. Biomed. Eng., vol. 49, no. 4, pp. 277–288,Apr. 2002.

[49] K. Jerbi, S. Baillet, J. C. Mosher, G. Nolte, L. Garnero, and R. M.Leahy, “Localization of realistic cortical activity in MEG using currentmultipoles,” Neuroimage, vol. 22, pp. 779–793, Jun. 2004.

[50] R. D. Sidman, D. J. Vincent, D. B. Smith, and L. Lee, “Experimentaltests of the cortical imaging technique–Applications to the responseto median nerve stimulation and the localization of epileptiform dis-charges,” IEEE Trans. Biomed. Eng., vol. 39, no. 5, pp. 437–444, May1992.

[51] J. Le and A. Gevins, “Method to reduce blur distortion from EEG’susing a realistic head model,” IEEE Trans. Biomed. Eng., vol. 40, no.6, pp. 517–528, Jun. 1993.

[52] A. Gevins, J. Le, N. K. Martin, P. Brickett, J. Desmond, and B. Reutter,“High resolution EEG: 124-channel recording, spatial deblurring andMRI integration methods,” Electroencephalogr. Clin. Neurophysiol.,vol. 90, pp. 337–358, May 1994.

[53] B. He, Y. Wang, and D. Wu, “Estimating cortical potentials from scalpEEG’s in a realistically shaped inhomogeneous head model by meansof the boundary element method,” IEEE Trans. Biomed. Eng., vol. 46,no. 10, pp. 1264–1268, Oct. 1999.

[54] B. He, J. Lian, K. M. Spencer, J. Dien, and E. Donchin, “A corticalpotential imaging analysis of the P300 and novelty P3 components,”Hum. Brain Mapp., vol. 12, pp. 120–130, Feb. 2001.

[55] B. He, X. Zhang, J. Lian, H. Sasaki, D. Wu, and V. L. Towle,“Boundary element method-based cortical potential imaging of so-matosensory evoked potentials using subjects’ magnetic resonanceimages,” Neuroimage, vol. 16, pp. 564–576, Jul. 2002.

[56] F. Babiloni, C. Babiloni, F. Carducci, L. Fattorini, C. Anello, P. Ono-rati, and A. Urbano, “High resolution EEG: A new model-dependentspatial deblurring method using a realistically-shaped MR-constructedsubject’s head model,” Electroencephalogr. Clin. Neurophysiol., vol.102, pp. 69–80, Feb. 1997.

[57] Y. Wang and B. He, “A computer simulation study of cortical imagingfrom scalp potentials,” IEEE Trans. Biomed. Eng., vol. 45, no. 6, pp.724–735, Jun. 1998.

[58] X. Zhang, W. van Drongelen, K. E. Hecox, V. L. Towle, D. M.Frim, A. B. McGee, and B. He, “High-resolution EEG: Corticalpotential imaging of interictal spikes,” Clin. Neurophysiol., vol. 114,pp. 1963–1973, Oct. 2003.

[59] R. N. Kavanagh, T. M. Darcey, D. Lehmann, and D. H. Fender,“Evaluation of methods for three-dimensional localization of electricalsources in the human brain,” IEEE Trans. Biomed. Eng., vol. 25, no.9, pp. 421–429, Sep. 1978.

[60] M. Scherg and D. Von Cramon, “Two bilateral sources of the late AEPas identified by a spatio-temporal dipole model,” Electroencephalogr.Clin. Neurophysiol., vol. 62, pp. 32–44, Jan. 1985.

[61] B. He, T. Musha, Y. Okamoto, S. Homma, Y. Nakajima, and T. Sato,“Electrical dipole tracing in the brain by means of the boundary elementmethod and its accuracy,” IEEE Trans. Biomed. Eng., vol. BME-34, no.6, pp. 406–414, Jun. 1987.

[62] B. N. Cuffin, “A method for localizing EEG sources in realistic headmodels,” IEEE Trans. Biomed. Eng., vol. 42, no. 1, pp. 68–71, Jan.1995.

[63] X. Bai and B. He, “Estimation of number of independent brain electricsources from the scalp EEGs,” IEEE Trans. Biomed. Eng., vol. 53, no.10, pp. 1883–1892, Oct. 2006.

[64] S. Rush and D. A. Driscoll, “EEG electrode sensitivity–An applicationof reciprocity,” IEEE Trans. Biomed. Eng., vol. BME-16, no. 1, pp.15–22, Jan. 1969.

[65] F. Perrin, O. Bertrand, and J. Pernier, “Scalp current density mapping:Value and estimation from potential data,” IEEE Trans. Biomed. Eng.,vol. BME-34, no. 4, pp. 283–288, Apr. 1987.

[66] B. N. Cuffin and D. Cohen, “Comparison of the magnetoencephalo-gram and electroencephalogram,” Electroencephalogr. Clin. Neuro-physiol., vol. 47, pp. 132–146, Aug. 1979.

[67] M. S. Hämäläinen and J. Sarvas, “Realistic conductivity geometrymodel of the human head for interpretation of neuromagnetic data,”IEEE Trans. Biomed. Eng., vol. 36, no. 2, pp. 165–171, Feb. 1989.

[68] M. Fuchs, R. Drenckhahn, H. A. Wischmann, and M. Wagner, “An im-proved boundary element method for realistic volume-conductor mod-eling,” IEEE Trans. Biomed. Eng., vol. 45, no. 8, pp. 980–997, Aug.1998.

[69] Y. Yan, P. L. Nunez, and R. T. Hart, “Finite-element model of thehuman head: Scalp potentials due to dipole sources,” Med. Biol. Eng.Comput., vol. 29, pp. 475–481, Sep. 1991.


[70] Y. Zhang, L. Ding, W. van Drongelen, K. Hecox, D. M. Frim, andB. He, “A cortical potential imaging study from simultaneous extra-and intracranial electrical recordings by means of the finite elementmethod,” in Neuroimage, Jul. 15, 2006, vol. 31, pp. 1513–1524.

[71] L. A. Neilson, M. Kovalyov, and Z. J. Koles, “A computationallyefficient method for accurately solving the EEG forward problem ina finely discretized head model,” Clin. Neurophysiol., vol. 116, pp.2302–2314, Oct. 2005.

[72] T. F. Oostendorp, J. Delbeke, and D. F. Stegeman, “The conductivity ofthe human skull: Results of in vivo and in vitro measurements,” IEEETrans. Biomed. Eng., vol. 47, no. 11, pp. 1487–1492, Nov. 2000.

[73] Y. Lai, W. van Drongelen, L. Ding, K. E. Hecox, V. L. Towle, D. M.Frim, and B. He, “Estimation of in vivo human brain-to-skull conduc-tivity ratio from simultaneous extra- and intra-cranial electrical poten-tial recordings,” Clin. Neurophysiol., vol. 116, pp. 456–465, Feb. 2005.

[74] Y. Zhang, W. van Drongelen, and B. He, “Estimation of in vivobrain-to-skull conductivity ratio in humans,” Appl. Phys. Lett., vol. 89,pp. 223903–2239033, 2006.

[75] P. Metheral, D. C. Barber, R. H. Smallwood, and H. B. Brown, “Threedimensional electrical impedance tomography,” Nature, vol. 380, pp.509–512, 1996.

[76] O. Kwon, E. J. Woo, J. R. Yoon, and J. K. Seo, “Magnetic resonanceelectrical impedance tomography (MREIT): Simulation study of J-sub-stitution algorithm,” IEEE Trans. Biomed. Eng., vol. 49, no. 2, pp.160–167, Feb. 2002.

[77] P. J. Basser, J. Mattiello, and D. LeBihan, “MR diffusion tensor spec-troscopy and imaging,” Biophys. J., vol. 66, pp. 259–267, Jan. 1994.

[78] D. S. Tuch, V. J. Wedeen, A. M. Dale, J. S. George, and J. W. Bel-liveau, “Conductivity tensor mapping of the human brain using diffu-sion tensor MRI,” in Proc. Nat. Acad. Sci. U.S.A., Sep. 25, 2001, vol.98, pp. 11697–11701.

[79] C. H. Wolters, A. Anwander, X. Tricoche, D. Weinstein, M. A. Koch,and R. S. MacLeod, “Influence of tissue conductivity anisotropy onEEG/MEG field and return current computation in a realistic headmodel: A simulation and visualization study using high-resolutionfinite element modeling,” in Neuroimage, Apr. 15, 2006, vol. 30, pp.813–826.

[80] K. Wang, S. Zhu, B. Mueller, K. Lim, Z. Liu, and B. He, “A new methodto derive white matter conductivity from diffusion tensor MRI,” IEEETrans. Biomed. Eng., vol. 55, no. 10, pp. 2481–2486, Oct. 2008.

[81] M. S. Hämäläinen and R. J. Ilmoniemi, Interpreting Measured Mag-netic Fields of the Brain: Estimate of Current Distributions. Helsinki,Finland: Helsinki Univ. of Technol., 1984.

[82] J. Z. Wang, S. J. Williamson, and L. Kaufman, “Magnetic source im-ages determined by a lead-field analysis: The unique minimum-normleast-squares estimation,” IEEE Trans. Biomed. Eng., vol. 39, no. 7, pp.665–675, Jul. 1992.

[83] R. D. Pascual-Marqui, C. M. Michel, and D. Lehmann, “Low resolutionelectromagnetic tomography: A new method for localizing electricalactivity in the brain,” Int. J. Psychophysiol., vol. 18, pp. 49–65, Oct.1994.

[84] R. D. Pascual-Marqui, “Standardized low resolution brain electromag-netic tomography (sLORETA): Technical detail,” Meth. Find. Exp.Clin. Pharm., vol. 24D, pp. 5–12, 2002.

[85] I. F. Gorodnitsky, J. S. George, and B. D. Rao, “Neuromagnetic sourceimaging with FOCUSS: A recursive weighted minimum norm algo-rithm,” Electroencephalogr. Clin. Neurophysiol., vol. 95, pp. 231–251,Oct. 1995.

[86] B. D. Rao and K. Kreutz-Delgado, “An affine scaling methodology forbest basis selection,” IEEE Trans. Signal Process., vol. 47, no. 1, pp.187–200, Jan. 1999.

[87] K. Uutela, M. Hamalainen, and E. Somersalo, “Visualization of mag-netoencephalographic data using minimum current estimates,” Neu-roimage, vol. 10, pp. 173–180, Aug. 1999.

[88] K. Matsuura and Y. Okabe, “Selective minimum-norm solution of thebiomagnetic inverse problem,” IEEE Trans. Biomed. Eng., vol. 42, no.6, pp. 608–615, Jun. 1995.

[89] M. X. Huang, A. M. Dale, T. Song, E. Halgren, D. L. Harrington, I.Podgorny, J. M. Canive, S. Lewis, and R. R. Lee, “Vector-based spatial-temporal minimum L1-norm solution for MEG,” in Neuroimage, Jul.1, 2006, vol. 31, pp. 1025–1037.

[90] L. Ding and B. He, “Sparse source imaging in electroencephalog-raphy with accurate field modeling,” Hum. Brain Mapp., vol. 29, pp.1053–1067, Sep. 2008.

[91] R. Beucker and H. A. Schlitt, On Minimal lp-Norm Solutions of theBiomagnetic Inverse Problem Technical Report KFA-ZAM-IB-9614,1996, vol. 7.

[92] K. Uutela, M. Hamalainen, and R. Salmelin, “Global optimization inthe localization of neuromagnetic sources,” IEEE Trans. Biomed. Eng.,vol. 45, no. 6, pp. 716–723, Jun. 1998.

[93] T. Musha and Y. Okamoto, “Forward and inverse problems of EEGdipole localization,” Crit. Rev. Biomed. Eng., vol. 27, pp. 189–239,1999.

[94] J. C. Mosher, P. S. Lewis, and R. M. Leahy, “Multiple dipole mod-eling and localization from spatio-temporal MEG data,” IEEE Trans.Biomed. Eng., vol. 39, no. 6, pp. 541–557, Jun. 1992.

[95] J. C. Mosher and R. M. Leahy, “Source localization using recursivelyapplied and projected (RAP) MUSIC,” IEEE Trans. Signal Process.,vol. 47, no. 2, pp. 332–340, Feb. 1999.

[96] K. Sekihara, D. Poeppel, A. Marantz, H. Koizumi, and Y. Miyashita,“Noise covariance incorporated MEG-MUSIC algorithm: A methodfor multiple-dipole estimation tolerant of the influence of backgroundbrain activity,” IEEE Trans. Biomed. Eng., vol. 44, no. 9, pp. 839–847,Sep. 1997.

[97] X. L. Xu, B. Xu, and B. He, “An alternative subspace approach to EEGdipole source localization,” in Phys. Med. Biol., Jan. 21, 2004, vol. 49,pp. 327–343.

[98] L. Ding and B. He, “Spatio-temporal EEG source localization usinga three-dimensional subspace FINE approach in a realistic geometryinhomogeneous head model,” IEEE Trans. Biomed. Eng., vol. 53, no.9, pp. 1732–1739, Sep. 2006.

[99] L. Ding, G. A. Worrell, T. D. Lagerlund, and B. He, “Ictal source anal-ysis: Localization and imaging of causal interactions in humans,” inNeuroimage, Jan. 15, 2007, vol. 34, pp. 575–586.

[100] B. D. Van Veen, W. van Drongelen, M. Yuchtman, and A. Suzuki, “Lo-calization of brain electrical activity via linearly constrained minimumvariance spatial filtering,” IEEE Trans. Biomed. Eng., vol. 44, no. 9, pp.867–880, Sep. 1997.

[101] K. Sekihara, S. S. Nagarajan, D. Poeppel, A. Marantz, and Y.Miyashita, “Reconstructing spatio-temporal activities of neuralsources using an MEG vector beamformer technique,” IEEE Trans.Biomed. Eng., vol. 48, no. 7, pp. 760–771, Jul. 2001.

[102] J. Gross, J. Kujala, M. Hamalainen, L. Timmermann, A. Schnitzler, andR. Salmelin, “Dynamic imaging of coherent sources: Studying neuralinteractions in the human brain,” in Proc. Nat. Acad. Sci. U.S.A., Jan.16, 2001, vol. 98, pp. 694–699.

[103] S. S. Dalal, A. G. Guggisberg, E. Edwards, K. Sekihara, A. M. Findlay,R. T. Canolty, M. S. Berger, R. T. Knight, N. M. Barbaro, H. E. Kirsch,and S. S. Nagarajan, “Five-dimensional neuroimaging: Localization ofthe time-frequency dynamics of cortical activity,” in Neuroimage, May1, 2008, vol. 40, pp. 1686–1700.

[104] Y. Yao and P. A. Dewald, “Evaluation of different cortical source lo-calization methods using simulation and experimental EEG data,” Neu-roimage, vol. 25, pp. 369–382, 2005.

[105] X. Bai, V. L. Towle, E. J. He, and B. He, “Evaluation of cortical currentdensity imaging methods using intracranial electrocorticograms andfunctional MRI,” in Neuroimage, Apr. 1, 2007, vol. 35, pp. 598–608.

[106] C. Grova, J. Daunizeau, J. M. Lina, C. G. Benar, H. Benali, and J.Gotman, “Evaluation of EEG localization methods using realistic sim-ulations of interictal spikes,” in Neuroimage, Feb. 1, 2006, vol. 29, pp.734–753.

[107] N. K. Logothetis, “What we can do and what we cannot do with fMRI,”in Nature, Jun. 12, 2008, vol. 453, pp. 869–878.

[108] R. L. Buckner, P. A. Bandettini, K. M. O’Craven, R. L. Savoy, S. E.Petersen, M. E. Raichle, and B. R. Rosen, “Detection of cortical acti-vation during averaged single trials of a cognitive task using functionalmagnetic resonance imaging,” in Proc. Nat. Acad. Sci. U.S.A., Dec. 10,1996, vol. 93, pp. 14878–14883.

[109] A. M. Dale and R. L. Buckner, “Selective averaging of rapidly pre-sented individual trials using fMRI,” Human Brain Mapping, vol. 5,pp. 329–340, 1997.

[110] K. J. Friston, P. Jezzard, and R. Turner, “Analysis of functional MRItime-series,” Humaan Brain Mapping, vol. 1, pp. 153–171, 1994.

[111] K. J. Friston, A. P. Holmes, K. J. Worsley, J. P. Pollin, C. D. Frith, andS. J. Frackowiak, “Statistical parametric maps in functional imaging: Ageneral linear approach,” Human Brain Mapping, vol. 2, pp. 189–210,1995.


[112] K. J. Friston, W. Penny, C. Phillips, S. Kiebel, G. Hinton, and J. Ash-burner, “Classical and Bayesian inference in neuroimaging: Theory,”Neuroimage, vol. 16, pp. 465–483, Jun. 2002.

[113] M. J. McKeown, S. Makeig, G. G. Brown, T. P. Jung, S. S. Kinder-mann, A. J. Bell, and T. J. Sejnowski, “Analysis of fMRI data by blindseparation into independent spatial components,” Human Brain Map-ping, vol. 6, pp. 160–188, 1998.

[114] V. D. Calhoun, T. Adali, G. D. Pearlson, and J. J. Pekar, “Spatial andtemporal independent component analysis of functional MRI data con-taining a pair of task-related waveforms,” Human Brain Mapping, vol.13, pp. 43–53, May 2001.

[115] G. M. Boynton, S. A. Engel, G. H. Glover, and D. J. Heeger, “Linearsystems analysis of functional magnetic resonance imaging in humanV1,” in J. Neurosci., Jul. 1, 1996, vol. 16, pp. 4207–4221.

[116] K. J. Friston, P. Fletcher, O. Josephs, A. Holmes, M. D. Rugg, andR. Turner, “Event-related fMRI: Characterizing differential responses,”Neuroimage, vol. 7, pp. 30–40, Jan. 1998.

[117] A. L. Vazquez and D. C. Noll, “Nonlinear aspects of the BOLDresponse in functional MRI,” Neuroimage, vol. 7, pp. 108–118, Feb.1998.

[118] A. F. Cannestra, N. Pouratian, M. H. Shomer, and A. W. Toga, “Refrac-tory periods observed by intrinsic signal and fluorescent dye imaging,”J. Neurophysiol., vol. 80, pp. 1522–1532, Sep. 1998.

[119] C. Janz, S. P. Heinrich, J. Kornmayer, M. Bach, and J. Hennig, “Cou-pling of neural activity and BOLD fMRI response: New insights bycombination of fMRI and VEP experiments in transition from singleevents to continuous stimulation,” Magn. Reson. Med., vol. 46, pp.482–486, Sep. 2001.

[120] R. M. Birn, Z. S. Saad, and P. A. Bandettini, “Spatial heterogeneityof the nonlinear dynamics in the FMRI BOLD response,” Neuroimage,vol. 14, pp. 817–826, Oct. 2001.

[121] A. Devor, A. K. Dunn, M. L. Andermann, I. Ulbert, D. A. Boas, and A.M. Dale, “Coupling of total hemoglobin concentration, oxygenation,and neural activity in rat somatosensory cortex,” in Neuron, Jul. 17,2003, vol. 39, pp. 353–359.

[122] S. A. Sheth, M. Nemoto, M. Guiou, M. Walker, N. Pouratian, and A. W.Toga, “Linear and nonlinear relationships between neuronal activity,oxygen metabolism, and hemodynamic responses,” in Neuron, Apr. 22,2004, vol. 42, pp. 347–355.

[123] N. Zhang, X. H. Zhu, and W. Chen, “Investigating the sourceof BOLD nonlinearity in human visual cortex in response topaired visual stimuli,” Neuroimage, Jul. 8, 2008, doi:10.1016/j.neu-roimage.2008.06.033.

[124] M. Lauritzen and L. Gold, “Brain function and neurophysiological cor-relates of signals used in functional neuroimaging,” in J. Neurosci.,May 15, 2003, vol. 23, pp. 3972–3980.

[125] X. Wan, J. Riera, K. Iwata, M. Takahashi, T. Wakabayashi, and R.Kawashima, “The neural basis of the hemodynamic response nonlin-earity in human primary visual cortex: Implications for neurovascularcoupling mechanism,” in Neuroimage, Aug. 15, 2006, vol. 32, pp.616–625.

[126] C. Nangini, F. Tam, and S. J. Graham, “A novel method for integratingMEG and BOLD fMRI signals with the linear convolution model inhuman primary somatosensory cortex,” Human Brain Mapping, vol.29, pp. 97–106, Jan. 2008.

[127] Z. Liu, “Multimodal Functional Neuroimaging Integrating FunctionalMagnetic Resonance Imaging and Electroencephalography,” Ph.D. dis-sertation, Univ. of Minnesota, , 2008.

[128] N. Zhang, Z. Liu, B. He, and W. Chen, “Noninvasive study of neurovas-cular coupling during graded neuronal suppression,” J. Cereb. BloodFlow Metab., vol. 28, pp. 280–290, Feb. 2008.

[129] S. Ogawa, T. M. Lee, R. Stepnoski, W. Chen, X. H. Zhu, and K.Ugurbil, “An approach to probe some neural systems interaction byfunctional MRI at neural time scale down to milliseconds,” in Proc.Nat. Acad. Sci. U.S.A., Sep. 26, 2000, vol. 97, pp. 11026–11031.

[130] M. E. Raichle, A. M. MacLeod, A. Z. Snyder, W. J. Powers, D. A.Gusnard, and G. L. Shulman, “A default mode of brain function,” inProc. Nat. Acad. Sci. U.S.A., Jan. 16, 2001, vol. 98, pp. 676–682.

[131] S. P. Ahlfors, G. V. Simpson, A. M. Dale, J. W. Belliveau, A. K. Liu, A.Korvenoja, J. Virtanen, M. Huotilainen, R. B. Tootell, H. J. Aronen, andR. J. Ilmoniemi, “Spatiotemporal activity of a cortical network for pro-cessing visual motion revealed by MEG and fMRI,” J. Neurophysiol.,vol. 82, pp. 2545–2555, Nov. 1999.

[132] A. Korvenoja, J. Huttunen, E. Salli, H. Pohjonen, S. Martinkauppi, J.M. Palva, L. Lauronen, J. Virtanen, R. J. Ilmoniemi, and H. J. Aronen,“Activation of multiple cortical areas in response to somatosensorystimulation: Combined magnetoencephalographic and functional mag-netic resonance imaging,” Human Brain Mapping, vol. 8, pp. 13–27,1999.

[133] N. Fujimaki, T. Hayakawa, M. Nielsen, T. R. Knosche, and S.Miyauchi, “An fMRI-constrained MEG source analysis with proce-dures for dividing and grouping activation,” Neuroimage, vol. 17, pp.324–343, Sep. 2002.

[134] M. Wagner, M. Fuchs, and J. Kastner, “fMRI-constrained dipole fitsand current density reconstructions,” Proc. 12th Int. Conf. Biomag., pp.785–788, 2000.

[135] F. Babiloni, C. Babiloni, F. Carducci, G. L. Romani, P. M. Rossini, L.M. Angelone, and F. Cincotti, “Multimodal integration of high-resolu-tion EEG and functional magnetic resonance imaging data: A simula-tion study,” Neuroimage, vol. 19, pp. 1–15, May 2003.

[136] S. P. Ahlfors and G. V. Simpson, “Geometrical interpretation offMRI-guided MEG/EEG inverse estimates,” Neuroimage, vol. 22, pp.323–332, May 2004.

[137] M. A. Sato, T. Yoshioka, S. Kajihara, K. Toyama, N. Goda, K. Doya,and M. Kawato, “Hierarchical Bayesian estimation for MEG inverseproblem,” Neuroimage, vol. 23, pp. 806–826, Nov. 2004.

[138] C. Phillips, J. Mattout, M. D. Rugg, P. Maquet, and K. J. Friston, “Anempirical Bayesian solution to the source reconstruction problem inEEG,” in Neuroimage, Feb. 15, 2005, vol. 24, pp. 997–1011.

[139] Z. Liu, F. Kecman, and B. He, “Effects of fMRI-EEG mismatches incortical current density estimation integrating fMRI and EEG: A simu-lation study,” Clin. Neurophysiol., vol. 117, pp. 1610–1622, Jul. 2006.

[140] J. Mattout, C. Phillips, W. D. Penny, M. D. Rugg, and K. J. Friston,“MEG source localization under multiple constraints: An extendedBayesian framework,” in Neuroimage, Apr. 15, 2006, vol. 30, pp.753–767.

[141] S. Twomey, “On the numerical solution of Fredholm integral equa-tion of the first kind by the inversion of the linear system producedby qudarature,” J Assoc. Comput. Mach., vol. 10, pp. 97–101, 1963.

[142] D. J. Felleman and D. C. Van Essen, “Distributed hierarchical pro-cessing in the primate cerebral cortex,” Cereb. Cortex, vol. 1, pp. 1–47,Jan.-Feb. 1991.

[143] J. R. Ives, S. Warach, F. Schmitt, R. R. Edelman, and D. L. Schomer,“Monitoring the patient’s EEG during echo planar MRI,” Electroen-cephalogr. Clin. Neurophysiol., vol. 87, pp. 417–420, Dec. 1993.

[144] J. Gotman, C. Grova, A. Bagshaw, E. Kobayashi, Y. Aghakhani, and F.Dubeau, “Generalized epileptic discharges show thalamocortical acti-vation and suspension of the default state of the brain,” in Proc. Nat.Acad. Sci. U.S.A., Oct. 18, 2005, vol. 102, pp. 15236–15240.

[145] R. I. Goldman, J. M. Stern, J. Engel, Jr, and M. S. Cohen, “Simulta-neous EEG and fMRI of the alpha rhythm,” in Neuroreport, Dec. 20,2002, vol. 13, pp. 2487–2492.

[146] H. Laufs, A. Kleinschmidt, A. Beyerle, E. Eger, A. Salek-Haddadi, C.Preibisch, and K. Krakow, “EEG-correlated fMRI of human alpha ac-tivity,” Neuroimage, vol. 19, pp. 1463–1476, Aug. 2003.

[147] M. Moosmann, P. Ritter, I. Krastel, A. Brink, S. Thees, F. Blanken-burg, B. Taskin, H. Obrig, and A. Villringer, “Correlates of alpharhythm in functional magnetic resonance imaging and near infraredspectroscopy,” Neuroimage, vol. 20, pp. 145–158, Sep. 2003.

[148] C. Kaufmann, R. Wehrle, T. C. Wetter, F. Holsboer, D. P. Auer, T.Pollmacher, and M. Czisch, “Brain activation and hypothalamic func-tional connectivity during human non-rapid eye movement sleep: AnEEG/fMRI study,” Brain, vol. 129, pp. 655–667, Mar. 2006.

[149] S. G. Horovitz, M. Fukunaga, J. A. de Zwart, P. van Gelderen, S. C.Fulton, T. J. Balkin, and J. H. Duyn, “Low frequency BOLD fluc-tuations during resting wakefulness and light sleep: A simultaneousEEG-fMRI study,” Human Brain Mapping, vol. 29, pp. 671–682, Jun.2008.

[150] T. Eichele, K. Specht, M. Moosmann, M. L. Jongsma, R. Q. Quiroga,H. Nordby, and K. Hugdahl, “Assessing the spatiotemporal evolution ofneuronal activation with single-trial event-related potentials and func-tional MRI,” in Proc. Nat. Acad. Sci. U.S.A., Dec. 6, 2005, vol. 102,pp. 17798–17803.

[151] S. Debener, M. Ullsperger, M. Siegel, and A. K. Engel, “Single-trialEEG-fMRI reveals the dynamics of cognitive function,” Trends Cogn.Sci., vol. 10, pp. 558–563, Dec. 2006.


[152] R. I. Goldman, J. M. Stern, J. Engel, Jr, and M. S. Cohen, “Acquiringsimultaneous EEG and functional MRI,” Clin. Neurophysiol., vol. 111,pp. 1974–1980, Nov. 2000.

[153] P. J. Allen, O. Josephs, and R. Turner, “A method for removing imagingartifact from continuous EEG recorded during functional MRI,” Neu-roimage, vol. 12, pp. 230–239, Aug. 2000.

[154] P. J. Allen, G. Polizzi, K. Krakow, D. R. Fish, and L. Lemieux, “Iden-tification of EEG events in the MR scanner: The problem of pulse arti-fact and a method for its subtraction,” Neuroimage, vol. 8, pp. 229–239,Oct. 1998.

[155] L. Lemieux, P. J. Allen, F. Franconi, M. R. Symms, and D. R. Fish,“Recording of EEG during fMRI experiments: Patient safety,” Magn.Reson. Med., vol. 38, pp. 943–952, Dec. 1997.

[156] K. Krakow, P. J. Allen, M. R. Symms, L. Lemieux, O. Josephs, andD. R. Fish, “EEG recording during fMRI experiments: Image quality,”Human Brain Mapping, vol. 10, pp. 10–15, May 2000.

[157] P. Ritter and A. Villringer, “Simultaneous EEG-fMRI,” Neurosci.Biobehav. Rev., vol. 30, pp. 823–838, 2006.

[158] H. Laufs, J. Daunizeau, D. W. Carmichael, and A. Kleinschmidt, “Re-cent advances in recording electrophysiological data simultaneouslywith magnetic resonance imaging,” in Neuroimage, Apr. 1, 2008, vol.40, pp. 515–528.

[159] H. Mandelkow, P. Halder, P. Boesiger, and D. Brandeis, “Synchro-nization facilitates removal of MRI artefacts from concurrent EEGrecordings and increases usable bandwidth,” Neuroimage, vol. 32, pp.1120–1126, Sep. 2006.

[160] S. B. Baumann and D. C. Noll, “A modified electrode cap for EEGrecordings in MRI scanners,” Clin. Neurophysiol., vol. 110, pp.2189–2193, Dec. 1999.

[161] C. E. Vasios, L. M. Angelone, P. L. Purdon, J. Ahveninen, J. W.Belliveau, and G. Bonmassar, “EEG/(f)MRI measurements at 7Tesla using a new EEG cap (“InkCap”),” Neuroimage, vol. 33, pp.1082–1092, Dec. 2006.

[162] J. Van Audekerkea, R. Peeters, M. Verhoye, J. Sijbers, and A. Vander Linden, “Special designed RF-antenna with integrated non-inva-sive carbon electrodes for simultaneous magnetic resonance imagingand electroencephalography acquisition at 7 T,” Magn. Reson. Imaging,vol. 18, pp. 887–891, Sep. 2000.

[163] C. Benar, Y. Aghakhani, Y. Wang, A. Izenberg, A. Al-Asmi, F.Dubeau, and J. Gotman, “Quality of EEG in simultaneous EEG-fMRIfor epilepsy,” Clin. Neurophysiol., vol. 114, pp. 569–580, Mar. 2003.

[164] J. Sijbers, I. Michiels, M. Verhoye, J. Van Audekerke, A. Van derLinden, and D. Van Dyck, “Restoration of MR-induced artifacts insimultaneously recorded MR/EEG data,” Magn. Reson. Imaging, vol.17, pp. 1383–1391, Nov. 1999.

[165] A. Hoffmann, L. Jager, K. J. Werhahn, M. Jaschke, S. Noachtar, andM. Reiser, “Electroencephalography during functional echo-planarimaging: Detection of epileptic spikes using post-processing methods,”Magn. Reson. Med., vol. 44, pp. 791–798, Nov. 2000.

[166] G. Garreffa, M. Carni, G. Gualniera, G. B. Ricci, L. Bozzao, D. DeCarli, P. Morasso, P. Pantano, C. Colonnese, V. Roma, and B. Mar-aviglia, “Real-time MR artifacts filtering during continuous EEG/fMRIacquisition,” Magn. Reson. Imaging, vol. 21, pp. 1175–1189, Dec.2003.

[167] M. Negishi, M. Abildgaard, T. Nixon, and R. T. Constable, “Removalof time-varying gradient artifacts from EEG data acquired duringcontinuous fMRI,” Clin. Neurophysiol., vol. 115, pp. 2181–2192, Sep.2004.

[168] R. K. Niazy, C. F. Beckmann, G. D. Iannetti, J. M. Brady, and S. M.Smith, “Removal of FMRI environment artifacts from EEG data usingoptimal basis sets,” Neuroimage, vol. 28, pp. 720–737, Nov. 15, 2005.

[169] M. L. Ellingson, E. Liebenthal, M. V. Spanaki, T. E. Prieto, J. R.Binder, and K. M. Ropella, “Ballistocardiogram artifact reductionin the simultaneous acquisition of auditory ERPS and fMRI,” Neu-roimage, vol. 22, pp. 1534–1542, Aug. 2004.

[170] G. Srivastava, S. Crottaz-Herbette, K. M. Lau, G. H. Glover, and V.Menon, “ICA-based procedures for removing ballistocardiogram arti-facts from EEG data acquired in the MRI scanner,” Neuroimage, vol.24, pp. 50–60, Jan. 1, 2005.

[171] W. Nakamura, K. Anami, T. Mori, O. Saitoh, A. Cichocki, and S.Amari, “Removal of ballistocardiogram artifacts from simultaneouslyrecorded EEG and fMRI data using independent component analysis,”IEEE Trans. Biomed. Eng., vol. 53, no. 7, pp. 1294–1308, Jul. 2006.

[172] G. Bonmassar, D. P. Schwartz, A. K. Liu, K. K. Kwong, A. M. Dale,and J. W. Belliveau, “Spatiotemporal brain imaging of visual-evokedactivity using interleaved EEG and fMRI recordings,” Neuroimage, vol.13, pp. 1035–1043, Jun. 2001.

[173] K. Anami, T. Mori, F. Tanaka, Y. Kawagoe, J. Okamoto, M. Yarita,T. Ohnishi, M. Yumoto, H. Matsuda, and O. Saitoh, “Stepping stonesampling for retrieving artifact-free electroencephalogram duringfunctional magnetic resonance imaging,” Neuroimage, vol. 19, pp.281–295, Jun. 2003.

[174] C. H. Im, Z. M. Liu, N. Zhang, W. Chen, and B. He, “Functional cor-tical source imaging from simultaneously recorded ERP and fMRI,” J.Neurosci. Method., vol. 157, pp. 118–123, 2006.

[175] V. S. Zotev, A. N. Matlashov, P. L. Volegove, A. V. Urbaitis, M. A.Espy, and R. H. , Jr, “SQUID-based instrumentation for ultralow-fieldMRI,” Supercond. Sci. Technol., vol. 20, pp. S 367–S 373, 2007.

[176] G. L. Shulman, J. A. Fiez, M. Corbetta, R. L. Buckner, F. M. Miezin,M. E. Raichle, and S. E. Petersen, “Common blood flow changes acrossvisual tasks: II. Decreases in cerebral cortex,” J. Cogn. Neurosic., vol.9, pp. 648–663, 1997.

[177] R. B. Tootell, J. D. Mendola, N. K. Hadjikhani, A. K. Liu, and A. M.Dale, “The representation of the ipsilateral visual field in human cere-bral cortex,” in Proc. Nat. Acad. Sci. U.S.A., Feb. 3, 1998, vol. 95, pp.818–824.

[178] J. D. Allison, K. J. Meador, D. W. Loring, R. E. Figueroa, and J. C.Wright, “Functional MRI cerebral activation and deactivation duringfinger movement,” in Neurology, Jan. 11, 2000, vol. 54, pp. 135–142.

[179] A. T. Smith, K. D. Singh, and M. W. Greenlee, “Attentional suppres-sion of activity in the human visual cortex,” in Neuroreport, Feb. 7,2000, vol. 11, pp. 271–277.

[180] A. T. Smith, A. L. Williams, and K. D. Singh, “Negative BOLD in thevisual cortex: Evidence against blood stealing,” Human Brain Map-ping, vol. 21, pp. 213–220, Apr. 2004.

[181] A. Shmuel, E. Yacoub, J. Pfeuffer, P. F. Van de Moortele, G. Adriany,X. Hu, and K. Ugurbil, “Sustained negative BOLD, blood flow andoxygen consumption response and its coupling to the positive responsein the human brain,” in Neuron, Dec. 19, 2002, vol. 36, pp. 1195–1210.

[182] A. Shmuel, M. Augath, A. Oeltermann, and N. K. Logothetis, “Nega-tive functional MRI response correlates with decreases in neuronal ac-tivity in monkey visual area V1,” Nat. Neurosci., vol. 9, pp. 569–577,Apr. 2006.

[183] B. Stefanovic, J. M. Warnking, and G. B. Pike, “Hemodynamic andmetabolic responses to neuronal inhibition,” Neuroimage, vol. 22, pp.771–778, Jun. 2004.

[184] A. Amedi, R. Malach, and A. Pascual-Leone, “Negative BOLD differ-entiates visual imagery and perception,” in Neuron, Dec. 8, 2005, vol.48, pp. 859–872.

[185] D. A. Gusnard, M. E. Raichle, and M. E. Raichle, “Searching for abaseline: Functional imaging and the resting human brain,” Nat. Rev.Neurosci., vol. 2, pp. 685–694, Oct. 2001.

[186] L. Gold and M. Lauritzen, “Neuronal deactivation explains decreasedcerebellar blood flow in response to focal cerebral ischemia or sup-pressed neocortical function,” in Proc. Nat. Acad. Sci. U.S.A., May 28,2002, vol. 99, pp. 7699–7704.

[187] N. Harel, S. P. Lee, T. Nagaoka, D. S. Kim, and S. G. Kim, “Origin ofnegative blood oxygenation level-dependent fMRI signals,” J. Cereb.Blood Flow Metab., vol. 22, pp. 908–917, Aug. 2002.

[188] S. S. Kannurpatti and B. B. Biswal, “Negative functional response tosensory stimulation and its origins,” J. Cereb. Blood Flow Metab., vol.24, pp. 703–712, Jun. 2004.

[189] M. D. Fox, A. Z. Snyder, J. L. Vincent, M. Corbetta, D. C. Van Essen,and M. E. Raichle, “The human brain is intrinsically organized intodynamic, anticorrelated functional networks,” in Proc. Nat. Acad. Sci.U.S.A., Jul. 5, 2005, vol. 102, pp. 9673–9678.

[190] J. R. Binder, J. A. Frost, T. A. Hammeke, P. S. Bellgowan, S. M. Rao,and R. W. Cox, “Conceptual processing during the conscious restingstate. A functional MRI study,” J. Cogn. Neurosci., vol. 11, pp. 80–95,Jan. 1999.

[191] J. M. Kilner, J. Mattout, R. Henson, and K. J. Friston, “Hemodynamiccorrelates of EEG: A heuristic,” Neuroimage, vol. 28, pp. 280–286, Oct.15, 2005.

[192] M. Siegel and P. Konig, “A functional gamma-band defined by stim-ulus-dependent synchronization in area 18 of awake behaving cats,” inJ. Neurosci., May 15, 2003, vol. 23, pp. 4251–4260.


[193] S. I. Goncalves, J. C. de Munck, P. J. Pouwels, R. Schoonhoven, J.P. Kuijer, N. M. Maurits, J. M. Hoogduin, E. J. Van Someren, R. M.Heethaar, and F. H. Lopes da Silva, “Correlating the alpha rhythm toBOLD using simultaneous EEG/fMRI: Inter-subject variability,” Neu-roimage, vol. 30, pp. 203–213, Mar. 2006.

[194] B. Horwitz, “The elusive concept of brain connectivity,” Neuroimage,vol. 19, pp. 466–470, Jun. 2003.

[195] A. A. Ioannides, “Dynamic functional connectivity,” Curr. Opin. Neu-robiol., vol. 17, pp. 161–170, Apr. 2007.

[196] C. Buchel and K. J. Friston, “Modulation of connectivity in visual path-ways by attention: Cortical interactions evaluated with structural equa-tion modelling and fMRI,” Cereb. Cortex, vol. 7, pp. 768–778, Dec.1997.

[197] A. R. McIntosh and F. Gonzalez-Lima, “Structural equation modellingand its application to network analysis in functional brain imaging,”Human Brain Mapping, vol. 2, pp. 2–22, 1994.

[198] R. Schlosser, T. Gesierich, B. Kaufmann, G. Vucurevic, S. Hunsche, J.Gawehn, and P. Stoeter, “Altered effective connectivity during workingmemory performance in schizophrenia: A study with fMRI and struc-tural equation modeling,” Neuroimage, vol. 19, pp. 751–763, Jul. 2003.

[199] O. J. Arthurs, T. Donovan, D. J. Spiegelhalter, J. D. Pickard, and S.J. Boniface, “Intracortically distributed neurovascular coupling rela-tionships within and between human somatosensory cortices,” Cereb.Cortex, vol. 17, pp. 661–668, Mar. 2007.

[200] A. Otte and U. Halsband, “Brain imaging tools in neurosciences,” J.Physiol. Paris, vol. 99, pp. 281–292, Jun. 2006.

[201] G. Buzsaki and A. Draguhn, “Neuronal oscillations in cortical net-works,” in Science, Jun. 25, 2004, vol. 304, pp. 1926–1929.

[202] A. S. Gevins, B. A. Cutillo, S. L. Bressler, N. H. Morgan, R. M. White,J. Illes, and D. S. Greer, “Event-related covariances during a bimanualvisuomotor task. II. Preparation and feedback,” Electroencephalogr.Clin. Neurophysiol., vol. 74, pp. 147–160, Mar.-Apr. 1989.

[203] A. Brovelli, M. Ding, A. Ledberg, Y. Chen, R. Nakamura, and S. L.Bressler, “Beta oscillations in a large-scale sensorimotor cortical net-work: Directional influences revealed by Granger causality,” in Proc.Nat. Acad. Sci. U.S.A., Jun. 29, 2004, vol. 101, pp. 9849–9854.

[204] K. Jerbi, J. P. Lachaux, K. N’Diaye, D. Pantazis, R. M. Leahy, L. Gar-nero, and S. Baillet, “Coherent neural representation of hand speed inhumans revealed by MEG imaging,” in Proc. Nat. Acad. Sci. U.S.A.,May 1, 2007, vol. 104, pp. 7676–7681.

[205] J. P. Lachaux, E. Rodriguez, J. Martinerie, and F. J. Varela, “Measuringphase synchrony in brain signals,” Human Brain Mapping, vol. 8, pp.194–208, 1999.

[206] S. H. Strogatz, “Exploring complex networks,” in Nature, Mar. 8, 2001,vol. 410, pp. 268–276.

[207] M. Kaminski and K. Blinowska, “A new method of the description ofthe information flow in the brain structures,” Biol. Cybern., vol. 65, pp.203–210, 1991.

[208] L. Astolfi, F. Cincotti, C. Babiloni, F. Carducci, A. Basilisco, P. M.Rossini, S. Salinari, D. Mattia, S. Cerutti, D. B. Dayan, L. Ding, Y.Ni, B. He, and F. Babiloni, “Estimation of the cortical connectivityby high-resolution EEG and structural equation modeling: Simulationsand application to finger tapping data,” IEEE Trans. Biomed. Eng., vol.52, no. 5, pp. 757–768, May 2005.

[209] L. A. Baccala and K. Sameshima, “Partial directed coherence: A newconcept in neural structure determination,” Biol. Cybern., vol. 84, pp.463–474, Jun. 2001.

[210] L. Astolfi, F. de Vico Fallani, F. Cincotti, D. Mattia, M. G. Marciani, S.Bufalari, S. Salinari, A. Colosimo, L. Ding, J. C. Edgar, W. Heller, G.A. Miller, B. He, and F. Babiloni, “Imaging functional brain connec-tivity patterns from high-resolution EEG and fMRI via graph theory,”Psychophysiology, vol. 44, pp. 880–893, Nov. 2007.

[211] Y. C. Zhang, S. A. Zhu, and B. He, “A high-order finite element algo-rithm for solving the three-dimensional EEG forward problem,” Phys.in Med. and Biol., vol. 49, pp. 2975–2987, 2004.

Bin He (S’87–M’88–SM’97–F’04) received theB.S. degree (with the highest honors) in electricalengineering from Zhejiang University, Hangzhou,China, in 1982 and the Ph.D. degree (with thehighest honors) in bioelectrical engineering fromthe Tokyo Institute of Technology, Tokyo, Japan, in1988. He was a Postdoctoral Fellow in biomedicalengineering at Harvard University—MassachusettsInstitute of Technology (MIT).

After working as a Research Scientist at MIT,he joined the faculty of the University of Illinois,

Chicago, where he became a Professor of Bioengineering, Electrical and Com-puter Engineering, and Computer Science. Since 2004, he has been a Professorof Biomedical Engineering, Electrical Engineering, and Neuroscience at theUniversity of Minnesota at Twin Cities, where he also serves as the Directorof Center for Neuroengineering and Director of NIH Training Program onNeuroimaging. His research interests include biomedical functional imaging,neural engineering, cardiovascular engineering, and bioelectromagnetism.

Dr. He is a Fellow of the American Institute of Medical and Biological Engi-neering (AIMBE), the recipient of the National Science Foundation (NSF) CA-REER Award, the American Heart Association Established Investigator Award,the University of Illinois University Scholar Award, Tejima Prize, and is listed inWho’s Who in America and Who’s Who in the World. He was the Vice Presidentfor Publications (2005–2007) of the IEEE Engineering in Medicine and BiologySociety (EMBS), the President of the International Society of Bioelectromag-netism (2002–2005), and the President-Elect of the IEEE EMBS (2008). Dr. Heis the Editor of book, Neural Engineering (Kluwer, 2005) and an associate ed-itor or guest editor of over ten international journals in biomedical engineering.

Zhongming Liu (S’05) received the B.S. and M.S.degrees in electrical engineering from ZhejiangUniversity, Hangzhou, China, in 2000 and 2003, andthe Ph.D. degree in biomedical engineering fromthe University of Minnesota (UMN), Twin Cities, in2008. He is currently a postdoctoral associate in theDepartment of Biomedical Engineering at the UMN.

His current research interests include multimodalfunctional neuroimaging, functional magnetic reso-nance imaging, and electroencephalography.

read it interesting

Documents