Multimodal Brain Imaging

Wellcome Trust Centre for Neuroimaging, University College, London

Guillaume Flandin, CEA, ParisNelson Trujillo-Barreto, CNC, Havana

Will Penny

ExperimentalManipulation

Neuronal Activity

MEG,EEG OpticalImaging

PETfMRI

Single/multi-unitrecordings

Spatialconvolution via Maxwell’sequations

Temporal convolutionvia Hemodynamic/Balloon models

FORWARD MODELS

Sensorimotor MemoryLanguageEmotionSocial cognition

ExperimentalManipulation

Neuronal Activity

MEG,EEG

fMRI

Spatialdeconvolution via beamformers

INVERSION

MEG/EEG: Source reconstructionMEG/EEG: Source reconstructionSource model

Forward model

Inverse method

Registration

‘Imaging’‘ECD’

Data Anatomy

Sources

‘Imaging’‘Equivalent Current

Dipoles’ (ECD)

Source Reconstruction

MEG data

EEG data

MEG/EEG: Source reconstructionMEG/EEG: Source reconstruction

ExperimentalManipulation

Neuronal Activity

MEG,EEG

fMRI

Spatialdeconvolution via beamformers

Temporal deconvolutionvia model fitting/inversion

INVERSION

fMRI Forward Model

[ , , , ]s f v qx

Seconds

fMRI model fittingfMRI model fitting

MotionCorrection Smoothing

SpatialNormalisation

General Linear Model

Effect Map

fMRI time-seriesParameter Estimates

Design matrix

Anatomical Reference

ExperimentalManipulation

Neuronal Activity

MEG,EEG

fMRI

Spatialdeconvolution via beamformers

Temporal deconvolutionvia model fitting/inversion

INVERSION

1. Spatio-temporal deconvolution

2. Probabilistic treatment

OverviewOverview

Spatio-temporal deconvolution for M/EEG

Spatio-temporal deconvolution for fMRI

Towards models for multimodal imaging

Spatio-temporal deconvolution for M/EEG

Add temporal constraints in the form of a General Linear Model to describe the temporal evolution of the signal.

Puts M/EEG analysis into same framework as PET/fMRIanalysis.

Work with Nelson. Described in chapter of new SPMbook:

Friston et al. Statistical Parametric Mapping, Elsevier, London, 2006

Approximate Bayesian InferenceApproximate Bayesian Inference

Repeat

• Update source estimates, q(j)• Update regression coefficients, q(w)• Update spatial precisions, q()• Update temporal precisions, q()• Update sensor precisions, q()

Until change in F is small

L

F

KL

1 2

, 0

Pp q q q

L M F KL q p KL

θ v v v v

v θ v θ v

1ˆ ˆ ˆ ˆ ˆ ˆT T T Tt t t

j K ΩK Λ K Ωy ΛW x

ˆTtyK Ω

ˆˆ T TtxW Λ

Corr(R3,R4)=0.47

1.00 0.16 0.09 0.110.16 1.00 0.290.09 0.86 1.000.11 0.29 0.47 1.

0.860.4

007

Corr

o

+

500ms

LowSymmetry

LowAsymmetry

HighSymmetry

HighAsymmetry

Phase 1

Time

600ms

+ 700ms

+

o

2456ms

+

Fa

+

Sb

Ub

+

Sa

Henson R. et al., Cerebral Cortex, 2005

B8

A1 Faces minus Scrambled Faces

170ms post-stimulus

B8 A1

Faces

Scrambled Faces

Daubechies Cubic Splines

Wavelets

28 Basis Functions 30 Basis Functions

Daubechies-4

ERP Faces

ERPScrambled

t = 170 ms

t = 170 ms

Faces – Scrambled faces: Difference of absolute values

Spatio-temporal deconvolution for fMRI

Temporal evolution is described by GLM in the usual way.

Add spatial constraints on regression coefficients in the form of a spatial basis set eg. spatial wavelets.

Automatically select the appropriate basis subset using a mixture prior which switches off irrelevant bases.

Embed this in a probabilistic model.

Work with Guillaume. In Neuroimage.

Spatial Model eg. Wavelets

Mixture prior on wavelet coefficients

(1) Wavelet switches: d=1 if coefficient is ON. Occurs with probability (2) If switch is on, draw z from the fat Gaussian.

Probabilistic Model for fMRI

fMRI data

General LinearModel

Waveletcoefficients

TemporalModel

Spatial Model

Waveletswitches

Switchpriors

Compare to (i) GMRF prior used in M/EEG and (ii) no prior

Inversion using wavelet priors is faster than using standard EEG priors

Results on face fMRI data

Towards multimodal imaging

Use simultaneous EEG- fMRI to identify relationship Between EEG and BOLD (MMN and Flicker paradigms)

EEG is compromised -> artifact removal

Testing the `heuristic’

Start work on specifying generative models

Ongoing work with Felix Blankenburg and James Kilner

fMRI results

fMRI results

We have “synchronized sEEG-fMRI” – MR clock triggers both fMRI and EEG acquisition; after each trigger we get 1 slice of fMRI and 65ms worth of EEG. Synchronisation makes removal of GA artefact easier

MRI Gradient artefact removal from EEG

Ballistocardiogram removal

Could identify QRS complex from ECG to set up a ‘BCG window’ for subsequent processing

Ballistocardiogram removal

Ballistocardiogram removal

The EEG-BOLD heuristic (Kilner, Mattout, Henson & Friston) contends that increases in average EEG frequency predict BOLD activation.

g(w) = spectral density

Testing the heuristic

Log of Bayes factor for Heuristic versus Null

Log of Bayes factor for Heuristic versus Alpha

Probabilistic model for EEG-fMRI

THANK-YOU FOR

YOUR ATTENTION !