Semi-Supervised State Space Models

A Big Thanks To

Prof. Jason BohlandQuantitative Neuroscience LaboratoryBoston University

Istavan (Pisti) Morocz, Harvard, MNI

Firdaus Janoos, OSU/Harvard,MIT/Exxon

Sources

http://neufo.org/lecture_eventsNIPS 2011

A Running Example

Difficulty in learning arithmetic that cannot be explained by mental retardation, inappropriate schooling, or poor social environment

Core conceptual deficit dealing with numbers

Very common : 3-6% of school-age children

Heterogeneous

Dyscalculia DyslexiaSelective inability to build a visual representation of a word, used in subsequent language processing, in the absence of general visual impairment or speech disorders

Affects 5-10% of the populationSpelling, phonological processing, word retrievalDisorder of the visual word form systemMultiple varietiesOccipital, temporal, frontal, cerebellum

Experimental protocolsEvent-related designs- single stimuli/“events” at any

time point- Periodic or spread across

frequencies- Require rapidly acquired

data(small TR)- Rapid events (less than ~20s

apart) give rise to temporal summation of BOLD response

- Summation is close to linear, but non-linearities are evident for small ISIs.

Stimulus function (s(t))

Mental Arithmetic Paradigm

Mental ArithmeticInvolves basic manipulation of number and

quantities

Magnitude based system – bilateral IPS

Verbal based system – left AG

Attentional system – ps Parietal Lobule

Other systems – SMA, primary visual cortex, liPFC, insula, etc

Cascadic Recruitment

Classical fMRI Pipeline

State-of-the-Art - ROI

Janoos et al., EuroVis2009

Another Way ?

Multi-voxel pattern analysis

Traditional analyses focus has focused on relationship between task and individual brain voxels (or regions)

MVPA uses patterns of observed activation across sets of voxels to decode represented information– Relies on machine learning / pattern classification

algorithms– Claim: more sensitive detection of cognitive states (Mind

Reading)– Does not employ spatial smoothing– Typically conducted within individual subjects

http://www.mrc-cbu.cam.ac.uk/people/nikolaus.kriegeskorte/infonotacti.html

Inter-voxel differences contain information!

Brain States

Brain States

Inspiration

Haxby, 2001

Mitchell, 2008

Functional Networks

Functional / Effective Connectivity

Standard analysis of fMRI data conforms to a functional segregation approach to brain function

i.e. brain regions are active for a stimulus typeAssumes the inputs have access to all brain regions

Pertinent Question: How do active brain regions interact with one another? [ functional integration ]

Effective Connectivity = the functional strength of a specific anatomical connection during a particular cognitive task; i.e. the influence that one region has on another. ( Inferred )

Functional Connectivity = the temporal correlation between signal from two brain regions during a cognitive task ( Measured )[ But these are exceptionally fuzzy terms ]

A Solution – State Space Models

Functional Distance ?

Zt1 Zt2

Zt3

Is Zt1 < Zt2 ,or Zt2 < Zt3 ,orSort Zt1, Zt2, Zt3

State Space Model

Comprehensive Model

State-Space Model

Janoos et al., MICCAI 2010

Computational Workflow

Feature Space Estimation

Functional Distance

Transportation Distance

Functional Distance

Zt – activation patternsf - transportation

Transportation Distance

Functional Connectivity Estimation

Gaussian smoothing

HAC until f ≈0.25N

Cluster-wise Correlation Estimation and Shrinkage

Voxel-wise Correlation Estimation

Clustering in Functional Space10

0s 4s 8s0s 4s 8s

Bra

in S

tate

Lab

el

5

0

10

5

0

CritiqueNo neurophysiologic model

Point estimatesHemodynamic uncertainty Temporal structure

Functional distance - an optimization problemNo metric structureExpensive !

Embeddings

A SolutionDistortion minimizing

Feature Space Φ

Orthogonal Bases Graph Partitioning

Normalized graph Laplacian of F

Working in Feature Space Φ

Feature SelectionY

Φ

Rtimes

Resampling with Replacement

Basis Vector φ(l,m) Computation

Bootstrap Distribution of Correlations ρ (l,m)

Feature SelectionRetain φ(l,m) if Pr[ρ (l,m) ≥ τΦ] ≥ 0.75

Functional Network Estimation

Model Size Selection

Strike balance between model complexity and model fit

Information theoretic or Bayesian criteriaNotion of model complexity

Cross-validationIID Assumption

Estimation

Chosen Method

Model Estimation

State Sequence Estimation

Φ Feature-Space Transformation

y

Until convergence

θ

Until convergence

s

K, λWError Rate

HyperparameterSelection

x

YfMRI Data

Feature-space basis

E-stepCompute q(n)(x,z) from p(y,z,x|θ(n))

M-stepEstimate θ(n+1) : L(q(n), θ(n+1)) > L(q(n), θ(n))

E-stepCompute q(n)(z) from p(z| y,x(n),θ)

M-stepx(n+1) = argmax L(q(n), x)

Stimulus Parameters

Hyperparameters

Premise - EM Algorithm

Generalized EM Algorithm

http://mplab.ucsd.edu/tutorials/EM.pdf

Mean Field Approximation

Experimental Conditions

Comprehensive Model

Comparisons

HRFs

Optimal States

Spatial Maps

Population Studies (sort of)

Interpretation

Janoos et al., NeuroImage, 2011

Control Dyscalculic

Dyslexic

MDS Plots

MDS Plots

Control MaleControl Female

Dyslexic FemaleDyslexic Male

Dyscalculic MaleDyscalculic Female

Stage-wise Error Plots

Phase 1

Phase 2

Phase 1: Product Size

Phase 2: Problem Difficulty

Stage-wise MDS Plots

What Else ?

Maximally Predictive Criteria

Multiple spatio-temporal patterns in fMRINeurophysiological

task related vs. default networksExtraneous

Breathing, pulsatile, scanner driftSelect a model that is maximally

predictive with respect to taskPredictability of optimal state-

sequence from stimulus, s

“Resting State”Rather than evoked responses, rs-fMRI looks at random, low-

frequency fluctuations of BOLD activity (Biswal, 1995) “industry standard” filters data at ~0.01 < f < 0.08 Hz

“Default mode” network (Raichle et al., 2001) Set of regions with correlated BOLD activity Activation decreases when subjects perform an explicit task Ventromedial PFC, precuneus, temporal-parietal junction…

But the default mode is only one network that emerges from the correlation structure of resting state networks

Smith et al (2009) showed various task-active networks emerge from ICA based interrogation of rs-fMRI data

Summary

Process model for fMRI Spatial patterns and the temporal structureIdentification of internal mental processes

Neurophysiologically plausibleTest for the effects of experimental

variablesParameter interpretation

Comparison of mental processesAbstract representation of patterns

Thank You for Putting Up with me for 9 Lectures