model-based and model-free connectivity methods for electrical neuroimaging
TRANSCRIPT
Directed dynamical connectivity in electricalneuroimaging: which tools should I use?
A very partial and personal overview, in good faith but still
Daniele Marinazzo
Department of Data Analysis, Faculty of Psychology and Educational Sciences,Ghent University, Belgium
7 @dan marinazzohttp://users.ugent.be/~dmarinaz/
Daniele Marinazzo Directed connectivity in electrical neuroimaging
At least two distinct ways one can think of causality
Temporal precedence, i.e. causes precede their consequences
Physical influence (control), i.e. changing causes changes theirconsequences
Daniele Marinazzo Directed connectivity in electrical neuroimaging
At least two distinct ways one can think of causality
Temporal precedence, i.e. causes precede their consequences
Physical influence (control), i.e. changing causes changes theirconsequences
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Two classes of methods
Assume independent measurements at each node
Inference of networks from temporally correlated data (dynam-ical networks)
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Using temporal dynamics
We model a dynamical system at each node
Two main approaches:
Dynamic Bayesian networks (Hidden Markov Models)
Model-free and model-based investigation of temporal correla-tion
Daniele Marinazzo Directed connectivity in electrical neuroimaging
What to expect from ”causality” measures in neuroscience
Causal measures in neuroscience should reflect effective con-nectivity, i.e. the underlying physiological influences exertedamong neuronal populations in different brain areas. → Dy-namic Causal Models
Different but complementary goal: to reflect directed dynam-ical connectivity without requiring that the resulting networksrecapitulate the underlying physiological processes. → GrangerCausality, Transfer Entropy
The same underlying (physical) network structure can give riseto multiple distinct dynamical connectivity patterns
In practice it is always unfeasible to measure all relevant vari-ables
Bressler and Seth 2010
Daniele Marinazzo Directed connectivity in electrical neuroimaging
What to expect from ”causality” measures in neuroscience
Causal measures in neuroscience should reflect effective con-nectivity, i.e. the underlying physiological influences exertedamong neuronal populations in different brain areas. → Dy-namic Causal Models
Different but complementary goal: to reflect directed dynam-ical connectivity without requiring that the resulting networksrecapitulate the underlying physiological processes. → GrangerCausality, Transfer Entropy
The same underlying (physical) network structure can give riseto multiple distinct dynamical connectivity patterns
In practice it is always unfeasible to measure all relevant vari-ables
Bressler and Seth 2010
Daniele Marinazzo Directed connectivity in electrical neuroimaging
What to expect from ”causality” measures in neuroscience
Causal measures in neuroscience should reflect effective con-nectivity, i.e. the underlying physiological influences exertedamong neuronal populations in different brain areas. → Dy-namic Causal Models
Different but complementary goal: to reflect directed dynam-ical connectivity without requiring that the resulting networksrecapitulate the underlying physiological processes. → GrangerCausality, Transfer Entropy
The same underlying (physical) network structure can give riseto multiple distinct dynamical connectivity patterns
In practice it is always unfeasible to measure all relevant vari-ables
Bressler and Seth 2010
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Basic idea of Dynamic Causal Models
We have several neural populations ..
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Basic idea of Dynamic Causal Models
.. with interactions among and within them
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Basic idea of Dynamic Causal Models
What we see and what we don’t
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Basic idea of Dynamic Causal Models
Forward model
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Basic idea of Dynamic Causal Models
Bayesian framework
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Basic idea of Dynamic Causal Models
Bayesian framework
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Basic idea of Dynamic Causal Models
Model inference
Prior: what connections are included in the model
Likelihood: Incorporates the generative model and predictionerrors
Model evidence: Quantifies the goodness of a model (i.e.,accuracy minus complexity). Used to draw inference on modelstructure.
Posterior: Probability density function of the parameters giventhe data and model. Used to draw inference on model param-eters.
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Basic idea of Dynamic Causal Models
Inference on model structure
Which model (or family of models) has highest evidence?
Inference on model parameters
Which parameters are statistically significant, and what is theirsize/sign?
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Basic idea of Dynamic Causal Models
Inference on model structure
Which model (or family of models) has highest evidence?
Inference on model parameters
Which parameters are statistically significant, and what is theirsize/sign?
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Inference on model structure
A necessary step, unless strong prior knowledge about structure
Bayesian model comparison (BMS) compares the (log) modelevidence of different models (i.e., probability of the data givenmodel)
log model evidence is approximated by free energy
The Kullback - Leibler divergence between the real and approx-imate conditional density minus the log-evidence
A Bayesian Expectation Maximization
ok, a model fit
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Inference on model structure
A necessary step, unless strong prior knowledge about structure
Bayesian model comparison (BMS) compares the (log) modelevidence of different models (i.e., probability of the data givenmodel)
log model evidence is approximated by free energy
The Kullback - Leibler divergence between the real and approx-imate conditional density minus the log-evidence
A Bayesian Expectation Maximization
ok, a model fit
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Inference on model structure
A necessary step, unless strong prior knowledge about structure
Bayesian model comparison (BMS) compares the (log) modelevidence of different models (i.e., probability of the data givenmodel)
log model evidence is approximated by free energy
The Kullback - Leibler divergence between the real and approx-imate conditional density minus the log-evidence
A Bayesian Expectation Maximization
ok, a model fit
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Inference on model structure
A necessary step, unless strong prior knowledge about structure
Bayesian model comparison (BMS) compares the (log) modelevidence of different models (i.e., probability of the data givenmodel)
log model evidence is approximated by free energy
The Kullback - Leibler divergence between the real and approx-imate conditional density minus the log-evidence
A Bayesian Expectation Maximization
ok, a model fit
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Inference on model parameters
Often a second step in DCM studies
Inference on the parameters of the clear winning model (if thereis one)
If no clear winning model (or if optimal model structure differsbetween groups) then Bayesian model averaging (BMA) isan option
Final parameters are weighted average of individual model pa-rameters and posterior probabilities
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Group level inference
Different DCMs are fitted to the data for every subject.
Group inference on the models (or groups of models: in DCMterminology families of models e.g. all models with input toregion A vs. input to region B, or vs. both, three families):Bayesian model selection
Winning model/family is the one with highest exceedance prob-ability
Group inference on model parameter: Either on the winningmodel or Bayesian model averaging (BMA) across models (withina winning family or all models when BMS reveal no clear win-ner)
(BMA) Parameter(s) of interest are harvested for every subjectand subjected to frequentist inference (e.g. t-test)
Daniele Marinazzo Directed connectivity in electrical neuroimaging
DCM for ERPs/ERFs
Bottom-up: connection from low to high hierarchical areastop-down: connection from high to low hierarchical areas (Felle-man 1991)
Lateral: same level in hierarchical organization (e.g. interhemi-spheric connection)
Prior on connection: forward → backward → lateral
Layers within regions interact via intrinsic connections
Daniele Marinazzo Directed connectivity in electrical neuroimaging
DCM inference: summary
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Influences in multivariate datasets
We must condition the measure to the effect of other variables
The most straightforward solution is the conditioned approach,starting from Geweke et al 1984
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Influences in multivariate datasets
We must condition the measure to the effect of other variables
The most straightforward solution is the conditioned approach,starting from Geweke et al 1984
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Beyond conditioning: joint information
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Transfer entropy and Markov property
Absence of causality: generalized Markov property
p(x |X ,Y ) = p(x |X )
Transfer Entropy
Transfer entropy (Schreiber 2000) quantifies the violation of thegeneralized Markov property
T (Y → X ) =
∫p(x |X ,Y ) log
p(x |X ,Y )
p(x |X )dx dX dY
T measures the information flowing from one series to the other.
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Transfer entropy and regression
Risk functional
The minimizer of the risk functional
R [f ] =
∫dX dx (x − f (X ))2 p(X , x)
represents the best estimate of x given X , and corresponds to theregression function
f ∗(X ) =
∫dxp(x |X ) x
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Transfer entropy and regression
Markov property for uncorrelated variables
The best estimate of x , given X and Y is now:
g∗(X ,Y ) =
∫dxp(x |X ,Y ) x
p(x |X ,Y ) = p(x |X )⇒ f ∗(X ) = g∗(X ,Y )
and the knowledge of Y does not improve the prediction of x
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Transfer entropy and regression
Transfer entropy (entropy rate)
SX = −∫
dx dX p(x ,X ) log[p(x |X )]
SXY = −∫
dx dX dY p(x ,X ,Y ) log[p(x |X ,Y )]
Regression
EX =
∫dx dX p(x ,X ) (x −
∫dx ′ p(x ′|X ) x ′)2
EX ,Y =
∫dx dX dY p(x ,X ,Y ) (x −
∫dx ′ p(x ′|X ,Y ) x ′)2
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Granger causality and Transfer entropy
GC and TE are equivalent for Gaussian variables and otherquasi-Gaussian distributions(Barnett et al 2009, Hlavackova-Schindler 2011, Barnett andBossomaier 2012)
In this case they both measure information transfer.
Unified approach (model based and model free)
Mathematically more treatable
Allows grouping variables according to their predictive content(Faes et al. 2014)
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Granger causality and Transfer entropy
GC and TE are equivalent for Gaussian variables and otherquasi-Gaussian distributions(Barnett et al 2009, Hlavackova-Schindler 2011, Barnett andBossomaier 2012)
In this case they both measure information transfer.
Unified approach (model based and model free)
Mathematically more treatable
Allows grouping variables according to their predictive content(Faes et al. 2014)
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Joint information
Let’s go for an operative and practical definition
Relation (B and C) → A
synergy: (B and C) contributes to A with more informationthan the sum of its variables
redundancy: (B and C) contributes to A with less informationthan the sum of its variables
Stramaglia et al. 2012, 2014, 2016
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Joint information
Let’s go for an operative and practical definition
Relation (B and C) → A
synergy: (B and C) contributes to A with more informationthan the sum of its variables
redundancy: (B and C) contributes to A with less informationthan the sum of its variables
Stramaglia et al. 2012, 2014, 2016
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Joint information
Let’s go for an operative and practical definition
Relation (B and C) → A
synergy: (B and C) contributes to A with more informationthan the sum of its variables
redundancy: (B and C) contributes to A with less informationthan the sum of its variables
Stramaglia et al. 2012, 2014, 2016
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Generalization of GC for sets of driving variables
Conditioned Granger Causality in a multivariate system
δX(B → α) = logε (xα|X \ B)
ε (xα|X)
Unnormalized version
δuX(B → α) = ε (xα|X \ B)− ε (xα|X)
An interesting property
If {Xβ}β∈B are statistically independent and their contributions in
the model for xα are additive, then δuX(B → α) =∑β∈B
δuX(β → α).
This property does not hold for the standard definition of GC, neitherfor entropy-rooted quantities, because logarithm.
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Generalization of GC for sets of driving variables
Conditioned Granger Causality in a multivariate system
δX(B → α) = logε (xα|X \ B)
ε (xα|X)
Unnormalized version
δuX(B → α) = ε (xα|X \ B)− ε (xα|X)
An interesting property
If {Xβ}β∈B are statistically independent and their contributions in
the model for xα are additive, then δuX(B → α) =∑β∈B
δuX(β → α).
This property does not hold for the standard definition of GC, neitherfor entropy-rooted quantities, because logarithm.
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Generalization of GC for sets of driving variables
Conditioned Granger Causality in a multivariate system
δX(B → α) = logε (xα|X \ B)
ε (xα|X)
Unnormalized version
δuX(B → α) = ε (xα|X \ B)− ε (xα|X)
An interesting property
If {Xβ}β∈B are statistically independent and their contributions in
the model for xα are additive, then δuX(B → α) =∑β∈B
δuX(β → α).
This property does not hold for the standard definition of GC, neitherfor entropy-rooted quantities, because logarithm.
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Question from the audience:
What does it ever mean to have an unnormalized measure ofGranger causality?
Don’t you lose any link with information?
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Question from the audience:
What does it ever mean to have an unnormalized measure ofGranger causality?
Don’t you lose any link with information?
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Define synergy and redundancy in this framework
Synergy
δuX(B → α) >∑β∈B δ
uX\B,β(β → α)
Redundancy
δuX(B → α) <∑β∈B δ
uX\B,β(β → α)
Stramaglia et al. IEEE TransBiomed. Eng. 2016
Pairwise syn/red index
ψα(i , j) = δuX\j(i → α)− δuX(i → α)
= δuX({i , j} → α)− δuX(i → α)− δuX(j → α)
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Define synergy and redundancy in this framework
Synergy
δuX(B → α) >∑β∈B δ
uX\B,β(β → α)
Redundancy
δuX(B → α) <∑β∈B δ
uX\B,β(β → α)
Stramaglia et al. IEEE TransBiomed. Eng. 2016
Pairwise syn/red index
ψα(i , j) = δuX\j(i → α)− δuX(i → α)
= δuX({i , j} → α)− δuX(i → α)− δuX(j → α)
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Define synergy and redundancy in this framework
Synergy
δuX(B → α) >∑β∈B δ
uX\B,β(β → α)
Redundancy
δuX(B → α) <∑β∈B δ
uX\B,β(β → α)
Stramaglia et al. IEEE TransBiomed. Eng. 2016
Pairwise syn/red index
ψα(i , j) = δuX\j(i → α)− δuX(i → α)
= δuX({i , j} → α)− δuX(i → α)− δuX(j → α)
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Do it yourself!
Statistical Parametric Mapping - DCM http://www.fil.ion.
ucl.ac.uk/spm/
MVGC (State-Space robust implementation) http://users.
sussex.ac.uk/~lionelb/MVGC/
BSmart (Time-varying, Brain-oriented) http://www.brain-smart.org/
MuTE (Multivariate Transfer Entropy, GC in the covariancecase) http://mutetoolbox.guru/
emVAR (Frequency Domain) http://www.lucafaes.net/emvar.html
ITS (Information Dynamics) http://www.lucafaes.net/its.html
Daniele Marinazzo Directed connectivity in electrical neuroimaging
Thanks
Hannes Almgren, Ale Montalto and Frederik van de Steen (UGent)
Sebastiano Stramaglia (Bari)
Pedro Valdes Sosa (CNeuro and UESTC)
Laura Astolfi and Thomas Koenig
Daniele Marinazzo Directed connectivity in electrical neuroimaging
References
David et al., 2006: Dynamical causal modelling of evoked reponses in EEG and MEG (NI)
Stephan et al., 2010: Ten simple rules for dynamic causal modeling (NI)
Penny et al., 2004: Comparing Dynamic causal models (NI)
Litvak et al., 2008: EEG and MEG Data Analysis in SPM8 (CIN)
Bressler and Seth, 2010: Wiener-Granger causality, a well-established methodology (NI)
Montalto et al., 2014: MuTE: A MATLAB Toolbox to Compare Established and Novel Estimators of theMultivariate Transfer Entropy (PLOS One)
Bastos and Schoffelen, 2016: A Tutorial Review of Functional Connectivity Analysis Methods and TheirInterpretational Pitfalls (Front N Sys)
Stramaglia et el. 2106: Synergetic and Redundant Information Flow Detected by Unnormalized GrangerCausality (IEEE TBME)
Daniele Marinazzo Directed connectivity in electrical neuroimaging