Download - Bayesian inference
![Page 1: Bayesian inference](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813bd0550346895da4f689/html5/thumbnails/1.jpg)
Bayesian inference
Jean Daunizeau
Wellcome Trust Centre for Neuroimaging
16 / 05 / 2008
![Page 2: Bayesian inference](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813bd0550346895da4f689/html5/thumbnails/2.jpg)
Overview of the talk
1 Probabilistic modelling and representation of uncertainty
1.1 Introduction
1.2 Bayesian paradigm
1.3 Specification of priors
1.4 Hierarchical models
2 Numerical Bayesian inference methods
2.1 Sampling methods
2.2 Variational methods (EM, VB)
3 SPM applications
3.1 aMRI segmentation
3.2 fMRI time series analysis with spatial priors
3.3 Dynamic causal modelling
3.4 EEG source reconstruction
![Page 3: Bayesian inference](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813bd0550346895da4f689/html5/thumbnails/3.jpg)
Overview of the talk
1 Probabilistic modelling and representation of uncertainty
1.1 Introduction
1.2 Bayesian paradigm
1.3 Specification of priors
1.4 Hierarchical models
2 Numerical Bayesian inference methods
2.1 Sampling methods
2.2 Variational methods (EM, VB)
3 SPM applications
3.1 aMRI segmentation
3.2 fMRI time series analysis with spatial priors
3.3 Dynamic causal modelling
3.4 EEG source reconstruction
![Page 4: Bayesian inference](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813bd0550346895da4f689/html5/thumbnails/4.jpg)
Introduction
![Page 5: Bayesian inference](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813bd0550346895da4f689/html5/thumbnails/5.jpg)
Overview of the talk
1 Probabilistic modelling and representation of uncertainty
1.1 Introduction
1.2 Bayesian paradigm
1.3 Specification of priors
1.4 Hierarchical models
2 Numerical Bayesian inference methods
2.1 Sampling methods
2.2 Variational methods (EM, VB)
3 SPM applications
3.1 aMRI segmentation
3.2 fMRI time series analysis with spatial priors
3.3 Dynamic causal modelling
3.4 EEG source reconstruction
![Page 6: Bayesian inference](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813bd0550346895da4f689/html5/thumbnails/6.jpg)
Bayesian paradigm (1) : theory of probability
Degree of plausibility desiderata:- should be represented using real numbers (D1)- should conform with intuition (D2)- should be consistent (D3)
a=2b=5
a=2
• normalization:
• marginalization:
• conditioning :(Bayes rule)
![Page 7: Bayesian inference](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813bd0550346895da4f689/html5/thumbnails/7.jpg)
Bayesian paradigm (2) : Likelihood and priors
generative model m
Likelihood:
Prior:
Bayes rule:
![Page 8: Bayesian inference](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813bd0550346895da4f689/html5/thumbnails/8.jpg)
Bayesian paradigm (3) : Model comparison
“Occam’s razor” :
Principle of parsimony :« plurality should not be assumed without necessity »
mo
de
l evi
de
nce
p(y
|m)
space of all data sets
y=f(
x)y
= f(
x)
x
Model evidence:
![Page 9: Bayesian inference](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813bd0550346895da4f689/html5/thumbnails/9.jpg)
Overview of the talk
1 Probabilistic modelling and representation of uncertainty
1.1 Introduction
1.2 Bayesian paradigm
1.3 Specification of priors
1.4 Hierarchical models
2 Numerical Bayesian inference methods
2.1 Sampling methods
2.2 Variational methods (EM, VB)
3 SPM applications
3.1 aMRI segmentation
3.2 fMRI time series analysis with spatial priors
3.3 Dynamic causal modelling
3.4 EEG source reconstruction
![Page 10: Bayesian inference](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813bd0550346895da4f689/html5/thumbnails/10.jpg)
Specification of priors
lack of information/entropy
order/structure
priors = population behaviour / information available before having observed the data
• subjectivist approach : “informative” priors• objectivist approach : “non-informative” priors
Principle of maximum entropy :
find the probability distribution functionwhich maximizes the entropy under some constraints (normalization, expectation, …)
![Page 11: Bayesian inference](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813bd0550346895da4f689/html5/thumbnails/11.jpg)
Overview of the talk
1 Probabilistic modelling and representation of uncertainty
1.1 Introduction
1.2 Bayesian paradigm
1.3 Specification of priors
1.4 Hierarchical models
2 Numerical Bayesian inference methods
2.1 Sampling methods
2.2 Variational methods (EM, VB)
3 SPM applications
3.1 aMRI segmentation
3.2 fMRI time series analysis with spatial priors
3.3 Dynamic causal modelling
3.4 EEG source reconstruction
![Page 12: Bayesian inference](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813bd0550346895da4f689/html5/thumbnails/12.jpg)
Hierarchical models (1) : principle
••• hierarchy
causality
![Page 13: Bayesian inference](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813bd0550346895da4f689/html5/thumbnails/13.jpg)
Hierarchical models (2) : directed acyclic graphs (DAGs)
![Page 14: Bayesian inference](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813bd0550346895da4f689/html5/thumbnails/14.jpg)
Hierarchical models (3) : univariate linear hierarchical model
•••
prior posterior
![Page 15: Bayesian inference](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813bd0550346895da4f689/html5/thumbnails/15.jpg)
Overview of the talk
1 Probabilistic modelling and representation of uncertainty
1.1 Introduction
1.2 Bayesian paradigm
1.3 Specification of priors
1.4 Hierarchical models
2 Numerical Bayesian inference methods
2.1 Sampling methods
2.2 Variational methods (EM, VB)
3 SPM applications
3.1 aMRI segmentation
3.2 fMRI time series analysis with spatial priors
3.3 Dynamic causal modelling
3.4 EEG source reconstruction
![Page 16: Bayesian inference](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813bd0550346895da4f689/html5/thumbnails/16.jpg)
Overview of the talk
1 Probabilistic modelling and representation of uncertainty
1.1 Introduction
1.2 Bayesian paradigm
1.3 Specification of priors
1.4 Hierarchical models
2 Numerical Bayesian inference methods
2.1 Sampling methods
2.2 Variational methods (EM, VB)
3 SPM applications
3.1 aMRI segmentation
3.2 fMRI time series analysis with spatial priors
3.3 Dynamic causal modelling
3.4 EEG source reconstruction
![Page 17: Bayesian inference](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813bd0550346895da4f689/html5/thumbnails/17.jpg)
Sampling methods
MCMC example: Gibbs sampling
![Page 18: Bayesian inference](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813bd0550346895da4f689/html5/thumbnails/18.jpg)
Overview of the talk
1 Probabilistic modelling and representation of uncertainty
1.1 Introduction
1.2 Bayesian paradigm
1.3 Specification of priors
1.4 Hierarchical models
2 Numerical Bayesian inference methods
2.1 Sampling methods
2.2 Variational methods (EM, VB)
3 SPM applications
3.1 aMRI segmentation
3.2 fMRI time series analysis with spatial priors
3.3 Dynamic causal modelling
3.4 EEG source reconstruction
![Page 19: Bayesian inference](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813bd0550346895da4f689/html5/thumbnails/19.jpg)
Variational methods
VB/EM/ReML: find (iteratively) the “variational” posterior q(θ)which maximizes the free energy F(q)under some mean-field approximation:
![Page 20: Bayesian inference](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813bd0550346895da4f689/html5/thumbnails/20.jpg)
Overview of the talk
1 Probabilistic modelling and representation of uncertainty
1.1 Introduction
1.2 Bayesian paradigm
1.3 Specification of priors
1.4 Hierarchical models
2 Numerical Bayesian inference methods
2.1 Sampling methods
2.2 Variational methods (EM, VB)
3 SPM applications
3.1 aMRI segmentation
3.2 fMRI time series analysis with spatial priors
3.3 Dynamic causal modelling
3.4 EEG source reconstruction
![Page 21: Bayesian inference](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813bd0550346895da4f689/html5/thumbnails/21.jpg)
Overview of the talk
1 Probabilistic modelling and representation of uncertainty
1.1 Introduction
1.2 Bayesian paradigm
1.3 Specification of priors
1.4 Hierarchical models
2 Numerical Bayesian inference methods
2.1 Sampling methods
2.2 Variational methods (EM, VB)
3 SPM applications
3.1 aMRI segmentation
3.2 fMRI time series analysis with spatial priors
3.3 Dynamic causal modelling
3.4 EEG source reconstruction
![Page 22: Bayesian inference](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813bd0550346895da4f689/html5/thumbnails/22.jpg)
aMRI segmentation
grey matterPPM of belonging to… CSFwhite matter
1
2
3
212
223
iiy
class variances
class priorfrequencies
ith voxel label
class means
ith voxel value
1
2
3
212
223
iiy
1
2
3
212
223
iiy
class variances
class priorfrequencies
ith voxel label
class means
ith voxel value
![Page 23: Bayesian inference](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813bd0550346895da4f689/html5/thumbnails/23.jpg)
Overview of the talk
1 Probabilistic modelling and representation of uncertainty
1.1 Introduction
1.2 Bayesian paradigm
1.3 Specification of priors
1.4 Hierarchical models
2 Numerical Bayesian inference methods
2.1 Sampling methods
2.2 Variational methods (EM, VB)
3 SPM applications
3.1 aMRI segmentation
3.2 fMRI time series analysis with spatial priors
3.3 Dynamic causal modelling
3.4 EEG source reconstruction
![Page 24: Bayesian inference](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813bd0550346895da4f689/html5/thumbnails/24.jpg)
fMRI time series analysiswith spatial priors
observations
GLM coeff
prior varianceof GLM coeff
prior varianceof data noise
AR coeff(correlated noise)
ML estimate of W VB estimate of W
aMRI smoothed W (RFT)
![Page 25: Bayesian inference](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813bd0550346895da4f689/html5/thumbnails/25.jpg)
Overview of the talk
1 Probabilistic modelling and representation of uncertainty
1.1 Introduction
1.2 Bayesian paradigm
1.3 Specification of priors
1.4 Hierarchical models
2 Numerical Bayesian inference methods
2.1 Sampling methods
2.2 Variational methods (EM, VB)
3 SPM applications
3.1 aMRI segmentation
3.2 fMRI time series analysis with spatial priors
3.3 Dynamic causal modelling
3.4 EEG source reconstruction
![Page 26: Bayesian inference](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813bd0550346895da4f689/html5/thumbnails/26.jpg)
Dynamic causal modelling
state-space formulation:
![Page 27: Bayesian inference](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813bd0550346895da4f689/html5/thumbnails/27.jpg)
Overview of the talk
1 Probabilistic modelling and representation of uncertainty
1.1 Introduction
1.2 Bayesian paradigm
1.3 Specification of priors
1.4 Hierarchical models
2 Numerical Bayesian inference methods
2.1 Sampling methods
2.2 Variational methods (EM, VB)
3 SPM applications
3.1 aMRI segmentation
3.2 fMRI time series analysis with spatial priors
3.3 Dynamic causal modelling
3.4 EEG source reconstruction
![Page 28: Bayesian inference](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813bd0550346895da4f689/html5/thumbnails/28.jpg)
EEG source reconstruction
![Page 29: Bayesian inference](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813bd0550346895da4f689/html5/thumbnails/29.jpg)
Homoapriorius
Homopragmaticus
Homofrequentistus
Homosapiens
Homobayesianis
y ,y
y
y