markov random fields and gibbs distributions
DESCRIPTION
Markov Random Fields and Gibbs Distributions. Qiang He School Of EE & CS Oregon State University. 1. Introduction. Markov random fields (MRFs). A statistical theory for analyzing spatial & contextual dependencies of physical phenomena. A Bayesian labeling problem - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/1.jpg)
Markov Random Fields and Gibbs Distributions
Qiang HeSchool Of EE & CS
Oregon State University
![Page 2: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/2.jpg)
2
Contents1. Introduction
2. Nondirected graphs
3. Markov Random Fields
4. Gibbs Random Fields
5. Markov-Gibbs Equivalence
6. Inference tasks
7. Summary
![Page 3: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/3.jpg)
3
1. Introduction
![Page 4: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/4.jpg)
4
A statistical theory for analyzing spatial & contextual dependencies of physical phenomena.
A Bayesian labeling problem
A method to establish the probabilistic distributions of interacting labels
Widely used in image processing and computer vision
Markov random fields (MRFs)
![Page 5: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/5.jpg)
5
Properties of MRF
Not ad hoc, can be solved based on sound mathematical principles (maximum a posterior probability, MAP)
Incorporating prior contextual information
Using local properties, which can be implemented in parallel
![Page 6: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/6.jpg)
6
An example: image restoration using MRF
![Page 7: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/7.jpg)
7
Image restoration process Build the neighborhood systems and cliques
Define the clique potentials for prior probability
Derive the likelihood energy
Compute the posterior energy
Solve the MAP
Goals
Restore degraded and noisy images
Infer the true pixels from noisy ones
![Page 8: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/8.jpg)
8
Definition for symbols
f
r
S
= hidden “true “ pixel
= observed “noisy “ pixel
= set of sites or nodes
N
),( NS
= neighbors
= a nondirected graph
![Page 9: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/9.jpg)
9
2. Nondirected graphs
![Page 10: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/10.jpg)
10
A neighborhood system for is defined as
where is the set of sites neighboring . The neighboring
relationship has the following properties:
(1) a site is not neighboring to itself
(2) the neighboring relationship is mutual
S
Neighborhood Systems
}|{ SNN ii
iN
},),(|{ iidpixelpixeldistSiN iii
i
![Page 11: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/11.jpg)
11
Neighborhood Systems
![Page 12: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/12.jpg)
12
Cliques
SA clique is defined as a subset of sites in , where every pair
of sites are neighbors of each other. The collections of single-
site, double-site, and triple-site cliques are denoted by , ,
and ,…
A collection of cliques is 1C 2C
3C
...321 CCCC
![Page 13: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/13.jpg)
13
Cliques
![Page 14: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/14.jpg)
14
3. Markov Random Fields
![Page 15: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/15.jpg)
15
Random field: A family of rvs
defined on the set
Configuration: a value assignment on a random field
Probability:
--discrete case: joint probability
--continuous case: joint PDF
},...,{ 1 mFFF
S
),...,()()( 11 mm fFfFPfFPfP
},...,{ 1 mfffF
Basics
![Page 16: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/16.jpg)
16
Positivity:
FffP ,0)(
Markov random fields
Markovianity:
)|()|( }{ iNiiSi ffPffP
Homogeneity: probability independent of positions of sites
Isotropy: probability independent of orientations of sites
![Page 17: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/17.jpg)
17
)(/)()|()|( rpfPfrprfP
Bayesian labeling problem
)|(maxarg* rfFPfSf
![Page 18: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/18.jpg)
18
4. Gibbs Random Fields (GRFs)
![Page 19: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/19.jpg)
19
Partition function:
))(1
exp(*)( 1 fUT
ZfP
Gibbs distribution:
Ff
fUT
Z ))(1
exp(
Temperature: T
Energy function: )()( fVfUCc
c
Clique potentials: )( fVc
Special case: Gaussian distribution
![Page 20: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/20.jpg)
20
5. Markov-Gibbs Equivalence
![Page 21: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/21.jpg)
21
Proof: MRF=GRF
LfiS
iSiiSi
ifP
fP
fP
ffPffP
' )(
)(
)(
),()|(
}{
}{}{
' ))(exp(
))(exp()|( }{
if Cc c
Cc ciSi fV
fVffP
' )})(exp(*))({exp(
))(exp(*))(exp()|( }{
if Bc cAc c
Bc cAc ciSi fVfV
fVfVffP
' ))(exp(
))(exp()|( }{
if Ac c
Ac ciSi fV
fVffP
Conditional probability:
Extended from clique
potentials: BAC
Factor into two terms
Containing i or not:
Remove the term
containing i:
![Page 22: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/22.jpg)
22
MRF prior and Gibbs distribution
...),()()()(21 },{2
}{1
ji
Cjii
CiCcc ffVfVfVfU
![Page 23: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/23.jpg)
23
Posterior MRF energy
))(exp(*)|( 1 fEZrfFp E
))|(exp(*)|( 1 frUZfFrp r
)|(/)()|()( frUTfUrfUfE
)|(minarg* rfUfSf
Likelihood function:
)|( frULikelihood energy:
Posterior probability:
Posterior energy:
MAP solution:
![Page 24: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/24.jpg)
24
6. Inference tasks
![Page 25: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/25.jpg)
25
Solve MRF prior probability through Gibbs distribution (since MRF=GRF)
Solve likelihood function by estimating the likelihood energy and the posterior energy: coding method or least square error method
Solve the MAP
Solve the Bayesian labeling problem, that is, find the maximum a posterior (MAP) configuration under the observation (simulated annealing process)
Compute a marginal probability
(Gibbs sampling)
Goals
Parameter estimation
)|( rfp
![Page 26: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/26.jpg)
26
Look back at image restoration
Build the neighborhood systems and cliques 4-neighborhood system and two-site cliques
},),(|{ ijrpixelpixeldistSjN iji
},|},{{2 SiNjjiCC i
)(*),( 202 jiji ffgvffV
Define the prior clique potentials
)](1[)( ji ffxg
![Page 27: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/27.jpg)
27
Compute the likelihood energy
nfr ii
Compute the posterior energy
),0(~ Nn
Si
ii frfrU /)()|( 2
Si
iiSi Nj
ji frTffgvrfUi
/)(/)(*)|( 220
![Page 28: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/28.jpg)
28
7. Summary
![Page 29: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/29.jpg)
29
The MRF modeling is to solve the Bayesian labeling problem, that is, find the maximum a posterior (MAP) configuration under the observation
The MRF factors joint distribution into a product of clique potentials
The MRF modeling provides a systematic approach in solving image processing and computer vision problems
![Page 30: Markov Random Fields and Gibbs Distributions](https://reader036.vdocuments.us/reader036/viewer/2022062315/5681572e550346895dc4ca59/html5/thumbnails/30.jpg)
Thank you very much!