edge preserving spatially varying mixtures for image segmentation
DESCRIPTION
Edge Preserving Spatially Varying Mixtures for Image Segmentation. by. Giorgos Sfikas, Christophoros Nikou, Nikolaos Galatsanos. (CVPR 2008). Presented by Lihan He ECE, Duke University Feb 23, 2009. Outline. Introduction Edge preserving spatially varying GMM Inference using MAP-EM - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Edge Preserving Spatially Varying Mixtures for Image Segmentation](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815967550346895dc6a53a/html5/thumbnails/1.jpg)
Edge Preserving Spatially Varying Mixturesfor Image Segmentation
Giorgos Sfikas, Christophoros Nikou, Nikolaos Galatsanos
(CVPR 2008)
Presented by Lihan He
ECE, Duke University
Feb 23, 2009
by
![Page 2: Edge Preserving Spatially Varying Mixtures for Image Segmentation](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815967550346895dc6a53a/html5/thumbnails/2.jpg)
Introduction
Edge preserving spatially varying GMM
Inference using MAP-EM
Experimental results
Conclusion
Outline
2/15
![Page 3: Edge Preserving Spatially Varying Mixtures for Image Segmentation](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815967550346895dc6a53a/html5/thumbnails/3.jpg)
Introduction
3/15
Image segmentation
GMM: no prior knowledge is exploited
Adjacent pixels most likely belong to the same cluster;
Edge of objectives.
SVGMM (spatially variant GMM):
Clustering pixels or super pixels such that the same group has common characteristics (same objective, similar texture)
Spatial smoothness is imposed in the neighborhood of each pixel based on the Markov random field;
Without considering the edge of textures
![Page 4: Edge Preserving Spatially Varying Mixtures for Image Segmentation](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815967550346895dc6a53a/html5/thumbnails/4.jpg)
Introduction
4/15
In this paper
Hierarchical Bayesian model;
Spatially varying GMM: mixing weights are different for different pixels;
Difference of mixing weights for two neighbored pixels follows a student-t distribution;
Heavy tailed student-t preserves edges of textures;
MAP-EM is used for model inference.
![Page 5: Edge Preserving Spatially Varying Mixtures for Image Segmentation](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815967550346895dc6a53a/html5/thumbnails/5.jpg)
St-SVGMM
Nnxn ,...,1, Feature vector for each pixel:
SVGMM:
Each pixel has its own mixing weights
Each pixel xn: },,...,,{ 21nK
nn
},,...,,{ 21nK
nn zzz
weights
indicator variables
Likelihood:
Prior:
5/15
K
j
nj
nj zz
11 },1,0{
![Page 6: Edge Preserving Spatially Varying Mixtures for Image Segmentation](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815967550346895dc6a53a/html5/thumbnails/6.jpg)
St-SVGMM
Prior for mixing weight π:
nx
d=2
d=1
d: neighborhood adjacency type
d=1: horizontald=2: vertical
γd(n): the set of neighbors of pixel n, with respect to the dth adjacency type
K×D different student-t distributions are introduced, with hyperparameters
Joint prior for π:
6/15
![Page 7: Edge Preserving Spatially Varying Mixtures for Image Segmentation](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815967550346895dc6a53a/html5/thumbnails/7.jpg)
St-SVGMM
The student-t distribution can be modeled by introducing the latent variable
)(,:1,:1,:1}{ nkDdKjNnnkj d
uU
nkju plays an important role:
,nkju ,k
jnj neighboring pixels n, k belong to the same cluster
,0nkju ,k
jnj n, k are at the edge of two clusters
n – edge location k (d) – adjacency type (horizontal or vertical)j – cluster index (edges of which cluster)
7/15
![Page 8: Edge Preserving Spatially Varying Mixtures for Image Segmentation](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815967550346895dc6a53a/html5/thumbnails/8.jpg)
St-SVGMM
),;(~,,| jjnnn xNzx
Model summary
)(~ nn lmultinomiaz
8/15
![Page 9: Edge Preserving Spatially Varying Mixtures for Image Segmentation](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815967550346895dc6a53a/html5/thumbnails/9.jpg)
Inference
MAP-EM algorithm for model inference.
Complete log-likelihood
Model parameters:
E-step (update Z, U)
9/15
![Page 10: Edge Preserving Spatially Varying Mixtures for Image Segmentation](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815967550346895dc6a53a/html5/thumbnails/10.jpg)
Inference
M-step ( update )
=
,,,,
10/15
![Page 11: Edge Preserving Spatially Varying Mixtures for Image Segmentation](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815967550346895dc6a53a/html5/thumbnails/11.jpg)
Results
U-variable maps
nkju
j=1: sky
j=2: roof & shadows
j=3: building
d=1: horizontal d=2: vertical
n – edge location k (d) – adjacency typej – cluster index
K=3 clusters
Brighter regions represent lower values – edges.
11/15
![Page 12: Edge Preserving Spatially Varying Mixtures for Image Segmentation](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815967550346895dc6a53a/html5/thumbnails/12.jpg)
Results
Comparison on 300 images of the Berkeley image database
Statistics on the Rand Index (RI) (measuring the consistency between the ground truth and the segmentation map); higher is better.
Statistics on the boundary displacement error (BDE) (measuring error of boundary displacement with respect to the ground truth); lower is better.
12/15
![Page 13: Edge Preserving Spatially Varying Mixtures for Image Segmentation](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815967550346895dc6a53a/html5/thumbnails/13.jpg)
Results
Segmentation examples
K=5 K=15K=10original image
13/15
![Page 14: Edge Preserving Spatially Varying Mixtures for Image Segmentation](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815967550346895dc6a53a/html5/thumbnails/14.jpg)
Results
K=5 K=15K=10original image 14/15
![Page 15: Edge Preserving Spatially Varying Mixtures for Image Segmentation](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815967550346895dc6a53a/html5/thumbnails/15.jpg)
Conclusion
15/15
Proposed a GMM-based clustering algorithm for image segmentation;
Used smoothness prior to consider the adjacent pixels belonging to the same cluster;
Also captured the image edge structure (no smoothness enforced across segment boundaries);
All required parameters are estimated from the data (no requirement of empirical parameter selection).
Next: automatically estimating the number of components K.