graph cut segmentation - ele.puc-rio.brraul/imageanalysis/graph cut segmentatio… · shi, j. and...
TRANSCRIPT
![Page 1: Graph Cut Segmentation - ele.puc-rio.brraul/ImageAnalysis/GRAPH CUT SEGMENTATIO… · Shi, J. and Malik, J., (1998), Motion Segmentation and Tracking Using Normalized Cuts, Proc](https://reader034.vdocuments.us/reader034/viewer/2022050423/5f92b35b2b333d667e16f2ef/html5/thumbnails/1.jpg)
Graph Cut Segmentation
Raul Queiroz Feitosa
![Page 2: Graph Cut Segmentation - ele.puc-rio.brraul/ImageAnalysis/GRAPH CUT SEGMENTATIO… · Shi, J. and Malik, J., (1998), Motion Segmentation and Tracking Using Normalized Cuts, Proc](https://reader034.vdocuments.us/reader034/viewer/2022050423/5f92b35b2b333d667e16f2ef/html5/thumbnails/2.jpg)
2
Basic Idea
Images are represented by weighted graphs, whereby
pixels are tokens.
Cut up this graph to get sub graphs with strong
interior links and weaker exterior links
11/28/2019 Graph Cut Segmentation
![Page 3: Graph Cut Segmentation - ele.puc-rio.brraul/ImageAnalysis/GRAPH CUT SEGMENTATIO… · Shi, J. and Malik, J., (1998), Motion Segmentation and Tracking Using Normalized Cuts, Proc](https://reader034.vdocuments.us/reader034/viewer/2022050423/5f92b35b2b333d667e16f2ef/html5/thumbnails/3.jpg)
Weighted Graphs
The set of points in an arbitrary feature space are represented as a weighted undirected graph G (V,E)
where V is the set of points in the feature space, and
E is the set of edges formed between every pair of nodes.
The weight on each edge, w(i,j) is a function of the similarity between nodes i and j.
3
1720
1600
7643
2041
0031
Weight Matrix: W
a
e
d
c
b
11/28/2019 Graph Cut Segmentation
![Page 4: Graph Cut Segmentation - ele.puc-rio.brraul/ImageAnalysis/GRAPH CUT SEGMENTATIO… · Shi, J. and Malik, J., (1998), Motion Segmentation and Tracking Using Normalized Cuts, Proc](https://reader034.vdocuments.us/reader034/viewer/2022050423/5f92b35b2b333d667e16f2ef/html5/thumbnails/4.jpg)
4
A graph G can be
partitioned into two disjoint
sets of points A and B such
that:
• A B =
• A B =V
The degree of dissimilarity
(~affinity) between A and B
can be computed as the
total weight of the edges
that have been removed.
* From Khurram Hassan-Shafique CAP5415 Computer Vision 2003
Minimum Cut
11/28/2019 Graph Cut Segmentation
![Page 5: Graph Cut Segmentation - ele.puc-rio.brraul/ImageAnalysis/GRAPH CUT SEGMENTATIO… · Shi, J. and Malik, J., (1998), Motion Segmentation and Tracking Using Normalized Cuts, Proc](https://reader034.vdocuments.us/reader034/viewer/2022050423/5f92b35b2b333d667e16f2ef/html5/thumbnails/5.jpg)
5
Minimum Cut and Clustering
From Khurram Hassan-Shafique CAP5415 Computer Vision 2003
The optimal bi-partitioning of a
graph is the one that minimizes this
cut value.
11/28/2019 Graph Cut Segmentation
![Page 6: Graph Cut Segmentation - ele.puc-rio.brraul/ImageAnalysis/GRAPH CUT SEGMENTATIO… · Shi, J. and Malik, J., (1998), Motion Segmentation and Tracking Using Normalized Cuts, Proc](https://reader034.vdocuments.us/reader034/viewer/2022050423/5f92b35b2b333d667e16f2ef/html5/thumbnails/6.jpg)
𝐶1𝐶2
Image Segmentation & Minimum Cut
11/28/2019 SLIC 6
...
...
...
...
...
...
...
...
...
...
...
...
...
...
An image is modeled as an undirected graph 𝐺 =𝑉, 𝐸 ,
where:
𝑉 = {𝑣𝑖}: vertex ( )
𝐸 = { 𝑣𝑖 , 𝑣𝑗 }: Edge ( )
𝑤 𝑣𝑖 , 𝑣𝑗 : weight associated to an edge (affinity
measure).
...
...
...
...
...
...
...
...
...
...
...
...
...
...
𝐶3𝑆: Segmentation
𝐶1, 𝐶2, …: Segments
high 𝒘
l𝒐𝒘 𝒘
![Page 7: Graph Cut Segmentation - ele.puc-rio.brraul/ImageAnalysis/GRAPH CUT SEGMENTATIO… · Shi, J. and Malik, J., (1998), Motion Segmentation and Tracking Using Normalized Cuts, Proc](https://reader034.vdocuments.us/reader034/viewer/2022050423/5f92b35b2b333d667e16f2ef/html5/thumbnails/7.jpg)
7
Measuring Affinity
Spectral
Texture
Distance
aff x, y exp 12 i
2
I x I y
2
aff x, y exp 12 d
2
x y
2
aff x, y exp 12 t
2
c x c y
2
Maximum affinity = 1.
11/28/2019 Graph Cut Segmentation
![Page 8: Graph Cut Segmentation - ele.puc-rio.brraul/ImageAnalysis/GRAPH CUT SEGMENTATIO… · Shi, J. and Malik, J., (1998), Motion Segmentation and Tracking Using Normalized Cuts, Proc](https://reader034.vdocuments.us/reader034/viewer/2022050423/5f92b35b2b333d667e16f2ef/html5/thumbnails/8.jpg)
8
Drawbacks of Minimum Cut
Weight of cut is directly proportional to the number
of edges in the cut.
Ideal Cut
Cuts with
lesser weight
than the
ideal cut
From Saksoy 2018
11/28/2019 Graph Cut Segmentation
![Page 9: Graph Cut Segmentation - ele.puc-rio.brraul/ImageAnalysis/GRAPH CUT SEGMENTATIO… · Shi, J. and Malik, J., (1998), Motion Segmentation and Tracking Using Normalized Cuts, Proc](https://reader034.vdocuments.us/reader034/viewer/2022050423/5f92b35b2b333d667e16f2ef/html5/thumbnails/9.jpg)
9
Normalized Cut
Instead of looking at the value of total edge weight
connecting the two partitions, compute the cut cost as
a fraction of the total edge.
where
is the total connection from nodes in A to all nodes in
the graph. Similarly for assoc(B,V).
11/28/2019 Graph Cut Segmentation
![Page 10: Graph Cut Segmentation - ele.puc-rio.brraul/ImageAnalysis/GRAPH CUT SEGMENTATIO… · Shi, J. and Malik, J., (1998), Motion Segmentation and Tracking Using Normalized Cuts, Proc](https://reader034.vdocuments.us/reader034/viewer/2022050423/5f92b35b2b333d667e16f2ef/html5/thumbnails/10.jpg)
10
Finding Minimum Normalized-Cut
“Although there are an exponential number of such
partitions, finding the minimum cut of a graph is a
well-studied problem and there exist efficient
algorithms for solving it”.
Shi & Malik, 2000
11/28/2019 Graph Cut Segmentation
![Page 11: Graph Cut Segmentation - ele.puc-rio.brraul/ImageAnalysis/GRAPH CUT SEGMENTATIO… · Shi, J. and Malik, J., (1998), Motion Segmentation and Tracking Using Normalized Cuts, Proc](https://reader034.vdocuments.us/reader034/viewer/2022050423/5f92b35b2b333d667e16f2ef/html5/thumbnails/11.jpg)
Finding Minimum Normalized-Cut
Example: Color Images
11
From Derek Hoiem
11/28/2019 Graph Cut Segmentation
![Page 12: Graph Cut Segmentation - ele.puc-rio.brraul/ImageAnalysis/GRAPH CUT SEGMENTATIO… · Shi, J. and Malik, J., (1998), Motion Segmentation and Tracking Using Normalized Cuts, Proc](https://reader034.vdocuments.us/reader034/viewer/2022050423/5f92b35b2b333d667e16f2ef/html5/thumbnails/12.jpg)
Pros & ConsPros
– Generic framework, can be used with many
different features and affinity formulations
– Provides regular segments
Cons– Need to chose number of segments
– High storage requirement and time complexity
– Bias towards partitioning into equal sized
segments
Usage– Use for oversegmentation when you want
regular segments
From Derek Hoiem
11/28/2019 Graph Cut Segmentation
![Page 13: Graph Cut Segmentation - ele.puc-rio.brraul/ImageAnalysis/GRAPH CUT SEGMENTATIO… · Shi, J. and Malik, J., (1998), Motion Segmentation and Tracking Using Normalized Cuts, Proc](https://reader034.vdocuments.us/reader034/viewer/2022050423/5f92b35b2b333d667e16f2ef/html5/thumbnails/13.jpg)
References
Shi, J. and Malik, J., (2000) Normalized Cuts and Image Segmentation,
IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22
(8), pp.888-905 .
Shi, J. and Malik, J., (1998), Motion Segmentation and Tracking Using
Normalized Cuts, Proc. Int'l Conf. Computer Vision, pp. 154-160.
1311/28/2019 Graph Cut Segmentation
![Page 14: Graph Cut Segmentation - ele.puc-rio.brraul/ImageAnalysis/GRAPH CUT SEGMENTATIO… · Shi, J. and Malik, J., (1998), Motion Segmentation and Tracking Using Normalized Cuts, Proc](https://reader034.vdocuments.us/reader034/viewer/2022050423/5f92b35b2b333d667e16f2ef/html5/thumbnails/14.jpg)
11/28/2019 Morphological Image Processing 14
Next Topic
Representation and
Description