24 cv meanshift-gcuts 0605users.ics.forth.gr/~argyros/cs472_spring20/24_cv... · 2020. 5. 6. ·...
TRANSCRIPT
24 – Mean shift and Graph Cuts
CS472 – Computer Vision, spring 20201
Mean Shift
Wednesday, 06/05/2020
Antonis Argyrose-mail: [email protected]
Mean shift clustering and segmentation
• An advanced and versatile technique for
clustering-based segmentation
D. Comaniciu and P. Meer, Mean Shift: A Robust Approach toward Feature
Space Analysis, PAMI 2002.
24 – Mean shift and Graph Cuts
CS472 – Computer Vision, spring 20202
What is Mean Shift ?
PDF in feature space
• Color space
• …
• Actually any feature space you can conceive
A tool for:
Finding modes in a set of data samples, manifesting an
underlying probability density function (PDF) in RN
Intuitive Description
Distribution of identical billiard balls
Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
24 – Mean shift and Graph Cuts
CS472 – Computer Vision, spring 20203
Intuitive Description
Distribution of identical billiard balls
Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
Intuitive Description
Distribution of identical billiard balls
Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
24 – Mean shift and Graph Cuts
CS472 – Computer Vision, spring 20204
Intuitive Description
Distribution of identical billiard balls
Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
Intuitive Description
Distribution of identical billiard balls
Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
24 – Mean shift and Graph Cuts
CS472 – Computer Vision, spring 20205
Intuitive Description
Distribution of identical billiard balls
Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
Intuitive Description
Distribution of identical billiard balls
Region of
interest
Center of
mass
Objective : Find the densest region
24 – Mean shift and Graph Cuts
CS472 – Computer Vision, spring 20206
What is Mean Shift ?
Non-parametric
Density Estimation
Non-parametric
Density GRADIENT Estimation
(Mean Shift)
Data
Discrete PDF Representation
PDF Analysis
A tool for:
Finding modes in a set of data samples, manifesting an
underlying probability density function (PDF) in RN
Non-Parametric Density Estimation
Assumption : The data points are sampled from an underlying PDF
Assumed Underlying PDF Real Data Samples
Data point density
implies PDF value !
24 – Mean shift and Graph Cuts
CS472 – Computer Vision, spring 20207
Assumed Underlying PDF Real Data Samples
Non-Parametric Density Estimation
Assumed Underlying PDF Real Data Samples
Non-Parametric Density Estimation
24 – Mean shift and Graph Cuts
CS472 – Computer Vision, spring 20208
Parametric Density Estimation
Assumption : The data points are sampled from an underlying PDF
Assumed Underlying PDF
2
2
( )
2
i
PDF( ) =
i
i
ic e
x-μ
x
Estimate
Real Data Samples
Real Modality Analysis
Tessellate the space
with windowsRun the procedure in parallel
24 – Mean shift and Graph Cuts
CS472 – Computer Vision, spring 20209
Real Modality Analysis
The blue data points were traversed by the windows towards the mode
Clustering
Attraction basin : the region for which all trajectories lead to the same mode
Cluster : All data points in the attraction basin of a mode
Mean Shift : A robust Approach Toward Feature Space Analysis, by Comaniciu, Meer
24 – Mean shift and Graph Cuts
CS472 – Computer Vision, spring 202010
Mean Shift Mode Detection
Updated Mean Shift Procedure:
• Find all modes using the Simple Mean Shift Procedure
• Prune modes by perturbing them (find saddle points and plateaus)
• Prune nearby – take highest mode in the window
What happens if we
reach a saddle point
?
Perturb the mode position
and check if we return back
ClusteringSynthetic Examples
Simple Modal Structures
Complex Modal Structures
24 – Mean shift and Graph Cuts
CS472 – Computer Vision, spring 202011
ClusteringReal Example
L*u*v space representation
ClusteringReal Example
Initial window
centers
Modes found Modes after
pruning
Final clusters
Feature space:
L*u*v representation
24 – Mean shift and Graph Cuts
CS472 – Computer Vision, spring 202012
SegmentationExample
…when feature space is only
gray levels…
SegmentationExample
24 – Mean shift and Graph Cuts
CS472 – Computer Vision, spring 202013
SegmentationExample
SegmentationExample
24 – Mean shift and Graph Cuts
CS472 – Computer Vision, spring 202014
SegmentationExample
SegmentationExample
24 – Mean shift and Graph Cuts
CS472 – Computer Vision, spring 202015
SegmentationExample
Mean-Shift Object TrackingTarget Representation
Choose a
reference
target model
Quantized
Color Space
Choose a
feature space
Represent the
model by its
PDF in the
feature space
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
1 2 3 . . . m
color
Pro
bab
ilit
y
Kernel Based Object Tracking, by Comaniniu, Ramesh, Meer
24 – Mean shift and Graph Cuts
CS472 – Computer Vision, spring 202016
,f p y q � �
Mean-Shift Object TrackingTarget Localization Algorithm
Start from the
position of the
model in the
current frame
q�
Search in the
model’s
neighborhood
in next frame
p y�
Find best
candidate by
maximizing a
similarity func.
Mean-Shift Object TrackingResults
24 – Mean shift and Graph Cuts
CS472 – Computer Vision, spring 202017
Tracking Through Scale SpaceResults
Fixed-scale
Tracking through scale space
Mean shift
• Pros:– Does not assume shape on clusters
– One parameter choice (window size, aka “bandwidth”)
– Generic technique
– Find multiple modes
• Cons:– Selection of window size
– Does not scale well with dimension of feature space
24 – Mean shift and Graph Cuts
CS472 – Computer Vision, spring 202018
Image Segmentation with Graph Cuts
q
Images as graphs
• Fully-connected graph
– node (vertex) for every pixel
– link between every pair of pixels, p,q
– affinity weight wpq for each link (edge)
• wpq measures similarity
• similarity is inversely proportional to difference (in color and
position…)
p
wpq
w
Source: Steve Seitz
24 – Mean shift and Graph Cuts
CS472 – Computer Vision, spring 202019
Segmentation by Graph Cuts
• Break Graph into Segments
– Want to delete links that cross between segments
– Easiest to break links that have low similarity (low weight)
• similar pixels should be in the same segments
• dissimilar pixels should be in different segments
w
A B C
Source: Steve Seitz
q
p
wpq
Measuring affinity
• One possibility:
Small sigma:
group only
nearby points
Large sigma:
group distant
points
24 – Mean shift and Graph Cuts
CS472 – Computer Vision, spring 202020
Measuring affinity
σ=.1 σ=.2 σ=1
σ=.2
Data points
Affinity
matrices
Slide credit: Kristen Grauman
Cuts in a graph: Min cut
• Link Cut
– set of links whose removal makes a graph
disconnected
Cost of a cut:
AB
Find minimum cut• gives you a segmentation
• fast algorithms exist for doing this
Source: Steve Seitz
BqAp
qpwBAcut,
,),(
24 – Mean shift and Graph Cuts
CS472 – Computer Vision, spring 202021
Minimum cut
• Problem with minimum cut:
Weight of cut proportional to number of edges in the cut;
tends to produce small, isolated components.
J. Shi and J. Malik, Normalized Cuts and Image Segmentation, CVPR, 1997