graph cut algorithms for binocular stereo with occlusions
DESCRIPTION
Graph Cut Algorithms for Binocular Stereo with Occlusions. Vladimir Kolmogorov, Ramin Zabih. Overview:. Traditional Stereo Methods Energy Minimization via Graph Cuts Stereo with Occlusions Voxel Labeling Algorithm Pixel Labeling Algorithm Results and Conclusions. - PowerPoint PPT PresentationTRANSCRIPT
Graph Cut Algorithms for Binocular Stereo with Occlusions
Vladimir Kolmogorov, Ramin Zabih
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
2/21
Evelyn Gutschier, Markus Sareika
Overview:
Traditional Stereo Methods Energy Minimization via Graph Cuts Stereo with Occlusions Voxel Labeling Algorithm Pixel Labeling Algorithm Results and Conclusions
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
3/21
Evelyn Gutschier, Markus Sareika
Traditional Stereo Methods
pixel correspondences labeling (disparity)
Traditional Stereo Problem
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
4/21
Evelyn Gutschier, Markus Sareika
Traditional Stereo Methods Disparity
depth
disparity
disparity ~ depthground truth disparity
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
5/21
Evelyn Gutschier, Markus Sareika
Traditional Stereo MethodsBinocular Stereo
goal is to compute pixels correspondences traditional stereo problem pixel labeling problem advantage: can be solved by graph cuts problem is formulated as energy term new goal: find the minimizing labeling
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
6/21
Evelyn Gutschier, Markus Sareika
Traditional Stereo MethodsEnergy Function
E fp P
D p f pp , q N
V p , q f p , f q
f f 1 , ... , f P
cost for assigning labels smoothness term
find labeling that minimizes
we assign the label to pixel p when p of image I corresponds to p + in I‘f p
f p
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
7/21
Evelyn Gutschier, Markus Sareika
Traditional Stereo MethodsEnergy Function
D p f p I p I ' p f p ²
V f p , f q T f p f q
other models:absolute distancequadratic
• data cost – gives penalty for different intensities
• smoothness term – gives penalty for discontinuities (Potts model)
V min K , f p f qV min K , f p f q ²
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
8/21
Evelyn Gutschier, Markus Sareika
Energy Minimization via Graph Cuts
Max-flow / Min-Cut(Ford and Fulkerson Algorithm, Push-Relabel Method)
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
9/21
Evelyn Gutschier, Markus Sareika
Energy Minimization via Graph Cuts convex V vs.
metric / semimetric α-β-swap move α-expansion move: assigning label α to an
arbitrary set of pixels
Initial Labeling α-expansionα-β-swap
V , V ,V , 0
V , V , V ,
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
10/21
Evelyn Gutschier, Markus Sareika
Stereo with Occlusions
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
11/21
Evelyn Gutschier, Markus Sareika
Stereo with Occlusions
treat input symmetrically scene elements only visible in single view
physically correct scenes geometric constraints occlusions physically possible labelings
introduce constraints in the problem formulation
graph cuts perform unconstrained energy minimization
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
12/21
Evelyn Gutschier, Markus Sareika
Voxel Labeling Algorithm
discrete scene of voxels
voxel v is active when visible from both cameras
uniqueness constraint – 1:1 correspondence of pixels
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
13/21
Evelyn Gutschier, Markus Sareika
Voxel Labeling AlgorithmEnergy Function
E g E data g E occ g E smooth g
C occ P occ gmatching penalty
(only active voxels)
occlusion penalty set of occluded pixels
smoothness term(Potts model)
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
14/21
Evelyn Gutschier, Markus Sareika
Pixel Labeling AlgorithmEnergy Function
E f E data f E smooth f
E datav p , q N
f p f q d v D vlike traditional stereo but for both images e.g. Potts modelactive ?
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
15/21
Evelyn Gutschier, Markus Sareika
Minimizing the Energy
convert constrained into unconstrained minimization problem write as sum over pairs form of energy function = standard stereo problem
minimization with α-expansion algorithm modified definition of α-expansion move for voxel labeling
E g E data g E smooth g E valid g(0=valid, else ∞)
uniqueness
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
16/21
Evelyn Gutschier, Markus Sareika
Results and Conclusions
ground truth
voxel labeling pixel labelingtraditional s.p.
Tsukuba ref. image
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
17/21
Evelyn Gutschier, Markus Sareika
Results and Conclusions
efficient energy minimization polynominal time instead of exponential time
traditional stereo algorithm is faster pixel labeling better than voxel labeling:
prohibits ‚holes‘ in the scene allows to use other effective smoothness terms
algorithms can be extended for multiple cameras
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
18/21
Evelyn Gutschier, Markus Sareika
Multi-view Stereo via Volumetric Multi-view Stereo via Volumetric Graph CutsGraph Cuts
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
19/21
Evelyn Gutschier, Markus Sareika
Recent Work
Graph-cut-based stereo matching using image segmentation with symmetrical treatment of occlusions, 2006 TUW
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
20/21
Evelyn Gutschier, Markus Sareika
Questions?
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
21/21
Evelyn Gutschier, Markus Sareika
References
M. Bleyer, M. Gelautz, „Graph-cut-based stereo matching using image segmentation with symmetrical treatment of occlusions“, 2007
Y. Boykov, O. Veksler, R. Zabih, „Fast Approximate Energy Minimization via Graph Cuts“, 2001
V. Kolmogorov, R. Zabih, „Graph Cut Algorithms for Binocular Stereo with Occlusions“, 2005
V. Kolmogorov, R. Zabih, „What energy functions can be minimized via graph cuts“, 2004
V. Kolmogorov, R. Zabih, „Generalized multi-camera scene reconstruction using graph cuts“, July 2003
V. Kolmogorov, R. Zabih, „Multi-camera Scene Reconstruction via Graph Cuts“, 2002 S. Seits, C. Dyer, „Photorealistic Scene Reconstruction by Voxel Coloring“, 1997 R.Szeliski, R. Zabih, „An Experimental Comparison of Stereo Algorithms“, 1999