cs 223b 1 more on stereo and correspondence. cs 223b 2 =?f g mostpopular for each window, match to...
Post on 15-Jan-2016
214 views
TRANSCRIPT
CS 223b1
More on stereo and More on stereo and correspondencecorrespondence
CS 223b2
==??
ff gg
MostMostpopularpopular
For each window, match to closest window on epipolar line in other image.
(slides O. Camps)
Comparing Windows:Comparing Windows:
CS 223b3
Window sizeWindow size
W = 3 W = 20
Better results with adaptive window• T. Kanade and M. Okutomi,
A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiment,, Proc. International Conference on Robotics and Automation, 1991.
• D. Scharstein and R. Szeliski. Stereo matching with nonlinear diffusion. International Journal of Computer Vision, 28(2):155-174, July 1998
Effect of window size
(S. Seitz)
CS 223b4
Dynamic Programming (Baker and Dynamic Programming (Baker and Binford, 1981)Binford, 1981)
Find the minimum-cost path going monotonicallydown and right from the top-left corner of thegraph to its bottom-right corner.
• Nodes = matched feature points (e.g., edge points).• Arcs = matched intervals along the epipolar lines.• Arc cost = discrepancy between intervals.
CS 223b5
Dynamic Programming (Baker and Dynamic Programming (Baker and Binford, 1981)Binford, 1981)
Find the minimum-cost path going monotonicallydown and right from the top-left corner of thegraph to its bottom-right corner.
• Nodes = matched feature points (e.g., edge points).• Arcs = matched intervals along the epipolar lines.• Arc cost = discrepancy between intervals.
CS 223b6
Dynamic Programming (Ohta and Kanade, Dynamic Programming (Ohta and Kanade, 1985)1985)
Reprinted from “Stereo by Intra- and Intet-Scanline Search,” by Y. Ohta and T. Kanade, IEEE Trans. on Pattern Analysis and MachineIntelligence, 7(2):139-154 (1985). 1985 IEEE.
CS 223b7
Other constraintsOther constraints
Smoothness: disparity usually doesn’t change too quickly.
Unfortunately, this makes the problem 2D again.
Solved with a host of graph algorithms, Markov Random Fields, Belief Propagation, ….
Uniqueness constraint (each feature can at most have one match)
Occlusion and disparity are connected.
CS 223b8
Feature-based MethodsFeature-based Methods
Conceptually very similar to Correlation-based methods, but: They only search for correspondences of a sparse set of image features.
Correspondences are given by the most similar feature pairs.
Similarity measure must be adapted to the type of feature used.
CS 223b9
Feature-based Methods:Feature-based Methods:
Features most commonly used: Corners
Similarity measured in terms of: surrounding gray values (SSD, Cross-correlation) location
Edges, Lines Similarity measured in terms of:
orientation contrast coordinates of edge or line’s midpoint length of line
CS 223b10
Example: Comparing linesExample: Comparing lines
ll and lr: line lengths l and r: line orientations (xl,yl) and (xr,yr): midpoints cl and cr: average contrast along lines l m c : weights controlling influence
The more similar the lines, the larger S is!
CS 223b11
Correspondence By FeaturesCorrespondence By Features
RIGHT IMAGE
corner line
structure
Search in the right image… the disparity (dx, dy) is the displacement when the similarity measure is maximum
CS 223b12
Dense Stereo Matching: ExamplesDense Stereo Matching: Examples
View extrapolation results
input depth image novel view
[Matthies,Szeliski,Kanade’88]
CS 223b13
Dense Stereo MatchingDense Stereo Matching
Some other view extrapolation results
input depth imagenovel view
CS 223b14
Dense Stereo MatchingDense Stereo Matching
Compute certainty map from correlations
input depth map certainty map
CS 223b15
Ordering constraintOrdering constraint
Usually, order of points in two images is same.
Is this always true? If we match pixel i in image 1 to pixel j in image 2, no matches that follow will affect which are the best preceding matches.
Example with pixels (a la Cox et al.).
CS 223b16
The Ordering ConstraintThe Ordering Constraint
But it is not always the case ...This enables dynamic programming
Points on the epipolar lines appear in the same order
CS 223b17
Correspondence Problem 2Correspondence Problem 2
Correspondence fail for smooth surfaces
There is currently no good solution to the correspondence problem
CS 223b18
Correspondence Problem 3Correspondence Problem 3
Regions without texture Highly Specular surfaces Translucent objects
CS 223b19
Stereo Vision: OutlineStereo Vision: Outline
Basic Equations Epipolar Geometry Image Rectification Reconstruction Correspondence Active Range Imaging Technology Dense and Layered Stereo Smoothing With Markov Random Fields
CS 223b20
How can We Improve Stereo?How can We Improve Stereo?
Space-time stereo scanneruses unstructured light to aidin correspondence
Result: Dense 3D mesh (noisy)
CS 223b21
Prof Marc Levoy @ StanfordProf Marc Levoy @ Stanford
By James Davis, Honda Research,
Now UCSC
CS 223b22
rect
ified
Active Stereo (Structured Active Stereo (Structured Light)Light)
CS 223b23
DP for CorrespondenceDP for Correspondence
Does this always work? When would it fail?
Failure Example 1 Failure Example 2 Failure Example 3
CS 223b24
Stereo CorrespondencesStereo Correspondences
… …Left scanline Right scanline
Match
Match
MatchOcclusion Disocclusion
CS 223b25
Stereo CorrespondencesStereo Correspondences
… …Left scanline Right scanline
CS 223b26
Search Over CorrespondencesSearch Over Correspondences
Three cases: Sequential – cost of match Occluded – cost of no match Disoccluded – cost of no match
Left scanline
Right scanline
Occluded Pixels
Disoccluded Pixels
CS 223b27
Scan across grid computing optimal cost for each node given its upper-left neighbors.Backtrack from the terminal to get the optimal path.
Occluded Pixels
Left scanline
Dis-occluded Pixels
Right scanline
Terminal
Stereo Matching with Dynamic ProgrammingStereo Matching with Dynamic Programming
CS 223b28
Stereo Matching with Dynamic ProgrammingStereo Matching with Dynamic Programming
Dynamic programming yields the optimal path through grid. This is the best set of matches that satisfy the ordering constraint
Occluded Pixels
Left scanline
Dis-occluded Pixels
Right scanline
Start
End
CS 223b29
Scan across grid computing optimal cost for each node given its upper-left neighbors.Backtrack from the terminal to get the optimal path.
Occluded Pixels
Left scanline
Dis-occluded Pixels
Right scanline
Terminal
Stereo Matching with Dynamic ProgrammingStereo Matching with Dynamic Programming
CS 223b30
Scan across grid computing optimal cost for each node given its upper-left neighbors.Backtrack from the terminal to get the optimal path.
Occluded Pixels
Left scanline
Dis-occluded Pixels
Right scanline
Terminal
Stereo Matching with Dynamic ProgrammingStereo Matching with Dynamic Programming