stereo matching using dynamic programming
DESCRIPTION
Stereo Matching Using Dynamic Programming. Jim Rehg CS 4495/7495 Computer Vision Lecture 4 Mon Sept 2, 2002. Correspondence. It is fundamentally ambiguous, even with stereo constraints. Ordering constraint…. …and its failure. Occluded Pixels. Dis-occluded Pixels. - PowerPoint PPT PresentationTRANSCRIPT
Stereo Matching Using Dynamic Stereo Matching Using Dynamic ProgrammingProgramming
Jim RehgJim Rehg
CS 4495/7495 Computer VisionCS 4495/7495 Computer Vision
Lecture 4Lecture 4
Mon Sept 2, 2002Mon Sept 2, 2002
2 J. M. Rehg © 2002
CorrespondenceCorrespondence
It is fundamentally ambiguous, even with stereo It is fundamentally ambiguous, even with stereo constraintsconstraints
Ordering constraint… …and its failure
3 J. M. Rehg © 2002
Search Over CorrespondencesSearch Over Correspondences
Three cases:Three cases: Sequential – cost of matchSequential – cost of match Occluded – cost of no matchOccluded – cost of no match Disoccluded – cost of no matchDisoccluded – cost of no match
Left scanline
Right scanline
Occluded Pixels
Dis-occluded Pixels
4 J. M. Rehg © 2002
Stereo Matching with Dynamic Stereo Matching with Dynamic ProgrammingProgramming
Dynamic programming Dynamic programming yields the optimal path yields the optimal path through grid. This is through grid. This is the best set of the best set of matches that satisfy matches that satisfy the ordering constraintthe ordering constraint
Occluded Pixels
Left scanline
Dis-occluded Pixels
Right scanline
Start
End
5 J. M. Rehg © 2002
Dynamic ProgrammingDynamic Programming
1
2
3
1
2
3
1
2
3
1tC tC 1tC
12
22
32
Principle of Optimality for an n-stage assignment problem:
)(maxarg)( 1 iCjC tijit
2j
1i
2i
3i
6 J. M. Rehg © 2002
Dynamic ProgrammingDynamic Programming
1
2
3
1
2
3
1
2
3
1tC tC 1tC
2)2( tb
Principle of Optimality for an n-stage assignment problem:
)(maxarg)( 1 iCjC tijit
2j
1i
2i
3i
7 J. M. Rehg © 2002
Dynamic ProgrammingDynamic Programming
1
2
3
1
2
3
1
2
3
1tC tC 1tC
Principle of Optimality for an n-stage assignment problem:
)(maxarg)( 1 iCjC tijit
3j
1i
2i
3i
13
23
33
8 J. M. Rehg © 2002
Dynamic ProgrammingDynamic Programming
1
2
3
1
2
3
1
2
3
1tC tC 1tC
Principle of Optimality for an n-stage assignment problem:
)(maxarg)( 1 iCjC tijit
3j
1i
2i
3i1)3( tb
9 J. M. Rehg © 2002
Dynamic ProgrammingDynamic Programming
1
2
3
1
2
3
1
2
3
1tC tC 1tC
Principle of Optimality for an n-stage assignment problem:
)(maxarg)( 1 iCjC tijit
1i
2i
3i1)3( tb
2)2( tb
1j1)1( tb
10 J. M. Rehg © 2002
Dynamic ProgrammingDynamic Programming
1
2
3
1
2
3
1
2
3
2TC 1TC TC
Back-chaining recovers the optimal path and its cost:
...),(),(),(maxarg **1
***TTTTTTTjT sbssCCjCs
1i
2i
3i
)2(
2*
*
TT
T
CC
s
11 J. M. Rehg © 2002
Stereo Matching with Dynamic Stereo Matching with Dynamic ProgrammingProgramming
Scan across grid Scan across grid computing optimal cost computing optimal cost for each node given its for each node given its upper-left neighbors.upper-left neighbors.Backtrack from the Backtrack from the terminal to get the terminal to get the optimal path.optimal path.
Occluded Pixels
Left scanline
Dis-occluded Pixels
Right scanline
Terminal
12 J. M. Rehg © 2002
Stereo Matching with Dynamic Stereo Matching with Dynamic ProgrammingProgramming
Scan across grid Scan across grid computing optimal cost computing optimal cost for each node given its for each node given its upper-left neighbors.upper-left neighbors.Backtrack from the Backtrack from the terminal to get the terminal to get the optimal path.optimal path.
Occluded Pixels
Left scanline
Dis-occluded Pixels
Right scanline
Terminal
13 J. M. Rehg © 2002
Stereo Matching with Dynamic Stereo Matching with Dynamic ProgrammingProgramming
Scan across grid Scan across grid computing optimal cost computing optimal cost for each node given its for each node given its upper-left neighbors.upper-left neighbors.Backtrack from the Backtrack from the terminal to get the terminal to get the optimal path.optimal path.
Occluded Pixels
Left scanline
Dis-occluded Pixels
Right scanline
Terminal
14 J. M. Rehg © 2002
Stereo Matching with Dynamic Stereo Matching with Dynamic ProgrammingProgramming
Scan across grid Scan across grid computing optimal cost computing optimal cost for each node given its for each node given its upper-left neighbors.upper-left neighbors.Backtrack from the Backtrack from the terminal to get the terminal to get the optimal path.optimal path.
Occluded Pixels
Left scanline
Dis-occluded Pixels
Right scanline
Terminal
15 J. M. Rehg © 2002
Stereo Matching with Dynamic Stereo Matching with Dynamic ProgrammingProgramming
Scan across grid Scan across grid computing optimal cost computing optimal cost for each node given its for each node given its upper-left neighbors.upper-left neighbors.Backtrack from the Backtrack from the terminal to get the terminal to get the optimal path.optimal path.
Occluded Pixels
Left scanline
Dis-occluded Pixels
Right scanline
Terminal
16 J. M. Rehg © 2002
Stereo Matching with Dynamic Stereo Matching with Dynamic ProgrammingProgramming
Scan across grid Scan across grid computing optimal cost computing optimal cost for each node given its for each node given its upper-left neighbors.upper-left neighbors.Backtrack from the Backtrack from the terminal to get the terminal to get the optimal path.optimal path.
Occluded Pixels
Left scanline
Dis-occluded Pixels
Right scanline
Terminal
17 J. M. Rehg © 2002
Stereo Matching with Dynamic Stereo Matching with Dynamic ProgrammingProgramming
Scan across grid Scan across grid computing optimal cost computing optimal cost for each node given its for each node given its upper-left neighbors.upper-left neighbors.Backtrack from the Backtrack from the terminal to get the terminal to get the optimal path.optimal path.
Occluded Pixels
Left scanline
Dis-occluded Pixels
Right scanline
Terminal
18 J. M. Rehg © 2002
Computing CorrespondenceComputing Correspondence
Another approach is to match Another approach is to match edgesedges rather than rather than windows of pixels:windows of pixels:
Which method is better?Which method is better? Edges tend to fail in dense texture (outdoors)Edges tend to fail in dense texture (outdoors) Correlation tends to fail in smooth featureless areasCorrelation tends to fail in smooth featureless areas
19 J. M. Rehg © 2002
Computing CorrespondencesComputing Correspondences
Both methods fail for smooth surfacesBoth methods fail for smooth surfaces
There is currently no good solution to the There is currently no good solution to the correspondence problemcorrespondence problem