epipolar lines
Post on 22-Feb-2016
23 Views
Preview:
DESCRIPTION
TRANSCRIPT
Epipolar lines
epipolar linesepipolar lines
BaselineO O’
epipolar plane
𝑝 ′𝑇 𝐸𝑝=0
Rectification
• Rectification: rotation and scaling of each camera’s coordinate frame to make the epipolar lines horizontal and equi-height,by bringing the two image planes to be parallel to the baseline
• Rectification is achieved by applying homography to each of the two images
Rectification
BaselineO O’
𝑞 ′𝑇𝐻 𝑙−𝑇 𝐸𝐻𝑟
−1𝑞=0
𝐻 𝑙 𝐻𝑟
Cyclopean coordinates• In a rectified stereo rig with baseline of length ,
we place the origin at the midpoint between the camera centers.
• a point is projected to:– Left image: , – Right image: ,
• Cyclopean coordinates:
Disparity
• Disparity is inverse proportional to depth• Constant disparity constant depth• Larger baseline, more stable reconstruction of depth
(but more occlusions, correspondence is harder)
(Note that disparity is defined in a rectified rig in a cyclopean coordinate frame)
Random dot stereogram
• Depth can be perceived from a random dot pair of images (Julesz)
• Stereo perception is based solely on local information (low level)
Moving random dots
Compared elements
• Pixel intensities• Pixel color• Small window (e.g. or ), often using
normalized correlation to offset gain• Features and edges (less common)• Mini segments
Dynamic programming
• Each pair of epipolar lines is compared independently
• Local cost, sum of unary term and binary term– Unary term: cost of a single match– Binary term: cost of change of disparity (occlusion)
• Analogous to string matching (‘diff’ in Unix)
String matching• Swing → String
S t r i n g
S w i n g
Start
End
String matching• Cost: #substitutions + #insertions + #deletions
S t r i n g
S w i n g
Dynamic Programming
• Shortest path in a grid• Diagonals: constant disparity• Moving along the diagonal – pay unary cost
(cost of pixel match)• Move sideways – pay binary cost, i.e. disparity
change (occlusion, right or left)• Cost prefers fronto-parallel planes. Penalty is
paid for tilted planes
Dynamic Programming
Start
, Complexity?
Probability interpretation: Viterbi algorithm
• Markov chain
• States: discrete set of disparity
• Log probabilities: product sum• Maximum likelihood: minimize sum of negative
logs• Viterbi algorithm: equivalent to shortest path
Markov Random Field
• Graph • In our case: graph is
a 4-connected grid
• Minimize energy of the form
• Interpreted as negative log probabilities
Iterated Conditional Modes (ICM)
• Initialize states (= disparities) for every pixel• Update repeatedly each pixel by the most likely
disparity given the values assigned to its neighbors:
• Markov blanket: the state of a pixel only depends on the states of its immediate neighbors
• Similar to Gauss-Seidel iterations• Slow convergence to bad local minimum
Graph cuts: expansion moves
• Assume is non-negative and is metric:
• We can apply more semi-global moves using minimal s-t cuts
• Converges faster to a better (local) minimum
α-Expansion
• Expansion move allows in one shot each pixel to either change its state to α or to maintain its current state
• We apply expansion moves for all possible disparity values and repeat this to convergence
• At convergence energy is within a scale factor from the global optimum:
where
α-Expansion (1D example)
α-Expansion (1D example)
𝛼
𝛼
α-Expansion (1D example)
𝛼
𝛼
α-Expansion (1D example)
𝐷𝑝(𝛼) 𝐷𝑞 (𝛼)
𝛼
𝛼
𝑉 𝑝𝑞 (𝛼 ,𝛼 )=0
α-Expansion (1D example)
𝛼
𝛼 𝐷𝑝(𝑑𝑝) 𝐷𝑞 (𝑑𝑞)
But what about?
α-Expansion (1D example)
𝛼
𝛼 𝐷𝑝(𝑑𝑝) 𝐷𝑞 (𝑑𝑞)
𝑉 𝑝𝑞(𝑑𝑝 ,𝑑𝑞)
α-Expansion (1D example)
𝛼
𝛼 𝐷𝑝(𝑑𝑝)
𝑉 𝑝𝑞(𝑑𝑝 ,𝛼)𝐷𝑞 (𝛼)
α-Expansion (1D example)
𝛼
𝛼 𝐷𝑞 (𝑑𝑞)
𝑉 𝑝𝑞(𝛼 ,𝑑𝑞)𝐷𝑝(𝛼)
α-Expansion (1D example)
𝛼
𝛼
𝑉 𝑝𝑞(𝛼 ,𝑑𝑞)𝑉 𝑝𝑞(𝑑𝑝 ,𝛼)
𝑉 𝑝𝑞(𝑑𝑝 ,𝑑𝑞)
Such a cut cannot be obtained due to triangle inequality:
Common Metrics
• Potts model:
• Truncated :
• Truncated squared difference is not a metric
Reconstruction with graph-cuts
Original Result Ground truth
top related