feature matching and ransac - umass amherstelm/teaching/ppt/370/370_10...feature matching •...
TRANSCRIPT
![Page 1: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/1.jpg)
Feature Matching and RANSAC
15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from
Steve Seitz and Rick Szeliski
© Krister Parmstrand
![Page 2: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/2.jpg)
Feature matching
?
![Page 3: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/3.jpg)
SIFT keypoints On the previous slide, the red points are all of the SIFT
keypoints.
![Page 4: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/4.jpg)
Feature matching
?
![Page 5: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/5.jpg)
RANSAC
![Page 6: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/6.jpg)
Color coding of points on previous slide • Red points
• points without a “good” match in the other image • In this image, the goodness of the match is decided by
looking at the ratio of the distances to the second nearest neighbor and first nearest neighbor. If this ration is high (above some threshold), it is considered a “good” match.
• Blue points • These are points with a “good” match in which the
match was wrong, meaning it connected two points that did not actually correspond in the world.
• Yellow points • These are correct matches. We need to run RANSAC
until it randomly picked 4 yellow points from among the blue and yellow points (the matches estimated to be “good”).
![Page 7: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/7.jpg)
Feature matching • Exhaustive search
• for each feature in one image, look at all the other features in the other image(s)
• Hashing • compute a short descriptor from each feature vector, or hash
longer descriptors (randomly)
• Nearest neighbor techniques • kd-trees and their variants
![Page 8: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/8.jpg)
What about outliers?
?
![Page 9: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/9.jpg)
Feature-space outlier rejection Let’s not match all features, but only these that have “similar enough” matches?
How can we do it? • SSD(patch1,patch2) < threshold • How to set threshold?
![Page 10: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/10.jpg)
Feature-space outlier rejection A better way [Lowe, 1999]:
• 1-NN: SSD of the closest match • 2-NN: SSD of the second-closest match • Look at how much better 1-NN is than 2-NN, e.g. 1-NN/2-NN • That is, is our best match so much better than the rest?
![Page 11: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/11.jpg)
Feature-space outliner rejection
Can we now compute H from the blue points? • No! Still too many outliers… • What can we do?
![Page 12: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/12.jpg)
Matching features
What do we do about the “bad” matches?
![Page 13: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/13.jpg)
RAndom SAmple Consensus
Select one match, count inliers
![Page 14: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/14.jpg)
RAndom SAmple Consensus
Select one match, count inliers
![Page 15: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/15.jpg)
Least squares fit
Find “average” translation vector
![Page 16: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/16.jpg)
RANSAC for estimating homography
RANSAC loop: 1. Select four feature pairs (at random) 2. Compute homography H (exact) 3. Compute inliers where SSD(pi’, H pi) < ε 4. Keep largest set of inliers 5. Re-compute least-squares H estimate on all of the
inliers
![Page 17: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/17.jpg)
RANSAC
![Page 18: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/18.jpg)
18
Under what conditions can you know where to translate each point of image A to where it would appear in camera B (with calibrated cameras), knowing nothing about image depths?
Camera A Camera B
![Page 19: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/19.jpg)
19
(a) camera rotation
![Page 20: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/20.jpg)
20
and (b) imaging a planar surface
![Page 21: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/21.jpg)
Aligning images: translation?
Translations are not enough to align the images
left on top right on top
![Page 22: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/22.jpg)
Homography example
![Page 23: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/23.jpg)
UMass Lab for Perceptual Robotics
![Page 24: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/24.jpg)
Example: Recognising Panoramas
M. Brown and D. Lowe, University of British Columbia
![Page 25: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/25.jpg)
Why “Recognising Panoramas”?
![Page 26: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/26.jpg)
Why “Recognising Panoramas”?
1D Rotations (θ) • Ordering ⇒ matching images
![Page 27: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/27.jpg)
Why “Recognising Panoramas”?
1D Rotations (θ) • Ordering ⇒ matching images
![Page 28: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/28.jpg)
Why “Recognising Panoramas”?
1D Rotations (θ) • Ordering ⇒ matching images
![Page 29: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/29.jpg)
Why “Recognising Panoramas”?
• 2D Rotations (θ, φ) – Ordering ⇒ matching images
1D Rotations (θ) • Ordering ⇒ matching images
![Page 30: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/30.jpg)
Why “Recognising Panoramas”?
1D Rotations (θ) • Ordering ⇒ matching images
• 2D Rotations (θ, φ) – Ordering ⇒ matching images
![Page 31: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/31.jpg)
Why “Recognising Panoramas”?
1D Rotations (θ) • Ordering ⇒ matching images
• 2D Rotations (θ, φ) – Ordering ⇒ matching images
![Page 32: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/32.jpg)
Why “Recognising Panoramas”?
![Page 33: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/33.jpg)
Overview Feature Matching Image Matching Bundle Adjustment Multi-band Blending Results Conclusions
![Page 34: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/34.jpg)
RANSAC for Homography
![Page 35: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/35.jpg)
RANSAC for Homography
![Page 36: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/36.jpg)
RANSAC for Homography
![Page 37: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/37.jpg)
Finding the panoramas
![Page 38: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/38.jpg)
Finding the panoramas
![Page 39: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/39.jpg)
Finding the panoramas
![Page 40: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/40.jpg)
Finding the panoramas
![Page 41: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/41.jpg)
Bundle Adjustment New images initialised with rotation, focal length of best
matching image
![Page 42: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/42.jpg)
Multi-band Blending Burt & Adelson 1983
• Blend frequency bands over range ∝ λ
![Page 43: Feature Matching and RANSAC - UMass Amherstelm/Teaching/ppt/370/370_10...Feature matching • Exhaustive search • for each feature in one image, look at all the other features in](https://reader030.vdocuments.us/reader030/viewer/2022040509/5e53c0b3396ec75a616485c5/html5/thumbnails/43.jpg)
Results