local features - khu.ac.krcvlab.khu.ac.kr/cvlecture16.pdf · 2018. 11. 12. · overview of keypoint...
TRANSCRIPT
![Page 1: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/1.jpg)
Local Features
Computer Vision
![Page 2: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/2.jpg)
Correspondence Problem & Keypoint
• Correspondence: matching points, patches, edges, or regions across images
≈
![Page 3: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/3.jpg)
Overview of Keypoint Matching
K. Grauman, B. Leibe
Af Bf
A1
A2 A3
Tffd BA ),(
1. Find a set of
distinctive key-
points
3. Extract and
normalize the
region content
2. Define a region
around each
keypoint
4. Compute a local
descriptor from the
normalized region
5. Match local
descriptors
![Page 4: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/4.jpg)
![Page 5: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/5.jpg)
![Page 6: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/6.jpg)
![Page 7: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/7.jpg)
![Page 8: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/8.jpg)
![Page 9: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/9.jpg)
![Page 10: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/10.jpg)
![Page 11: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/11.jpg)
DoG – Efficient Computation
• Computation in Gaussian scale pyramid
K. Grauman, B. Leibe
s
Original image 4
1
2s
Sampling with
step s4 =2
s
s
s
![Page 12: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/12.jpg)
Find local maxima in position-scale space of Difference-of-Gaussian
K. Grauman, B. Leibe
)()( ss yyxx LL
s
s2
s3
s4
s5
List of (x, y, s)
![Page 13: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/13.jpg)
Results: Difference-of-Gaussian
K. Grauman, B. Leibe
![Page 14: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/14.jpg)
SIFT
• Scale-space extrema detection
• Keypoint localization
• Orientation assignment
• Keypoint descriptor
![Page 15: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/15.jpg)
Scale-space extrema detection
• Find the points, whose surrounding patches (with some scale) are distinctive
• An approximation to the scale-normalized Laplacian of Gaussian
Maxima and minima in a
3*3*3 neighborhood
![Page 16: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/16.jpg)
Eliminating edge points
![Page 17: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/17.jpg)
Orientation assignment
• Assign an orientation to each keypoint, the keypoint descriptor can be represented relative to this orientation and therefore achieve invariance to image rotation
• Compute magnitude and orientation on the Gaussian smoothed images
![Page 18: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/18.jpg)
T. Tuytelaars, B. Leibe
Orientation normalization
• Compute orientation histogram
• Select dominant orientation
• Normalize: rotate to fixed orientation
0 2 p
![Page 19: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/19.jpg)
Reduce effect of illumination • 128-dim vector normalized to 1
• Threshold gradient magnitudes to avoid excessive influence of high gradients
– after normalization, clamp gradients >0.2
– renormalize
![Page 20: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/20.jpg)
A result
![Page 21: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/21.jpg)
Structure from Motion
Computer Vision
![Page 22: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/22.jpg)
![Page 23: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/23.jpg)
![Page 24: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/24.jpg)
Depth from disparity
x’ z
f
x
C C’
X
baseline
f
(X – X’) / f = baseline / z X – X’ = (baseline*f) / z z = (baseline*f) / (X – X’)
![Page 25: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/25.jpg)
General case, with calibrated cameras
• The two cameras need not have parallel optical axes.
Vs.
![Page 26: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/26.jpg)
• Given p in left image, where can corresponding point p’ be?
Stereo correspondence constraints
![Page 27: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/27.jpg)
Stereo correspondence constraints
![Page 28: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/28.jpg)
Geometry of two views constrains where the
corresponding pixel for some image point in the first view
must occur in the second view.
• It must be on the line carved out by a plane
connecting the world point and optical centers.
Epipolar constraint
![Page 29: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/29.jpg)
• Epipolar Plane
Epipole
Epipolar Line
Baseline
Epipolar geometry
Epipole
![Page 30: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/30.jpg)
• Baseline: line joining the camera centers
• Epipole: point of intersection of baseline with image plane
• Epipolar plane: plane containing baseline and world point
• Epipolar line: intersection of epipolar plane with the image plane
• All epipolar lines intersect at the epipole
• An epipolar plane intersects the left and right image planes in epipolar lines
Epipolar geometry: terms
Why is the epipolar constraint useful?
![Page 31: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/31.jpg)
Epipolar constraint
This is useful because it reduces the correspondence
problem to a 1D search along an epipolar line.
Image from Andrew Zisserman
![Page 32: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/32.jpg)
Example
![Page 33: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/33.jpg)
What do the epipolar lines look like?
Ol Or
Ol Or
1.
2.
![Page 34: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/34.jpg)
Example: converging cameras
Figure from Hartley & Zisserman
![Page 35: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/35.jpg)
Figure from Hartley & Zisserman
Example: parallel cameras
Where are the
epipoles?
![Page 36: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/36.jpg)
Example: Forward motion
What would the epipolar lines look like if the camera moves directly forward?
![Page 37: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/37.jpg)
e
e’
Example: Forward motion
Epipole has same coordinates in both
images.
Points move along lines radiating from e:
“Focus of expansion”
![Page 38: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/38.jpg)
Fundamental matrix
• Let p be a point in left image, p’ in right image
• Epipolar relation – p maps to epipolar line l’ – p’ maps to epipolar line l
• Epipolar mapping described by a 3x3 matrix F
• It follows that
l’ l
p p’
![Page 39: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/39.jpg)
Fundamental matrix
• This matrix F is called – the “Essential Matrix”
• when image intrinsic parameters are known
– the “Fundamental Matrix” • more generally (uncalibrated case)
• Can solve for F from point correspondences
– Each (p, p’) pair gives one linear equation in entries of F
– F has 9 entries, but really only 7 or 8 degrees of freedom. – With 8 points it is simple to solve for F, but it is also
possible with 7. See Marc Pollefey’s notes for a nice tutorial
![Page 40: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/40.jpg)
Active stereo with structured light
• Project “structured” light patterns onto the object – Simplifies the correspondence problem
– Allows us to use only one camera
camera
projector
L. Zhang, B. Curless, and S. M. Seitz. Rapid Shape Acquisition Using Color Structured
Light and Multi-pass Dynamic Programming. 3DPVT 2002
![Page 41: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/41.jpg)
Structured light
https://www.youtube.com/watch?v=jmffVdU
JYv4
![Page 42: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/42.jpg)
Kinect: Structured infrared light
http://bbzippo.wordpress.com/2010/11/28/kinect-in-infrared/
![Page 43: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/43.jpg)
Photometry stereo
https://www.youtube.com/watch?v=qlq3n5r1Xy0
Helmholtz Stereopsis
![Page 44: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/44.jpg)
Structure from Motion
![Page 45: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/45.jpg)
Structure from motion
• Given a set of corresponding points in two or more images, compute the camera parameters and the 3D point coordinates
Camera 1 Camera 2 Camera 3
R1,t1 R2,t2 R3,t3
? ? ? Slide credit:
Noah Snavely
?
![Page 46: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/46.jpg)
https://www.youtube.com/watch?v=i7ierVk
XYa8
![Page 47: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/47.jpg)
Structure from motion ambiguity
• If we scale the entire scene by some factor k and, at the same time, scale the camera matrices by the factor of 1/k, the projections of the scene points in the image remain exactly the same:
It is impossible to recover the absolute scale of the scene!
)(1
XPPXx kk
![Page 48: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/48.jpg)
How do we know the scale of image content?
![Page 49: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/49.jpg)
![Page 50: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/50.jpg)
Bundle adjustment
• Non-linear method for refining structure and motion
• Minimizing reprojection error
2
1 1
,),(
m
i
n
j
jiijDE XPxXP
x1j
x2j
x3j
Xj
P1
P2
P3
P1Xj
P2Xj
P3Xj
![Page 51: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/51.jpg)
Photo synth
Noah Snavely, Steven M. Seitz, Richard Szeliski, "Photo tourism: Exploring
photo collections in 3D," SIGGRAPH 2006
![Page 52: Local Features - khu.ac.krcvlab.khu.ac.kr/CVLecture16.pdf · 2018. 11. 12. · Overview of Keypoint Matching K. Grauman, B. Leibe f A f B A 1 A 2 A 3 d( f A, f B) T 1. Find a set](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc3fa6b3ee56447305ce520/html5/thumbnails/52.jpg)
https://www.youtube.com/watch?v=IgBQC
oEfiMs