2d alignment: linear least squares
TRANSCRIPT
![Page 1: 2D Alignment: Linear Least Squares](https://reader030.vdocuments.us/reader030/viewer/2022012804/61bd26da61276e740b0fe3cf/html5/thumbnails/1.jpg)
2D Alignment: Linear Least Squares16-385 Computer Vision (Kris Kitani)
Carnegie Mellon University
![Page 2: 2D Alignment: Linear Least Squares](https://reader030.vdocuments.us/reader030/viewer/2022012804/61bd26da61276e740b0fe3cf/html5/thumbnails/2.jpg)
Extract features from an image …
what do we do next?
![Page 3: 2D Alignment: Linear Least Squares](https://reader030.vdocuments.us/reader030/viewer/2022012804/61bd26da61276e740b0fe3cf/html5/thumbnails/3.jpg)
Feature matching (object recognition, 3D reconstruction, augmented reality, image stitching)
How do we estimate the transformation?
![Page 4: 2D Alignment: Linear Least Squares](https://reader030.vdocuments.us/reader030/viewer/2022012804/61bd26da61276e740b0fe3cf/html5/thumbnails/4.jpg)
{xi,x0i}
Given a set of matched feature points
x
0 = f(x;p)
and a transformation
Find the best estimate of
p
parameterstransformation function
point in one image
point in the other image
![Page 5: 2D Alignment: Linear Least Squares](https://reader030.vdocuments.us/reader030/viewer/2022012804/61bd26da61276e740b0fe3cf/html5/thumbnails/5.jpg)
Model fitting
![Page 6: 2D Alignment: Linear Least Squares](https://reader030.vdocuments.us/reader030/viewer/2022012804/61bd26da61276e740b0fe3cf/html5/thumbnails/6.jpg)
Recover the transformation
f(x,y) g(x,y)
Given f and g, how would you recover the transform T? (user will provide correspondences)
How many do we need?
T
![Page 7: 2D Alignment: Linear Least Squares](https://reader030.vdocuments.us/reader030/viewer/2022012804/61bd26da61276e740b0fe3cf/html5/thumbnails/7.jpg)
Translation
f(x,y) g(x,y)
T
• How many Degrees of Freedom? • How many correspondences needed? • What is the transformation matrix?
![Page 8: 2D Alignment: Linear Least Squares](https://reader030.vdocuments.us/reader030/viewer/2022012804/61bd26da61276e740b0fe3cf/html5/thumbnails/8.jpg)
Euclidean
f(x,y) g(x,y)
T
• How many Degrees of Freedom? • How many correspondences needed for translation+rotation? • What is the transformation matrix?
![Page 9: 2D Alignment: Linear Least Squares](https://reader030.vdocuments.us/reader030/viewer/2022012804/61bd26da61276e740b0fe3cf/html5/thumbnails/9.jpg)
Affine
f(x,y) g(x,y)
T
• How many Degrees of Freedom? • How many correspondences needed for affine? • What is the transformation matrix?
![Page 10: 2D Alignment: Linear Least Squares](https://reader030.vdocuments.us/reader030/viewer/2022012804/61bd26da61276e740b0fe3cf/html5/thumbnails/10.jpg)
Projective
f(x,y) g(x,y)
T
• How many Degrees of Freedom? • How many correspondences needed for projective? • What is the transformation matrix?
![Page 11: 2D Alignment: Linear Least Squares](https://reader030.vdocuments.us/reader030/viewer/2022012804/61bd26da61276e740b0fe3cf/html5/thumbnails/11.jpg)
Suppose we have two triangles: ABC and DEF.
What transformation will map A to D, B to E, and C to F?
How can we get the parameters?
A
D
B E
F
C
![Page 12: 2D Alignment: Linear Least Squares](https://reader030.vdocuments.us/reader030/viewer/2022012804/61bd26da61276e740b0fe3cf/html5/thumbnails/12.jpg)
Estimate transformation parameters using
Linear least squares
![Page 13: 2D Alignment: Linear Least Squares](https://reader030.vdocuments.us/reader030/viewer/2022012804/61bd26da61276e740b0fe3cf/html5/thumbnails/13.jpg)
{xi,x0i}
Given a set of matched feature points
x
0 = f(x;p)
and a transformation
Find the best estimate of
p
parameterstransformation function
point in one image
point in the other image
![Page 14: 2D Alignment: Linear Least Squares](https://reader030.vdocuments.us/reader030/viewer/2022012804/61bd26da61276e740b0fe3cf/html5/thumbnails/14.jpg)
x1
x2
x3
x
01
x
02
x
03
Given point correspondences
![Page 15: 2D Alignment: Linear Least Squares](https://reader030.vdocuments.us/reader030/viewer/2022012804/61bd26da61276e740b0fe3cf/html5/thumbnails/15.jpg)
x1
x2
x3
x
01
x
02
x
03
f
(xi;p)
How can you solve for the transformation?
Given point correspondences
![Page 16: 2D Alignment: Linear Least Squares](https://reader030.vdocuments.us/reader030/viewer/2022012804/61bd26da61276e740b0fe3cf/html5/thumbnails/16.jpg)
ELS =X
i
kf(xi;p)� x
0ik2
Least Squares Error
x1
x2
x3
x
01
x
02
x
03
f
(xi;p)
![Page 17: 2D Alignment: Linear Least Squares](https://reader030.vdocuments.us/reader030/viewer/2022012804/61bd26da61276e740b0fe3cf/html5/thumbnails/17.jpg)
ELS =X
i
kf(xi;p)� x
0ik2
What is this?
Least Squares Error
What is this?
x1
x2
x3
x
01
x
02
x
03
f
(xi;p)
![Page 18: 2D Alignment: Linear Least Squares](https://reader030.vdocuments.us/reader030/viewer/2022012804/61bd26da61276e740b0fe3cf/html5/thumbnails/18.jpg)
ELS =X
i
kf(xi;p)� x
0ik2
What is this?
Least Squares Error
What is this?
x1
x2
x3
x
01
x
02
x
03
f
(xi;p)
What is this?
![Page 19: 2D Alignment: Linear Least Squares](https://reader030.vdocuments.us/reader030/viewer/2022012804/61bd26da61276e740b0fe3cf/html5/thumbnails/19.jpg)
ELS =X
i
kf(xi;p)� x
0ik2
predicted location
Least Squares Error
measured location
x1
x2
x3
x
01
x
02
x
03
f
(xi;p)
Euclidean (L2) normEuclidean (L2) norm
squared!
||x|| =q
x
21 + x
22 + · · ·+ x
2n
![Page 20: 2D Alignment: Linear Least Squares](https://reader030.vdocuments.us/reader030/viewer/2022012804/61bd26da61276e740b0fe3cf/html5/thumbnails/20.jpg)
ELS =X
i
kf(xi;p)� x
0ik2
Residual (projection error)
x1
x2
x3
x
01
x
02
x
03
f
(xi;p)
Least Squares Error
![Page 21: 2D Alignment: Linear Least Squares](https://reader030.vdocuments.us/reader030/viewer/2022012804/61bd26da61276e740b0fe3cf/html5/thumbnails/21.jpg)
ELS =X
i
kf(xi;p)� x
0ik2
What is the free variable? What do we want to optimize?
x1
x2
x3
x
01
x
02
x
03
f
(xi;p)
Least Squares Error
![Page 22: 2D Alignment: Linear Least Squares](https://reader030.vdocuments.us/reader030/viewer/2022012804/61bd26da61276e740b0fe3cf/html5/thumbnails/22.jpg)
x1
x2
x3
x
01
x
02
x
03
f
(xi;p)
p̂ = argminp
X
i
kf(xi;p)� x
0ik2
Find parameters that minimize squared error
![Page 23: 2D Alignment: Linear Least Squares](https://reader030.vdocuments.us/reader030/viewer/2022012804/61bd26da61276e740b0fe3cf/html5/thumbnails/23.jpg)
General form of linear least squares
ELLS =X
i
|aix� bi|2
= kAx� bk2 (matrix form)
(Warning: change of notation. x is a vector of parameters!)
This function is quadratic. How do you find the root of a quadratic?
![Page 24: 2D Alignment: Linear Least Squares](https://reader030.vdocuments.us/reader030/viewer/2022012804/61bd26da61276e740b0fe3cf/html5/thumbnails/24.jpg)
General form of linear least squares
ELLS =X
i
|aix� bi|2
= kAx� bk2 (matrix form)
(Warning: change of notation. x is a vector of parameters!)
ELLS = x
>(A>A)x� 2x>(A>b) + kbk2
Expand
Take derivative, set to zero (A>A)x = A>
b
(normal equation)
Minimize the error:
Solve for xx = (A>A)�1A>
b
![Page 25: 2D Alignment: Linear Least Squares](https://reader030.vdocuments.us/reader030/viewer/2022012804/61bd26da61276e740b0fe3cf/html5/thumbnails/25.jpg)
x
0 = Mx
x
0
y
0
�=
p1 p2 p3
p4 p5 p6
�2
4x
y
1
3
5
x
0
y
0
�=
x y 1 0 0 00 0 0 x y 1
� 2
6666664
p1p2p3p4p5p6
3
7777775
x
0
y
0
� x y 1 0 0 00 0 0 x y 1
�
x
0
y
0
� x y 1 0 0 00 0 0 x y 1
�
For the Affine transformation
Vectorize transformation parameters
......
A xbNotation in general form
x
0 = f(x;p)
![Page 26: 2D Alignment: Linear Least Squares](https://reader030.vdocuments.us/reader030/viewer/2022012804/61bd26da61276e740b0fe3cf/html5/thumbnails/26.jpg)
Linear
least squares
estimation
only works
when the
transform function
is
?
![Page 27: 2D Alignment: Linear Least Squares](https://reader030.vdocuments.us/reader030/viewer/2022012804/61bd26da61276e740b0fe3cf/html5/thumbnails/27.jpg)
Linear
least squares
estimation
only works
when the
transform function
is
linear!
![Page 28: 2D Alignment: Linear Least Squares](https://reader030.vdocuments.us/reader030/viewer/2022012804/61bd26da61276e740b0fe3cf/html5/thumbnails/28.jpg)
Also
doesn’t
deal well
with
outliers