3d from photographs: structure from motion · 2017-04-28 · 3d from photographs: structure from...
TRANSCRIPT
![Page 2: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/2.jpg)
Note: in these slides the optical center is placed back to simplify drawing and understanding.
![Page 3: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/3.jpg)
3D model
3D from PhotographsAutomatic
Matching of Images
Camera Calibration
Dense Matching
Photographs
Surface Reconstruction
![Page 4: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/4.jpg)
3D model
3D from PhotographsAutomatic
Matching of Images
Camera Calibration
Dense Matching
Photographs
Surface Reconstruction
![Page 5: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/5.jpg)
Camera Pose Calibration
![Page 6: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/6.jpg)
Camera Pose Calibration
• We have seen methods for estimating the intrinsic matrix K, and the extrinsic matrix G = [R | t] using a calibration pattern:
• DLT
• Zhang’s algorithm
![Page 7: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/7.jpg)
How do we get the camera’s pose without
the pattern?
![Page 8: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/8.jpg)
Camera Pose Calibration
• Let’s assume that:
• We have K for each photograph.
• We have matches between images.
![Page 9: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/9.jpg)
A Two-Camera Example
![Page 10: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/10.jpg)
A Two-Camera ExampleM
m1
m2
C1
C2
Camera 1
Camera 2
![Page 11: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/11.jpg)
A Two-Camera Example: Epipolar Geometry
M
m1
m2
C1
C2
Camera 1
Camera 2
C1C2 is called baseline
![Page 12: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/12.jpg)
A Two-Camera Example: Epipolar Geometry
M
m1
m2
C1
C2
Camera 1
Camera 2
C1C2 is called baseline
e2
e1
![Page 13: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/13.jpg)
A Two-Camera Example: Epipolar Geometry
M
m1
m2
C1
C2
Camera 1
Camera 2
e1m1 is an epipolar line
e2
e1
![Page 14: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/14.jpg)
A Two-Camera Example: Epipolar Geometry
• The epipolar line is defined as
• where an epipole ei is defined as
e1 ' P1 ·C2
e2 ' P2 ·C1
m1(t) = (Q1 ·Q�12 ) · t+ e1
P1[Q1|q1] P2[Q2|q2]
m1(t) ' (Q1 ·Q�12 ) · t+ e1
![Page 15: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/15.jpg)
A Two-Camera Example• We have K1 and K2.
• Let’s assume that G1 is set in the origin and aligned with the reference frame:
Note that we need to estimate both R and t !
G1 = [I|0] ! P1 = K1 ·G1
P2 = K2 · [R|t]
![Page 16: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/16.jpg)
A Two-Camera Example
• To simplify, let’s remove K matrices:
• To points as well:
P 01 = K�1
1 · P1 = [I|0]P 02 = K�1
2 · P2 = [R|t]
m1 = K�11 ·m1
m2 = K�12 ·m2
![Page 17: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/17.jpg)
A Two-Camera Example
• To simplify, let’s remove K matrices:
• To points as well:
P 01 = K�1
1 · P1 = [I|0]P 02 = K�1
2 · P2 = [R|t]
Normalized coordinates
m1 = K�11 ·m1
m2 = K�12 ·m2
![Page 18: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/18.jpg)
A Two-Camera Example• Given the Longuet-Higgins equation, we know that:
• where:
• and:
E = [t]⇥ ·R
[t]⇥ =
2
40 �t3 t2t3 0 �t1�t2 t1 0
3
5
m>2 · E · m1 = 0
![Page 19: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/19.jpg)
The Essential Matrix• E is called the essential matrix, and it is a 3×3
matrix.
• If we have the K matrices and apply the Longuet-Higgins equation we obtain:
• F is called the fundamental matrix:
F = K�>2 · E ·K�1
1
m>1 · F ·m2 = 0
![Page 20: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/20.jpg)
The Essential Matrix: 8-points algorithm
• From:
• We can define a linear system as
• Given enough matches we can solve the system using the SVD. How many do we need? 8 is the minimum, as usual the more the better!
• This method is called 8-points algorithm.
A · b = 0
A =
2
664
(p11)
> ⌦ (p12)
>
(p21)
> ⌦ (p22)
>
. . .(pn
1 )> ⌦ (pn
2 )>
3
775 b = vec(E)
m>2 · E · m1 = 0
A =
2
64(m1
1)> ⌦ (m1
2)>
...(mn
1 )> ⌦ (mn
2 )>
3
75 b = vec(E)
![Page 21: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/21.jpg)
The Essential Matrix• The Kronecker product is defined as
• where A is m×n matrix, and B is a r×s matrix.
A⌦B =
2
6664
a1,1 ·B . . . a1,n ·Ba2,1 ·B . . . a2,n ·B
... . . ....
am,1 ·B . . . am,n ·B
3
7775
![Page 22: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/22.jpg)
The Essential Matrix: Practice
• Typically, we do not estimate E directly, but F. Then, we compute E from F, K1, and K2.
• When estimation F, we use homogenous coordinates for mi, such that ui ∈ [0, w] and vi ∈ [0, h].
• However, solving the linear system with such values we can get numerical instabilities!
![Page 23: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/23.jpg)
The Essential Matrix: Practice
• For removing numerical instabilities, it would be nice to have values with average distance from the origin.
• Given the input n points mi, we compute:
u =1
n
nX
i=1
ui v =1
n
nX
i=1
vi mi =
2
4ui
vi1
3
5
s =1
np2
nX
i=1
p(vi � u)2 + (ui � v)2
p2
![Page 24: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/24.jpg)
The Essential Matrix: Practice
• Finally, we shift and scale all n points using the following:
• We can now solve the linear system!
• Note that this operation, shift and scale, needs to be done for estimating a homography as well!
ui =ui � u
s
vi =vi � v
s
![Page 25: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/25.jpg)
Non-Linear Optimization• As seen before, we need to refine E using a geometric
error, note that we compute E indirectly so we minimize F:
• where d𝜋 is the distance point-line, and n is the number of matched points.
• Again we can solve it with Nelder-Mead method (fminsearch in MATLAB).
argminE
nX
i=1
d⇡(F ·mi1,m
i2)
2 + d⇡(F> ·mi
2,mi1)
2
argminE
nX
i=1
d⇡(F ·mi1,m
i2)
2 + d⇡(F> ·mi
2,mi1)
2
![Page 26: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/26.jpg)
Non-Linear Optimization• As seen before, we need to refine E using a geometric
error, note that we compute E indirectly so we minimize F:
• where d𝜋 is the distance point-line, and n is the number of matched points.
• Again we can solve it with Nelder-Mead method (fminsearch in MATLAB).
argminE
nX
i=1
d⇡(F ·mi1,m
i2)
2 + d⇡(F> ·mi
2,mi1)
2
argminE
nX
i=1
d⇡(F ·mi1,m
i2)
2 + d⇡(F> ·mi
2,mi1)
2
This is a line
![Page 27: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/27.jpg)
Non-Linear Optimization• As seen before, we need to refine E using a geometric
error, note that we compute E indirectly so we minimize F:
• where d𝜋 is the distance point-line, and n is the number of matched points.
• Again we can solve it with Nelder-Mead method (fminsearch in MATLAB).
argminE
nX
i=1
d⇡(F ·mi1,m
i2)
2 + d⇡(F> ·mi
2,mi1)
2
argminE
nX
i=1
d⇡(F ·mi1,m
i2)
2 + d⇡(F> ·mi
2,mi1)
2
This is a line This is a line
![Page 28: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/28.jpg)
Now we have E, and so what?
![Page 29: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/29.jpg)
E Factorization• Once we have estimated E, we would like estimate
R and t to get the pose of the camera:
• As you may notice we have:
• [t]× = S is an anti-symmetric matrix.
• R is orthogonal matrix.
E = [t]⇥ ·R
![Page 30: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/30.jpg)
E Factorization• Given a m×n matrix A, its SVD decomposition is defined
as:
• where:
• U is an m×m orthogonal matrix.
• Σ is a diagonal m×n matrix.
• V* is the conjugate transpose of an orthogonal matrix.
SVD(A) = U · ⌃ · V ⇤
![Page 31: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/31.jpg)
E Factorization• Theorem: “A 3×3 matrix is an essential matrix if
and only if two singular values are equal and the third is zero”.
• This means that:
• Note that
SVD(E) = U · diag(1, 1, 0) · V >
diag(1, 1, 0) = W · Z
W =
2
40 �1 01 0 00 0 1
3
5 Z =
2
40 1 0�1 0 00 0 0
3
5
![Page 32: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/32.jpg)
E Factorization
• Lemma: Given R a rotation matrix, and U and V two orthogonal matrices, we have that:
• R' is still a rotation matrix!
R0 = det(U · V >) · U ·R ·R>
![Page 33: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/33.jpg)
E Factorization
• Given that:
• We can have four possible factorizations of E such that E = S . R:
SVD(E) = U · diag(1, 1, 0) · V >
S = U · (±Z) · U>
R = U ·W · V >or R = U ·W> · V >
![Page 34: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/34.jpg)
E Factorization: The Four Cases
C1 C2 C1 C2
![Page 35: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/35.jpg)
E Factorization: The Four Cases
C1
C2 C1
C2
![Page 36: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/36.jpg)
Which is the correct configuration?
![Page 37: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/37.jpg)
C1 C2
![Page 38: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/38.jpg)
Why?
![Page 39: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/39.jpg)
Both points are seen by the cameras!
![Page 40: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/40.jpg)
How do we find it?
![Page 41: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/41.jpg)
We need to find a case in which all 3D points are in the positive
frustum of both cameras!
![Page 42: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/42.jpg)
Triangulation
![Page 43: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/43.jpg)
Triangulation
• Input: n matched 2D feature points in two images and their P matrices (i.e., we know K, G, and t).
• Output: n 3D points.
![Page 44: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/44.jpg)
Triangulation: Pure Rotational Motion Case
X
Z M
m1 m2
M =
2
664
x
y
z
1
3
775
mi =
2
4ui
vi1
3
5
There is no displacement —> The same lines for intersection —> no 3D
![Page 45: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/45.jpg)
Triangulation: Pure Translational Motion Case
X
Z M
m1 m2
f
b
M =
2
664
x
y
z
1
3
775
mi =
2
4ui
vi1
3
5
![Page 46: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/46.jpg)
Triangulation
• We first fix the frame of reference to one of the two cameras. Then, we know that:
• So, we can obtain:
(f
z
= �u1x
f
z
= � u2x�b
z =b · f
u2 � u1
![Page 47: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/47.jpg)
Triangulation: The General Case
M
m1
m2
C1
C2
Camera 1
Camera 2
![Page 48: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/48.jpg)
Similar to DLT but different!
![Page 49: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/49.jpg)
Triangulation: Eigen Method
P =
2
4p>1
p>2
p>3
3
5
8<
:u = p>
1 ·Mp>
3 ·M
v = p>2 ·M
p>3 ·M
![Page 50: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/50.jpg)
Triangulation: Eigen Method
P =
2
4p>1
p>2
p>3
3
5
8<
:u = p>
1 ·Mp>
3 ·M
v = p>2 ·M
p>3 ·M
known!
![Page 51: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/51.jpg)
Triangulation: Eigen Method
P =
2
4p>1
p>2
p>3
3
5
8<
:u = p>
1 ·Mp>
3 ·M
v = p>2 ·M
p>3 ·M
known! unknown!
![Page 52: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/52.jpg)
• This leads to:
• Given that:
Triangulation: Eigen Method
((p1 � u · p1)> ·M = 0
(p2 � v · p1)> ·M = 0
Pi =
2
4(pi
1)>
(pi2)
>
(pi3)
>
3
5 mi =
2
4ui
vi1
3
5
![Page 53: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/53.jpg)
Triangulation: Eigen Method• We obtain:
• For l cameras, this leads to:
2
664
(p11 � u1 · p1
3)>
(p12 � v1 · p1
3)>
(p21 � u2 · p2
3)>
(p22 � v2 · p2
3)>
3
775 ·M = 0
2
666664
(p11 � u1 · p1
3)>
(p12 � u1 · p1
3)>
...(pl
1 � u1 · pl3)
>
(pl2 � u1 · pl
3)>
3
777775·M = 0
![Page 54: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/54.jpg)
Triangulation: Eigen Method• Again, we solve this linear system using SVD; i.e.,
the kernel of V.
• Again, we minimized an algebraic error without a geometric meaning!
• Again, we use this initial solution for a non-linear method that minimizes a geometric error:
argminM
nX
j=1
✓uj �
(pj1)
> ·M(pj
3)> ·M
◆2
+
✓vj �
(pj2)
> ·M(pj
3)> ·M
◆2l
![Page 55: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/55.jpg)
Triangulation: Example
C1
C2
Camera 1
Camera 2
Projected MCorner mi
![Page 56: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/56.jpg)
Triangulation: Example
C1
C2
Camera 1
Camera 2
Projected MCorner mi
![Page 57: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/57.jpg)
Structure From
Motion
![Page 58: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/58.jpg)
Structure From Motion
• Input: n matched points (corners computed with Harris algorithm) between two images, and K for all cameras.
• Output: n 3D points, and G for the two cameras.
![Page 59: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/59.jpg)
Structure From Motion
• The algorithm is:
• Estimation of E.
• Factorization of E to obtain G.
• Triangulation of the n matched points using P1 and P2.
![Page 60: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/60.jpg)
So far we have only used only a two cameras!
![Page 61: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/61.jpg)
Structure From Motion: Multi-View
• We compute G for different views using the previous algorithm.
• We use a reference view for computing the different G matrices. For example, we can use the first image.
![Page 62: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/62.jpg)
Structure From Motion: Multi-View Example
P1
P2P3
![Page 63: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/63.jpg)
Structure From Motion: Multi-View Example
P1
P2 ReferenceP3
![Page 64: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/64.jpg)
Structure From Motion: Multi-View Example
P1
P2P3 Reference
We compute G12
![Page 65: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/65.jpg)
Structure From Motion: Multi-View Example
P1
P2P3 Reference
We compute G13
![Page 66: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/66.jpg)
Hang on, was it a good reference the one before?
![Page 67: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/67.jpg)
Hang on, what can possibly go wrong?
![Page 68: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/68.jpg)
We are accumulating error, and we will drift
from the solution!
![Page 69: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/69.jpg)
Structure From Motion: Multi-View
• To avoid error accumulation, we minimize in a non-linear way at the same time both poses estimation and 3D points generation:
• where d is the Euclidian distance, l is the number of cameras, and n is the number of points.
arg minRi,ti,Mj
lX
i=1
nX
j=1
d
✓Ki · [Ri|ti] ·Mj ,mj
i
◆2
![Page 70: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/70.jpg)
Structure From Motion: Multi-View
• Typically, the method is difficult to minimize as a whole thing. This is because there are many parameters to minimize.
• A two-step approach:
• First, minimize (or viceversa) all extrinsic parameters (G) without modifying the 3D points.
• Then, minimize (or viceversa) 3D points coordinates without modifying G.
![Page 71: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/71.jpg)
Structure From Motion: Example
![Page 72: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/72.jpg)
Structure From Motion: Example
![Page 73: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/73.jpg)
Structure From Motion: Example
![Page 74: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/74.jpg)
Structure From Motion: Example
![Page 75: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/75.jpg)
Structure From Motion: Multi-View
• To obtain something of interesting:
• we need to feed into the system hundreds of images.
• we need to manage thousands of features (corners)!
• Even the two-step approach would struggle a bit.
![Page 76: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/76.jpg)
Structure From Motion: Multi-View
• To make the problem computational tractable, we can notice this:
1
2 4
3 5
6 8
7 9
A B C D
![Page 77: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/77.jpg)
Structure From Motion: Multi-View
• To make the problem computational tractable, we can notice this:
1
2 4
3 5
6 8
7 9
A B C D
![Page 78: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/78.jpg)
Structure From Motion: Multi-View
• To make the problem computational tractable, we can notice this:
1
2 4
3 5
6 8
7 9
A B C D
![Page 79: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/79.jpg)
Structure From Motion: Multi-View
• To make the problem computational tractable, we can notice this:
1
2 4
3 5
6 8
7 9
A B C D
![Page 80: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/80.jpg)
Structure From Motion: Multi-View
• The idea is to divide the scene into clusters.
• For each cluster we compute SfM.
• We combine all 3D reconstructions and camera poses together.
![Page 81: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/81.jpg)
Structure From Motion: Conclusions
• Advantages:
• It requires only photographs/videos: cheap and fast.
• We can recover color information from photographs!
• Disadvantages:
• The output model may be skewed; it is hard to keep two things going at the same time (3D points and cameras’ poses).
• We do not have a scale!
![Page 82: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/82.jpg)
One thing…
![Page 83: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/83.jpg)
![Page 84: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/84.jpg)
RANSAC• Random sample consensus (RANSAC) is an
iterative method for estimating the parameters of a model in a robust way.
• The main idea is to get a subset of the set of samples and to estimate the model with this subset:
• We estimate the model using the best subset of samples!
![Page 85: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/85.jpg)
RANSAC
• Input: a set of n samples S, and a model 𝜋.
• Output: parameters, P, for the model 𝜋.
![Page 86: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/86.jpg)
RANSAC• e = +∞ and Sb = ∅
• For each iteration:
• Si ⊂ S where Si is random.
• Estimate Pi for 𝜋 using Si
• Compute the error ei for Pi
• if ei < e then
• e = ei and Sb = Si
![Page 87: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/87.jpg)
RANSAC: Example𝜋: a straight line
Si and Sb
![Page 88: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/88.jpg)
RANSAC: Example
Iteration 0 e = +∞
𝜋: a straight line
Si and Sb
![Page 89: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/89.jpg)
RANSAC: Example
Iteration 1 e = +∞
𝜋: a straight line
Si and Sb
![Page 90: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/90.jpg)
RANSAC: Example
Iteration 1 e = +∞
𝜋: a straight line
Si and Sb
![Page 91: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/91.jpg)
RANSAC: Example
Iteration 1 e = +∞
ei = 10
𝜋: a straight line
Si and Sb
![Page 92: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/92.jpg)
RANSAC: Example
Iteration 2 e = 10
𝜋: a straight line
Si and Sb
![Page 93: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/93.jpg)
RANSAC: Example
Iteration 2 e = 10
𝜋: a straight line
Si and Sb
![Page 94: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/94.jpg)
RANSAC: Example
Iteration 2 e = 10
𝜋: a straight line
Si and Sb
![Page 95: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/95.jpg)
RANSAC: Example
Iteration 2 e = 10
ei = 1𝜋: a straight line
Si and Sb
![Page 96: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/96.jpg)
RANSAC: Example
Iteration 2 e = 1
𝜋: a straight line
Si and Sb
![Page 97: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/97.jpg)
and we continue for n iterations…
![Page 98: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/98.jpg)
how many?
![Page 99: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/99.jpg)
RANSAC: Iterations
• n has to be large; i.e., we need to have at least one subset containing only inliers Sinliers:
• We are interested for P = 1.
P (|Si| = c) = 1�✓1�
✓1� |S
outliers
||S|
◆c◆n
Si ✓ Sinliers
![Page 100: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/100.jpg)
RANSAC• When do we need to use it?
• Estimation of the fundamental/essential matrix.
• Estimation of a homography in the general case.
• When we do not:
• DLT and Zhang’s algorithm: corners are extracted in an accurate way using a calibration pattern!
![Page 101: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/101.jpg)
RANSAC: Fundamental Matrix Estimation• The algorithm is modified a bit:
• We count the inliers of each set given a threshold:
• terr takes into account this constraint:
• If we have a set with more inliers of the previous one it is accepted.
• We compute the F using only the inliers!
m>1 · F ·m2 = 0
![Page 102: 3D from Photographs: Structure From Motion · 2017-04-28 · 3D from Photographs: Structure From Motion Dr Francesco Banterle francesco.banterle@isti.cnr.it. Note: ... 3D model 3D](https://reader034.vdocuments.us/reader034/viewer/2022043020/5f3c206ca7e5766b384657d7/html5/thumbnails/102.jpg)
that’s all folks!