Two-view geometry• epipolar points/lines• fundamental matrix• essential matrix• estimation of relative camera pose

Two-view geometry• Also known as epipolar geometry

𝐱𝐱′

𝐞𝐞′

𝐗

𝐥′𝐥

e, e0

l, l0

: epipoles(the images of camera centers)

: epipolar lines(intersections of the plane XCCʼwith the image planes)

C

C 0

Two-view geometry• Specifying , you have

C

C 0

x l0

𝐱 𝐱′𝐞

𝐞′

𝐗

𝐥′

Two-view geometry

A case when the epipole is at infinity

[Hartley-Zisserman03]

x

l0

• Specifying , you havex

l0

Fundamental matrix• The epipolar line specified by is given as

• is a constant 3x3 matrix defined for a pair of views, which is called the fundamental matrix

x

l0

l

0 = Fx

F

𝐱𝐱′

𝐞′

𝐥′

𝛑

CC 0

Suppose a plane that does not pass or

⇡C C 0

Then there is a 2D projective transformation

x

0 / Hx

is given by l0l

0 = [e0]⇥x0

= [e0]⇥Hx

= ([e0]⇥H)x

F = ([e0]⇥H)

A matrix representing vector cross product• For a 3-vector , is defined to be a 3x3 matrix satisfying

• It is given by

• is a skew-symmetric matrix, i.e., • has rank , because

v [v]⇥

v ⇥ x = [v]⇥x

2 [v]⇥v = 0

[v]⇥ =

2

40 �v3 v2v3 0 �v1�v2 v1 0

3

5

[v]⇥[v]⇥

[v]>⇥ = �[v]⇥

Properties of fundamental matrix• gives the relation for the reverse order of views:

• , because• and , because any epipolar line passes ;

thus , and ; this should hold for any• has rank 2, because

• Any matrix of rank 2 can be a fundamental matrix; proof omitted

• The DoF of is seven; 3 x 3 – 1(scaling) – 1(rank=2) = 7 • represents the geometric relation of a pair of uncalibrated

cameras in a complete and concise manner

F>

l

0 = Fx l = F>x0

x

0>Fx = 0x

0>l

0 = 0

Fe = 0 F>e0 = 0 l el>e = 0 x

0>Fe = 0 x

0

F F = [e0]⇥H

F

F

Deriving camera matrices from • Proposition: Given a fundamental matrix of two views, the

camera matrices of the two views are given as

• where is any skew-symmetric matrix and is the epipole:

• Remark: the above gives a projective reconstruction

• Lemma of the proposition: If is a fundamental matrix of two views having camera matrices and , then is skew-symmetric, and vice versa

F

F

P =⇥I 0

⇤P0 =

⇥SF e0

⇤

S e0

e0>F = 0>

FP P0 P0>FP

A> = �ASkew-symmetric:

Deriving camera matrices from • Proof of Lemma: If a matrix is skew-symmetric, then it holds

that for any , and vice versa• Therefore, if is skew-symmetric, for

any , and vice versa • We may set and , resulting in

• Proof of proposition: We only need to show is skew-symmetric; this is done as follows

F

P0>FP

x

>Ax = 0

A

x

X

x

0 / P0Xx / PX x

0>Fx = 0

P0>FP =⇥SF e0

⇤>F⇥I 0

⇤

=

�F>SF 0e0>F 0

�=

�F>SF 00> 0

�

P0>FP

X>P0>FPX = 0

Essential matrix• is called the essential matrix

• gives a two-view relation similar to when the camera(s) are calibrated• Substituting and

into , we have

• Properties:• Denote the coordinate trans. between the two camera coord.

by ; then • DoF of is five (rotation + translation – scaling)• A 3x3 matrix is an essential matrix if and only if the two of its

singular values are identical and the rest is zero; proof omitted

x

0>Fx = 0

E F

E

E

E = [t]⇥R

x / KX̃ = K⇥X Y Z

⇤>

x

0>Fx = X̃

0>K0>FKX̃ = X̃

0>(K0>FK)X̃ = 0

x

0 / K0X̃0 = K0⇥X 0 Y 0 Z 0⇤>

E ⌘ K0>FK

X̃0 = RX̃+ t

Essential matrix• Proof of

K�1x

(t)

(RK�1x)

CC 0

(K0�1x

0)

X(X0)

x

x

0

E = [t]⇥R

The three colored vectors lie on a plane:

Coordinate trans. from 1st

to 2nd camera:

(K0�1x

0)>(t⇥ RK�1x) = 0

x

0>K0�>t⇥ RK�1

x = 0

E = [t]⇥R

x

0>K0�>[t]⇥RK�1

x = 0

X̃0 = RX̃+ t

Essential matrix• If is an essential matrix, then its singular values is

• is of rank 2 just like

• Given , we can obtain and as shown below

• The SVD of is given as

• Suppose the camera matrix of 1st view to be

• Then, 2nd camera matrix should be one of the four matrices:

[s, s, 0]E

E F

E t R

EE = U

2

41

10

3

5 V>

P = K⇥I 0

⇤

P0 = K0⇥UWV> u3

⇤

P0 = K0⇥UWV> �u3

⇤

P0 = K0⇥UW>V> u3

⇤

P0 = K⇥UW>V> �u3

⇤ W ⌘

2

4�1

11

3

5 u3 = U

2

4001

3

5

where

Obtaining R and t from E• Four solutions and relative camera poses

• A single solution is physically possible: 3D points will be in front of both the cameras for only one case (“chirality check”)

1 2

12 1 2

12

chirality

Application of epipolar (two-view) geometry• Finding point correspondences between two images

• Starting from (seven point matches) or (five)• Once epipolar geometry is obtained, you need only to search along

the epipolar line for the corresponding point • Robust correspondence estimation: RANSAC etc.

• Reconstructing 3D structure or camera poses• Projective reconstruction from

• Similarity reconstruction from

EF

x

l0

P =⇥I 0

⇤P0 =

⇥SF e0

⇤F

E

Summary: matrix decompositions• SVD; singular value decomposition

• Any matrix can be decomposed as• and are orthogonal:• is diagonal whose diagonal entries are called singular values• Unique if singular values are sorted in descending order

• QR decomposition• Any matrix can be decomposed as• is orthogonal ( ) and is an upper-right triangular matrix• Decomposition is unique

• Cholesky decomposition• Any positive definite symmetric matrix can be

decomposed as• L is a lower-left triangular matrix

＝

＝

＝

m⇥ n

m⇥m

A = UWV>

m⇥m

V>V = IU>U = IU V

W

A = QR

A

A

Q Q>Q = I R

AA = LL>