lecture 2 camera models - silvio savarese€¦ · silvio savarese lecture 2 - 14-jan-14...

Lecture 2 -Silvio Savarese 14-Jan-14

Professor Silvio Savarese

Computational Vision and Geometry Lab

Lecture 2Camera Models

Lecture 2 -Silvio Savarese 14-Jan-14

Prerequisites: any questions?

This course requires knowledge of linear algebra, probability, statistics, machine learning and computer vision, as well as decent programming skills. Though not an absolute requirement, it is encouraged and preferred that you have at least taken either CS221 or CS229 or CS131A or have equivalent knowledge.

Topics such as linear filters, feature detectors and descriptors, low level segmentation, tracking, optical flow, clustering and PCA/LDA techniques for recognition won’t be covered in CS231

We will provide links to background material related to CS131A (or discuss during TA sessions) so students can refresh or study those topics if needed

We will leverage concepts from machine learning (CS229) (e.g., SVM, basic Bayesian inference, clustering, etc…) which we won’t cover in this class either. Again, we will supply links to related material for background reading.

Announcements

Lecture 2 -Silvio Savarese 14-Jan-14

Next TA session: Fridays from 2:15-3:05pm

Announcements

Lecture 2 -Silvio Savarese 14-Jan-14

• Pinhole cameras• Cameras & lenses• The geometry of pinhole cameras• Other camera models

Lecture 2Camera Models

Reading: [FP] Chapter 1 “Cameras” [FP] Chapter 2 “Geometric Camera Models”[HZ] Chapter 6 “Camera Models”

Some slides in this lecture are courtesy to Profs. J. Ponce, S. Seitz, F-F Li

How do we see the world?

• Let’s design a camera– Idea 1: put a piece of film in front of an object

– Do we get a reasonable image?

Pinhole camera

• Add a barrier to block off most of the rays

– This reduces blurring

– The opening known as the aperture

Milestones:

• Leonardo da Vinci (1452-1519):

first record of camera obscura

Some history…

Milestones:

• Leonardo da Vinci (1452-1519):

first record of camera obscura

• Johann Zahn (1685): first

portable camera

Some history…

Photography (Niepce, “La

Table Servie,” 1822)

Milestones:

• Leonardo da Vinci (1452-1519):

first record of camera obscura

• Johann Zahn (1685): first

portable camera

• Joseph Nicephore Niepce (1822):

first photo - birth of photography

Some history…

Photography (Niepce, “La

Table Servie,” 1822)

Milestones:

• Leonardo da Vinci (1452-1519):

first record of camera obscura

• Johann Zahn (1685): first

portable camera

• Joseph Nicephore Niepce (1822):

first photo - birth of photography

• Daguerréotypes (1839)

• Photographic Film (Eastman, 1889)

• Cinema (Lumière Brothers, 1895)

• Color Photography (Lumière Brothers,

1908)

Some history…

Let’s also not forget…

Motzu

(468-376 BC)

Aristotle

(384-322 BC)

Also: Plato, Euclid

Al-Kindi (c. 801–873)

Ibn al-Haitham

(965-1040)Oldest existent

book on geometry

in China

Pinhole perspective projectionPinhole cameraf

f = focal length

o = aperture = pinhole = center of the camera

o

z

yf'y

z

xf'x

y

xP

z

y

x

P

Pinhole camera

Derived using similar triangles

O

P = [x, z]

P’=[x’, f ]

f

z

x

f

x

i

k

Pinhole camera

Common to draw image plane in front of

the focal point. What’s the transformation between these 2 planes?

f f

Pinhole camera

z

yf'y

z

xf'x

Kate lazuka ©

Pinhole camera

Is the size of the aperture important?

Shrinking

aperture

size

- Rays are mixed up

Adding lenses!

-Why the aperture cannot be too small?

-Less light passes through

-Diffraction effect

Cameras & Lenses

• A lens focuses light onto the film

• A lens focuses light onto the film

– There is a specific distance at which objects are “in

focus”

– Related to the concept of depth of field

“circle of

confusion”

Cameras & Lenses

• A lens focuses light onto the film

– There is a specific distance at which objects are “in

focus”

– Related to the concept of depth of field

Cameras & Lenses

• A lens focuses light onto the film

– All parallel rays converge to one point on a plane

located at the focal length f

– Rays passing through the center are not deviated

focal point

f

Cameras & Lenses

Thin Lenses

Snell’s law:

n1 sin 1 = n2 sin 2

Small angles:

n1 1 n2 2

n1 = n (lens)

n1 = 1 (air)

zo

z

y'z'y

z

x'z'x

ozf'z

)1n(2

Rf

Focal length

For details see lecture

on cameras in CS131A

Issues with lenses: Radial Distortion

Pin cushion

Barrel (fisheye lens)

No distortion

Pin cushion

Barrel (fisheye lens)

Issues with lenses: Radial Distortion

– Deviations are most noticeable for rays that pass through

the edge of the lens

Image magnification

decreases with distance

from the optical axis

Lecture 2 -Silvio Savarese 14-Jan-14

• Pinhole cameras• Cameras & lenses• The geometry of pinhole cameras

• Intrinsic• Extrinsic

• Other camera models

Lecture 2Camera Models

Pinhole perspective projectionPinhole cameraf

f = focal length

o = center of the camera

o

)z

yf,

z

xf()z,y,x(

2E

3

From retina plane to images

Pixels, bottom-left coordinate systems

Coordinate systems

xc

yc

Converting to pixels

x

y

xc

yc

C=[cx, cy]

)cz

yf,c

z

xf()z,y,x( yx

1. Off set

Converting to pixels

)cz

ylf,c

z

xkf()z,y,x( yx

1. Off set

2. From metric to pixels

x

y

xc

yc

C=[cx, cy] Units: k,l : pixel/m

f : m: pixel

, Non-square pixels

• Matrix form?

A related question:

• Is this a linear transformation?x

y

xc

yc

C=[cx, cy]

)cz

y,c

z

x()z,y,x( yx

Converting to pixels

Is this a linear transformation?

No — division by z is nonlinear

How to make it linear?

)z

yf,

z

xf()z,y,x(

Homogeneous coordinates

homogeneous image

coordinates

homogeneous scene

coordinates

• Converting from homogeneous

coordinates

For details see lecture on

transformations in CS131A

Camera Matrix

)cz

y,c

z

x()z,y,x( yx

1

z

y

x

0100

0c0

0c0

z

zcy

zcx

X y

x

y

x

x

y

xc

yc

C=[cx, cy]

Perspective Projection

Transformation

1

z

y

x

0100

00f0

000f

z

yf

xf

X

z

yf

z

xf

iX

XMX

M

3H

4

XMX

X0IK

Camera Matrix

1

z

y

x

0100

0c0

0c0

X y

x

Camera

matrix K

Finite projective cameras

1

z

y

x

0100

0c0

0cs

X y

x

Skew parameter

x

y

xc

yc

C=[cx, cy]K has 5 degrees of freedom!

Lecture 2 -Silvio Savarese 14-Jan-14

• Pinhole cameras• Cameras & lenses• The geometry of pinhole cameras

• Intrinsic• Extrinsic

• Other camera models

Lecture 2Camera Models

World reference system

Ow

iw

kw

jwR,T

•The mapping so far is defined within the camera

reference system

• What if an object is represented in the world

reference system

2D Translation

P

P'

t

• For details see lecture on

transformations in CS131A

• See also TA session on Friday

2D Translation Equation

P

x

y

tx

ty

P’

t

)ty,tx(' yx tPP

),(

),(

yx tt

yx

t

P

2D Translation using Homogeneous Coordinates

1100

10

01

1

' y

x

t

t

ty

tx

y

x

y

x

P

)1,,(),(

)1,,(),(

yxyx tttt

yxyx

t

P

t P

1

I tP T P

0

P

x

y

tx

ty

P’

t

Scaling

P

P'

Scaling Equation

P

x

y

sx x

P’sy y

1100

00

00

1

' y

x

s

s

ys

xs

y

x

y

x

P

)1,,(),('

)1,,(),(

ysxsysxs

yxyx

yxyx

P

P

S

'

1

S 0P S P

0

)ys,xs(')y,x( yx PP

Scaling & Translating

'

'' '

P S P

P T P

'' ' ( ) P T P T S P T S P A P

P

P''

Scaling & Translating

1 0 0 0

'' 0 1 0 0

0 0 1 0 0 1 1

0

01

0 0 1 1 1 1

x x

y y

x x x x

y y y y

t s x

t s y

s t x x s x t

s t y y s y t

P T S P

S t

0

Α

Rotation

P

P'

Rotation Equations

• Counter-clockwise rotation by an angle 𝜃

y

x

y

x

cossin

sincos

'

'P

x

y’

P’

x’

y

PRP'

' cos sinx x y

' cos siny y x

Degrees of Freedom

Note: R is an orthogonal matrix and satisfies many interesting properties:

R is 2x2 4 elements

' cos sin

' sin cos

x x

y y

1)det(

R

IRRRRTT

Rotation + Scale + Translation

1 0 cos in 0 0 0

' 0 1 sin cos 0 0 0

0 0 1 0 0 1 0 0 1 1

x x

y y

t s s x

t s y

P T R S P

cos in 0 0

sin cos 0 0

0 0 1 0 0 1 1

x x

y y

s t s x

t s y

1 1 11 1

x x

y y

R t S 0 R S t

0 0 0

If sx = sy, this is asimilarity transformation!

' ( ) P T R S P

3D Rotation of PointsRotation around the coordinate axes, counter-clockwise:

100

0cossin

0sincos

)(

cos0sin

010

sin0cos

)(

cossin0

sincos0

001

)(

z

y

x

R

R

R

p

x

Y’

p’

x’

y

z

wXTRK

World reference system

Ow

iw

kw

jwR,T

wXTR

X

4410

In 4D homogeneous coordinates:

Internal parameters External parameters

XIKX 0' wXTR

IK

4410

0

M

X

X’

wx XMX 4313

14w4333 XTRK

Projective cameras

Ow

iw

kw

jwR,T

How many degrees of freedom?

5 + 3 + 3 =11!

100

c0

cs

K y

x

X

X’

Projective cameras

Ow

iw

kw

jwR,T

w13 XMX

14w4333 XTRK

3

2

1

m

m

m

M

W

W

W

W

m

m

m

m

m

m

X

X

X

X

3

2

1

3

2

1

),(3

2

3

1

w

w

w

w

X

X

X

X

m

m

m

m

E

X

X’

Theorem (Faugeras, 1993)

][ bATKRKTRKM

3

2

1

a

a

a

A

100

0 y

x

c

cs

K

lf

;kf

Properties of Projection• Points project to points

• Lines project to lines

• Distant objects look smaller

Properties of Projection•Angles are not preserved

•Parallel lines meet! Vanishing point

One-point perspective• Masaccio, Trinity,

Santa Maria

Novella, Florence,

1425-28

Credit slide S. Lazebnik

Next lecture

•How to calibrate a camera?