Camera Models and Imaging

Leow Wee Kheng CS4243 Computer Vision and Pattern Recognition

CS4243 Camera Models and Imaging 1

Through our eyes… Through our eyes we see the world.

CS4243 Camera Models and Imaging 2

CS4243 Camera Models and Imaging 3

lens

pupil

cornea

iris

retina

optic nerves

focus light

control amount of light entering the eye

captures image

transmit image

Through their eyes… Camera is computer’s eye.

CS4243 Camera Models and Imaging 4

Camera is structurally the same the eye.

CS4243 Camera Models and Imaging 5

lens lens | cornea

sensor | retina

CS4243 Camera Models and Imaging 6

lens | cornea

lens

aperture | pupil

CS4243 Camera Models and Imaging 7

aperture | pupil

aperture plates | iris

Imaging Images are 2D projections of real world scene. Images capture two kinds of information:

Geometric: positions, points, lines, curves, etc. Photometric: intensity, colour.

Complex 3D-2D relationships. Camera models approximate relationships.

CS4243 Camera Models and Imaging 8

Camera Models Pinhole camera model Orthographic projection Scaled orthographic projection Paraperspective projection Perspective projection

CS4243 Camera Models and Imaging 9

Pinhole Camera Model

CS4243 Camera Models and Imaging 10

image plane pinhole virtual image

plane 3D

object

focal length

2D image

Math representation

CS4243 Camera Models and Imaging 11

world frame / camera frame

image frame

camera centre

image centre

optical axis

projection line

By default, use right-handed coordinate system.

CS4243 Camera Models and Imaging 12

3D point P = (X, Y, Z)T projects to 2D image point p = (x, y)T .

By symmetry,

i.e.,

Simplest form of perspective projection.

CS4243 Camera Models and Imaging 13

Orthographic Projection 3D scene is at infinite distance from camera. All projection lines are parallel to optical axis. So,

CS4243 Camera Models and Imaging 14

Scaled Orthographic Projection

CS4243 Camera Models and Imaging 15

Scene depth << distance to camera. Z is the same for all scene points, say Z0

Perspective Projection

CS4243 Camera Models and Imaging 16

camera frame

world frame

image frame is parallel to camera’s X-Y plane

Intrinsic Parameters Pinhole camera model

Camera sensor’s pixels not exactly square x, y : coordinates (pixels) k, l : scale parameters (pixels/m) f : focal length (m or mm)

CS4243 Camera Models and Imaging 17

f, k, l are not independent. Can rewrite as follows (in pixels):

So,

Image centre or principal point c may not be at origin. Denote location of c in image plane as cx, cy. Then,

Along optical axis, X = Y = 0, and x = cx, y = cy.

CS4243 Camera Models and Imaging 18

Image frame may not be exactly rectangular. Let θ denote skew angle between x- and y-axis. Then,

Combine all parameters yield

CS4243 Camera Models and Imaging 19

homogeneous coordinates

Intrinsic parameter matrix

Denoting ρ = Z, we get

Some books and papers absorb ρ into :

giving Be careful.

A simpler form of K uses skew parameter s:

CS4243 Camera Models and Imaging 20

Extrinsic Parameters Camera frame is not aligned with world frame. Rigid transformation between them:

Coordinates of 3D scene point in camera frame. Coordinates of 3D scene point in world frame. Rotation matrix of world frame in camera frame. Position of world frame’s origin in camera frame.

CS4243 Camera Models and Imaging 21

frame

object

Translation matrix

Rotation matrix

CS4243 Camera Models and Imaging 22

Rotation matrix is orthonormal:

So, . In particular, let

Then,

CS4243 Camera Models and Imaging 23

Combine intrinsic and extrinsic parameters:

Simpler notation:

CS4243 Camera Models and Imaging 24

Lens Distortion Lens can distort images,

especially at short focal length.

CS4243 Camera Models and Imaging 25

barrel pin-cushion fisheye

Radial distortion is modelled as

with . Actual image coordinates

CS4243 Camera Models and Imaging 26

distorted coordinates

undistorted coordinates

distortion parameters

Camera Calibration Compute intrinsic / extrinsic parameters. Capture calibration pattern from various views.

CS4243 Camera Models and Imaging 27

Detect inner corners in images.

Run calibration program (available in OpenCV). CS4243 Camera Models and Imaging 28

Homography A transformation between projective planes. Maps straight lines to straight lines.

CS4243 Camera Models and Imaging 29

Homography equation:

Affine is a special case of homography:

CS4243 Camera Models and Imaging 30

image point

image point homography

Scaling H by s does not change equation:

Homography is defined up to unspecified scale. So, can set h33 = 1. Expanding homography equation gives

CS4243 Camera Models and Imaging 31

Assembling equations over all points yields

System of linear equations. Easy to solve.

CS4243 Camera Models and Imaging 32

Example Circles: input corresponding points. Black dots: computed points, match input points.

CS4243 Camera Models and Imaging 33

Before homographic transform

CS4243 Camera Models and Imaging 34

After homographic transform

CS4243 Camera Models and Imaging 35

Summary Simplest camera model: pinhole model. Most commonly used model: perspective model. Intrinsic parameters:

Focal length, principal point.

Extrinsic parameters: Camera rotation and translation.

Lens distortion Homography: maps lines to lines.

CS4243 Camera Models and Imaging 36

Further Reading Camera models: [Sze10] Section 2.1.5, [FP03]

Section 1.1, 1.2, 2.2, 2.3. Lens distortion: [Sze10] Section 2.1.6, [BK08]

Chapter 11. Camera calibration: [BK08] Chapter 11, [Zha00]. Image undistortion: [BK08] Chapter 11.

CS4243 Camera Models and Imaging 37

References Bradski and Kaehler. Learning OpenCV. O’Reilly, 2008.

D. A. Forsyth and J. Ponce. Computer Vision: A Modern Approach. Pearson Education, 2003.

R. Szeliski. Computer Vision: Algorithms and Applications. Springer, 2010.

Z. Zhang. A flexible new technique for camera calibration. IEEE Trans. PAMI, 22(11):1330–1334, 2000.

CS4243 Camera Models and Imaging 38