planar homographies nov. 15, '04

(c) 2004 F.J. Estrada & A.D. Jepson & D. Fleet

Planar Homographies Nov. 15, ’04

References:

• Your lecture notes on coordinate frames

• Your lecture notes on epipolar geometry

• http://www.robots.ox.ac.uk/˜vgg/projects/SingleView/

Planar Homographies 1


A quick review of projection

We need to determine the parameters that govern the projection

from points in the world to points in an image, a quick review of

coordinate frames and transformations is in order.

Homogeneous Coordinates:

Let ~x = (x1, x2, x3)T be a 3-D point in some reference

frame, we can express ~x in Homogeneous coordinates as

~xH = (x1, x2, x3, 1)T .

A general 3D rigid transformation can be written as a 4 by 4

matrix multiplication using Homogeneous coordinates:

M3D−trans =

(

R3,3~d3,1

01,3 1

)

(1)

Where R is a 3 by 3 rotation matrix, and d = (dx, dy, dz)T ,

specifies translation components for the x, y, and z coordinates.

The rotation matrix in turn can be decomposed into three

matrices, each specifying a rotation around each of the coordinate

axes.

A transformation of the type described above is used to convert

coordinates between different coordinate frames. The first set

A quick review of projection 2


of parameters that governs the mapping of points in the world

to points in the image is given by a transformation from world

coordinates to camera coordinates:

Mext =(

R3,3 −R3,3~d3,1

)

(2)

Where R is the rotation from world to camera coordinates, and~d contains the world coordinates of the camera’s nodal point.

Notice that Mext is a 3 by 4 matrix with 12 D.O.F., R and ~d are

known as the extrinsic camera parameters.



A quick review of projection

The second set of parameters that govern the mapping

from world points to image points is given by the specific

characteristics of the projection of light rays onto the image.

Figure 1 illustrates the projection process in a pinhole camera.

In this figure, C is the center of projection, X, Y , and Z are

the coordinate axes in camera coordinates, u and v are the

image coordinate axes, f is the focal length of the camera, and

a point ~xcam in camera coordinates projects to point ~xim in

image coordinates.



The transformation that maps from camera coordinates to

image coordinates has 2 components, the first component

converts camera coordinates to image coordinates. If ~xcam =

(x, y, z)T , then ~xim = (u, v) = (fx/z, fy/z). This

operation can be written as the 3 by 3 perspective projection

matrix:

Mproj =

f 0 0

0 f 0

0 0 1

(3)

The second component accounts for the size and shape of pixels,

and for the image coordinates. These effects are specified with

a 3 by 3 matrix:

Mim =

1/l1 0 pc,1

0 1/l2 pc,2

0 0 1

(4)

Where l1 is the width of the pixels, l2 is the height of the

pixels, and pc = (pc,1, pc,2) is the point (in image coordinates)

where the optical axis of the camera intersects the image plane.

The parameters in the previous 2 matrices are known as the

camera’s intrinsic parameters. The complete mapping from a

point W in (homogeneous) world coordinates to point Pim in



image coordinates is:

Pim = Mim · Mproj · Mext ·~W T (5)



2-D Homographies

2-D homographies are projective transformations that map

points from one plane to another plane (for example the

transformation mapping points in a planar surface in the world

to the image plane). Figure 2 illustrates the geometry involved

in this process.

2-D Homographies 7


2-D Homographies

Consider the problem of determining the homography that

maps points in one image to the corresponding points in a

second image.

Assuming that we can identify corresponding points in both

images (let’s say, by detecting and matching interest points),

such a homography exists and can be computed, consider the

homography Hl mapping points on a plane to image points on

the left side image:

~xl,i = αHl · ~qi, α > 0 (6)

2-D Homographies 8


Where ~qi = (u, v, 1)T is a point on some plane π, and ~xl,i =

(u′

l, v′

l, 1)T its projection onto the left image. In a similar

fashion, consider the homography Hr that maps points on π

to image points on the right side image:

~xr,i = βHr · ~qi, β > 0 (7)

Solving for ~qi in Eq. and substituting in Eq. we have:

~xl,i =α

βHl · H

−1r · ~xr,i = γH~xr,i (8)

Where H = Hl ·H−1r is the homography that maps points on

the right side image to points on the left side image. Notice that

we can do this without ever knowing the location of the points

~qi. Figure 5 shows the result of applying the homography to

the left side image, and a composite of both images showing

the correspondence.

2-D Homographies 9


Range: [20.6, 254] Dims: [384, 256]

Range: [−173, 215] Dims: [384, 256]

2-D Homographies 10


2-D Homographies

A 2-D homography is defined as a 3 by 3 homogeneous matrix

such that for any point ~xi = (γu, γv, γ) on π, and its

corresponding point ~xi′ on π′:

~xi′ = H · ~xi (9)

2-D homographies have 8 D.O.F. (9 entries in the H matrix, but

the common scale factor is not relevant), hence, to determine

the homography we require 4 pairs of corresponding points.

Notice though that 3 collinear points in either plane result in

a configuration with no unique solution.

Homographies can be applied to many problems in computer

vision including stereo reconstruction, image mosaics, and

applications using perspective geometry. Figure 3 shows an

example of using 2-D planar homographies to rectify an image

region (remove distortion due to perspective projection).

2-D Homographies 11


In the above example, the 4 corners of the window are

mapped to the corners of a rectangular polygon, however, any

rectangular polygon will offer 4 correspondence pairs. The

aspect ratio for the rectangular polygon can not be determined

from one view of the 4 corners of the window.

2-D Homographies 12


Perspective Structure

Finally, let’s take a look at an image taken under perspective

projection and observe some of its characteristics.

One important property of perspective projection is that

parallel lines converge to some point in the image (though

the point of convergence may be at infinity as is the case for

horizontal and vertical lines in the image above).

The image location of these intersection points (usually called

vanishing points) determine the orientation in 3D of the set of

parallel lines that converge to it. It is also possible to determine

Perspective Structure 13


from pairs of vanishing points the 3D orientation of planar

surfaces bounded by parallel lines converging to either of the

vanishing points. This enables us to do take the above image

and turn it into a 3D model as shown below.

..

This type of reconstruction relies heavily on planar

homographies to determine the correct mapping between image

pixels and reconstructed 3D surfaces. For more information on

this type of reconstruction, visit the link shown in the first

slide.

Perspective Structure 14

planar homographies nov. 15, '04

Documents