University of Sao Paulo

PSI 5787

Virtual Reality

The mathematics of aprojection system

Author:

Alexandre Martins

Professors:

Prof. Roseli Lopes

Prof. Marcelo Zuffo

December, 2012

Contents

1 Introduction 2

2 The problem 2

3 A few solutions 3

3.1 First approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

3.1.1 From the camera to the display . . . . . . . . . . . . . . . . 4

3.1.2 Homography estimation . . . . . . . . . . . . . . . . . . . . 8

3.1.3 Linear algebra review . . . . . . . . . . . . . . . . . . . . . . 11

3.1.4 Stereographic projection . . . . . . . . . . . . . . . . . . . . 14

3.1.5 Working demo . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3.2 Camera matrix formulation . . . . . . . . . . . . . . . . . . . . . . 17

3.3 Piecewise-Linear approximation . . . . . . . . . . . . . . . . . . . . 20

3.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.3.2 Convex combination . . . . . . . . . . . . . . . . . . . . . . 20

3.3.3 Quadrilaterals . . . . . . . . . . . . . . . . . . . . . . . . . . 21

3.3.4 Triangles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

4 Future work 24

5 Conclusion 25

1

1 Introduction

In this rather informal text, we’re going to discuss the mathematics of a projection

system for a spherical display. Specifically, given a camera, a spherical display

surface and an image projected on it, we turn our attention towards finding a

transform from the camera coordinates to the coordinates of the image displayed

on the surface. We provide multiple solutions to the problem. Let us discuss all

that in more detail.

2 The problem

We need a projection system for a spherical display surface. We have:

• A spherical display surface D;

• N projectors: P1, ..., PN ;

• A camera C.

In order to display an image I on the surface D, we’ll somehow “align” the

projectors, warping the projected images. This is known as geometric correction.

Thus, to be able to perform such a correction, we need to determine how the

individual projectors’ images map to I. For projector Pi, let Gi be this mapping.

First of all, let us state a few points:

• D, which is a spherical display, is a parametric display surface;

• the camera C can observe projected features from each Pi, thus establishing

a mapping Hi from the ith projector coordinates to the camera coordinates;

• the camera C, which observes D, has its own coordinate system. Let FCD

be the transform from the camera coordinates to the display coordinates;

• we display image I on D. Let FDI be the mapping from the display coordi-

nates to the image coordinates;

• finally, we can combine FCD and FDI to form F , a transform from the camera

coordinates to the image coordinates.

2

Hence, Gi, which is a mapping from the ith projector coordinates to the image

coordinates, is merely the composition between F and Hi:

Gi = F ·Hi

In order to find Hi, one might use some sort of binary encoding approach, as

described in [Ferreira, 2011] or in [Majumder and Brown, 2007].

For the VR class, however, we just want to find a transform from camera to

image. That is, we want to

find F

One quickly realizes that, although this work is part of a projection system, no

projectors are involved in it.

Now we discuss a few solutions to the problem.

3 A few solutions

In the solutions given below, we’ll be working with the pinhole camera model,

which “describes the mathematical relationship between the coordinates of a 3D

point and its projection onto the image plane of an ideal pinhole camera”. [Wikipedia]

3.1 First approach

In the first approach we came up with, we considered a set of geometric entities

and found some relationships between them.

In order to find F , we need to find two auxiliary transforms:

• FCD: camera to display surface

• FDI : display surface to image

Then, we set

F = FDI · FCD

3

3.1.1 From the camera to the display

Let:

• S be the unit sphere in R3 centered at the origin O;

• π be the plane z = z0, where −1 < z0 < 0;

• C = (u, v, w) be the position of the optical center of the camera, where we

enforce w < z0.

Figure 1: Our model.

Notice that, in this formulation, the values z0, u, v, w are given a priori.

Furthermore, we’ll say that the base B of the sphere S is the circle in π with

radius (1 − z20)1/2 centered at (0, 0, z0). Points in S having z-coordinate less than

z0 are said to be “invalid”. Points that are not invalid are called “valid”.

4

Figure 2: The base B is highlighted in blue.

Now, given q = (qx, qy), the position of a pixel in the image of the camera, we

want to find the coordinates of the corresponding point P = (x, y, z) in the sphere

S. That is, we want to find P such that P = FCD(q).

Assume for a moment that there is a bijective transform H from the camera

coordinates to the plane π, taking q to some P ′ such that P ′ is the intersection

between the line CP and the plane π.

Figure 3: Points C, P ′ and P .

In order to find P , we use a very straightforward sequence of three steps:

1. Let P ′ = (x0, y0, z0) ∈ π be H applied to q;

• that is, P ′ = H q.

5

2. If P ′ /∈ B, then return null ;

• in other words, return null if x20 + y20 > 1− z20 ;

• in this case, we’ll say that pixel q is garbage.

3. Walk from C on the direction P ′ − C until you find, in S, a valid point,

which will be P .

The last step may be performed analytically. By noticing that, by construction,

points C, P ′ and P are collinear, one realizes that vectors (P ′ − C) and (P − P ′)are parallel. That is,

(P ′ − C)× (P − P ′) = 0

Simple algebraic manipulation provides:

0 =

(P ′ − C)× (P − P ′) =

∣∣∣∣∣∣∣i j k

x0 − u y0 − v z0 − wx− x0 y − y0 z − z0

∣∣∣∣∣∣∣ =

∣∣∣∣∣ y0 − v z0 − wy − y0 z − z0

∣∣∣∣∣ i−∣∣∣∣∣ x0 − u z0 − wx− x0 z − z0

∣∣∣∣∣ j +

∣∣∣∣∣ x0 − u y0 − vx− x0 y − y0

∣∣∣∣∣ kTherefore, (P ′ − C)× (P − P ′) = 0 implies:

x = x0 + ( z−z0z0−w )(x0 − u)

y = y0 + ( z−z0z0−w )(y0 − v)

Now, plug-in the sphere equation and solve for (x, y, z):

6

x = x0 + ( z−z0z0−w )(x0 − u)

y = y0 + ( z−z0z0−w )(y0 − v)

x2 + y2 + z2 = 1

In order to solve this system of equations, let:

a = 1 + ( x0−uz0−w )2 + ( y0−v

z0−w )2

b = 2{[x0 − ( x0−uz0−w )z0](

x0−uz0−w ) + [y0 − ( y0−v

z0−w )z0](y0−vz0−w )}

c = −1 + [x0 − ( x0−uz0−w )z0]

2 + [y0 − ( y0−vz0−w )z0]

2

In order to find P , one must first solve az2 + bz + c = 0 for z. However, this

equation has at most two real solutions. Since P ′ ∈ B, only one results in a valid

point (i.e., not below the base) in S: the greater one.

Figure 4: The point below π is invalid.

Let us state that, if one wants to find the inverse transform FCD−1, one must

realize that, since P , P ′ and C are on the same line, then (P ′−C)× (P −C) = 0,

which implies:

x0 = u+ ( z0−wz−w )(x− u)

7

y0 = v + ( z0−wz−w )(y − v)

In other words, given z0, P = (x, y, z) and C = (u, v, w), one can find P ′ =

(x0, y0, z0) ∈ π. Then, one applies H−1 to P ′ to return to the camera space.

Earlier, however, we assumed that there is a bijective transform H from the

camera space to plane π. In order to compute such a transform, we need to study

a new subject called homography estimation.

3.1.2 Homography estimation

Recall that RP 2, the 2-dimensional projective space, is the set of lines in R3 passing

through the origin (except the origin itself). [Gomes and Velho, 2008]

We say that a homography is a bijective transformation from a projective space

to itself that maps straight lines to straight lines. [Wikipedia]

Consider two coordinate systems represented by planes:

• the image plane I(x, y) and

• the world plane W (x′, y′).

Figure 5: Homography. Extracted from [mmLAB Wiki].

Now, since we’re in 2D projective space, let us pick points in I and in W in

homogeneous coordinates. We may design an homography as a 3× 3 matrix:

8

s x

s y

s

=

H−1︷ ︸︸ ︷ a1 a2 a3

a4 a5 a6

a7 a8 a9

x′

y′

1

where s 6= 0 is an arbitrary scale factor. Remember that multiplying the

homogeneous coordinates of some projective point by any non-zero scalar results

in the same projective point.

Given the system of equations displayed above, we may write:

x =s x

s=a1x

′ + a2y′ + a3

a7x′ + a8y′ + a9

y =s y

s=a4x

′ + a5y′ + a6

a7x′ + a8y′ + a9

That is,

a1x′ + a2y + a3 − a7x′x− a8y′x− a9x = 0

a4x′ + a5y

′ + a6 − a7x′y − a8y′y − a9y = 0

Which means, in matrix form, that

[x′ y′ 1 0 0 0 −x′x −y′x −x0 0 0 x′ y′ 1 −x′y −y′y −y

] a1...

a9

= 0

Given a set of m correspondences (xi, yi) ←→ (x′i, y′i), we want to estimate

H−1, which maps world to image (clearly, H = (H−1)−1

). Hence,

9

x′1 y′1 1 0 0 0 −x′1x1 −y′1x1 −x10 0 0 x′1 y′1 1 −x′1y1 −y′1y1 −y1x′2 y′2 1 0 0 0 −x′2x2 −y′2x2 −x20 0 0 x′2 y′2 1 −x′2y2 −y′2y2 −y2

...

x′m y′m 1 0 0 0 −x′mxm −y′mxm −xm0 0 0 x′m y′m 1 −x′mym −y′mym −ym

a1

a2

a3

a4...

a8

a9

= 0

Estimating H−1 boils down to estimating vector a = [a1 a2 . . . a9]T . However,

since the homography matrix is defined up to a scale factor, this last system of

equations has a trivial solution: a = 0. Clearly, this is not what we want. We need

to add a new constraint to the system of equations: for example, we might enforce

‖a‖ = 1. Other authors, such as [Ferreira, 2011], enforce a9 = 1. By adopting the

latter strategy, we rewrite the system of equations to:

x′1 y′1 1 0 0 0 −x′1x1 −y′1x10 0 0 x′1 y′1 1 −x′1y1 −y′1y1x′2 y′2 1 0 0 0 −x′2x2 −y′2x20 0 0 x′2 y′2 1 −x′2y2 −y′2y2

...

x′m y′m 1 0 0 0 −x′mxm −y′mxm0 0 0 x′m y′m 1 −x′mym −y′mym

a1

a2

a3

a4...

a7

a8

=

x1

y1

x2

y2...

xm

ym

and, given m ≥ 4 correspondences (xi, yi)←→ (x′i, y

′i), we solve it for a. How-

ever, a brief look at the equations shows that an exact solution might not exist. In

order to deal with this situation and estimate a, we’ll review some linear algebra.

10

3.1.3 Linear algebra review

Notation: ‖v‖2 = ‖v‖ is the Euclidean norm of vector v.

Let:

• A ∈ Rm×n, m ≥ n;

• b ∈ Rm.

We realize that the system of equations

Ax = b

may not have a solution. Then, let’s find x that minimizes ‖b− Ax‖2.

Figure 6: there’s no x such that Ax = b.

In the picture displayed above, the blue line represents the image of matrix A,

Im(A), which is the set of vectors spanned by the columns of A. Furthermore,

point b may not belong to Im(A).

Let y be the orthogonal projection of b onto Im(A): y = Pb, where P projects

b orthogonally onto Im(A). That is, matrix P is an orthogonal projector (P = P 2

and P = P T ) such that Im(P ) = Im(A).

11

Figure 7: y = P b.

Let us now prove a few theorems.

Theorem: A vector x minimizes the 2-norm of the error r = (b− Ax) if, and

only if, Pb = Ax, where P projects b orthogonally onto Im(A).

Proof : we’ll show that y = Pb is the only element that minimizes ‖z − b‖among all z’s in Im(A).

This has to be the case, because, by taking any z 6= y in Im(A), we have:

‖z − b‖2 =

> 0︷ ︸︸ ︷‖y − z‖2 +‖y − b‖2 > ‖y − b‖2

Therefore, y is strictly better than z.

Figure 8: clearly, ‖y − b‖2 < ‖z − b‖2.

12

Theorem: x is such that Pb = Ax if, and only if, r ⊥ Im(A).

Proof :

1. (⇐) Suppose that r ⊥ Im(A). We know that b = y+ r for some y ∈ Im(A).

That said, we have y = Pb. Now, since Pb ∈ Im(A), we can just set x such

that Pb = Ax.

2. (⇒) Suppose that Pb = Ax. Observe that (b − Pb)⊥Im(A). Now, Atr =

At(b− Ax) = At(b− Pb) = 0.

Fact: r ⊥ Im(A) ⇐⇒ AtAx = Atb.

Proof :

Atr = 0

At(b− Ax) = 0

Atb− AtAx = 0

Atb = AtAx

Therefore, a vector x minimizes the 2-norm of the error r = b−Ax if, and only

if, that last equation is true.

Finally, if AtA is non-singular, then the last fact suggests picking:

x = (AtA)−1At b

in order to minimize the error ‖b− Ax‖.

Note: (AtA)−1At is known as pseudo-inverse matrix.

13

One may, alternatively, compute the pseudo-inverse matrix by using QR fac-

torization or singular value decomposition (SVD). We won’t go into the details,

though.

By using this result to compute the homography, we’re able to compute FCD:

the transformation from the camera coordinates to the coordinates of the display

surface. Now we want to compute FDI : the mapping from the surface to the image

displayed on it. In order to do that, we’ll study a new subject called stereographic

projection.

3.1.4 Stereographic projection

Stereographic projection is a bijective, smooth mapping that projects a sphere

onto a plane. [Wikipedia]

Let:

• S be the unit sphere in R3 centered at the origin;

• Q = (0, 0,−1) be the “south pole”;

• π be the plane z = 1.

That said, our stereographic projection is defined on S − {Q}. In order to

compute FDI , let:

• P = (x, y, z) be a point in S − {Q};

• r be the line connecting P and Q.

We define the stereographic projection P ′ = (x′, y′) of P as r ∩ π.

14

Figure 9: stereographic projection from the “south pole”.

Now, given P = (x, y, z) ∈ S−{Q}, we want P ′ = (x′, y′) ∈ π: that is, we want

the stereographic projection, from the “south pole”, of P . By using elementary

algebra, we find that:

x′ = 2x1+z

y′ = 2 y1+z

We also find the inverse transform:

x = 4x′

4+(x′)2+(y′)2y = 4 y′

4+(x′)2+(y′)2z = 4−(x′)2−(y′)2

4+(x′)2+(y′)2

That said, computing FDI boils down to applying a very simple algorithm:

1. Compute the stereographic projection;

2. Rescale the projection to a suitable region, say, [−1, 1]× [−1, 1].

We have FCD and FDI . Therefore, we have F .

Note: one might use a different parametrization for the image (e.g., geodesic

polar coordinates).

15

3.1.5 Working demo

There’s a working JavaScript demo at:

• www.ime.usp.br/~alemart/psi5787/spheree/

Given a picture taken from a webcam, we just apply function F (camera to

image), where F is designed according to the approach we have just presented:

Figure 10: A picture taken from a webcam

Figure 11: The image displayed on the sphere

16

In practice, this formulation requires:

• At least 4 correspondences between camera pixels and points in the sphere;

• The diameter of the sphere and of the base of the sphere (then, one finds the

value of z0);

• The position of the camera in 3D space (u, v, w).

Finally, we emphasize that function F is invertible, since (we have seen that)

both FCD and FDI are invertible.

3.2 Camera matrix formulation

Although the previous approach delivers reasonable results, it requires the position

of the camera in relation to the center of the sphere. That is, the position must

be given. That said, one might ask: isn’t there a different formulation where this

position can be figured out automatically by an algorithm? This question inspired

this camera matrix formulation.

A camera matrix C is a 3×4 matrix used to describe the transform of a pinhole

camera from points in 3D space to 2D points in the image. Given the position

P ′ = (x′, y′) of a pixel in the image and a point P = (x, y, z) in 3D space, we take

both in homogeneous coordinates and write:

s x′

s y′

s

=

C︷ ︸︸ ︷ a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

x

y

z

1

where s 6= 0 is an arbitrary scale factor. Some known facts:

• C has 11 degrees of freedom;

• C encodes intrinsics and extrinsics of the camera;

• We use at least 6 correspondences (x′, y′)↔ (x, y, z) to estimate it.

17

Matrix C, which is defined up to a scale (featuring 11 DoF), encodes intrinsic

camera parameters (such as focal lengths) as well as extrinsic camera parameters

(e.g., rotation and position in 3D space). C can be estimated by means of similar

calculations compared to what we did in the homography estimation section (i.e.,

set a34 = 1 and use the pseudo-inverse technique to estimate a11, a12, . . . , a33).

Again, in order to find F , the transform from camera space to image space, we

need to find the functions:

• FCD: camera to display surface;

• FDI : display surface to image.

Mapping FDI can be designed as before, using stereographic projection. On

the other hand, once C has been estimated, building FCD boils down to solving

the following system of equations:

s x′

s y′

s

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

x

y

z

1

x2 + y2 + z2 = 1

Using the camera matrix equation, write x and y as a function of z (note that

x′, y′, a11, . . . , a34 are known):

x = α1 z + α0

y = β1 z + β0

Clearly, α0, α1, β0, β1 ∈ R are constants which can be determined analytically.

Now, function FCD (camera → display) can be translated to the following algo-

rithm:

1. let ∆ = (α0α1 + β0β1)2 − (1 + α2

1 + β21)(α2

0 + β20 − 1);

2. if ∆ < 0, return null ;

• P ′ may be discarded. You’re not looking at the sphere.

18

3. else, let zk be:

zk =−(α0α1 + β0β1) + (−1)k

√∆

1 + α21 + β2

1

4. and sk be:

sk =

a11(α1zk+α0)+a12(β1zk+β0)+a13zk+a14a31(α1zk+α0)+a32(β1zk+β0)+a33zk+a34

a21(α1zk+α0)+a22(β1zk+β0)+a23zk+a24a31(α1zk+α0)+a32(β1zk+β0)+a33zk+a34

5. there are two choices for zk: z0, which represents the upper half of the sphere,

and z1, representing the lower half. Which one should you pick? I claim that

you should pick zk̂.

6. set k̂ to argmin k

∥∥∥∥∥[x′

y′

]− sk

∥∥∥∥∥;

7. return point P = (x, y, z), where:

• x = α1zk̂ + α0;

• y = β1zk̂ + β0;

• z = zk̂.

And, surprisingly, that’s all there is to it.

In practice, this formulation requires:

• At least 6 correspondences camera ←→ sphere;

• No need to specify the position of the camera!

Clearly, function F is still invertible.

19

3.3 Piecewise-Linear approximation

3.3.1 Introduction

The idea of this formulation is to cover any parametric surface with a set of

linear patches.

Figure 12: The surface is covered with patches.

In this text, we’ll briefly study algorithms that cover the surface with quadri-

laterals and with triangles. However, we’ll first review what a convex combination

is.

3.3.2 Convex combination

Given a finite set of points {u1, u2, . . . , uk} in 3D space, their convex combina-

tion is the point:

α1u1 + α2u2 + . . .+ αkuk

where:

• scalar αi ≥ 0 for all i;

•∑k

j=1 αj = 1.

20

3.3.3 Quadrilaterals

In [Ferreira, 2011], it was discussed the possibility of using homography matrices

to map non-linear regions by approximating them as a set of smaller (linear)

quadrilaterals. Although this is, indeed, an insightful idea, we do not recommend

using this approach for a spherical display, for a reason we’ll see later. Still, we

have devised a sketch of an algorithm based on this idea.

You are given:

• a point p′ = (x′, y′) in the camera space.

• two dense sets of quads (i.e., convex quadrilaterals):

1. a set of quads in cam. space: {q1, . . . , qm};

2. a set of quads in display space: {Q1, . . . , Qm};

3. we assume that quad qi corresponds to Qi, for all i.

You want:

• transform p′ from the camera space to the display space.

We now provide the sketch of the algorithm:

1. for all j:

(a) let:

• cj be the centroid of quad qj;

• Cj be the centroid of quad Qj.

(b) estimate homography Hj using quads (qj, Qj);

• coord system: origin at the centroids

2. return a convex combination of {Cj + Rj Hj (p′ − cj)}

• where Rj converts to the plane (in 3D) formed by Qj

As we’ll see, different convex combinations may be used.

21

Linear patches. One might use indicator variables for the scalars of the convex

combination:

αj =

{1 if p′ ∈ qj0 otherwise

By using this, the display surface will be approximated by linear patches.

Bell-shaped curves. Alternatively, one might use the following scalars:

αj =

1m∑i=1

exp (− 12( diσ )

2)

exp (−12

(djσ

)2)

where:

• σ is a fixed constant (say, 1).

• di = ‖p′ − ci‖ for all i.

The display surface will be described, then, by some sort of non-linear behavior.

Figure 13: Plot for exp (−x2

2).

Unfortunately, the algorithm we have sketched is much more complicated than

we have described it, as it hides lots of gory maths. Also, it might not work at all:

22

given four points on the spherical display surface, there might not exist a plane

passing through all of them, and hence, these points might not form a quadrilateral.

Thus, the (already) complicated input would have to be either “correct” (the user

would have to ensure it is correct), or it would have to be pre-processed somehow,

which could decrease (abruptly) the quality of the approximation.

In spite of the drawbacks, we believe that, for planar (linear) displays, as well

as for cylindrical displays, the described approach would work just fine, providing

indeed very good accuracy and precision.

There’s a different approach, based on [Majumder and Brown, 2007], in which

one covers the parametric surface with triangles instead of quadrilaterals. There-

fore, homography matrices do not apply. Also, the input is simpler.

3.3.4 Triangles

We want to find FCD.

You are given:

• a point p′ = (x′, y′) in the camera space.

• a dense set of correspondences camera ↔ display.

You want:

• move p′ to display space.

We now provide a sketch of the algorithm:

1. compute a triangulation in the camera space;

2. move that triangulation to display space;

• that is, connect the points of the display space exactly in the same way

you connected the points of the camera space

3. suppose that p′ is in triangle T ′:

23

• p′ can be written as a convex combination of the vertices:

p′ = α1 v1 + α2 v2 + (1− α1 − α2) v3 (1)

4. move p′ to the corresponding triangle T in the display space.

One quickly realizes that unknowns α1 and α2 can be found in constant time,

as (1) features two equations and two unknowns. That’s all there is to it.

As a final observation, we notice that it’s worth working with a different

parametrization for the images (both the camera image and the image that is

projected on the display surface). Instead of working with pixels, one attributes

an identification number to each triangle. Then, by realizing that a given point

can only be inside one triangle (whose id is, say, j - resolve disputes arbitrarily),

and that that point can be written as a convex combination of the vertices (with

parameters, say, αj1 and αj2), then the location of that point may be described as

the tuple (j, αj1, αj2). Hence, mapping FDI can just be set to be the identity.

Since FCD and FDI are both invertible, it results that F is also invertible.

4 Future work

I expected that we would have the projection system working by the end of the

VR class. Unfortunately, that didn’t happen. By the time the hardware was built

on the cave and connected to a computer, it was late november / early december,

when we are concerned with final examinations and final coursework.

Although this text has provided, in reasonable detail, a few mathematical for-

mulations for the projection system, some man-hours are still required to make

things actually work on the hardware. You probably need a dedicated person

working on the project. Below, I provide the outline of the stuff I would do if the

VR class was still going on:

1. Once the actual wide-angle lenses are available, one has to figure out how

to deal with the distortion (radial and tangential). I believe that OpenCV

can be very helpful in this. Please take a look at this tutorial: http://docs.

opencv.org/trunk/doc/tutorials/calib3d/camera_calibration/camera_

calibration.html

24

2. If one is going to use a formulation described in this text, it’s important to

establish a few correspondences between the camera space and the sphere.

This can be done either manually, or through the use of some automatic de-

tection of visual markers placed on the sphere. I don’t know which approach

you guys prefer. Pick one.

3. During the implementation effort, one has to work closely with Fernando, so

that there is proper integration and troubleshooting.

5 Conclusion

We studied the mathematics of a projection system for a spherical display. We

considered N projectors P1, . . . , PN , a camera C, a parametric (spherical) display

surface D and the projected image I. In order to perform the geometric correction,

so that the multiple projectors “speak the same language” (that is, so that they

are aligned), one has to figure out how each projector Pi is related to the projected

image I. This relationship, which we called Gi, can be described by two auxiliary

mappings: Hi (projector to camera) and F (camera to projected image). Mapping

F , which is what we studied in detail, can be decomposed into two other auxiliary

transforms: FCD (camera to display surface) and FDI (display surface to projected

image). Schematically,

PiGi

−−−−−−→ I

PiHi

−−−−−−→ CF

−−−−−−→ I

PiHi

−−−−−−→ CFCD

−−−−−−→ DFDI

−−−−−−→ I

In order to find function F , we came up with a few approaches. Each one has

advantages and disadvantages. Below, we briefly discuss them:

25

• First approach

– The first formulation we came up with.

– It requires:

∗ At least 4 correspondences cam ↔ sphere;

∗ The diameter of the sphere & of the base;

∗ The position of the camera in 3D space.

• Camera matrix formulation

– An improvement of the first approach.

– It requires:

∗ At least 6 correspondences cam ↔ sphere.

• Piecewise-Linear approximation

– It models any parametric surface.

– It requires:

∗ A dense set of correspondences cam ↔ sphere.

In all three cases, mapping F can be made invertible.

I hope this all makes sense. Hopefully, this text will be useful to the Spheree

project.

Thank you

26

References

[Ferreira, 2011] Ferreira, F. T. (2011). Um sistema de multiprojecao escalavel.

Doctoral dissertation.

[Majumder and Brown, 2007] Majumder, A. and Brown, M. S. (2007). Practical

multi-projector display design.

[Gomes and Velho, 2008] Gomes, J. and Velho, L. (2008). Fundamentos da Com-

putacao Grafica.

[Wikipedia] Wikipedia. Pinhole camera model.

http://en.wikipedia.org/wiki/Pinhole camera model

[Wikipedia] Wikipedia. Homography. http://en.wikipedia.org/wiki/Homography

[mmLAB Wiki] mmLAB Wiki. Additional Material for the Computer Vision

Course. http://mmlab.disi.unitn.it/wiki

[Wikipedia] Wikipedia. Stereographic Projection.

http://en.wikipedia.org/wiki/Stereographic projection

27