photometric and photogeometric stereo - purdue university

Photometric and Photogeometric Stereo

CS635 Spring 2010

Daniel G. AliagaDepartment of Computer Science

Purdue University

Photometric Stereo

• A technique for estimating the surface normals of objects by observing that object under different lighting conditions– Then, using the surface normals, a plausible surface geometry can be reconstructed

– Woodham in 1980

• Related: when using a single image, it is called shape from shading– B. K. P. Horn in 1989

Photometric Stereo

surface

point

in ],1[ Pi

What are the values for ?in

Photometric Stereo

surface

point

n

l

cln )(

Lambertian (single channel) model:

Photometric Stereo

surface

point

n

l

clnlnln zzyyxx )(Lambertian (single channel) model:

How many unknowns per point?

How to get more equations?

3+1

More lights!

(known lights)

Photometric Stereo

surface

point

n

1l

1111 )( clnlnln zzyyxx

Photometric Stereo

surface

point

n

2l



Photometric Stereo

surface

point

n3l




Photometric Stereo1111 )( clnlnln zzyyxx



Using 1l Using 2l Using 3l




Using 1l Using 2l Using 3l

1c 2c 3c




What is ? zyx nnn ,,

cLn Write as and solve cLn 1

Where Tzyx nnnn

Tcccc 321

What is the surface normal at the point? nn /

What is (the albedo at the surface point)? nn

Lambertian Photometric Stereo with Known Lights

• Take three pictures of a static Lambertian object with a static camera

• In each picture move the light to a different but known position– For distant lights, could know light direction instead of light position

• At pixel i, solve • Use normals to “integrate” a surface

ii cLn 1

Normals ‐> Surface

• How?

Normals ‐> Surface),( yxzSurface (height field) is

Tyx zzyxn 1),( Surface normal is

))y,x(z)y,1x(z(zx

))y,x(z)1y,x(z(zy and

1122

yxyx

zyx zzzz

nnn

1 x

z

x znn

1 y

z

y znn

As we saw before

So and or zxx nnz / zyy nnz /and

Altogether: xzz nyxznyxzn ),(),1(yzz nyxznyxzn ),()1,(

Normals ‐> Surfacexzz nyxznyxzn ),(),1(yzz nyxznyxzn ),()1,(

Can setup as a large over-constrained linear system: bAz Azb

where has a bunch of normal values are the unknown heights (z-values)is a zero vector

and solve (e.g., use “lsqr” method)bAz 1

Normals ‐> Surface

[Basri et al., IJCV, 2007]

Fundamental Ambiguity

CNL

CLNRR 1

CLNAA 1

CLANA ))(( 1

RGA

Rotation matrix

Generalized Bas Relief (GBR) Ambiguity matrix

Fundamental Ambiguity

Belhumeur et al. 1999

GBR Transform

• Consider– This means “flatten” by and add a plane ( )

• When this is classical “bas‐relief”• Else, it is a generalized bas‐relief that can be written as

yxyxzyxz ),(),(

,

0

010001

G

10000

11

G

Photometric Stereo Ambiguity

CLANA ))(( 1

010001

RRGA

• Interpretation: given a solution for the normals, we can transform the normals by A and the lights by A-1

and the resulting picture looks the same…

Photometric Stereo with Unknown Lights

• Question:– What if both the surface normals and the light directions are unknown?

– Can we reconstruct the light directions, surface normals, and thus surface geometry (up to the aforementioned ambiguity)?

• Answer:– Yes, and the problem is still linear!

PS with Unknown LightingcLn Recall

1)( 2/1 nnTThink of the normals as located at the origin, then we care about a linear transformation of the sphere

Note PccLn 1

So 1)( QccPcPcPcPc TTTT

And Q is a 3x3 symmetric positive definite matrix:

Geometrically, this means intensity’s c lie on an ellipsoid whose equation is

01222 3223311321122

3332

2222

111 ccqccqccqcqcqcq

332313

232212

131211

qqqqqqqqq

which only has unknowns6

PS with Unknown Lighting

• Given at least six pixels, can solve for q’s• In general, a large overconstrained linear solution is used

3231212

32

22

1

1312131112112

132

122

11

222..................

222

nnnnnnnnn ccccccccc

cccccccccM

Tqqqqqqq 231312332211

1...1b

bMq 1 bMMMq TT 1)( or

PS with Unknown Lighting

• Given Q, we can plug back and get the angles between light directions and their strengths

• Then “lights are known” and solve for normals as before…– but need to choose a reasonable R and G

Photogeometric Approach

• Combine photometric stereo with geometric stereo– High resolution of photometric stereo– Accuracy of geometric method– Can lead to self‐calibration of entire acquisition process

Photogeometric Upsampling

1. Integrate surface normals

photo-surface


2. Compute sparse geometric model

geo-surface


3. Warp photometric surface to geometric surface

geo-surface

photo-surface


3. Warp photometric surface to geometric surfacephoto-geo surface


4. Triangulate and proceed to optimizationphoto-geo surface

true surface

Photogeometric Optimization

• Linear system in the unknown 3D points (pi)• Supports multi‐view reconstruction• Weighted combination of three error terms:

where

eg = error of reprojection

ep = error of perpendicularity of normal‐to‐tangent

er = error of relative distance change

Photogeometric Optimization• Linear system in the unknown 3D points (pi)• Supports multi‐view reconstruction• Weighted combination of three error terms:

where

Photogeometric Reconstruction

photographs reconstruction

photometric and photogeometric stereo - purdue university

Documents