sources and shading - vision labsvision.psych.umn.edu/users/schrater/schrater_lab/... · computer...

Computer Vision - A Modern ApproachSet: Sources, shadows and shading

Slides by D.A. Forsyth

Sources and shading

• How bright (or what colour) areobjects?

• One more definition: Exitanceof a source is– the internally generated power

radiated per unit area on theradiating surface

• similar to radiosity: a sourcecan have both– radiosity, because it reflects– exitance, because it emits

• General idea:

• But what aspects of theincoming radiance will wemodel?

†

B(x) = E (x) +radiosity due toincoming radiance

Ï Ì Ó

¸ ˝ ˛ dw

W

Ú



Radiosity due to a point sources

• small, distant sphere radius eand exitance E, which is faraway subtends solid angle ofabout

†

ped

Ê Ë

ˆ ¯

2



Radiosity due to a point source

• Radiosity is

†

B x( ) = pLo x( )= rd x( ) Li x,w( )cosq idw

W

Ú

= rd x( ) Li x,w( )cosq idwDÚ

ª rd x( ) solid angle( ) Exitance term( )cosq i

=rd x( )cosq i

r x( )2 Exitance term and some constants( )



Standard nearby point source model

• N is the surface normal• rho is diffuse albedo• S is source vector - a vector

from x to the source, whoselength is the intensity term– works because a dot-product is

basically a cosine

†

rd x( ) N x( ) •S x( )r x( )2

Ê

Ë Á

ˆ

¯ ˜



Standard distant point source model

• Issue: nearby point source getsbigger if one gets closer– the sun doesn’t for any

reasonable binding of closer• Assume that all points in the

model are close to each otherwith respect to the distance tothe source. Then the sourcevector doesn’t vary much, andthe distance doesn’t vary mucheither, and we can roll theconstants together to get:

†

rd x( ) N x( )• Sd x( )( )



Shadows cast by a point source

• A point that can’t see the sourceis in shadow

• For point sources, the geometryis simple



Line sources

radiosity due to line source varies with inverse distance, if the source is long enough



Area sources

• Examples: diffuser boxes, whitewalls.

• The radiosity at a point due toan area source is obtained byadding up the contribution overthe section of view hemispheresubtended by the source– change variables and add up

over the source



Radiosity due to an area source

• rho is albedo• E is exitance• r(x, u) is

distancebetweenpoints

• u is acoordinate onthe source

†

B x( ) = rd x( ) Li x,u Æ x( )cosqidwW

Ú

= rd x( ) Le x, u Æ x( )cosq idwW

Ú

= rd x( ) E u( )p

Ê

Ë Á ˆ

¯ cosqidw

W

Ú

= rd x( )E u( )

p

Ê

Ë Á ˆ

¯ sourceÚ cosqi cosq s

dAu

r(x, u)2Ê Ë Á ˆ

¯

= rd x( ) E u( ) cosqi cosqs

pr(x,u)2 dAusourceÚ



Area Source Shadows



Shading models

• Local shading model– Surface has radiosity due only

to sources visible at each point– Advantages:

• often easy to manipulate,expressions easy

• supports quite simpletheories of how shapeinformation can beextracted from shading

• Global shading model– surface radiosity is due to

radiance reflected from othersurfaces as well as fromsurfaces

– Advantages:• usually very accurate

– Disadvantage:• extremely difficult to infer

anything from shadingvalues



Photometric stereo

• Assume:– a local shading model– a set of point sources that are infinitely distant– a set of pictures of an object, obtained in exactly the same

camera/object configuration but using different sources– A Lambertian object (or the specular component has been

identified and removed)



Projection model for surface recovery - usually calleda Monge patch



Image model

• For each point source, we knowthe source vector (byassumption). We assume weknow the scaling constant of thelinear camera. Fold the normaland the reflectance into onevector g, and the scalingconstant and source vector intoanother Vj

• Out of shadow:

• In shadow:

†

I j(x,y) = 0

†

I j (x, y) = kB(x, y)

= kr(x, y) N(x, y) •S j( )= g(x, y)• Vj



Dealing with shadows

†

I12(x, y)

I22(x, y)

..In

2(x, y)

Ê

Ë

Á Á Á Á

ˆ

¯

˜ ˜ ˜ ˜

=

I1(x, y) 0 .. 00 I2(x, y) .. .... .. .. 00 .. 0 In(x, y)

Ê

Ë

Á Á Á Á

ˆ

¯

˜ ˜ ˜ ˜

V1T

V2T

..Vn

T

Ê

Ë

Á Á Á Á

ˆ

¯

˜ ˜ ˜ ˜

g(x, y)

Known Known Known Unknown



Recovering normal and reflectance

• Given sufficient sources, we can solve the previousequation (most likely need a least squares solution) for

g(x, y)• Recall that N(x, y) is the unit normal• This means that r(x,y) is the magnitude of g(x, y)• This yields a check

– If the magnitude of g(x, y) is greater than 1, there’s a problem• And N(x, y) = g(x, y) / r(x,y)



Example figures



Recovered reflectance



Recovered normal field



Recovering a surface from normals - 1

• Recall the surface is written as

• This means the normal has theform:

• If we write the known vector gas

• Then we obtain values for thepartial derivatives of thesurface:

†

(x, y, f (x, y))

†

g(x, y) =

g1(x, y)g2 (x, y)g3(x, y)

Ê

Ë

Á Á

ˆ

¯

˜ ˜

†

fx (x, y) = g1(x, y) g3(x, y)( )

†

fy(x, y) = g2(x, y) g3(x, y)( )

†

N(x, y) =1

fx2 + fy

2 +1Ê

Ë Á

ˆ

¯ ˜

- fx

- fy

1

Ê

Ë

Á Á

ˆ

¯

˜ ˜



Recovering a surface from normals - 2

• Recall that mixed secondpartials are equal --- this givesus a check. We must have:

(or they should be similar, atleast)

• We can now recover the surfaceheight at any point byintegration along some path,e.g.

†

∂ g1(x, y) g3(x, y)( )∂y

=

∂ g2(x, y) g3(x, y)( )∂x

†

f (x, y) = fx (s, y)ds0

x

Ú +

fy (x, t)dt0

y

Ú + c



Surface recovered by integration



Curious Experimental Fact

• Prepare two rooms, one with white walls and whiteobjects, one with black walls and black objects

• Illuminate the black room with bright light, the white roomwith dim light

• People can tell which is which (due to Gilchrist)

• Why? (a local shading model predicts they can’t).


Slides by D.A. ForsythFigure from “Mutual Illumination,” by D.A. Forsyth and A.P. Zisserman, Proc. CVPR, 1989, copyright 1989 IEEE

A view of awhite room,under dim light.Below, we see across-section ofthe imageintensitycorresponding tothe line drawnon the image.



A view of a blackroom, underbright light.Below, we see across-section ofthe imageintensitycorresponding tothe line drawnon the image.



What’s going on here?

• local shading model is a poor description of physicalprocesses that give rise to images– because surfaces reflect light onto one another

• This is a major nuisance; the distribution of light (inprinciple) depends on the configuration of every radiator;big distant ones are as important as small nearby ones(solid angle)

• The effects are easy to model• It appears to be hard to extract information from these

models



Interreflections - a global shading model

• Other surfaces are now area sources - this yields:

• Vis(x, u) is 1 if they can see each other, 0 if they can’t

†

Radiosity at surface = Exitance + Radiosity due to other surfaces

B x( ) = E x( )+ rd x( ) B u( )cosqi cosqs

pr(x,u)2 Vis x,u( )dAuall othersurfaces

Ú



What do we do about this?

• Attempt to build approximations– Ambient illumination

• Study qualitative effects– reflexes– decreased dynamic range– smoothing

• Try to use other information to control errors



Ambient Illumination

• Two forms– Add a constant to the radiosity at every point in the scene to

account for brighter shadows than predicted by point source model• Advantages: simple, easily managed (e.g. how would you

change photometric stereo?)• Disadvantages: poor approximation (compare black and white

rooms– Add a term at each point that depends on the size of the clear

viewing hemisphere at each point (see next slide)• Advantages: appears to be quite a good approximation, but

jury is out• Disadvantages: difficult to work with



At a point inside a cube or room, the surface sees light in alldirections, so add a large term. At a point on the base of a groove,the surface sees relatively little light, so add a smaller term.



Reflexes

• A characteristic feature of interreflections is little brightpatches in concave regions– Examples in following slides– Perhaps one should detect and reason about reflexes?– Known that artists reproduce reflexes, but often too big and in the

wrong place



At the top, geometry of asemi-circular bump on aplane; below, predictedradiosity solutions, scaledto lie on top of each other,for different albedos ofthe geometry. Whenalbedo is close to zero,shading follows a localmodel; when it is close toone, there are substantialreflexes.



Radiosity observed in animage of this geometry;note the reflexes, whichare circled.

Figure from “Mutual Illumination,” by D.A. Forsyth and A.P. Zisserman, Proc. CVPR, 1989, copyright 1989 IEEE



At the top, geometry of agutter with triangularcross-section; below,predicted radiositysolutions, scaled to lie ontop of each other, fordifferent albedos of thegeometry. When albedois close to zero, shadingfollows a local model;when it is close to one,there are substantialreflexes.



Radiosity observed in animage of this geometry;above, for a black gutterand below for a white one




At the top, geometry of agutter with triangularcross-section; below,predicted radiositysolutions, scaled to lie ontop of each other, fordifferent albedos of thegeometry. When albedois close to zero, shadingfollows a local model;when it is close to one,there are substantialreflexes.



Radiosity observed in animage of this geometryfor a white gutter.




Smoothing

• Interreflections smooth detail– E.g. you can’t see the pattern of a stained glass window by looking

at the floor at the base of the window; at best, you’ll see colouredblobs.

– This is because, as I move from point to point on a surface, thepattern that I see in my incoming hemisphere doesn’t change allthat much

– Implies that fast changes in the radiosity are local phenomena.



Fix a small patch near alarge radiator carrying aperiodic radiosity signal;the radiosity on the surfaceis periodic, and its amplitudefalls very fast with thefrequency of the signal. Thegeometry is illustratedabove. Below, we show agraph of amplitude as afunction of spatialfrequency, for differentinclinations of the smallpatch. This means that ifyou observe a highfrequency signal, it didn’tcome from a distant source.

sources and shading - vision labsvision.psych.umn.edu/users/schrater/schrater_lab/... · computer...

Documents