tuesday february 19 th, 2002 deep shadow maps tom lokovic & eric veach pixar animation studios...

Tuesday February 19th, 2002

Deep Shadow Deep Shadow MapsMaps

Tom Lokovic & Eric VeachTom Lokovic & Eric VeachPixar Animation StudiosPixar Animation Studios

Presented by Tom LechnerPresented by Tom Lechner

Tuesday February 19th, 2002 Deep Shadow Maps

OutlineOutlineTraditional Shadow Maps (TSMs)Traditional Shadow Maps (TSMs)Other Shadowing TechniquesOther Shadowing TechniquesDeep Shadow Maps (DSMs)Deep Shadow Maps (DSMs)

GenerationGenerationSamplingSamplingThe Transmittance FunctionThe Transmittance FunctionThe Visibility FunctionThe Visibility FunctionCompressionCompressionLookupsLookups

Comparing DSMs to TSMsComparing DSMs to TSMsImplementationsImplementationsExamplesExamples


So why are Shadow Maps So why are Shadow Maps Important?Important?


Traditional Shadow Maps Traditional Shadow Maps (TSMs)(TSMs)

GenerationGenerationShadow Camera (SC)Shadow Camera (SC)Rectangular array of depths to the closest Rectangular array of depths to the closest surface of a given pixelsurface of a given pixel

SamplingSamplingTransform P into SC coordinate systemTransform P into SC coordinate systemCompare point depth to shadow depthCompare point depth to shadow depthHigher quality images requireHigher quality images require

Percentage closer filteringPercentage closer filtering - examines depth - examines depth samples within a given filter region and computes samples within a given filter region and computes the fraction that are closer than a given depth the fraction that are closer than a given depth zzStratified sampling in both the original shadow map Stratified sampling in both the original shadow map and filtering subsetand filtering subset


Traditional Shadow Maps Traditional Shadow Maps (cont’d)(cont’d)

AdvantagesAdvantagesRenders large objects wellRenders large objects wellStores only one depth value per pixelStores only one depth value per pixel

DisadvantagesDisadvantagesPoorly Renders highly detailed geometries Poorly Renders highly detailed geometries (e.g. fur, hair)(e.g. fur, hair)Produces artifacts, esp. in animation (e.g. Produces artifacts, esp. in animation (e.g. “sparkling”)“sparkling”)Rendering time and memory allocation for Rendering time and memory allocation for detailed renders rapidly increase due to detailed renders rapidly increase due to super samplingsuper sampling


Other Shadowing Other Shadowing TechniquesTechniques

Ray CastingRay Casting Fuzzy objects (potentially millions of hairs) too Fuzzy objects (potentially millions of hairs) too expensive!expensive!No soft shadows (unless using an expensive area No soft shadows (unless using an expensive area light source)light source)Smoke and fog require ray marching for each hairSmoke and fog require ray marching for each hair

3D Texturing3D TexturingHas had some success for volume datasets (clouds, Has had some success for volume datasets (clouds, fog, medical imaging)fog, medical imaging)Relatively course resolutionRelatively course resolutionLow accuracy in Low accuracy in zz (creates bias problems) (creates bias problems)Become prohibitively large as detail increasesBecome prohibitively large as detail increases

Multi-layer Multi-layer ZZ-buffers-buffersRenders opaque surfaces from differing viewpointsRenders opaque surfaces from differing viewpointsSimilar drawbacks to TSMsSimilar drawbacks to TSMs


Deep Shadow Maps Deep Shadow Maps (DSMs)(DSMs)

GenerationGenerationRectangular array of pixels in which every Rectangular array of pixels in which every pixel stores a pixel stores a visibility functionvisibility function

Every value is a function of Every value is a function of transmittancetransmittance - the - the fraction of light that penetrates to a given depth fraction of light that penetrates to a given depth zzThe original beam can be shaped and filtered to The original beam can be shaped and filtered to any particular filterany particular filter

A visibility function is calculated by filtering A visibility function is calculated by filtering the nearby transmittance functions and re-the nearby transmittance functions and re-sampling at the pixel centersampling at the pixel center

= transmittance function

= desired band limiting pixel filter (centered around the origin)

= filter radius


Deep Shadow Maps Deep Shadow Maps (cont’d)(cont’d)

Generation (cont’d)Generation (cont’d)Visibility functions are closely related to Visibility functions are closely related to alpha channelsalpha channelsThus a DSM is equivalent to computing the Thus a DSM is equivalent to computing the approximate value ‘1 - approximate value ‘1 - ’ at all depths, and ’ at all depths, and storing result as a function of storing result as a function of zzContains the combined attenuation and Contains the combined attenuation and coverage information for every depthcoverage information for every depth

A stack of semitransparent objects

Partial coverage of opaque blockers

Volume attenuation due to smoke



SamplingSamplingSelect a set of sample points across the Select a set of sample points across the shadow camera’s image planeshadow camera’s image planeFor each sample point, determine the For each sample point, determine the corresponding transmittance functioncorresponding transmittance function

Given an image point (Given an image point (xx,,yy) compute the surfaces ) compute the surfaces and volume elements intersected by the and volume elements intersected by the corresponding primary raycorresponding primary raysurface transmission functionsurface transmission function * * volume volume transmission functiontransmission function = transmission function of = transmission function of ((xx,,yy))

For each pixel, compute its visibility function For each pixel, compute its visibility function by taking a weighted combination of the by taking a weighted combination of the transmittance function at nearby sample transmittance function at nearby sample points.points.



The Transmittance FunctionThe Transmittance FunctionSurface TransmittanceSurface Transmittance

Each Each surface hitsurface hit has a depth value, has a depth value, zziiss, and an opacity, , and an opacity, OOii

Start with a transparency of 1 and then multiply by 1 – Start with a transparency of 1 and then multiply by 1 – OOii at at every surface hit (resulting in piecewise constant function every surface hit (resulting in piecewise constant function TTss))

Volumetric TransmittanceVolumetric TransmittanceSample atmospheric density at regular intervals along Sample atmospheric density at regular intervals along primary rayprimary rayEach volume sample has a depth value, Each volume sample has a depth value, zzii

vv, and an extinction , and an extinction coefficient, coefficient, kkii, that measures light falloff per unit distance, that measures light falloff per unit distanceLinearly interpolate between the samples to get the Linearly interpolate between the samples to get the extinction functionextinction function, , kk

Since not piecewise linear, approximate by evaluating Since not piecewise linear, approximate by evaluating transmittance at each vertex of the extinction function and transmittance at each vertex of the extinction function and linearly interpolatinglinearly interpolating

Composite like surface transparencies to find Composite like surface transparencies to find TTvv, except this , except this time interpolate between vertices rather than forcing time interpolate between vertices rather than forcing discrete stepsdiscrete steps



The Transmittance Function (cont’d)The Transmittance Function (cont’d)Merge surface and volume transmission Merge surface and volume transmission functionsfunctions

Since result is not piecewise linear, evaluate Since result is not piecewise linear, evaluate it at the combined vertices of it at the combined vertices of TTss and and TTvv, , interpolating linearly between theminterpolating linearly between them



The Visibility FunctionThe Visibility FunctionAt each depth At each depth zz, the nearby transmittance , the nearby transmittance functions are filtered like ordinary image samplesfunctions are filtered like ordinary image samples

Results in a piecewise linear function with Results in a piecewise linear function with approx. n times as many vertices as the approx. n times as many vertices as the transmittance functionstransmittance functionsTakes into account the fractional coverage of Takes into account the fractional coverage of semitransparent surfaces and fog as well as light semitransparent surfaces and fog as well as light attenuationattenuation

= number of transmittance functions within the filter radius = number of transmittance functions within the filter radius around (around (ii + + ½½, , jj + + ½½))= normalized filter weight for each corresponding = normalized filter weight for each corresponding sample point (sample point (xxkk, , yykk))



CompressionCompressionVisibility functions can have a large number Visibility functions can have a large number of vertices depending on filter size and of vertices depending on filter size and number of samples per shadow pixelnumber of samples per shadow pixelFortunately, functions tend to be very smoothFortunately, functions tend to be very smoothCompression must preserve Compression must preserve zz values of values of important features, since simple errors in important features, since simple errors in zz can cause self-shadowing artifactscan cause self-shadowing artifacts



Compression (cont’d)Compression (cont’d)



LookupsLookupsApply a reconstruction and re-sampling filter to a Apply a reconstruction and re-sampling filter to a rectangular array of pixels (similar to textures)rectangular array of pixels (similar to textures)Given a point (Given a point (xx,,yy,,zz) at which to perform a lookup ) at which to perform a lookup and a 2D filter kernel and a 2D filter kernel ff, the filtered shadow value , the filtered shadow value isis

Evaluating each visibility function requires Evaluating each visibility function requires searching through its data points to determine searching through its data points to determine which segment contains the which segment contains the zz-value-valueMay be implemented as a binary or linear searchMay be implemented as a binary or linear search

= filter weight for filter weight for pixel (pixel (ii, , jj))= sum over all pixels in sum over all pixels in filter radiusfilter radius


Comparing DSMs to TSMsComparing DSMs to TSMsSupport prefilteringSupport prefilteringFaster lookupsFaster lookupsMuch smaller in comparison to an equivalent Much smaller in comparison to an equivalent high resolution depth map (dependant upon high resolution depth map (dependant upon compression)compression)

Fortunately, at any sampling rate there is an error Fortunately, at any sampling rate there is an error tolerance that allows significant compression without tolerance that allows significant compression without compromising qualitycompromising qualityShadows of detailed geometry have an expected Shadows of detailed geometry have an expected error of about error of about OO((NN--½½))This error is a measure of the noise inherent in the This error is a measure of the noise inherent in the sampled visibility functionsampled visibility functionActually implemented with a tolerance of 0.25*Actually implemented with a tolerance of 0.25*((NN--½½) – ) – half of the maximum expected noise magnitudehalf of the maximum expected noise magnitudeTSM uses TSM uses OO((NN) storage, where DSM uses ) storage, where DSM uses OO((NN--½½), ), approaches approaches OO((NN-¼-¼) when functions are piecewise ) when functions are piecewise linearlinear


Comparing DSMs to TSMs Comparing DSMs to TSMs (cont’d)(cont’d)

Significantly more expensive to compute than a Significantly more expensive to compute than a regular shadow map at the same pixel regular shadow map at the same pixel resolutionresolutionBias artifacts possible, due to constant Bias artifacts possible, due to constant zz depths depths

Exacerbated by encouragement of large filter widthsExacerbated by encouragement of large filter widthsMight be useful, as it provides an extra degree of Might be useful, as it provides an extra degree of freedomfreedom

Shadow resolution should be chosen according to minimum Shadow resolution should be chosen according to minimum filter width desired (shadow detail)filter width desired (shadow detail)Number of samples per pixel should be determined by Number of samples per pixel should be determined by maximum acceptable noisemaximum acceptable noise

Note: Bias artifacts and computational expense Note: Bias artifacts and computational expense of DSMs are no worse off than TSMs that are of DSMs are no worse off than TSMs that are equivalent in terms of pixel resolution or equivalent in terms of pixel resolution or sample sizesample size



512x512 512x512 TSMTSM

4Kx4K TSM4Kx4K TSM 512x512 512x512 DSMDSM


ImplementationsImplementationsIncremental UpdatesIncremental Updates

Can be optimized to proceed in Can be optimized to proceed in OO((nnloglognn))

Colored ShadowsColored ShadowsAt expense of twice the storageAt expense of twice the storageAllows for some compression for gray shadowsAllows for some compression for gray shadows

Mip-MappingMip-MappingCan dramatically reduce lookup costs when Can dramatically reduce lookup costs when objects are viewed over a wide range of scalesobjects are viewed over a wide range of scales

Tiling and CachingTiling and CachingStored similar to textures and share in some of Stored similar to textures and share in some of their advantagestheir advantagesTile directoryTile directory

Motion BlurMotion Blur


ExamplesExamplesWith and without DSMsWith and without DSMs


Examples (cont’d)Examples (cont’d)


Questions?Questions?E.G.E.G.

Was there anything you didn’t understand Was there anything you didn’t understand about my explanation of the algorithms?about my explanation of the algorithms?Do you see the correlation between DSMs Do you see the correlation between DSMs and Light Fields?and Light Fields?Do you need clarification on the Do you need clarification on the implementations?implementations?

Thanks!Thanks!

tuesday february 19 th, 2002 deep shadow maps tom lokovic & eric veach pixar animation studios...

Documents

shadow maps important

original shadow map

tsms slide

function of transmittance

function of z

shadow cameras image

smoke slide

point depth