practical implementation of light scattering effects using ... · render coord texture detect depth...

Practical Implementation of Light Scattering Effects Using Epipolar Sampling and 1D Min/Max Binary Trees

Egor Yusov

Graphics Software Engineer – Intel Corporation

• Introduction

• Light Scattering Theory

• Solution to Inscattering Integral

• Epipolar Sampling

• 1D Min/Max Binary Trees

• Implementation

• Performance

Agenda

• Light scattering greatly enhances the realism

• It is hard to compute

• Achieving high-performance is challenging

Introduction

𝐿𝐼𝑛𝑃𝑡 𝐂, 𝐎, 𝐋 = 𝑒−𝑇 𝐏→𝐂 𝛽 𝐏 𝑝 𝜃

𝐼𝑒−𝑇 𝐋→𝐏

𝐋 − 𝐏 𝟐

𝑆𝑂

𝑆𝐶

𝑉 𝐏 𝑑𝑠

𝑇 𝐀 → 𝐁 = 𝛽 𝐏 𝑑𝑠𝐁

𝐀 𝑝𝑅 𝜃 =

16𝜋1 + cos2 𝜃

𝑝𝑀 𝜃 =1

3(1 − 𝑔2)

2(2 + 𝑔2)

1 + 𝑐𝑜𝑠2𝜃

1 + 𝑔2 − 2𝑔𝑐𝑜𝑠𝜃 3/2

• Hoffman & Preetham, 2002 – Alanalytical model for outdoor light scattering

• Sun et al, 2005 – Semi-analytical solution to airlight integral

• Tevs et al, 2008 – Using maximum mipmaps for dynamic height field rendering

• Chen et al, 2011 – Using 1d min-max mipmaps for volumetric shadows

• Engelhardt & Dachsbacher, 2010 – Epipolar Sampling

Previous Work

Inscattering Integral Derivation

Aerial Perspective

For homogeneous medium

𝐿𝐶𝑎𝑚 = 𝐿𝑂𝑏𝑗 ∙ 𝑒−𝑇 𝐎→𝐂 +𝐿𝐼𝑛

𝑇 𝐀 → 𝐁 = 𝛽 𝐏 𝑑𝑠𝐁

= 𝛽 𝐀 − 𝐁

Camera

Light source

Object

Radiant Intensity of a Point Light

• I – radiant intensity

• 𝑣 = 𝐎−𝐂𝐎−𝐂

– view direction

𝐿𝐼𝑛 = 𝐼

Attenuation and optical thickness

• Amount of energy reaching P is

− Inversely proportional to distance squared

− Attenuated by the extinction in the media

𝐿𝐼𝑛 = 𝐼 𝑒−𝑇 𝐋→𝐏

𝐋 − 𝐏 𝟐

Visibility factor

• 𝑉 𝐏 = 1 if point is lit and 0 if shadowed

− Filtered in practice

𝐋 − 𝐏 𝟐 𝑉 𝐏

Scattering coefficient

• Total scattered light (in any direction) is proportional to

− Section length 𝑑𝑠

− Scattering coefficient 𝛽 𝐏

𝐋 − 𝐏 𝟐 𝑉 𝐏 𝛽 𝐏 𝑑𝑠

𝑑𝑠

Phase function

• Phase function gives amount of light scattered towards the camera

𝐋 − 𝐏 𝟐 𝑉 𝐏 𝑝 𝜃 𝛽 𝐏 𝑑𝑠

P 𝑑𝐿𝐼𝑛

𝑑𝐿𝐼𝑛 𝜃

Attenuation of inscattered light

• Differential inscattered light is attenuated in the media

• Total inscattering is given by integrating differential amounts

𝐋 − 𝐏 𝟐 𝑉 𝐏 𝑝 𝜃 𝛽 𝐏 𝑒−𝑇 𝐏→𝐂 𝑑𝑠

P 𝑑𝐿𝐼𝑛

𝑒−𝑇 𝐏→𝐂

Directional light

• Phase function does not vary across the ray

• Light intensity is constant

𝐿𝐼𝑛 = 𝑝 𝜃 𝑒−𝑇 𝐏→𝐂 𝛽 𝐏 𝑉 𝐏 𝐸𝑆𝑢𝑛

𝑑𝑠

Scattering Properties of the Media

Rayleigh scattering

• Caused by molecules

• Wavelength-dependent

𝛽𝑅 . rgb = (5.8, 13.5, 33.1) × 10−6

• Almost isotropic:

𝑝𝑅 𝜃 =3

16𝜋(1 + cos2 𝜃)

Mie Scattering

• Caused by aerosols

• Wavelength-independent

𝛽𝑀. rgb = 2.0 × 10−5

• Anisotropic

𝑝𝑀 𝜃 =1

3(1 − 𝑔2)

2(2 + 𝑔2)

1 + 𝑔2 − 2𝑔𝑐𝑜𝑠𝜃 3/2

Scattering due to both types of particles

• 𝑇 𝐀 → 𝐁 = 𝛽Σ 𝐀 − 𝐁

𝛽Σ = 𝛽𝑅 + 𝛽𝑀

• 𝛽𝑝 𝜃 → 𝛽𝑅 ∙ 𝑝𝑅 𝜃 + 𝛽𝑀 ∙ 𝑝𝑀 𝜃 ≡ 𝑃Σ (𝜃)

Scattering Properties of the Media Isotropic phase

function

𝑝𝑀 𝜃 =1

Scattering Properties of the Media Anisotropic phase

function

𝑝𝑀 𝜃 = 1

3(1 − 𝑔2)

2(2 + 𝑔2)∙

1 + 𝑔2 − 2𝑔𝑐𝑜𝑠𝜃 3/2

Solution to Inscattering Integral

Directional light – fully analytical solution

𝐿𝐼𝑛𝐷𝑖𝑟 𝜃, 𝑆 =

𝐸𝑆𝑢𝑛 ∙ 𝑃Σ (𝜃)

𝛽Σ1 − 𝑒−𝛽Σ∙𝑆

𝑆 = 𝐎 − 𝐂

𝑠 ℎ

𝜃 𝐋0

Point light source – integral reformulation

𝐿𝐼𝑛𝑃𝑡 = 𝐿𝐼𝑛

𝑃𝑡 ℎ, 𝑆𝐶 , 𝑆𝑂 =

𝐼 𝑃Σ (𝜃) 𝑆𝑂

𝑆𝐶

∙𝑒−𝛽Σ 𝑠−𝑆𝐶 + ℎ2+𝑠2

ℎ2 + 𝑠2𝑑𝑠

𝐋 − 𝐏 2

𝐏 − 𝐂 + 𝐋 − 𝐏

cos 𝜃 = −𝑠

ℎ2 + 𝑠2

𝑆𝑂 𝑆𝐶

Point light source – look-up table

Precompute 2D LUT:

ℒ ℎ, 𝑆 = 𝐿𝐼𝑛𝑃𝑡 ℎ, 𝑆, +∞

Point light source – semi-analytical solution

𝐿𝐼𝑛𝑃𝑡 ℎ, 𝑆𝐶 , 𝑆𝑂 =

ℒ ℎ, 𝑆𝐶

−𝑒−𝛽Σ 𝑆𝑂−𝑆𝐶 ℒ ℎ, 𝑆𝑂

𝑇 𝐂 → 𝐎 𝑆𝑂 𝑆𝐶

Volumetric Shadows

𝑆0𝐸𝑛𝑡 𝑆0

𝐸𝑥𝑡 𝑆1𝐸𝑛𝑡 𝑆2

𝐸𝑛𝑡 𝑆2𝐸𝑥𝑡 𝑆1

𝐸𝑥𝑡

𝑒−𝛽Σ∙ 𝑆𝑖𝐸𝑛𝑡−𝑆𝐶 ℒ ℎ, 𝑆𝑖

𝐸𝑛𝑡

−𝑒−𝛽Σ∙ 𝑆𝑖𝐸𝑥𝑡−𝑆𝐶 ℒ ℎ, 𝑆𝑖

𝐸𝑥𝑡

𝐿𝐼𝑛𝑃𝑡 =

𝑛−1

𝑖=0

𝑇 𝑆𝑖𝐸𝑥𝑡 → 𝐂

𝑇 𝑆𝑖𝐸𝑛𝑡 → 𝐂

𝐿𝐼𝑛𝐷𝑖𝑟 = 𝐹(𝜃) ∙ 𝑒−𝛽Σ∙𝑆𝑖

𝐸𝑛𝑡− 𝑒−𝛽Σ∙𝑆𝑖

𝐸𝑥𝑡

𝑛−1

𝑖=0

𝐹 𝜃 =𝐸𝑆𝑢𝑛 ∙ 𝑃Σ (𝜃)

𝛽Σ

Volumetric Shadows

𝑆0𝐸𝑛𝑡 𝑆0

𝐸𝑥𝑡 𝑆1𝐸𝑛𝑡 𝑆2

𝐸𝑛𝑡 𝑆2𝐸𝑥𝑡 𝑆1

𝐸𝑥𝑡

Volumetric Shadows

Starting point - tracing the view ray in shadow map space

For each pixel:

•Project the view ray onto the shadow map

•Set up 𝑃𝑟𝑒𝑣𝐿𝐼𝑛. 𝑟𝑔𝑏 = ℒ ℎ, 𝑆𝐶 , 𝐿𝐼𝑛. 𝑟𝑔𝑏 = 0

•Step through each texel:

−𝐶𝑢𝑟𝑟𝐿𝐼𝑛. 𝑟𝑔𝑏 = 𝑒−𝛽Σ∙ 𝑆−𝑆𝐶 ℒ ℎ, 𝑆

−𝐿𝐼𝑛. 𝑟𝑔𝑏 += 𝑃𝑟𝑒𝑣𝐿𝐼𝑛. 𝑟𝑔𝑏 − 𝐶𝑢𝑟𝑟𝐿𝐼𝑛. 𝑟𝑔𝑏 ∙ 𝑉(𝐏)

−𝑃𝑟𝑒𝑣𝐿𝐼𝑛. 𝑟𝑔𝑏 = 𝐶𝑢𝑟𝑟𝐿𝐼𝑛. 𝑟𝑔𝑏

𝑃𝑟𝑒𝑣𝐿𝐼𝑛. 𝑟𝑔𝑏 𝐶𝑢𝑟𝑟𝐿𝐼𝑛. 𝑟𝑔𝑏

Volumetric Shadows

Perspective-correct interpolation

𝑧=1

𝑧𝐶+ 𝛼1

𝑧𝑂−1

𝑧𝐶

𝑧=𝑆𝐶𝑧𝐶+ 𝛼𝑆𝑂𝑧𝑂−𝑆𝐶𝑧𝐶

Shadow

Map plane

𝑆𝐶

𝑆𝑂

𝑧𝐶

𝑧𝑂

𝑧 𝛼

1 − 𝛼

Volumetric Shadows

Starting point - tracing the view ray in shadow map space

• Need to test all samples along the ray

• Too slow (> 90 ms on GeForce 680 GTX)

• Optimizations are necessary!

Epipolar Sampling

Sample generation

Exit point

Entry point

Exit point

Epipolar Sampling

Sample generation – improved sample placement

Too dense sampling

Much better sampling

Epipolar Sampling

Coordinate texture

Sample coordinate on each line

Epipolar Sampling

Sample classification

Initial ray marching

samples are placed

equidistantly

Additional ray marching

samples are placed at

depth breaks The remaining samples are interpolation

samples

Epipolar Sampling

Interpolation source texture

0 1 2 3 4 5 6 7 8

0,0 0,3 0,3 3,3 4,4 4,8 4,8 4,8 8,8

Epipolar Sampling

Transformation from epipolar to rectangular coordinates

• Cast epipolar line through the pixel

• Perfrom bilateral filtering

− One Gather() and two Sample() instructions are used

Epipolar Sampling

Fix-up pass

• Not all samples can be properly interpolated from epipolar coordinates

• These samples are marked in stencil

• Additional fix-up pass is performed without 1D min/max acceleration

Epipolar Sampling

91 ms -> 14 ms

We can do better!

1D min/max binary trees

1D height map & first level

Second level

Third level

Traversal optimization

𝑑𝑚𝑖𝑛

𝐀 𝐁

𝑑𝑚𝑎𝑥

max 𝑑𝐀, 𝑑𝐁

min 𝑑𝐂, 𝑑𝐃

𝐂 𝐃

𝐄 𝐅

• max 𝑑𝐀, 𝑑𝐁 < 𝑑𝑚𝑖𝑛: lit

• min 𝑑𝐂, 𝑑𝐃 > 𝑑𝑚𝑎𝑥: shadowed

• EF: neither lit nor shadowed

Shadow map

Slice Origin and Direction

𝐎𝑈𝑉 𝐷𝑈𝑉

• Location of the i-th sample is

𝐎𝑈𝑉 + 𝑖 ∙ 𝐷𝑈𝑉

𝐎𝑈𝑉 𝐷𝑈𝑉

𝑆 1

𝑆 0

Exit point View

𝑁𝐿𝑖𝑔ℎ𝑡

𝑁𝑆𝑙𝑖𝑐𝑒

𝑁𝑆𝑙𝑖𝑐𝑒 = 𝑆 1 × 𝑆 0

𝐷 = 𝑁𝑆𝑙𝑖𝑐𝑒 × 𝑁𝐿𝑖𝑔ℎ𝑡

𝑆 0 =𝐋 − 𝐂

𝐋 − 𝐂

𝑆 1 =𝐒𝐄𝐱𝐢𝐭 − 𝐂

𝐒𝐄𝐱𝐢𝐭 − 𝐂

𝑂 𝑆ℎ𝑎𝑑𝑜𝑤 𝑚𝑎𝑝 𝑝𝑙𝑎𝑛𝑒

𝑆 1

𝑆 0

𝑁𝐿𝑖𝑔ℎ𝑡

𝐏0 = 𝑐1𝑁𝑆𝑙𝑖𝑐𝑒 + 𝑐2𝑁𝐿𝑖𝑔ℎ𝑡 + 𝑡𝐷

𝐎 = 𝐏0 + 𝑡𝑚𝑖𝑛 ∙ 𝑫

𝑡𝑚𝑖𝑛 𝑡𝑚𝑎𝑥

𝑆ℎ𝑎𝑑𝑜𝑤 𝑚𝑎𝑝 𝑝𝑙𝑎𝑛𝑒

𝑆 1

−𝐿

𝑆 1

𝐷 𝑂

−𝐿

Colored Light Shafts

Implementation Details

Auxiliary textures

Name Format Dimension

Back Buffer RGBA_UNORM8 𝑊𝑆𝑐𝑟𝑒𝑒𝑛 × 𝐻𝑆𝑐𝑟𝑒𝑒𝑛

Camera Space Z R_FLOAT32 𝑊𝑆𝑐𝑟𝑒𝑒𝑛 × 𝐻𝑆𝑐𝑟𝑒𝑒𝑛

Camera Depth Stencil D24_S8 𝑊𝑆𝑐𝑟𝑒𝑒𝑛 × 𝐻𝑆𝑐𝑟𝑒𝑒𝑛

Coordinate texture RG_FLOAT32 𝑁𝑆𝑎𝑚𝑝𝑙𝑒𝑠 × 𝑁𝑆𝑙𝑖𝑐𝑒𝑠

Epipolar depth-stencil D24_S8 𝑁𝑆𝑎𝑚𝑝𝑙𝑒𝑠 × 𝑁𝑆𝑙𝑖𝑐𝑒𝑠

Epipolar camera space Z R_FLOAT32 𝑁𝑆𝑎𝑚𝑝𝑙𝑒𝑠 × 𝑁𝑆𝑙𝑖𝑐𝑒𝑠

Interpolation source RG_UINT16 𝑁𝑆𝑎𝑚𝑝𝑙𝑒𝑠 × 𝑁𝑆𝑙𝑖𝑐𝑒𝑠

Inscattering × 2 RGBA_FLOAT16 𝑁𝑆𝑎𝑚𝑝𝑙𝑒𝑠 × 𝑁𝑆𝑙𝑖𝑐𝑒𝑠

Slice End Points RGBA_FLOAT32 𝑁𝑆𝑙𝑖𝑐𝑒𝑠 ×1

Slice UV origin & direction RGBA_FLOAT32 𝑁𝑆𝑙𝑖𝑐𝑒𝑠 ×1

1D min/max shadow map × 2 RG_FLOAT16 𝐷𝑆ℎ𝑎𝑑𝑜𝑤𝑀𝑎𝑝 × 𝑁𝑆𝑙𝑖𝑐𝑒𝑠

Algorithm specific data

Implementation

1. Reconstruct camera space Z

Camera space Z

Camera depth buffer

3. Render coordinate texture

Epipolar depth stencil

Interpolation

source texture

Interpolation

source texture

5. Render Slice Origin and Direction in

Shadow Map Space

5. Render Slice Origin and Direction in

Shadow Map Space

2. Render Slice End Points in Screen Space

Slice End Points

Slice UV Orig & Dir 4. Refine Sample

Locations 4. Refine Sample

Locations

Epipolar camera space Z

Epipolar coordinates

Implementation

Epipolar coordinates

Epipolar depth stencil

Camera space Z

7. Mark Ray Marching Samples

8. Do Ray Marching

Initial inscattering

6. Build 1D min/max mipmap

Shadow map

1D min-max mipmap

Slice UV Orig & Dir

Implementation Epipolar

camera space Z

Camera space Z Initial inscattering

9. Interpolate inscattering

Interpolated inscattering

10. Transform to rectangular coordinates

Source Color Buffer

Slice End Points

Implementation

Initial inscattering

Source Color Buffer

12. Fix inscattering

Final image

Camera space Z

Shadow map

Discontinuities search with CS

• Each ray section is processed by one thread group

• Each thread processes one sample

Thread

Discontinuities search with CS

• Step 1: each thread sets 1-bit flag in group shared array indicating if there is a depth break

−InterlockedOr() must be used

• Step 2: find closest depth break or section end for each sample using bit manipulation and firstbitlow() and firstbithigh() intrinsics

0 0 0 1 0 0 0 0

Up to 6x speed-up

Min/max binary tree construction

• Step 1: construct 1st tree level using shadow map

− Use Gather() instructions to read min/max z for each pair of two adjacent samples

− Compute conservative min/max z

• Step 2: Construct coarser tree levels

− Use temporary texture to render to

One texel

Optimized sample placement

+1 2/H

Less efficient sample

placement

More efficient sample

placement

Gives more than 5% speedup

Performance

Reconstruct

Render coord

texture

Detect depth

breaks

Build 1D

min/max trees

Ray march Interpolate Transform to

rect coords

Inscattering fix-

2560 × 1600 resolution, NVIDIA GeForce 680 GTX, spot light

Profile # slices # samples SM Time, ms Mem, MB

Brute Force 40962 91.18

High Quality 2048 1024 40962 5.986 88.0

Balanced 1024 1024 20482 3.269 44.0

High performance 1024 512 10242 2.267 18.0

Brute Force Ray Marching

High Quality

Balanced

High Performance

Performance

Reconstruct

Render coord

texture

Detect depth

breaks

Build 1D

min/max trees

rect coords

Inscattering fix-

1366 × 768 resolution, Nvidia Quadro 1000M, spot light

Brute Force 20482 164

High Quality 1024 1024 20482 18.9 44.0

Balanced 512 512 10242 8.37 11.0

High Quality

Balanced

High Performance

Performance

Reconstruct

Render coord

texture

Detect depth

breaks

Build 1D

min/max trees

rect coords

Inscattering fix-

1024 × 768, Intel CPU with 3rd Gen HD Graphics, spot light

Brute Force 20482 184.79

High Quality 1024 1024 20482 28.83 44.0

Balanced 512 512 10242 12.85 11.0

High Quality

Balanced

High Performance

0.07 0.01 0.09 0.31

0.52 0.26

Reconstruct

Render slice

enter/exit

points

Render

texture

Detect depth

breaks

Render slice

origin and

Build 1D

min/max

rect coords

Inscattering

fix-up

Time, ms

Performance

• Spot light, balanced quality

• 1280 × 960 resolution, NVIDIA GeForce 480 GTX

• 1024 slices, 512 samples in slice, 2048 × 2048 Shadow Map

3.366 ms

0.07 0.01 0.10 0.31

Reconstruct

Render slice

enter/exit

points

Render

texture

Detect depth

breaks

Render slice

origin and

Build 1D

min/max

rect coords

Inscattering

fix-up

Time, ms

Performance

• Directional light, high quality

• 1280 × 960 resolution, NVIDIA GeForce 480 GTX

• 1024 slices, 512 samples in slice, 4096 × 4096 Shadow Map

4.076 ms

Ray marching, 5.49

Ray marching, 1.67

No binary trees With 1D min/max trees

Time, ms

Binary tree construction,

0.38 ms

Performance

> 2x faster

Performance

Colored light shafts

• Binary tree optimization is less efficient

−Ray marching: 1.67 ms -> 5.49 ms

−Total time: 3.36 ms -> 6.86 ms

• Still less than 7 ms on NVIDIA GeForce 480 GTX

• Semi-analytical solution is used to calculate the integral on each ray segment

− No sophisticated mathematics like SVD etc.

• Epipolar scattering is combined with 1D min/max binary trees

− Significant performance improvements

• Efficient implementation methods

Advantages Over Previous Approaches

• The technique is fully post-processing

• The only inputs are:

− Depth buffer

− Back buffer

− Shadow map

− [optional] Stained glass color

• The source code is available at http://software.intel.com/en-us/blogs/2013/03/18/gtd-light-scattering-sample-updated

http://software.intel.com/gamecode

Integration Into Existing Apps

• Large outdoor environments

• Integration with cascaded SM, VSM/EVSM

• Heterogeneous (dynamic) media

− Clouds

− Smoke

− Sparse octree/PRT

Extensions

• HOFFMAN N., PREETHAM A. J.: Rendering outdoor light scattering in real time. In Proceedings Game Developer Conference 2002.

• ENGELHARDT T., DACHSBACHER C.: Epipolar sampling for shadows and crepuscular rays in participating media with single scattering. In Proc. 2010 ACM SIGGRAPH symposium on Interactive 3D Graphics and Games, ACM, 119–125.

• CHEN J., BARAN I., DURAND F., JAROSZ W.: Real-time volumetric shadows using 1d min-max mipmaps. In Proceedings of the Symposium on Interactive 3D Graphics and Games, 39–46.

• GAUTRON P., MARVIE J.-E., FRANCOIS G.: Volumetric shadow mapping. ACM SIGGRAPH 2009 Sketches.

• TEVS A., IHRKE I., SEIDEL H.-P.: Maximum mipmaps for fast, accurate, and scalable dynamic height field rendering. In Symposium on Interactive 3D Graphics and Games (i3D’08), 183–190.

• SUN B., RAMAMOORTHI R., NARASIMHAN S. G., NAYAR S. K.: A practical analytic single scattering model for real time rendering. ACM Trans. Graph. 24, 3, 1040–1049.

References

The presenter would like to thank:

• Artem Brizitsky

• Jeffery A. Williams

• Doug Mcnabb

• Dave Bookout

• Charu Chandrasekaran

• Quentin Froemke

• Glen Lewis

• Kyle Weicht

Acknowledgments

Thank you!

practical implementation of light scattering effects using ... · render coord texture detect depth...

Documents

id & building services coord 2

claude gauvard - robert jacob (coords.), tes et rituels

sect. 3.11: transformation to lab coords

improving coord

3d transformations, homogeneous coords

europatria - oliveira, francisco (coord.)

enemy coords

polar coords

an algorithm for interpolate discrete time signals

the academy cafe life coord

wk13- stability of coord compounds

insul coord 800 kv.pdf

coord rectangulares polares

coord iss-08-05

nerc prot coord

system sector region coord

issn 2249-8524 original article interpolate the rate...

1 o path reference frame (x,y) coord r (r, ) coord x yr...

conversion coord utm a topograficas

insul coord 800 kv