scalable many-light methods jaroslav křivánek charles university in prague
TRANSCRIPT
![Page 1: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/1.jpg)
Scalable many-light methods
Jaroslav KřivánekCharles University in Prague
![Page 2: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/2.jpg)
• Approximate indirect illumination byVirtual Point Lights (VPLs)
1. Generate VPLs
2
Instant radiosity
2. Render with VPLs
![Page 3: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/3.jpg)
• Large number of VPLs required• True even for diffuse scenes• Scalability issues
3
Ground truth 1,000 VPLs 100,000 VPLs
Instant radiosity with glossy surfaces
![Page 4: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/4.jpg)
1. Generate many, many VPLs
2. Pick only the most relevant VPLs for rendering
4
Scalable many-light methods
![Page 5: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/5.jpg)
• Choosing the right VPLs– Per-pixel basis
• Lightcuts [Walter et al 05/06]– Per-image basis
• Matrix Row Column Sampling [Hašan et al. 07]
– Somewhere in-between• LightSlice [Ou & Pellacini 2011]• Importance caching [Georgiev et al. 2012]
5
Scalable many-light methods
![Page 6: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/6.jpg)
Scalable many-light rendering
LightcutsMultidimensional
LightcutsWalter et al., SIGGRAPH 2005/2006
Slides courtesy Bruce Walter:http://www.graphics.cornell.edu/~bjw/papers.html
![Page 7: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/7.jpg)
• http://www.graphics.cornell.edu/~bjw/papers.html
7
Lightcuts
![Page 8: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/8.jpg)
Complex Lighting
• Simulate complex illumination using VPLs– Area lights– HDR environment maps– Sun & sky light– Indirect illumination
• Unifies illumination
Area lights + Sun/sky + Indirect
![Page 9: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/9.jpg)
Scalable
• Scalable solution for many point lights– Thousands to millions– Sub-linear cost
0
100
200
300
400
500
600
0 1000 2000 3000 4000
Number of Point Lights
Tim
e (s
ecs)
Standard
Ward
Lightcut
Tableau Scene
![Page 10: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/10.jpg)
Lightcuts Problem
Visiblesurface
![Page 11: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/11.jpg)
Lightcuts Problem
![Page 12: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/12.jpg)
Lightcuts Problem
Camera
![Page 13: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/13.jpg)
Key Concepts
• Light Cluster– Approximate many lights by a single
brighter light (the representative light)
![Page 14: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/14.jpg)
Key Concepts
• Light Cluster• Light Tree
– Binary tree of lights and clusters
Clusters
IndividualLights
![Page 15: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/15.jpg)
Key Concepts
• Light Cluster• Light Tree• A Cut
– A set of nodes that partitions the lights into clusters
![Page 16: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/16.jpg)
Simple Example
#1 #2 #3 #4
1 2 3 4
1 4
Light Tree
Clusters
IndividualLights
RepresentativeLight
4
![Page 17: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/17.jpg)
Three Example Cuts
1 2 3 4
1 4
4
1 2 3 4
1 4
4
1 2 3 4
1 4
4
Three Cuts
#1 #2 #4 #1 #3 #4 #1 #4
![Page 18: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/18.jpg)
Three Example Cuts
1 2 3 4
1 4
4
1 2 3 4
1 4
4
1 2 3 4
1 4
4
Three Cuts
#1 #2 #4 #1 #3 #4 #1 #4
Good Bad Bad
![Page 19: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/19.jpg)
Three Example Cuts
1 2 3 4
1 4
4
1 2 3 4
1 4
4
1 2 3 4
1 4
4
Three Cuts
#1 #2 #4 #1 #3 #4 #1 #4
Bad Good Bad
![Page 20: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/20.jpg)
Three Example Cuts
1 2 3 4
1 4
4
1 2 3 4
1 4
4
1 2 3 4
1 4
4
Three Cuts
#1 #2 #4 #1 #3 #4 #1 #4
Good Good Good
![Page 21: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/21.jpg)
Algorithm Overview
• Pre-process– Convert illumination to point lights– Build light tree
• For each eye ray– Choose a cut to approximate the
illumination
![Page 22: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/22.jpg)
Convert Illumination• HDR environment map
– Importance sampling
• Indirect Illumination– Convert indirect to direct
illumination using Instant Radiosity [Keller 97]• Caveats: no caustics, clamping,
etc.– More lights = more indirect
detail
![Page 23: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/23.jpg)
Build light tree
• Cluster spatially close lights with similar orientation
1 2 3 4
1 4
Light Tree
Clusters
IndividualLights
RepresentativeLight
4
![Page 24: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/24.jpg)
Choose a cut
• Approximate illumination with a bounded error
• Different cut for each pixel
![Page 25: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/25.jpg)
Illumination Equation
Material term
result = Mi Gi Vi IiSlights
Geometric term
Visibility term
Light intensity
![Page 26: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/26.jpg)
Cluster Approximation
Cluster
result ≈ Mj Gj Vj IiSlights
j is the representative light
Sum pre-computed during light tree construction
![Page 27: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/27.jpg)
error < Mub Gub Vub Ii
Cluster Error Bound
Cluster
Slights
-• Bound each term
– Visibility <= 1 (trivial)
– Intensity is known– Bound material and
geometric terms using cluster bounding volume
ub = upper bound
![Page 28: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/28.jpg)
Perceptual Metric
• Weber’s Law– Contrast visibility threshold is fixed
percentage of signal– Used 2% in our results
• Ensure each cluster’s error < visibility threshold– Transitions will not be visible– Used to select cut
![Page 29: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/29.jpg)
Perceptual Metric
• Problem: – We don’t know the illumination so we
don’t know the threshold either • (because threshold = 2% illumination)
• Solution: – As we traverse the tree, gradually
improve the illumination estimate.– Stop the traversal if the error bound for
all cut nodes is below threshold.
![Page 30: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/30.jpg)
Cut Selection Algorithm
Cut
• Start with coarse cut (eg, root node)
![Page 31: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/31.jpg)
Cut Selection Algorithm
Cut
• Select cluster with largest error bound
![Page 32: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/32.jpg)
Cut Selection Algorithm
Cut
• Refine if error bound > 2% of total
![Page 33: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/33.jpg)
Cut Selection Algorithm
Cut
![Page 34: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/34.jpg)
Cut Selection Algorithm
Cut
![Page 35: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/35.jpg)
Cut Selection Algorithm
Cut
![Page 36: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/36.jpg)
Cut Selection Algorithm
Cut
• Repeat until cut obeys 2% threshold
![Page 37: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/37.jpg)
Lightcuts (128s) Reference (1096s)
Error Error x16
Kitchen, 388K polygons, 4608 lights (72 area sources)
![Page 38: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/38.jpg)
Combined Illumination
Lightcuts 128s
4 608 Lights(Area lights only)
Avg. 259 shadow rays / pixel
Lightcuts 290s
59 672 Lights(Area + Sun/sky + Indirect)
Avg. 478 shadow rays / pixel(only 54 to area lights)
![Page 39: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/39.jpg)
Lightcuts Reference
Error x 16Cut size
![Page 40: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/40.jpg)
Scalable
• Scalable solution for many point lights– Thousands to millions– Sub-linear cost
0
100
200
300
400
500
600
0 1000 2000 3000 4000 5000
Number of Point Lights
Tim
e (s
ecs)
Standard
Ward
Lightcuts
0
100
200
300
400
500
600
0 1000 2000 3000 4000
Number of Point Lights
Tim
e (s
ecs)
Standard
Ward
Lightcut
Tableau Scene Kitchen Scene
![Page 41: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/41.jpg)
• Data-driven stratification & importance sampling
• Stratification– Clustering of similar lights in the light
tree
• Importance sampling– Subdividing clusters with high
contribution
41
Why does it work so well?
![Page 42: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/42.jpg)
• Problem: Large cuts in dark areas
42
Main issue
![Page 43: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/43.jpg)
Lightcuts Recap
• Key ingredients
– Upper bound on error
– Refinement of the highest-error nodes first
![Page 44: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/44.jpg)
Multidimensional Lightcuts
http://www.graphics.cornell.edu/~bjw/papers.html
![Page 45: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/45.jpg)
Problem
• Simulate complex, expensive phenomena – Complex illumination– Anti-aliasing– Motion blur– Participating media– Depth of field
Pixel L(x,)...Lights
PixelArea
Aperture
Volume
Time
Pixel L(x,)...Lights
PixelArea
Time
![Page 46: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/46.jpg)
Problem
• Simulate complex, expensive phenomena – Complex illumination– Anti-aliasing– Motion blur– Participating media– Depth of field
Pixel L(x,)...Lights
PixelArea
Time
Volume
![Page 47: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/47.jpg)
Problem
• Simulate complex, expensive phenomena – Complex illumination– Anti-aliasing– Motion blur– Participating media– Depth of field
Pixel L(x,)...Lights
PixelArea
Time
Volume
Aperture
![Page 48: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/48.jpg)
• Complex integrals over multiple dimensions
– Requires many samples
Pixel L(x,)...Lights
PixelArea
Time
Volume
Aperture
camera
Problem
![Page 49: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/49.jpg)
Multidimensional Lightcuts
• Solves all integrals simultaneously
• Accurate• Scalable
0
600
1200
1800
0 100 200 300
Samples per pixel
Ima
ge
tim
e (
secs
) Supersampling
Multidimensional
![Page 50: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/50.jpg)
Direct only (relative cost 1x) Direct+Indirect (1.3x)
Direct+Indirect+Volume (1.8x) Direct+Indirect+Volume+Motion (2.2x)
![Page 51: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/51.jpg)
Camera
• Discretize full integral into 2 point sets– Light points (L)– Gather points (G)
Point Sets
Light points
![Page 52: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/52.jpg)
Camera
• Discretize full integral into 2 point sets– Light points (L)– Gather points (G)
Point Sets
Light points
![Page 53: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/53.jpg)
Camera
• Discretize full integral into 2 point sets– Light points (L)– Gather points (G)
Point Sets
Light points
Pixel
Gather points
![Page 54: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/54.jpg)
• Discretize full integral into 2 point sets– Light points (L)– Gather points (G)
Point Sets
Light points
Gather points
![Page 55: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/55.jpg)
• Sum over all pairs of gather and light points– Can be billions of pairs per pixel
Material term
Pixel = Sj Mji Gji Vji Ii( j,i) Î GxL
Geometric term
Visibility term
Light intensity
Gather strength
Discrete Equation
![Page 56: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/56.jpg)
Product Graph
• Explicit hierarchy would be too expensive– Up to billions of pairs per pixel
• Use implicit hierarchy – Cartesian product of two trees (gather &
light)
![Page 57: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/57.jpg)
Light tree
Gather tree
Product Graph
L0 L1 L2 L3
L4 L5
L6
G1G0
G2
L0L1 L2
L3
G0
G1
![Page 58: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/58.jpg)
Product Graph
Light tree
Gather tree
X =L0 L1 L2 L3
L4 L5
L6
G1G0
G2G1
G0
G2
L0 L4 L1 L6 L2 L5 L3
Product Graph
![Page 59: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/59.jpg)
Product Graph
Light tree
Gather tree
X =L0 L1 L2 L3
L4 L5
L6
G1G0
G2G1
G0
G2
L0 L4 L1 L6 L2 L5 L3
Product Graph
![Page 60: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/60.jpg)
Product Graph
G1
G0
G2
L0 L4 L1 L6 L2 L5 L3
Product Graph
![Page 61: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/61.jpg)
Product Graph
G1
G0
G2
L0 L4 L1 L6 L2 L5 L3
Product Graph
![Page 62: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/62.jpg)
Product Graph
Light tree
Gather tree
X =L0 L1 L2 L3
L4 L5
L6
G1G0
G2G1
G0
G2
L0 L4 L1 L6 L2 L5 L3
Product Graph
![Page 63: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/63.jpg)
Cluster Representatives
![Page 64: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/64.jpg)
Cluster Representatives
![Page 65: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/65.jpg)
• Collapse cluster-cluster interactions to point-cluster – Minkowski sums– Reuse bounds
from Lightcuts
• Compute maximum over multiple BRDFs– Rasterize into cube-maps
• More details in the paper
Error Bounds
![Page 66: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/66.jpg)
Algorithm Summary
• Once per image– Create lights and light tree
• For each pixel– Create gather points and gather tree for
pixel– Adaptively refine clusters in product
graph until all cluster errors < perceptual metric
![Page 67: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/67.jpg)
L6
G2
L1 L2 L3L4 L5L0
G1
G0
• Start with a coarse cut
– Eg, source node of product graph
Scalability
![Page 68: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/68.jpg)
L6
G2
L1 L2 L3L4 L5L0
G1
G0
• Choose node with largest error bound & refine
– In gather or light tree
Scalability
![Page 69: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/69.jpg)
L6
G2
L1 L2 L3L4 L5L0
G1
G0
• Choose node with largest error bound & refine
– In gather or light tree
Scalability
![Page 70: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/70.jpg)
L6
G2
L1 L2 L3L4 L5L0
G1
G0
• Repeat process
Scalability
![Page 71: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/71.jpg)
L6
G2
L1 L2 L3L4 L5L0
G1
G0
• Until all clusters errors < perceptual metric
– 2% of pixel value (Weber’s law)
Algorithm summary
![Page 72: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/72.jpg)
Results
• Limitations– Some types of paths not included
• Eg, caustics– Prototype only supports diffuse, Phong,
and Ward materials and isotropic media
![Page 73: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/73.jpg)
Roulette
7,047,430 Pairs per pixel Time 590 secsAvg cut size 174 (0.002%)
![Page 74: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/74.jpg)
0
400
800
1200
1600
0 50 100 150 200 250 300
Ima
ge
tim
e (
se
cs
)
Gather points (avg per pixel)
Image time vs. Gather points
Multidimensional
Original lightcuts
Eye rays only
Scalability
![Page 75: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/75.jpg)
Metropolis Comparison
Our resultTime 9.8min
MetropolisTime 148min (15x) Visible noise5% brighter (caustics etc.)
Zoomed insets
![Page 76: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/76.jpg)
Kitchen
5,518,900 Pairs per pixel Time 705 secsAvg cut size 936 (0.017%)
![Page 77: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/77.jpg)
180 Gather points X 13,000 Lights = 234,000 Pairs per pixel
Avg cut size 447 (0.19%)
![Page 78: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/78.jpg)
Scalable many-light rendering
Matrix Row-Column sampling
Hašan et al., SIGGRAPH 2007
Slides courtesy Miloš Hašan:http://www.cs.cornell.edu/~mhasan/
![Page 79: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/79.jpg)
• http://miloshasan.net/• Designed specifically for GPU
rendering (shadow mapping)79
Matrix Row-Column sampling
![Page 80: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/80.jpg)
Improving Scalability and Performance
10 min
13 min
20 min
3.8 sec 13.5 sec 16.9 sec
Brute force
(100k VPLs)
MRCS(our result)
80
![Page 81: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/81.jpg)
81
A Matrix Interpretation
Pixels(2,000,00
0)
Lights (100,000)
![Page 82: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/82.jpg)
• Compute sum of columns
82
Problem Statement
= Σ ( )Pi
xels
Lights
![Page 83: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/83.jpg)
83
Low-Rank Assumption
= Σ ( )
Pixe
ls
Lights
• Column space is (close to) low-dimensional
![Page 84: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/84.jpg)
84
Ray-tracing vs Shadow Mapping
Lights
Pixe
ls
Point-to-point visibility: Ray-tracing
Point-to-many-points visibility: Shadow-mapping
![Page 85: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/85.jpg)
85
Computing Column Visibility
Shadow map at light position
Surface samples
• Regular Shadow Mapping Lights
Pixe
ls
1 shadow map
![Page 86: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/86.jpg)
86
Row-Column Duality
Shadow map at sample position
• Rows: Also Shadow Mapping! Lights
Pixe
ls
1 shadow map
![Page 87: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/87.jpg)
Image as a Weighted Column Sum
Compute small subset of columns
(i.e. pick some lights)
compute weighted
sum
• The following is possible:
Use rows to choose a good set of columns (=lights)
87
Lights
Pixe
ls
![Page 88: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/88.jpg)
88
The Row-Column Sampling Idea
compute rows
(i.e. pick pixels, compute contrib from ALL lights
for each)
compute columns
(i.e. for the selected lights,
compute contribution to
all pixels)
weighted sum
?
choose columns (=lights) and
weights
how to choose columns and
weights?
Pixe
ls
Lights
![Page 89: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/89.jpg)
89
Clustering Approach
ClusteringChoose
representative columns
Columns
![Page 90: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/90.jpg)
90
Reduced Matrix
Reduced columns
![Page 91: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/91.jpg)
• Algorithm:1. Cluster reduced columns2. Choose a representative in each
cluster, with probability proportional to weight
3. Approximate other columns in cluster by (scaled) representative
• This is an unbiased Monte Carlo estimator (of the sum of matrix columns)
• Which clustering minimizes its variance?
91
Monte Carlo Estimator
![Page 92: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/92.jpg)
Weights and Information Vectors
• Weights wi
– Norms of reduced columns – Represent the “energy” of the light
• Information vectors xi
– Normalized reduced columns– Represent the “kind” of light’s
contribution
![Page 93: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/93.jpg)
93
Visualizing the Reduced Columns
Reduced columns: vectors
in high-dimensional
spacevisualize as …
radius = weight
position = information vector
![Page 94: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/94.jpg)
• Minimize:
• where:
94
The Clustering Objective
total cost of all clusters
cost of a cluster
sum over all pairs in
it
weights squared distance between
information vectors
![Page 95: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/95.jpg)
Clustering Illustration
Strong but similar columns
Weak columns can be clustered more easily
Columns with various intensities can be clustered
![Page 96: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/96.jpg)
How to minimize?
• Problem is NP-hard• Not much previous research• Should handle large input:
– 100,000 points– 1000 clusters
• We introduce 2 heuristics:– Random sampling– Divide & conquer
![Page 97: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/97.jpg)
Clustering by Random Sampling
Very fast (use optimized BLAS)Some clusters might be too small /
large
![Page 98: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/98.jpg)
Clustering by Divide & Conquer
Splitting small clusters is fast
Splitting large clusters is slow
![Page 99: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/99.jpg)
Combined Clustering Algorithm
![Page 100: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/100.jpg)
Combined Clustering Algorithm
![Page 101: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/101.jpg)
101
Full Algorithm
Compute rows (GPU)
Weighted sum
Assemble rows into reduced
matrix
Cluster reduced columns
Choose representatives
Compute columns (GPU)
Lights
Pixe
ls
![Page 102: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/102.jpg)
• 2.1m polygons• Mostly indirect & sky
illumination• Indirect shadows
102
Example: Temple
Our result: 16.9 sec
(300 rows + 900 columns)
Reference: 20 min (using all 100k
lights)
5x diff
![Page 103: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/103.jpg)
• 388k polygons• Mostly indirect illumination• Glossy surfaces• Indirect shadows
103
Example: Kitchen
Our result: 13.5 sec
(432 rows + 864 columns)
Reference: 13 min (using all 100k lights)
5x diff
![Page 104: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/104.jpg)
• 869k polygons• Incoherent geometry• High-frequency
lighting• Kajiya-Kay hair
shader
104
Example: Bunny
Our result: 3.8 sec
(100 rows + 200 columns)
Reference: 10 min (using all 100k lights)
5x diff
![Page 105: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/105.jpg)
105
Effect of exploration
# rows = 30028 sec
# rows = 90035 sec
Too few VPLs
#VPLs = 250k, # cols = 10k
![Page 106: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/106.jpg)
106
Comparison
Lightcuts (2M VPLs)30 sec
MRCS (250k VPLs)35 sec
![Page 107: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/107.jpg)
• Data-driven stratification & importance sampling– Same reason as for Lightcuts
• Stratification– Split clusters with dissimilar lights
• Importance sampling– Split clusters with high-contribution
lights
107
Why does it work so well?
![Page 108: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/108.jpg)
• Advantage– Takes visibility into account in light
selection (reduced matrix contains the full light contributions)
• Drawback– Impossible to capture localized effects
• Low likelihood of getting the right row sample
• Global light selection
108
Comparison to Lightcuts
![Page 109: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/109.jpg)
http://www.cs.dartmouth.edu/~fabio/publication.php?id=lightslice11
109
LightSlice
![Page 110: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/110.jpg)
• Get the better from Lightcuts and MRCS
– Lightcuts: Localized selection of relevant lights
– MRCS: Take visibility into account in light selection
110
LightSlice: Idea
![Page 111: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/111.jpg)
• Cannot use shadow maps for row/column sampling
• Clustering borrowed from the original MRCS paper
111
LightSlice: The Algorithm
![Page 112: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/112.jpg)
112
LightSlice: Matrix Slices
ImageMatrix slice visualization
![Page 113: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/113.jpg)
• Not really a fair comparison to MRCS (uses ray traced visibility)
113
LightSlice: Results
![Page 114: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/114.jpg)
114
Importance Caching
• Localized light (VPL) selection taking visibility into account
![Page 115: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/115.jpg)
1. Importance caching– Pick image pixels (2000 - 3000)– For each such pixel, compute and
cache contributions from all lights (i.e. matrix row)
2. Image rendering– For each image pixel
• Collect nearby importance records• Pick a VPL proportional to the cached
contributions
115
Importance Caching: Algorithm
![Page 116: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/116.jpg)
116
Cached light importance
Importance record 1Importance record 2Importance record 3
Contributions to the record locationof individual lights
![Page 117: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/117.jpg)
117
Possible problems
OK Occlusion Geometry factor Irrelevant
![Page 118: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/118.jpg)
118
Solution: Multiple importancesFull light contribution Unoccluded contribution
Unoccluded contribution w/ bounded G-term
Uniform distribution
![Page 119: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/119.jpg)
• Multiple importance sampling (MIS)– See CG III slides
http://cgg.mff.cuni.cz/~jaroslav/teaching/2011-pg3/slides/krivanek-07-npgr010-2011-mc2.pptx
• New combination heuristic proposed in the paper
119
Distribution combination
![Page 120: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/120.jpg)
120
Results
![Page 121: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/121.jpg)
121
Results: A 2-second rendering
![Page 122: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/122.jpg)
• Fewer initial VPLs than any of the previous algorithms (up to 10k)– Because of memory limitations– Must run in iterations if more VPLs
needed
• No data-driven stratification
122
Importance Caching: Limitations
![Page 123: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/123.jpg)
• Data-driven importance sampling may be dangerous
– Non-zero contribution may be sampled with very low, or even zero probability
– Being conservative is safer, but can reduce the advantage of data-driven IS
– For more information see [Owen & Zhou 2000] http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.36.2813
123
Take-home message
![Page 124: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/124.jpg)
• MRCS– In theory suffers from the same
problem, but– light importance averaged over full
columns, so it works ok– BTW: Why do you think there’s global
clustering in LightSlice
• Lightcuts– Has upper bounds so it knows which
lights can be safely skipped124
How ‘bout Lightcuts and MRCS?
![Page 125: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/125.jpg)
• Lightcuts [Walter et al 05/06]
• Matrix Row Column Sampling [Hašan et al. 07]
• LightSlice [Ou & Pellacini 2011]
• Importance caching [Georgiev et al. 2012]
125
Scalable many-light methods
![Page 126: Scalable many-light methods Jaroslav Křivánek Charles University in Prague](https://reader038.vdocuments.us/reader038/viewer/2022110323/56649d835503460f94a68c04/html5/thumbnails/126.jpg)
• [Segovia et al. 07]: Metropolis instant radiosity– http://www710.univ-lyon1.fr/~jciehl/bsegovia/bat710/public_html/p
apers/mir.html
Use Metropolis sampling to guide the VPL where they can contribute to the image
• [Georgiev & Slussalek 2010]– http://www.iliyan.com/p/publications.html– Use rejection sampling to reject VPLs that
do not contribute significantly126
Improved VPL distribution