matrix row-column sampling for the many-light problem
DESCRIPTION
Matrix Row-Column Sampling for the Many-Light Problem. Milo š Ha š an (Cornell University) Fabio Pellacini (Dartmouth College) Kavita Bala (Cornell University). Complex Illumination: A Challenge. Conversion to Many Lights. Area, indirect, sun/sky. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/1.jpg)
Matrix Row-Column Sampling for the Many-Light Problem
Matrix Row-Column Sampling for the Many-Light Problem
Miloš Hašan (Cornell University)
Fabio Pellacini (Dartmouth College)
Kavita Bala (Cornell University)
![Page 2: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/2.jpg)
Complex Illumination: A Challenge
![Page 3: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/3.jpg)
Conversion to Many Lights
Courtesy Walter et al., Lightcuts, SIGGRAPH 05/06
• Area, indirect, sun/sky
![Page 4: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/4.jpg)
A Matrix Interpretation
Pixels(2,000,000)
Lights (100,000)
![Page 5: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/5.jpg)
• Compute sum of columns
• Note: We don’t have the matrix data
Problem Statement
= Σ ( )P
ixel
s
Lights
![Page 6: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/6.jpg)
Indirect Illumination Many Lights
=
100,000 point lights
Σ ( )
![Page 7: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/7.jpg)
Environment Map Many Lights
=
100,000 point lights
Σ ( )
![Page 8: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/8.jpg)
Sun, Sky, Indirect Many Lights
=
100,000 point lights
Σ ( )
![Page 9: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/9.jpg)
Brute Force Takes Minutes
• Why not sum all columns? – With 100,000 lights, still several minutes
10 min 13 min 20 min
![Page 10: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/10.jpg)
Our Contribution• Fast, accurate, GPU-based approximation
• Application: Preview for lighting design
10 min 13 min 20 min
3.8 sec 13.5 sec 16.9 sec
Brute force:
Our result:
![Page 11: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/11.jpg)
Related Work• Many lights (CPU-based): Walter et al 05/06,
Ward 94, Paquette et al 98, Wald et al 03, …• Instant radiosity & related: Keller 97,
Dachsbacher & Stamminger 05/06, Laine et al 07, …• Environment maps: Agarwal et al 03,
Ostromoukhov et al 04, …• Precomputation-based: Sloan et al 02/03, Ng
et al 03/04, Ben-Artzi et al 06, Hasan et al 06, Ritschel et al 07, …
• Other global illumination: Ward et al 88, Jensen 96, Hanrahan et al 91, Christensen 97, Scheel 01/02, Gautron et al 05, Krivanek et al 06, Dachsbacher et al 07, …
![Page 12: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/12.jpg)
Insight #1: Matrix has structure
• Compute small subset of elements• Reconstruct
A simple scene
30 x 30 image The matrix
643 lights
900
pixe
ls
![Page 13: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/13.jpg)
Insight #2: Sampling Pattern Matters
Lights
Pix
els
Point-to-point visibility: Ray-tracingPoint-to-many-points visibility: Shadow-mapping
![Page 14: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/14.jpg)
Row-Column Duality
Shadow map at light position
Surface samples
• Columns: Regular Shadow Mapping
![Page 15: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/15.jpg)
Row-Column Duality
Shadow map at sample position
• Rows: Also Shadow Mapping!
![Page 16: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/16.jpg)
Image as a Weighted Column Sum
compute very small subset of columns
compute weighted sum
• The following is possible:
• Use rows to choose a good set of columns!
![Page 17: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/17.jpg)
Exploration and Exploitation
compute rows (explore)
compute columns (exploit)
weighted sum
?
choose columns and weights
how to choose columns and
weights?
![Page 18: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/18.jpg)
Reduced Matrix
Reduced columns
![Page 19: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/19.jpg)
Clustering Approach
Choose k clustersChoose
representative columns
Reduced columns
![Page 20: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/20.jpg)
Reduced Full
Use the same representatives for
the full matrix
Weighted sum
Representative columns
![Page 21: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/21.jpg)
Visualizing the Reduced Columns
Reduced columns: vectors in high-
dimensional space
visualize as …
radius = norm
![Page 22: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/22.jpg)
Clustering Illustration
Strong but similar columns
Weak columns can be clustered more easily
Columns with various intensities can be clustered
![Page 23: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/23.jpg)
• Minimize:
• where:
The Clustering Metric
total cost of all clusters
cost of a cluster
sum over all pairs in it
norms of the reduced columns
squared distance between normalized
reduced columns
![Page 24: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/24.jpg)
How to minimize?
• Problem is NP-hard• Not much previous research• Should handle large input:
– 100,000 points– 1000 clusters
• We introduce 2 heuristics:– Random sampling– Divide & conquer
![Page 25: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/25.jpg)
Clustering by Random Sampling
Very fast (use optimized BLAS)
Some clusters might be too small / large
![Page 26: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/26.jpg)
Clustering by Divide & Conquer
Splitting small clusters is fast
Splitting large clusters is slow
![Page 27: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/27.jpg)
Combined Clustering Algorithm
![Page 28: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/28.jpg)
Combined Clustering Algorithm
![Page 29: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/29.jpg)
Full Algorithm
Compute rows (GPU)
Weighted sum
Assemble rows into reduced matrix
Cluster reduced columns
Choose representatives
Compute columns (GPU)
![Page 30: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/30.jpg)
Results
• We show 5 scenes:
• Show reference and 5x difference image• All scenes have 100,000+ lights• Timings
– NVidia GeForce 8800 GTX– Light / surface sample creation not included
Temple BunnyKitchen Trees Grand Central
![Page 31: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/31.jpg)
Results: Kitchen• 388k polygons• Mostly indirect illumination• Glossy surfaces• Indirect shadows
Our result: 13.5 sec (432 rows + 864 columns)
Reference: 13 min (using all 100k lights)
5x diff
![Page 32: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/32.jpg)
Results: Temple
• 2.1m polygons• Mostly indirect & sky illumination• Indirect shadows
Our result: 16.9 sec (300 rows + 900 columns)
Reference: 20 min (using all 100k lights)
5x diff
![Page 33: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/33.jpg)
Results: Trees• 328k polygons• Complex incoherent geometry
Our result: 2.9 sec (100 rows + 200 columns)
Reference: 14 min (using all 100k lights)
5x diff
![Page 34: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/34.jpg)
Results: Bunny• 869k polygons• Incoherent geometry• High-frequency lighting• Kajiya-Kay hair shader
Our result: 3.8 sec (100 rows + 200 columns)
Reference: 10 min (using all 100k lights)
5x diff
![Page 35: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/35.jpg)
Results: Grand Central
• 1.5m polygons• Point lights between
stone blocks
Our result: 24.2 sec (588 rows + 1176 columns)
Reference: 44 min (using all 100k lights)
5x diff
![Page 36: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/36.jpg)
The Value of Exploration
Our result(432 rows + 864 columns)
No exploration(Using 1455 lights)
Equal time comparison
![Page 37: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/37.jpg)
The Value of Exploration
Our result No exploration
Equal time comparison: 5x difference from reference
![Page 38: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/38.jpg)
Conclusion
• Fast, high quality approximation for many lights– GPU-oriented– Sample rows to explore low-rank structure– Sample well-chosen columns
• Application: Preview for lighting design– Indirect illumination– Environment maps– Arbitrary lights and shaders
![Page 39: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/39.jpg)
Future Work
• How many rows + columns?– Pick automatically
• Row / column alternation
• Progressive algorithm:– stop when user likes the image
• Render multiple frames at once?
![Page 40: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/40.jpg)
Acknowledgments
• Veronica Sundstedt and Patrick Ledda– Temple scene
• Bruce Walter, PCG @ Cornell• NSF CAREER 0644175• Affinito-Stewart Award
![Page 41: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/41.jpg)
Thank You
![Page 42: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/42.jpg)
Discarded slides
![Page 43: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/43.jpg)
Indirect Illumination Many Lights
• shoot photons from light sources
• deposit on every bounce
• treat photons as point lights
• cosine-weighted emission
![Page 44: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/44.jpg)
Low Rank Assumption
• Worst case: lights with very local contribution
…
![Page 45: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/45.jpg)
The Value of Exploration
Our result(432 rows + 864 columns)
No exploration(Using 1992 lights)
Equal time comparison
![Page 46: Matrix Row-Column Sampling for the Many-Light Problem](https://reader035.vdocuments.us/reader035/viewer/2022062221/56813dad550346895da77523/html5/thumbnails/46.jpg)
The Value of Exploration
Our result No exploration
Equal time comparison: 5x difference image