![Page 1: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/1.jpg)
PRT Summary
![Page 2: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/2.jpg)
Motivation for Precomputed Transfer
• better light integration and light transport – dynamic, area lights – shadowing – interreflections
• in real-time
point lightpoint light area lightarea light
area lighting,area lighting,no shadowsno shadows
area lighting,area lighting,shadowsshadows
![Page 3: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/3.jpg)
Precomputed Radiance Transfer (PRT)
represent lighting using spherical harmonics low frequency = few coefficients
n=4n=4 n=9n=9 n=25n=25
n=676n=676 originaloriginal
![Page 4: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/4.jpg)
Tabulating PRT
LightinLighting Basis g Basis
ii
LightingLightingBasis Basis i+1i+1
LightingLightingBasis Basis i+2i+2
illuminateilluminate store store coefs on coefs on surfacesurface
......
......
PRT is a linear operator (matrix) per PRT is a linear operator (matrix) per surface point maps incident lighting to surface point maps incident lighting to
exit radianceexit radiance
![Page 5: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/5.jpg)
Self-Transfer Results (Diffuse)
No Shadows/Inter Shadows Shadows+InterNo Shadows/Inter Shadows Shadows+Inter
![Page 6: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/6.jpg)
Self-Transfer Results (Glossy)
No Shadows/Inter Shadows Shadows+InterNo Shadows/Inter Shadows Shadows+Inter
![Page 7: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/7.jpg)
PRT Terminology
![Page 8: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/8.jpg)
PRT Terminology
![Page 9: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/9.jpg)
PRT Terminology
![Page 10: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/10.jpg)
PRT Terminology
![Page 11: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/11.jpg)
( ) ( ) ( )T Tp p p p p pe v v v y e y M l
p pe M l
PRT as a Linear Operator
• l : light vector (in source basis)• Mp : source-to-exit transfer matrix• ep : exit radiance vector (in exit basis)• y(vp) : exit basis evaluated in direction vp
• ep(vp) : exit radiance in direction vp
![Page 12: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/12.jpg)
Tp pe y M l
PRT Special Case: Diffuse Objects
transfer vector rather than matrix
• independent of view (constant exit basis)• matrix is row vector• previous work uses different light bases• image relighting
[PRT02] SH[Xi03] Directional[Ng03] Haar[Ashikhmin02] Steerable
![Page 13: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/13.jpg)
( ) ( )Tp p p pe v v y M l
PRT Special Case: Surface Light Fields
transfer vector rather than matrix
• frozen lighting environment• matrix is column vector
[Miller98][Nishino99][Wood00][Chen02][Matusik02]
![Page 14: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/14.jpg)
Factoring PRT (BRDFs)
( ) ( )
p p p
Tp p p pe v v
e B R T l
y e
• Tp: source → transferred incident radiance
• Rp : rotate to local frame
• B: integrate against BRDF [Westin92]
• y(vp) • ep: evaluate exit radiance at vp
![Page 15: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/15.jpg)
Hemispherical Projection
• exit radiance is defined over hemisphere, not sphere• spherical harmonics not orthogonal over hemisphere• how to project hemispherical functions using SH?
– naïve projection assumes “underside” is zero– least squares projection minimizes approximation error
• see appendix
![Page 16: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/16.jpg)
Factoring PRT (BRDFs)
Technique LightB ExitB Note
[Sloan02] SH SH Phong
[Kautz02] SH Dir Arb
[Lehtinen03] SH Dir Lsq
[Matusik02] Dir Dir IBR
( ) ( )Tp p p p pe v v y B R T l
![Page 17: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/17.jpg)
Extending PRT to BSSRDFs
• already handled by original equation• use [Jensen02], only multiple scattering (matrix
with only 1 row)• mix with “conventional” BRDF
( ) ( )Tp p p p
p p p p
e v v
y M l
M S B R T
![Page 18: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/18.jpg)
Problems With PRT
• Big matrices at each surface point– 25-vectors for diffuse, x3 for spectral
– 25x25-matrices for glossy
– at ~50,000 vertices
• Slows glossy rendering (4hz)– Frozen View/Light can increase performance
– Not as GPU friendly
• Limits diffuse lighting order– Only very soft shadows
![Page 19: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/19.jpg)
Compression Goals
• Decode efficiently– As much on the GPU as possible– Render compressed representation directly
• Increase rendering performance– Make non-diffuse case practical
• Reduce memory consumption– Not just on disk
![Page 20: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/20.jpg)
Compression Example
Surface is curve, signal is normal
![Page 21: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/21.jpg)
Compression Example
Signal Space
![Page 22: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/22.jpg)
VQ
Cluster normals
![Page 23: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/23.jpg)
VQ
Replace samples with cluster mean
pp p C M M M
![Page 24: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/24.jpg)
PCA
Replace samples with mean + linear combination
0
1
Ni i
p p pi
w
M M M M
![Page 25: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/25.jpg)
0
1p p
Ni i
p p C p Ci
w
M M M M
CPCA
Compute a linear subspace in each cluster
![Page 26: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/26.jpg)
CPCA
• Clusters with low dimensional affine models• How should clustering be done?
• Static PCA– VQ, followed by one-time per-cluster PCA– optimizes for piecewise-constant reconstruction
• Iterative PCA– PCA in the inner loop, slower to compute– optimizes for piecewise-affine reconstruction
![Page 27: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/27.jpg)
Static vs. Iterative
![Page 28: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/28.jpg)
Related Work
• VQ+PCA [Kambhatla94] (static)• VQPCA [Khambhatla97] (iterative)• Mixture PC [Dony95] (iterative)• More sophisticated models exist
– [Brand03], [Roweis02]– Mapping to current GPUs is challenging
• Variable storage per vertex• Partitioning is more difficult (or requires more
passes)
![Page 29: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/29.jpg)
Equal Rendering Cost
VQVQ PCAPCA CPCACPCA
![Page 30: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/30.jpg)
0
1
( ) ( )p p
NT i i
p p p C p Ci
e v v w
y M l M l p
iCe 0
pCe
Rendering with CPCA
( ) ( )Tp p p pe v v y M l
0
1p p
Ni i
p C p Ci
w
M M M
0
1
( ) ( )p p
NT i i
p p p C p Ci
e v v w
y M M l
![Page 31: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/31.jpg)
0
1
( ) ( )p p
NT i i
p p p C p Ci
e v v w
y e e
Rendering with CPCA
Constant per cluster – precompute on the CPURendering is a dot productCompute linear combination of vectors
Only depends on # rows of M
![Page 32: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/32.jpg)
Non-Local Viewer
Assume: • vp constant across object (distant viewer)
Rendering independent of view & light orders- linear combination of colors
0
1
( ) ( )p p
NT i i
p p p C p Ci
e v v w
y M l M l
0
1
( ) ( )p p
NT i T i
p g C p g Ci
e v w v
y M l y M l
![Page 33: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/33.jpg)
Rendering
= + +
![Page 34: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/34.jpg)
Overdraw
1
1
1
3
3
22
2 2
22
22
22
2
2
2
faces belong to 1-3 clusters
OD = 1 face drawn onceOD = 2 face drawn 2xOD = 3 face drawn 3x
coherence optimization:• reclassification• superclustering
![Page 35: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/35.jpg)
PixelShader
TextureConstants
Eye PointXform Matrix
GPU Dataflow
Vertices
1
NormalTangent
InClusterp
N
p
C
w
w
Vertex Shader
0
p
Ni i
Ci
w
e e
local view vecv
1
1
1
0
1C
C
NC
ee
e
( )T vy e
( )T vy
Exit Rad.
![Page 36: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/36.jpg)
Demo
![Page 37: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/37.jpg)
Results
Model #Pts SPCA IPCA FPS
Buddha 49.9k 3m30s 1h51m 27
BuddhaSS 49.9k 6m12s 4h32m 27
Bird Anis 48.7k 6m34s 3h43m 45
Bird Diff 48.7k 43s 3m26s 227
Head 50k 4m20s 2h12m 58.5
All examples have 25x25 matrices, 256 clusters, 8 PCA vectors
![Page 38: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/38.jpg)
Conclusions
CPCA• works in “signal space”, not “surface space”• uses affine subspace per-cluster• compresses PRT well• is used directly without “blowing out” signal• requires small, uniform state storage• provides
– faster rendering– higher-frequency lighting
![Page 39: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/39.jpg)
Future Work
• time-dependent and parameterized geometry• higher-frequency lighting• combination with bi-scale rendering• better signal continuity
![Page 40: PRT Summary. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d2b5503460f949ffd33/html5/thumbnails/40.jpg)
Questions?
• DirectX SDK for PRT available soon.
• Jason Mitchell, Hugues Hoppe, Jason Sandlin, David Kirk
• Stanford, MPI for models