differentiable rendering theory and applicationsdifferentiable rendering theory and applications...
TRANSCRIPT
![Page 1: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/1.jpg)
Differentiable RenderingTheory and Applications
Cheng Zhang
Department of Computer ScienceUniversity of California, Irvine
![Page 2: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/2.jpg)
Outline
• Introduction• Definition• Motivations
• Related work• Our work• A Differential Theory of Radiative Transfer (SIGGRAPH ASIA 2019)
• Future work
![Page 3: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/3.jpg)
What is diff. rendering?
Rendering Image I
𝐹 𝝅Geometry
Camera
Material
Light
Scene Parameter 𝝅
![Page 4: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/4.jpg)
What is diff. rendering?
Derivative Image 𝑰$
𝜕&𝐹(𝝅)Geometry
Camera
Material
Light
Scene Parameter 𝝅
![Page 5: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/5.jpg)
Why is diff. rendering important?
Rendering Image I
Geometry
Camera
Material
Light
Scene Parameter 𝝅
Inverse
Rendering
![Page 6: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/6.jpg)
Why is diff. rendering important?
• Inverse rendering• Enable gradient-based optimization• Backpropagation through rendering (machine learning)
Scene Param.
Error
Derivative Img.
Current Img.
Target Img.
![Page 7: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/7.jpg)
Why is diff. rendering important?
• Inverse rendering• Enable gradient-based optimization• Backpropagation through rendering (machine learning)
![Page 8: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/8.jpg)
Related work
• Rasterization rendering• Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning (ICCV 2019)• Neural 3D Mesh Renderer (CVPR 2018)• TensorFlow, pytorch3D etc.
Soft Rasterizer Neural 3D Mesh Renderer
![Page 9: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/9.jpg)
Related work
Volume ScatteringGkioulekas et al. 2013, 2016
Human TeethVelinov et al. 2018
NLOS 3D ReconstructionTsai et al. 2019
FabricationSumin et al. 2019
Reflectance & Lighting EstimationAzinovic et al. 2019
Cloth RenderingKhungurn et al. 2015
Not General
![Page 10: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/10.jpg)
Related work
• Inverse transport networks, Che et at. [2018]• Volumetric scattering ✓• Geometry X
• Differentiable Monte Carlo ray tracing through edge sampling, Li et at. [2018]• Volumetric scattering X• Geometry ✓
![Page 11: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/11.jpg)
Our work
• A Differential Theory of Radiative Transfer (SIGGRAPH ASIA 2019)
• Differential theory of radiative transfer• Captures all surface and volumetric light transport effects• Supports derivative computation with respect to any parameters
• Monte Carlo estimator• Unbiased estimation• Analogous to volumetric path tracing
![Page 12: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/12.jpg)
Radiative Transfer
• Radiative Transfera mathematical model describing how light interacts within participating media (e.g. smoke) and translucent materials (e.g. marble and skin)
Kutz et al. 2017 Gkioulekas et al. 2013
![Page 13: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/13.jpg)
Radiative Transfer
• Radiative Transfera mathematical model describing how light interacts within participating media (e.g. smoke) and translucent materials (e.g. marble and skin)
• Now used in many areas• Astrophysics (light transport in space)• Biomedicine (light transport in human tissue)• Nuclear science & engineering (neutron transport)• Remote sensing• …
![Page 14: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/14.jpg)
Radiative Transfer
• Radiative Transfera mathematical model describing how light interacts within participating media (e.g. smoke) and translucent materials (e.g. marble and skin)
𝒙𝛚
![Page 15: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/15.jpg)
Radiative Transfer Equation (RTE)
𝐿 = 𝐾,𝐾-𝐿 + 𝑄
Collision operator
Transport operator Source
Radiative Transfer Equation(Operator Form)
![Page 16: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/16.jpg)
Transport Operator 𝐾,
𝐿 𝒙,𝝎 = 23
4𝑇 𝒙$, 𝒙 (𝐾-𝐿) 𝒙$, 𝝎 𝑑𝜏 + 𝑄
Transmittance
𝑇 𝑥$, 𝑥 = exp −23
?𝜎A 𝒙 − 𝜏$𝝎 𝑑 𝜏$
Extinction coefficient 𝜎A 𝒙controls how frequently light scatters and is also known as optical density
𝒙$= 𝒙-𝜏𝛚 𝒙𝛚
𝜏
𝐷
![Page 17: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/17.jpg)
Collision Operator 𝐾-• phase function 𝑓D 𝒙,𝝎𝒊,𝝎
A probability density over 𝕤G given 𝒙and 𝝎𝒊
• scattering coefficient 𝜎H 𝒙
𝐿 𝒙,𝝎 = 𝐾,𝜎H 𝒙 2𝕤I𝑓D 𝒙$, 𝝎𝒊, 𝝎 𝐿 𝒙$, 𝝎𝒊 𝒅𝝎𝒊 + 𝑄
=: 𝐿LMN(𝑥, 𝜔)in-scattered radiance
𝒙$ 𝒙𝛚
𝝎𝒊
![Page 18: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/18.jpg)
radiant emission
𝒙$ 𝒙𝛚
𝒙𝟎 𝑳𝒆
𝐷
𝑳𝒔
Source 𝑄• Absorption coefficient 𝜎T 𝒙• Interfacial radiance 𝐿H
Boundary condition
Interfacial radiance
Attenuation
𝐿 𝒙,𝝎 = 𝐾,𝐾-𝐿 + 23
4𝑇 𝒙$, 𝒙 𝜎T 𝒙 𝐿U 𝒙$, 𝝎 𝑑𝜏 + 𝑇(𝒙𝟎, 𝒙) 𝐿H(𝒙𝟎,𝝎)
![Page 19: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/19.jpg)
𝑲𝑻Transport Operator
𝑸Source
𝑲𝒄Collision Operator
Radiative Transfer Equation(Integral Form)
𝐿 𝒙,𝝎 = 23
4𝑇 𝒙$, 𝒙 𝜎H 𝒙 2
𝕤I𝑓D 𝒙$, 𝝎𝒊, 𝝎 𝐿 𝒙$, 𝝎𝒊 𝒅𝝎𝒊 𝑑𝜏
+23
4𝑇 𝒙$, 𝒙 𝜎T 𝒙$ 𝐿U 𝒙$, 𝝎 𝑑𝜏 + 𝑇(𝒙𝟎, 𝒙)𝐿H(𝒙𝟎,𝝎)
![Page 20: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/20.jpg)
Differentiating the RTE
Differentiating individual operators
𝐿 = 𝐾,𝐾-𝐿 + 𝑄
𝜕&𝐿 = 𝜕&(𝐾,𝐾-𝐿) + 𝜕&𝑄
Differentiatingboth sides
![Page 21: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/21.jpg)
𝐾𝑐𝐿 𝝎 = 𝜎H 2𝕤I𝑓D 𝝎𝒊,𝝎 𝐿 𝝎𝒊 d𝝎𝒊
𝑓 𝝎𝒊(𝒙 omitted fornotational simplicity)
𝜕& 2𝕤I𝑓 𝝎𝒊 d𝝎𝒊 = ?
Differentiating the Collision Operator
Scatteringcoefficient
Phasefunction
𝐿 = 𝐾,𝐾-𝐿 + 𝑄RTE:
Requires differentiatinga (spherical) integral
![Page 22: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/22.jpg)
Differentiating the Collision Operator
when 𝑓 has discontinuities that depend 𝜋≠ +2
𝕤I𝜕&𝑓 𝝎𝒊 d𝝎𝒊𝜕& 2
𝕤I𝑓 𝝎𝒊 d𝝎𝒊 = 2
𝕤𝒏,𝜕𝝎𝒊
𝜕𝜋 ∆𝑓(𝝎𝒊)d𝝎𝒊
Boundary term
𝐾-𝐿 𝝎 = 𝜎H 2𝕤I𝑓D 𝝎𝒊,𝝎 𝐿 𝝎𝒊 d𝝎𝒊
𝑓 𝝎𝒊
𝜕& 2𝕤I𝑓 𝝎𝒊 d𝝎𝒊(according to Reynolds transport theorem)
![Page 23: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/23.jpg)
Boundary term
change rate of discontinuity (in the normal direction)
∆𝑓 is the difference of integrand 𝑓 across the
discontinuity 𝝎𝒊
discontinuities of integrand f
2𝕤𝒏,𝜕𝝎𝒊
𝜕𝜋∆𝑓(𝝎𝒊)d𝝎𝒊
∆𝑓(𝝎𝒊) = 𝑓D 𝝎𝒊,𝝎 ∆𝐿 𝝎𝒊
when 𝑓D is continuous𝐾-𝐿 𝝎 = 𝜎H 2
𝕤I𝑓D 𝝎𝒊,𝝎 𝐿 𝝎𝒊 d𝝎𝒊
𝑓 𝝎𝒊
![Page 24: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/24.jpg)
Sources of Discontinuity
Visibility
𝜔`
Normal
2𝕤𝒏,𝜕𝝎𝒊𝜕𝜋
𝑓D 𝝎𝒊,𝝎 ∆𝐿 𝝎𝒊 d𝝎𝒊
The boundary term:
2𝕤𝒏,𝜕𝝎𝒊
𝜕𝜋𝑓D 𝝎𝒊,𝝎 ∆𝐿 𝝎𝒊 d𝝎𝒊
The boundary term:
𝜋
𝜔`
![Page 25: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/25.jpg)
Sources of Discontinuity
2𝕤𝒏,𝜕𝝎𝒊
𝜕𝜋𝑓D 𝝎𝒊,𝝎 ∆𝐿 𝝎𝒊 d𝝎𝒊
The boundary term:
𝜋
𝜔`Reduces to the changerate of 𝝎𝒊 (as an angle)
![Page 26: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/26.jpg)
Sources of Discontinuity
2𝕤𝒏,𝜕𝝎𝒊
𝜕𝜋𝑓D 𝝎𝒊,𝝎 ∆𝐿 𝝎𝒊 d𝝎𝒊
The boundary term:
𝜋
𝜔`
∆𝐿 𝜔` = 𝐿 − 𝐿( )(with the absence of attenuation)
![Page 27: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/27.jpg)
visualization of 𝐿 visualization of discontinuitycurves 𝕤
line integral
Discontinuities in 3D
2𝕤𝒏,𝜕𝝎𝒊
𝜕𝜋 𝑓D 𝝎𝒊,𝝎 ∆𝐿 𝝎𝒊 d𝝎𝒊
![Page 28: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/28.jpg)
Discontinuities curves:Projection of moving geometric edges onto the sphere
Discontinuities in 3D
2𝕤𝒏,𝜕𝝎𝒊
𝜕𝜋 𝑓D 𝝎𝒊,𝝎 ∆𝐿 𝝎𝒊 d𝝎𝒊
![Page 29: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/29.jpg)
Edge normal• in the tangent space of the sphere• perpendicular to the discontinuity curve at direction 𝝎𝒊
Discontinuities in 3D
2𝕤𝒏,𝜕𝝎𝒊
𝜕𝜋 𝑓D 𝝎𝒊,𝝎 ∆𝐿 𝝎𝒊 d𝝎𝒊
![Page 30: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/30.jpg)
Other Terms in the RTE
(𝐾,𝐾-𝐿) 𝑥, 𝜔 = 23
4𝑇 𝑥$, 𝑥 (𝐾-𝐿)(𝑥$, 𝜔)𝑑𝜏
Transport operator
𝐿 = 𝐾,𝐾-𝐿 + 𝑄
Transmittance
𝑄 = 𝑇(𝒙𝟎, 𝒙)𝐿H(𝒙𝟎,𝝎)Source
![Page 31: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/31.jpg)
Full Radiance Derivative
This is Eq. (32) of the paper
![Page 32: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/32.jpg)
Significance of the Boundary Terms
Orig. Image
𝑃bcde
𝑃fLghi 𝑃fLghi = 𝑃3 +0𝜋0
𝑦
𝑃bcde = 𝑃l +0𝜋0
Initial position(constant)
![Page 33: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/33.jpg)
Significance of the Boundary Terms
Orig. Image Deriv. Image Deriv. Image(no boundary term)
![Page 34: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/34.jpg)
𝐿 = 𝐾,𝐾-𝐿 + 𝑄
𝜕&𝐿 = 𝜕&(𝐾,𝐾-𝐿) + 𝜕&𝑄
Differentiating the RTE: Summary
Key:• Tracking discontinuities of integrands• Establishing boundary terms accordingly
![Page 35: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/35.jpg)
𝐿 = 𝐾,𝐾-𝐿 + 𝑄Differential RTE
𝜕&𝐿𝐿
= 𝐾,𝐾- 𝐾∗0 𝐾,𝐾-
𝜕&𝐿𝐿
+𝜕&𝑄𝑄
Boundary terms are included
𝜕&𝐿 = 𝜕&(𝐾,𝐾-𝐿) + 𝜕&𝑄
![Page 36: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/36.jpg)
Differentiable Volumetric Path Tracing
𝒙𝟎𝛚𝟎𝒙𝟏𝛚𝟏
𝒙𝟐𝛚𝟐
𝒙𝟑
Side Path 1
Side Path 2
∆𝐿
∆𝐿
∆𝐿
Component 2:Side paths (for estimating ∆𝐿)
Component 1:Derivative of path throughput
![Page 37: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/37.jpg)
Results
![Page 38: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/38.jpg)
Orig. Image
𝑃bcde
𝑃fLghi 𝑃fLghi = 𝑃3 +0𝜋0
𝑦
𝑃bcde = 𝑃l +0𝜋0
Initial position(constant)
Results: Validation
![Page 39: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/39.jpg)
Results: Validation
(equal-time comparison)
Orig. Image Ours
large spacing
small spacing
Finite Diff.Absolute
difference
![Page 40: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/40.jpg)
Results: Inverse Rendering
• Scene configurations• participating media• changing geometry
• Optimization• L2 loss for its simplicity• Any differentiable metric can be used with our method
![Page 41: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/41.jpg)
Apple in a Box
#iterations Time (CPU core minute per iteration)
80 12.2
Target Optimization processParameters
Cube roughness
Apple position
![Page 42: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/42.jpg)
Camera Pose
#iterations Time (CPU core minute per iteration)
220 9.3
Target Optimization Process
![Page 43: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/43.jpg)
None-Line-of-Sight
#iterations Time (CPU core minute per iteration)
100 9
Target Optimization ProcessDiff. View
(not used in optimization)
![Page 44: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/44.jpg)
None-Line-of-Sight
Heterogeneous medium
Density scaling(param)
Medium orientation(param)
![Page 45: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/45.jpg)
None-Line-of-Sight
#iterations Time (CPU core minute per iteration)
60 7.6
Target Optimization process Diff. View
![Page 46: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/46.jpg)
Design-Inspired
Spotlight ALight directionLight colorLight falloff angle
Spotlight BParameters
![Page 47: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/47.jpg)
Design-InspiredTarget Optimization process
#iterations Time (CPU core minute per iteration)
110 11.2
![Page 48: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/48.jpg)
Design-inspired
#iterations Time (CPU core minute per iteration)
100 27.2
Target Optimization Process Diff. View
![Page 49: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/49.jpg)
Future work
•Differentiable rendering is slow due to• Main term• Needs better sampling methods
• Boundary term• Detecting visibility changes (e.g., object silhouettes)• Tracing side paths
![Page 50: Differentiable Rendering Theory and ApplicationsDifferentiable Rendering Theory and Applications Cheng Zhang Department of Computer Science University of California, Irvine. ... •](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0f50ce7e708231d4438def/html5/thumbnails/50.jpg)
Thank you