09-towardblurryrasterizermunkberg-bps2011

7/29/2019 09-towardBlurryRasterizerMunkberg-BPS2011

1/52

Beyond Programmable Shading Cours

Toward a Blurry RasterizerJacob Munkberg

Intel Corporation



2/52


Realistic camera model

Finite aperture: Depth of field

Slow shutter: Motion blur

Ease of use

Set camera parameters andper-vertex motion vectors

High quality & low cost

Goals


3/52


Covers:

Stochastic rasterization for motion blur & depth

Not covering:Post-processing techniques [Potmesil81, Max85, Shinya93, Sous

Layered depth image approaches [Lee2009, Lee2010, ...]

Blurry ray tracing [Cook84, Walter2006, Wald2007, Hachisuka2008,Overbeck2009, Hou2010, Grnshlo2011, ...]

Analytical visibility [Korein83, Sung2002, Gribel2010, Gribel2011, ...]

Reconstruction & filtering [Egan2009, Soler2009, Parker2010, Leh

This Talk


4/52


Depth of Field

Direct viewers attention

Match footage with CGI

Motion Blur

Enhance effect of motion

Reduce temporal aliasing

Motivation


5/52


Temporal Aliasing


6/52


Motion Blur Rendering

x

yt

pixelt

x

y

Pixel color is an integral over(x,y,t)


7/52


Depth of Field Rendering

pixel

image plane(x,y)

focus planelens(u,v)

Pixel color is an integral over(x,y,u,v)

object in focus


8/52


Depth of Field and Motion Blur

pixel

image plane(x,y)

focus planelens(u,v)

object in focus

Pixel color is an integral over(x,y,u,v,t)


9/52


SamplingApproximating the visibility integral



10/52


Approximate Integral

Accumulation buffering

Render scene at many fixedtimes (and lens positions)[Korein83, Haeberli90]

Stochastic sampling [Cook86]Sample pixel at non-uniformly

spaced locations in (x,y,t,u,v)

Aliasing is replaced by noise

uniform

stochastic


11/52


Interleaved Sampling [Keller2001]

Nfixed sample times

Position triangle andsample sparsely in (x,y)

Improvement over acc.

buffering as (x,y) posvary from tile to tile

5D: use (ui,vi,ti) tuples[Fatahalian2009]

t=

t=

t=

t=


12/52


Sampling for Motion Blur (3D)

A set of 64 fixed timesis sufficient to avoid

banding artifacts

Many samples per pixel

needed to reduce noise

4 spp 8 spp 16 spp 32 spp


13/52


Sampling for DOF (4D)

Need many unique lens positions

InterleaveUV (64 lens pos) InterleaveUV (256 lens pos) 64 ra


14/52

InterleaveUVT Sobol sampling InterleaveUVT Sobol

4 spp 4 spp 16 spp 16Beyond Programmable Shading Cours

Sampling for MB+DOF (5D)

Need large number of unique (ui,vi,ti) tupl

Harder to find good sample distributions


15/52


VisibilityThe rasterizer stage



16/52


Recap: Standard Rasterizatio



17/52


The Graphics Pipeline

Vertex Shader

Input Assembler

Hull Shader

Tessellator

Domain Shader

Rasterizer

Pixel Shader

Output Merger


18/52


Rasterization of Static Triangle

Determine which pixels atriangle covers

1.Project triangle on screen

2.Traverse covered area

3.Test samples!"#$$%&!'("$

!

"


19/52



20/52

Test tile of pixels for overlap

Reject samples in tiles completely outside trian

Enables coarse occlusion culling, improves co


Tiled Traversal


21/52


Blurry Rasterization



22/52

Moving triangles

Triangles out of focus

q0

q1

q2

r1

r2

t=0.0 t=1.0t1=0.37


Blurry Rasterization


23/52


The Modified Graphics Pipeline

Vertex Shader

Input Assembler

Hull Shader

Tessellator

Domain Shader

Rasterizer

Pixel Shader

Output Merger

Sample in (x,y,u,v,t)

Output two vertex positions


24/52


Clipping Moving Triangles

Clipping moving primitives is very difficult

Vertices intersect the frustum at different times

z= 0

t=0

t=1

near plane

t=1

t=0.8

t=0.7t=0.6


25/52


Screen-Space Bounds

Bound full motion and all lens positions

Handle moving triangles intersectingz=0

t=0

t=1

z= 0

t=0t=1

image plane


26/52


Spatio-Temporal Visibility

Moving triangle may cover largeregion in (x,y,t)

More samples per pixel neededfor low noise levels

We need to quickly cull samplesin both space and time

"

#

S S C


27/52

"

#


Screen Space Convex Hull

Reduce spatial bounds [McGuire2010]

A tile is only visited once, enablingmultisampling & coarse Z culling

No temporal culling

I t l


28/52

"

#


Interval

UseNtime intervals[Cook90, Fatahalian2009]

Bound triangles movement in

each interval and test all samples

in (x,y,t) box

I t l d R t i ti


29/52

"

#


Interleaved Rasterization

UseNfixed sample times[Keller2001, Fatahalian2009]

Position triangle at each time

Rasterize coarsely in (x,y)

Restricted sampling patterns

Til B d T l


30/52

"

#


Tile-Based Traversal

Tile-test reduces samples to test

Trivially rejects tile or returnstemporal bounds for overlap[Laine2011, Munkberg2011b]

Preserves GPU traversal order

5D R t i ti


31/52


5D Rasterization

Interval [Cook90, Fatahalian2009]

Many strata in (u,v,t). Find (x,y) bounds for eac

Interleaved sampling [Keller2001, Fatahalian2009]

Generate fixed set of(u,v,t) tuples and

rasterize coarsely in (x,y)

Tile-based traversal with (u,v,t) bounds [La

Tile-test returns bounds foru,v and tindividual

Til O l T t


32/52


Tile-Overlap Tests

Find time interval when moving triangle o

with the screen space tile [Laine2011, Munkberg201

[0,1/4] [0,1/2] [1/4,3/4] [1/2,1] [3/4, 1]

t=0 t=1

Til O l T t f DOF


33/52


Tile-Overlap Test for DOF

pixel

image planefocus planelens

objects out of focus

focus planelens

objects out of focus

objects in focus

Idea:Only tesregion t

the objea certaispace t

Lens Bo nds Per Tile


34/52


Lens Bounds Per Tile

Separating plane tests determine active l

region [Akenine-Mller2011, Laine2011]lens separating

linestriangle

focus plane

tile

lens separat

lines

triangle


35/52


Shading


Shading Static Primitives


36/52


Shading Static Primitives

Shading after visibility

Shade per 2x2 pixels in screen space

Shading before visibility [Cook87]

Shade in object space per vertex

Decoupled shading and visibility

Shading Moving Primitives


37/52


Shading Moving Primitives

Supersampling

Shade each visibility sample

Prohibitively expensive

Extend MSAA [McGuire2010]

Motion blur exampleShade once per covered pixel at a certain time

Texture filters take motion into account [Loviscach2005]

MSAA Failure Case


38/52


MSAA Failure Case

Reverts to supersampling for large motio

Each sample hits a new triangle

t=0

t=1/4

t=1/2

t=3/4

Movement

Decoupled Sampling


39/52


Decoupled Sampling[Ragan-Kelley2011, B

Many-to-one mapping function

Between visibility and shading samples

Low shading rate

Even for large motion and defocus

Needs cache for shaded valuesRemapping logic per sample

Decoupled Sampling Example


40/52


Decoupled Sampling Example

T=0

T=1

t=0

t=1

screen space barycentric space shadin

remap to triangle at t=0

snapvisito c

samvisibility samples

Reconstruction


41/52


Reconstruction

Reduce noise in stochastic rendering

May drastically reduce required #samples per

Store additional information with samples [Lehtin

Reference (256 spp)ReconstructedInput image (1 spp)


42/52

View frustum

culledOcclusion

culled

Backface

culled


Culling


View Frustum Culling


43/52


View Frustum Culling

Solve for temporal interval when triangle

inside the view frustum [Laine2011, Munkberg2011b]

z= 0

t=0

t=1

near plane

t=0.6

t=1

Motion Blur Backface Culling


44/52

Backface test is cubic

polynomial in t[Munkberg2011a]

Not conservative to test

facing at t=0 and t=1


Motion Blur Backface Culling

p0

p1

p2

p0

p1

p2

p0

p1

p2

t=0 t=0.5 t=1

DOF Backface Culling


45/52



Lens

BF cull at center of

Correct BF cull



46/52


Lens


BF cull at center of

Correct BF cull

Occlusion Culling for Motion Bl


47/52


Occlusion Culling for Motion Bl

Standard z-culling less efficient

Spatio-temporal occlusion culling[Akenine-Mller2007, Boulos2010]

x

t

solid occludervalid for t=[0,1]

solid ovalid f

Conclusions


48/52


Conclusions

Stochastic rasterization does not come fo

Many more sample tests and shading evaluati

Higher color, depth & texture bandwidth

Less efficient culling and compression

Efficient shading is a requirementVery easy for end-user if implemented in

Q&A


49/52


Q&A

Thank You:

The Advanced Rendering Technology team at

The Graphics Group at Lund University

Mike Doggett, Jonathan Ragan-Kelley, Samuli

and Timo Aila for fruitful discussions

References


50/52


References

[Akenine-Mller2007]Tomas Akenine-Mller, Jacob Munkberg, and Jon Hasselgren,

"Stochastic Rasterization using Time-Continuous Triangles", Graphics Hardware, August 2007.

[Akenine-Mller2011]Tomas Akenine-Mller, Robert Toth, Jacob Munkberg, and Jon Hasselgren,

"Efficient Depth of Field Rasterization using a Tile Test based on Half-Space Culling", technical report, June 2011

[Boulos2010] Solomon Boulos, Edward Luong, Kayvon Fatahalian, Henry Moreton and Pat Hanrahan

Space-Time Hierarchical Occlusion Culling for Micropolygon Rendering with Motion Blur High Performance Graphics 2010

[Burns2010] Burns, C.A., Fatahalian, K., Mark, W.R. A Lazy Object-Space Shading Architecture With Decoupled Sampling. High Performance Gra

[Cook84] Robert L. Cook, Thomas Porter, Loren Carpenter, Distributed Ray Tracing SIGGRAPH 1984

[Cook86] Robert L. Cook. Stochastic Sampling in Computer Graphics. ACM Transactions on Graphics, Volume 6, Number 1, January 1996.

[Cook87] Robert L. Cook, Loren Carpenter, Edwin Catmull. The Reyes Rendering Architecture. 1987

[Cook90] Robert L. Cook, Thomas Porter, Loren Carpenter. Pseudo-random point sampling techniques in computer graphics. United States Patent

[Egan2009] Kevin Egan , Yu-Ting Tseng , Nicolas Holzschuch , Frdo Durand , Ravi Ramamoorthi,Frequency analysis and sheared reconstruction for rendering motion blur, ACM Transactions on Graphics (TOG), v.28 n.3, August 2009

[Fatahalian2009] Kayvon Fatahalian, Edward Luong, Solomon Boulos, Kurt Akeley, William R. Mark and Pat Hanrahan.

Data-Parallel Rasterization of Micropolygons with Defocus and Motion Blur High Performance Graphics 2009

[Fatahalian2011] Kayvon Fatahalian Evolving the Real-Time Graphics Pipeline for Micropolygon Rendering. Ph.D. Dissertation, Stanford University

[Grunschlo2011] Leonhard Grunschlo, Martin Stich, Sehera Nawaz, Alexander Keller. MSBVH: An Efficient Acceleration Data Structure for Ray

High Performance Graphics 2011

References cont.


51/52


References cont.

[Gribel2010] Carl Johan Gribel, Michael Doggett, and Tomas Akenine-Mller, "Analytical Motion Blur Rasterization using Compression",

High Performance Graphics, pp. 163-172, June 2010

[Gribel2011] Carl Johan Gribel, Rasmus Barringer, and Tomas Akenine-Mller, "High-Quality Spatio-Temporal Rendering using Semi-Analytical Visi

[Hachisuka2008]Toshiya Hachisuka, Wojciech Jarosz, Richard Peter Weistroffer, Kevin Dale, Greg Humphreys, Matthias Zwicker, Henrik Wann JenMultidimensional adaptive sampling and reconstruction for ray tracing SIGGRAPH 2008

[Hou2010] Qiming Hou, Hao Qin, Wenyao Li, Baining Guo, Kun Zhou. Micropolygon ray tracing with defocus and motion blur. SIGGRAPH 2010

[Haeberli90] Paul Haeberli , Kurt Akeley, The accumulation buffer: hardware support for high-quality rendering,

Proceedings of the 17th annual conference on Computer graphics and interactive techniques, p.309-318, 1990

[Korein83] Jonathan Korein , Norman Badler,Temporal anti-aliasing in computer generated animation, Computer Graphics, vol 17, no .3, p.377-388

[Keller2001]Alexander Keller , Wolfgang Heidrich, Interleaved Sampling, Eurographics Workshop on Rendering Techniques, p.269-276, 2001

[Laine2011] Samuli Laine, Timo Aila, Tero Karras and Jaakko Lehtinen. Clipless Dual-Space Bounds for Faster Stochastic Rasterization. SIGGRAPH

[Lee2009] Sungkill Lee, Elmar Eisemann and Hans-Peter Seidel. Depth-of-field rendering with multiview synthesis. SIGGRAPH Asia 2009[Lee2010] Sungkill Lee, Elmar Eisemann and Hans-Peter Seidel. Real-time lens blur effects and focus control. SIGGRAPH 2010

[Lehtinen2011] Jaakko Lehtinen, Timo Aila, Jiawen Chen, Samuli Laine and Frdo Durand.

Temporal Light Field Reconstruction for Rendering Distribution Effects. SIGGRAPH 2011

[Loviscach2005] Jrn Loviscach, Motion Blur for Textures by Means of Anisotropic Filtering, Eurographics Symposium on Rendering, 2005. pp. 105

[Max85] Max N. L., Lerner D. M.: A two-and-a-half-d motion-blur algorithm. SIGGRAPH '85

[McGuire2010] McGuire, Enderton, Shirley, and Luebke, Real-Time Stochastic Rasterization on Conventional GPU Architectures, High Performanc

References cont.


52/52


References cont.

[Munkberg2011a] Jacob Munkberg and Tomas Akenine-Mller, "Backface Culing for Motion Blur and Depth of Field", journal of gpu, graphics and ga

[Munkberg2011b] Jacob Munkberg, Petrik Clarberg, Jon Hasselgren, Robert Toth, Masamichi Sugihara, and Tomas Akenine-Mller,

Hierarchical Stochastic Motion Blur Rasterization, High Performance Graphics 2011.

[Navarro2011] Fernando Navarro, Francisco J. Sern and Diego Guitierrez.Motion Blur Rendering: State of the Art. Computer Graphics Forum Volume 30 (2011), number 1 , pp. 3-26

[Overbeck2009] Ryan S. Overbeck , Craig Donner , Ravi Ramamoorthi, Adaptive Wavelet Rendering, ACM Transactions on Graphics (TOG), v.28 n

[Potmesil81] Potmesil M. Chakravarty I: A lens and aperture camera model for synthetic image generation. SIGGRAPH 81 pp. 389-399

[Ragan-Kelley2011] Jonathan Ragan-Kelley, Jaakko Lehtinen, Jiawen Chen, Michael Doggett, Frdo Durand. Decoupled Sampling for Graphics Pip

[Shinya93] Shinya M.: Spatial anti-aliasing for animation sequences with spatio-temporal filtering. SIGGRAPH 93 pp. 289-296

[Shirley2011] Shirley, Aila, Cohen, Eric Enderton, Laine, Luebke, McGuire, A Local Image Reconstruction Algorithm for Stochastic Rendering, ACM

3D Graphics and Games (I3D 2011 proceedings), February 2011

[Soler2009] Cyril Soler, Kartic Subr, Frdo Durand, Nicolas Holzschuch, Franois Sillion.Fourier depth of field. ACM Transactions on Graphics (TOG) , Volume 28 Issue 2, 2009

[Sousa2008] Sousa T.: Crysis next-gen effects. In Proceedings of the Game Developers Conference 2008.

[Sung2002] K. Sung , A. Pearce , C. Wang, Spatial-Temporal Antialiasing, IEEE Transactions on Visualization and Computer Graphics, v.8 n.2, p.14

[Wald2007] Wald I., Mark. W. R., Gnther J., Boulos S., Ize T., Hunt W., Parker S.G.,Shirley P.:

State of the art in ray tracing animated scenes. Eurographics 2007 State of the Art Reports

[Walter2006] Bruce Walter, Adam Arbree, Kavita Bala, Donald P. Greenberg. Multidimensional Lightcuts. SIGGRAPH 2006

09-towardblurryrasterizermunkberg-bps2011

Documents