originally presented at: acm siggraph 2003 symposium on interactive 3d graphics interruptible...
TRANSCRIPT
Originally presented at:ACM SIGGRAPH 2003Symposium on Interactive 3D Graphics
Interruptible Rendering
Joint work with:David Luebke * Benjamin Watson† Abhinav Dayal†
* University of Virginia† Northwestern University
Master’s Project Presentation Cliff Woolley
University of Virginia
4/30/2003 Interruptible Rendering 3
Balance complexity with interactivity
Age-old tradeoff: detail vs. frame rate
Typical approach: Maintain a high, constant frame rate Create a mesh that can always be rendered in
time
4/30/2003 Interruptible Rendering 4
Improve on traditional LOD
Is a high, constant frame rate good enough?
Is fidelity ever more important than frame rate?
How can this be decided at runtime while still guaranteeing interactivity?
4/30/2003 Interruptible Rendering 5
Error metric unification
Inspired by [Meruvia 2000]
Combine notions of temporal and spatial error (i.e. “lateness” vs. “coarseness”)
Use to drive progressive refinement
4/30/2003 Interruptible Rendering 6
Unified error metrics
With interruptible rendering, we address this key question:
At what point does further refinement of the current frame become pointless?
4/30/2003 Interruptible Rendering 8
Methodology
Refine a stream of continuous LODs
Monitor input frequently
Minimize dynamic visual error
4/30/2003 Interruptible Rendering 9
Refine a stream of continuous LODs
Render progressive refinements on top of each other until “out of time”
Ensure that we can stop refining at any time and move on to the next frame
4/30/2003 Interruptible Rendering 10
Refinement
Three refinement schemes implemented: Splatting (Tech sketch, SIGGRAPH 2002)
Progressive polygonal hulls Progressive ray casting (Abhinav Dayal, I3D 2003)
Will work with any progressive rendering scheme
4/30/2003 Interruptible Rendering 11
Start with a very coarse rendering
Progressively render refinement stream over top of the coarse image
Progressive hull refinement
4/30/2003 Interruptible Rendering 12
Progressive hulls
Simplification method by [Sander et al. 2000]
Record a sequence of constrained edge collapses and play it back in reverse
Guarantees containment
V1
Vn
V4
V3V2
4/30/2003 Interruptible Rendering 15
Monitor input frequently
Ideally: input monitored continuously
Realistically: check every x ms
Allows quick reaction when sudden changes in input occur
Allows system to be self-tuning
4/30/2003 Interruptible Rendering 16
Minimize dynamic visual error
Always display image with least error
Sometimes: Further refinement is pointless
Temporal error (lateness) exceeds spatial error (coarseness) for the current frame
Front buffer “out of date” Dynamic visual error (spatial + temporal) in
front is greater than in the back
4/30/2003 Interruptible Rendering 17
Calculating dynamic visual error
Spatial error Screen-space size of
coarsest geometry visible
Temporal error Project bounding box of
object in screen-space Find the maximum
screen-space distance any corner has moved since frame began
clear front bufferIfront = Icurrent
start
Refine the current imagein the front buffer.
improve ifront
tfront = Icurrent – Ifront
sfront = Ifront – ifront
Refine the current image in theback buffer.
improve iback
tback = Icurrent – Iback
sback = Iback – iback
Start refining a new image towardmost current input in the back buffer.
clear back bufferIback = Icurrent
Back buffer now closer to Icurrent than front is.
swap buffersIfront = Iback
It is pointless tocontinue refining.
swap buffersIfront = Iback
sfront = sback
efront >= eback?no yesyesno
tfront > sfront?
yesnotback > sback?
Rendering to Back BufferRendering to Front Buffer
time
tfront = Icurrent – Ifront
efront = sfront + tfront
eback = sback + tback
Clear thefront buffer
start
Refine thefront buffer
Refine the back buffer
Clear the back buffer
Swap buffers
Swap buffers
ef >= eb?N YYN tf > sf?
YN tb > sb?
Rendering to Back BufferRendering to Front Buffer
time
Compute dynamic
visual error
4/30/2003 Interruptible Rendering 20
“Gold Standard” evaluation
Compares an ideal rendering to interactive approximations
Ideal rendering: full detail, zero delay
Interactive approximations are Unmanaged Constant fidelity (in pixels) Constant frame rate (fixed Hz) Interruptible rendering
4/30/2003 Interruptible Rendering 21
Frame generation
Record what the user sees and when Generate each approximation offline Record actual frames displayed over time
Account for: Render time Double buffering Frame locking Rendering into front buffer (interruptible)
4/30/2003 Interruptible Rendering 22
Comparing frames
Error measures RMS – Root Mean Square error Lindstrom's perceptually based error
Difft = Idealt - Renderedt
4/30/2003 Interruptible Rendering 23
Two test input streams
Rotation Model-centered Fixed angular velocity
User interaction Includes both view translation and rotation Includes both static and dynamic segments
Both input streams recorded to files, with timestamps
Ideal
Inter-ruptible
Un-managed
Frames:
Interaction sequence: ray caster
ConstantFidelity
33 5754514845423936 60
Interactive input stream Rotation input streamR
ay
ca
stin
gP
rog
ress
ive
hu
llser
ror(
rms)
erro
r(rm
s)
erro
r(rm
s)er
ror(
rms)
time(secs) time(secs)
time(secs) time(secs)
constant fidelityunmanaged
constant frame rateinterruptible
4/30/2003 Interruptible Rendering 29
Benefits
Accuracy Balancing spatial and temporal error
Interactivity Slow renderers, like ray caster Large models
Self-tuning feedback system Adapts to hardware
4/30/2003 Interruptible Rendering 30
Limitations
Overdraw in progressive renderer Progressive ray caster better here
Cost of monitoring temporal error
Rendering immersive models Requires reversing the containment criteria for
polygon rendering Not a problem for ray caster
4/30/2003 Interruptible Rendering 31
Future work
Temporal & spatial error Better estimates of each (e.g. color) Better comparisons of both
Textures, normal maps, etc.
Reducing overdraw
View-dependent refinement
Ray tracing: a promising candidate
4/30/2003 Interruptible Rendering 32
Acknowledgements
Peter Lindstrom for ltdiff
OpenRT Interactive Raytracing Project
Stanford 3D Scanning Repository
National Science Foundation Awards 0092973, 0093172, and 0112937
4/30/2003 Interruptible Rendering 34
Progressive ray casting
Progressive refinement Coarse to fine sampling/reconstruction
Sampling Where to shoot the ray
Adaptive Non-adaptive
Reconstruction Producing imagery from samples
4/30/2003 Interruptible Rendering 35
Sampling
Coarse to fine - quadtree approach Sample every quadtree node's center Breadth first traversal Predetermined random traversal per level
Ray casting – using OpenRT Per ray API ~ 600,000 rays per sec. Simple phong shading No shadows or specular highlights
4/30/2003 Interruptible Rendering 36
Reconstruction Placement
Place splat at center of each quadtree node
Shading Flat shaded quad
covering the node's screen space
Alpha-textured quads (smooth reconstruction) Quad size = twice of
node's screen space Texture: Gaussian blob
Transparent at edges Opaque at center