san diego supercomputer center university of california, san diego volume graphics technology to...
TRANSCRIPT
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Volume GraphicsTechnology to Tools
David R. Nadeau
Principal Scientist
San Diego Supercomputer Center
University of California, San Diego, USA
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Technology & Tools
• Technology = a useful idea– Polygons, images, volumes, transforms, shading, …
• Tool = an idea put to use– Visualization, art, story telling, games, …
• As a technology matures, it becomes a tool– We focus more on its use, than its mechanism
– Use really explodes when artists begin to use it
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Technology & Tools
• Written storytelling – dramatically advanced by…– Printing press, movable type, word processing, e-mail,
Internet, …
• Music composition – dramatically advanced by…– Standardized notation, equal-tempered tuning, piano,
electric guitar, synthesizer, digital signal processing, …
• Computer graphics – dramatically advanced by…– Hmmm…?
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Topics of the Talk
• History– What has happened so far?
• Rendering– What can we do now?– What would we like to do?– What might we be able to do in the future?– How might this affect the way we render volumes?
• Authoring– How might rendering issues affect how we create volumes?– What might future authoring tools look like?
HistoryRenderingAuthoring
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
HistoryRenderingAuthoring
A Brief Look atGraphics History…
• What are some major events?• What did they enable?
• Surface vs. Volume graphics
• Technology vs. Story telling vs. Games
HistoryRenderingAuthoring
Very
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Evolution of surface graphics
• Selected Technology events:1963 “Sketchpad” I.E Sutherland, "Sketchpad: A Man-Machine Graphical Communication System," Ph.D.
Thesis, MIT, 1962.
1965Digital line drawing
J.E. Bresenham, “Algorithm for computer control of a digital plotter,” IBM Systems Journal, 4(1), 1965.
1971 Gouraud shading H. Gouraud, “Continuous shading of curved surfaces,” IEEE Transactions on Computers, C-20, 1971.
1972Hidden surface removal
M.E. Newell, R.G. Newell, T.L. Sancha, “A Solution to the Hidden Surface Problem,” Proceedings of ACM National Meeting, 1972.
1974 Polygon clipping I.E. Sutherland, G.W. Hodgman, “Reentrant Polygon Clipping,” CACM, 17(1), 1974.
1975 Phong shading B.T. Phong, “Illumination for Computer Generated Pictures,” CACM, 18(6), 1975.
1976Scene graphsTexture mapping
J.H. Clark, “Hierarchical Geometric Models for Visible Surface Algorithms,” CACM, 19(10), 1976.J.F. Blinn, M.E. Newell, “Texture and Reflection in Computer Generated Images,” CACM, 19(10), 1976.
1977ShadowsStandard lighting model
F.C. Crow, “Shadow Algorithms for Computer Graphics,” Computer Graphics, 11(2), 1977.J.F. Blinn, “Models of Light Reflection for Computer Synthesized Pictures,” Computer Graphics, 11(2), 1977.
1978 Bump mapping J.F. Blinn, “Simulation of Wrinkled Surfaces,” Computer Graphics, 12(3), 1978.
HistoryRenderingAuthoring
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Evolution of surface graphics
• Selected Technology events:1980 Ray tracing T. Whitted, “An improved illumination model for shaded display,” CACM, 23(6), 1980.
1982Geometry VLSIAutoCAD
J.H. Clark, “The Geometry Engine: A VLSI geometry system for graphics,” SIGGRAPH 82.Autodesk Inc.
1983Particle systemsAlias
W.T. Reeves, “Particle Systems – A Technique for Modeling a Class of Fuzzy Objects,” SIGGRAPH 83.Alias Research Inc.
1984
RadiosityShadersSGI IRIS 1000Wavefront
C.M. Goral, K.E. Torrance, D.P. Greenberg, B. Battaile, “Modeling the Interaction of Light Between Diffuse Surfaces,” SIGGRAPH 84.R.L. Cook, “Shade trees,” SIGGRAPH 84.Silicon GraphicsWavefront Inc.
1986 Rendering eq. J.T. Kajiya, “The rendering equation,” SIGGRAPH 86.
1989 RenderMan PIXAR
1992 OpenGL OpenGL ARB
1993 Direct3D Microsoft
1994 GLINT 300SX 3Dlabs
1995 VRML VRML/Web3D Consortium
1997VoodooRiva 128
3dfxnVidia
HistoryRenderingAuthoring
Too many papers to list
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Evolution of surface graphics
• Selected Story Telling events:1982
TronWrath of Khan
DisneyParamount
1984Andre & Wally B.The Last Starfighter
PIXARUniversal
1985 Young Sherlock Holmes Paramount
1986 Luxo Jr. PIXAR
1987Red’s DreamStanley & Stella
PIXARSymbolics
1988 Tin Toy PIXAR
1989KnickknackThe Abyss
PIXARFox
1991Beauty and the BeastTerminator 2
DisneyTri-Star
1992AladdinLawnmower Man
DisneyNew Line
HistoryRenderingAuthoring
All images copyright by Disney, Fox, Paramount, New Line, PIXAR, and Tri-Star, as appropriate.
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Evolution of surface graphics
• Selected Story Telling events:1993 Jurassic Park Universal
1994The MaskRebootTrue Lies
New LineMainFrameFox
1995 Toy Story PIXAR
1996DragonheartTwister
UniversalWarner Bros.
1997
Geri’s GameLost WorldStar Wars, Special EditionTitanic
PIXARUniversalLucasfilmParamount
1998A Bug’s LifeAntz
PIXARDreamWorks
1999Toy Story 2Star Wars, Episode 1
PIXARLucasfilm
2000 Dinosaur Disney
HistoryRenderingAuthoring
All images copyright by Disney, DreamWorks, Paramount, Lucasfilm, MainFrame, PIXAR, and Universal, as appropriate.
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Evolution of surface graphics
• Selected Game events:1991 Catacomb 3-D Id Software
1992Wolfenstein 3-D7th Guest
Id SoftwareTilobyte Studios
1993DOOMMyst
Id SoftwareCyan Productions
1995 Dark Forces LucasArts
1996Nintendo 64QuakeTomb Raider
NintendoId SoftwareEidos Interactive
1997Jedi KnightQuake IIRiven
LucasArtsId SoftwareCyan Productions
1998Half-LifeThiefUnreal
SierraEidos InteractiveEpic MegaGames
1999 Quake III: Arena Id Software
2000 RealMyst Cyan Productions
Catacomb 3-D
DOOM
Quake
Unreal
Jedi Knight
HistoryRenderingAuthoring
Tomb Raider
Half-Life
Quake III: Arena
All images copyright by Id Software, Cyan Productions, Eidos Interactive, Epic MegaGames, LucasArts, and Sierra, as appropriate.
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Evolution of surface graphics
1960’s 1970’s 1980’s 1990’s 2000’s
TechnologyTechnology
Story TellingStory Telling
GamesGames
HistoryRenderingAuthoring
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Evolution of volume graphics
• Selected Technology events:1984
Volume ray casting
H. Tuy, L. Tuy, “Direct 2D display of 3D objects,” Computer Graphics and Applications, 4(10), 1984.
19853D texture generation
K. Perlin, “An image synthesizer,” SIGGRAPH 85.
1987 Marching cubes W.E. Lorensen, H.E. Cline, “Marching cubes: A high resolution 3D surface construction algorithm,” SIGGRAPH 87.
1989 Hypertexture K. Perlin, E.M. Hoffert, “Hypertexture,” SIGGRAPH 89.
1990 Splatting L. Westover, “Footprint evaluation for volume rendering,” SIGGRAPH 90.
1991 Volume sculpting T.A. Galyean, J.F. Hughes, “Sculpting: An interactive volumetric modeling technique,” SIGGRAPH 91.
1994Shear-warp3D texture mapping
P. Lacroute, M. Levoy, “Fast volume rendering using a shear-warp factorization of the viewing transformation,” SIGGRAPH 94.B. Cabral, N. Cam, J. Foran, “Accelerated volume rendering and tomographic reconstruction using texture mapping hardware,” Volume Visualization Symposium, 1994.
1999VolumeProGeForce 256
Real-Time Visualization, MitsubishinVidia
2000GeForce2DirectX 8.0
nVidiaMicrosoft
HistoryRenderingAuthoring
Too many papers to list
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Evolution of volume graphics
• Selected Story Telling events:1997 Contact Warner Bros.
1998 Sphere Warner Bros.
HistoryRenderingAuthoring
Contact
Sphere
All images copyright by Warner Bros., San Diego Supercomputer Center, and the American Museum of Natural History, as appropriate.
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Evolution of volume graphics
• Selected Game events:2001 ?
?
HistoryRenderingAuthoring
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Evolution of volume graphics
1960’s 1970’s 1980’s 1990’s 2000’s
TechnologyTechnology
Story TellingStory Telling
GamesGames
HistoryRenderingAuthoring
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Evolution of volume graphics
• Volume graphics today……is where surface graphics was 15 years ago
– We are at the start of a transition from technology to tool
• What enabled story telling and games for surface graphics?
• What might do the same for volume graphics?
HistoryRenderingAuthoring
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
• 1st business-level (expensive):– Attention-grabbing event: TRON
– Interactive rendering hardware: SGI
– Authoring tools: Alias, Wavefront
1982-4 1991-3 1997
• 1st consumer-level (cheap):– Attention-grabbing event: Catacomb3D, DOOM– Interactive rendering hardware: Voodoo
Enabling eventsHistoryRenderingAuthoring
Surface graphics1960’s 1970’s 1980’s 1990’s 2000’s
TechnologyTechnology
Story TellingStory Telling
GamesGames
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
• 1st business-level (expensive):– Attention-grabbing event: ?
– Interactive rendering hardware: VolumePro, SGI?
– Authoring tools: ?
1999
?2000
• 1st consumer-level (cheap):– Attention-grabbing event: ?– Interactive rendering hardware: nVidia?
Enabling eventsVolume graphics
HistoryRenderingAuthoring
1960’s 1970’s 1980’s 1990’s 2000’s
TechnologyTechnology
Story TellingStory Telling
GamesGames
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
HistoryRenderingAuthoring
Interactive Volume Rendering…
HistoryRenderingAuthoring
• What can we do now?
• What would we like to do?
• What might we be able to do in the future?
• How might this affect the way we render volumes?
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
What can we do now?
• Canonical volume visualizations…
• What can we do interactively?
HistoryRenderingAuthoring
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
What can we do now?
RTviz VolumePro
nVidia GeForce2Ultra
SGI IR2(1 pipe, 4 RMs)
Memory 256 MB 64 MB 64 MB
Fill rate N/A 1 Gpixels/s 896 Mpixels/s
Texture rate N/A 2 Gtexels/s 768 Mtexels/s
Max stored volume
16 x 2563
(scalar)2563
(RGBα)2563
(RGBα)
Max volume 2563 @ 30 fps 2563 @ 15 fps 2563 @ 12 fps
• But 2563 is a small volume…
HistoryRenderingAuthoring
Data source: Product literature
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
What resolution can we see?
• Eye’s lens focuses light onto retina– Fovea = focus area = center 20
• More receptors at the fovea– Mostly “Cones” (color) at fovea
– Mostly “Rods” (intensity) in surrounding areaFovea
Periphery
HistoryRenderingAuthoring
Data source: Perception, Sekuler & Blake, 2nd ed, 1990, McGraw-Hill.
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
What resolution can we see?
• Measure visual acuity by viewing “gratings” of parallel lines• How fine a grating can you see?
HistoryRenderingAuthoring
Gratings
Data source: Visual Perception, Spillmann & Werner, 1990, Academic Press.
• Normal vision:• 120 lines/degree in center 20 = 2,400 lines
• Legally blind:• 1/10th normal vision
• 12 lines/degree in center 20 = 240 lines
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
What resolution can we see?
• A 2563 volume legally blind!– Very low resolution
• Normal vision requires 2,4003!– If it only covers central 20 of visual field
HistoryRenderingAuthoring
• 180 visual field requires 21,6003!– 120 lines/degree in 180 = 21,600 lines!
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
What resolution can we show?
• We’re constrained by screen resolution– 1280x1024 on a large monitor
– 30 pixels/degree at normal distance
– ¼ normal vision = visually impaired
• For now…– 10243 is a minimum for effective screen use
• Smaller volumes are blurry (1 voxel covers multiple pixels)
– But we want much higher volume resolutions…
HistoryRenderingAuthoring
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
What resolution do we want?
• Medical imaging (cryosections)– State of the art: 2048 x 2048 images, 1/10mm apart
– 2048 x 2048 x 18000 for full body (180cm person)
– Visible Human Male: 2048 x 1216 x 1871• Re-digitization of images: 4096 x 2700 x 1871
– Visible Human Female: 2048 x 1216 x 5189
– Recent brain only: 1789 x 1472 x 1152
HistoryRenderingAuthoring
Head data from the Visible Human Project, www.nlm.nih.gov/research/visibleBrain data from Arthur Toga’s lab, LONI, UCLA, www.loni.ucla.edu
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
What resolution do we want?
• Medical imaging (CT scan)– State of the art: 2048 x 2048 images, ½mm apart
– 2048 x 2048 x 3600 for full body (180cm person)
– Visible Human Male: 512 x 512 x 1871
HistoryRenderingAuthoring
Head data from the Visible Human Project, www.nlm.nih.gov/research/visible
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
What resolution do we want?
• Microscopy imaging (fluorescence)– State of the art: 1024 x 1024, 0.15 microns apart
– 1024 x 1024 x 75 for a 10 micron cell
– Cell membranes: 768 x 768 x 72
HistoryRenderingAuthoring
Cell data from Hiro Tsukada, Eric Elenko, and Maria Pinhal, UCSD Cancer Center
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
What resolution do we want?
• Simulations– Tend to use all available memory on a supercomputer
– SDSC IBM SP “Blue Horizon”• 1152 processors (144 nodes, 8 cpus/node, +others), 1.7 teraflops
• 576 Gbytes total memory (4 Gbytes/node)
• Largest possible volume is 50003
– Ocean simulation: 2160 x 960 x 30
HistoryRenderingAuthoring
Ocean data from Detlef Stammer, UCSD, Scripps Institute of Oceanography
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
What resolution do we want?
• Combine to build volumetric scenes– Composite overlapping volumes
– Mosaic volumes to fill space
– Combine procedural elements
HistoryRenderingAuthoring
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
What resolution do we want?
110
1001,000
10,000100,000
1,000,00010,000,000
100,000,0001,000,000,000
10,000,000,000100,000,000,000
1,000,000,000,000
Nu
mb
er
of
vo
xe
ls
HistoryRenderingAuthoring
2563
• We’re a long way from what we want… why?
Full screenFoveaFull eye
It’s a logarithmic scale!
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Memory and bandwidth
• Memory needed grows with cube of volume’s width– 643 RGBα = 1 Mbyte
– 1283 RGBα = 8 Mbytes
– 2563 RGBα = 64 Mbytes
– 5123 RGBα = 512 Mbytes
– 10243 RGBα = 4096 Mbytes
• Bandwidth needed also grows with frame rate– 10 fps minimum
– 30 fps better
– 70 fps best
HistoryRenderingAuthoring
0
1024
2048
3072
4096
0 256 512 768 1024
Resolution
Mb
yte
s
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
What will volumes require?
Memory 10 fps 30 fps 70 fps
2563
(legally blind)64 MB 640 MB/s 1.9 GB/s 4.5 GB/s
10243
(full screen)4 GB 40 GB/s 120 GB/s
(30 Gpixl/s)180 GB/s
24003
(fovea)55 GB 550 GB/s 1.6 TB/s 3.8 TB/s
216003
(full eye)40 TB 400 TB/s 120 TB/s 280 TB/s
• 120 GB/s is > 30 times current bandwidths!
HistoryRenderingAuthoring
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Growth of pixel fill ratesHistoryRenderingAuthoring
Data source: Product literature
0
200
400
600
800
1000
1200
19
91
19
92
19
93
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
Fill
ra
te, M
pix
els
/s
SGI PC cards
1996 30 Mpixels/s 3Dlabs Permedia1997 100 Mpixels/s nVidia RIVA 1281998 366 Mpixels/s 3dfx Voodoo 31999 540 Mpixels/s ATI Rage Fury MAXX2000 1000 Mpixels/s nVidia GeForce2 Ultra
* Not counting custom hardware or special configurations
• Fill rates recently growing by x2 every year
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
How long to get there?
Bandwidth @ 30 fps Years
2563
(legally blind)1.9 GB/s(0.5 Gpixels/s)
Last year
10243
(full screen)120 GB/s(30 Gpixels/s)
5 years
24003
(fovea)1600 GB/s(400 Gpixels/s)
8 years
216003
(full eye)120000 GB/s(30,000 Gpixels/s)
14 years
HistoryRenderingAuthoring
• But, this is not very realistic…
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
How long to get there?
About trends…
“A frequent criticism of predictions of the future is that they rely on mindless extrapolation of current trends without consideration of forces that may terminate or alter that trend.”
– Ray Kurzweil, The Age of Spiritual Machines
HistoryRenderingAuthoring
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
How long to get there?
About the computer industry…
“If the automobile industry had made as much progress in the past fifty years, a car today would cost a hundredth of a cent and go faster than the speed of light.”
– Ray Kurzweil, The Age of Spiritual Machines
HistoryRenderingAuthoring
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
CPU performance growthHistoryRenderingAuthoring
Data source: SPEC benchmark database, www.spec.org
0
10
20
30
40
50
60
70
80
90
1994 1995 1996 1997 1998 1999 2000 2001
SP
EC
fp9
5
• Floating point power grows by x2 every 2 years
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
CPU performance growth
• At this rate, by 2020 transistor insulators will be just a few atoms thick– A possible end to growth using this technology
– What technology will arrive next?
• Meanwhile, back to the present…
HistoryRenderingAuthoring
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Memory speed growthHistoryRenderingAuthoring
• Memory bandwidth only grows by x2 every 5 years
Data source: Kingston Technology, www.kingston.com
0
20
40
60
80
100
120
140
19
85
19
86
19
87
19
88
19
89
19
90
19
91
19
92
19
93
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
Me
mo
ry b
us
sp
ee
d, M
Hz
• PC processors only
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Comparing growth ratesHistoryRenderingAuthoring
0
5
10
15
20
25
30
35
40
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
Inc
rea
se
fa
cto
r
Processor performance growth
Memory bus speed growth
Pixel fill rate growth
• Highly unlikely that fill rates can continue to grow faster than processor speed and memory bandwidth
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
What should we do?
• Don’t bet on rapid fill rate growth to satisfy our volume rendering needs– If fill rate growth drops to track memory bandwidth
growth, full screen volume rendering may arrive in25 years, not 5 years
• And full eye in 70 years!
HistoryRenderingAuthoring
• Faster hardware is no substitute for a better algorithm– How can we change the way we work?
– Lots of ways… but I’ll focus on a few…
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Change how we render
• Object-space rendering traversals are in common use– Scan through all voxels and project towards screen
• Splatting, shear-warp, 3D texture mapping, point clouds
– Memory streaming is possible• If the viewpoint is outside the volume
• O(n3) time & spacen = volume width
HistoryRenderingAuthoring
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Change how we render
• Image-space traversals are more time/space friendly– Scan through all screen pixels and project towards voxels
• Ray casting, …
– Memory streaming not possible• Nearly random access
• O(k r n) time & spacen = volume width
r = image resolution (width x height)
k = additional computation factor (for comparisons)
HistoryRenderingAuthoring
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
• Is volume ray casting really O(k r n)?– Worst case: each ray takes longest path = n
– Normal case: early ray termination reduces it
– Rays diverge, skipping voxels, introducing aliasing
– For accuracy: supersample
– For speed: use volume MIP-mapping• Upfront cost to build MIP-map volumes
• Increases memory needed (but not bandwidth)– Memory is cheap, bandwidth is not
Change how we renderHistoryRenderingAuthoring
3
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
• To interactively render larger volumes, we need to get on the better curves – such as ray casting
Rendering time growthHistoryRenderingAuthoring
0
10
24
20
48
30
72
40
96
51
20
61
44
71
68
81
92
Volume size
Re
nd
eri
ng
Tim
e
Object space traversalImage space traversal (k=5)Image space traversal (k=10)Image space traversal (k=15)Image space traversal (k=20)
full
scr
een
fove
a
lega
lly
bli
nd
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
What can we do?
• Changing our rendering algorithm can help– For “large data,” image-space traversals are better than
object-space traversals
– We have “large data” now
HistoryRenderingAuthoring
110
1001,000
10,000100,000
1,000,00010,000,000
100,000,0001,000,000,000
10,000,000,000100,000,000,000
1,000,000,000,000
VH Mal
e Cry
osect
ion
VH Mal
e (n
ew) C
ryose
ctio
n
VH Fem
ale
Cryose
ctio
n
Brain
Cry
osect
ion
Full Body
Cryose
ctio
n
VH Mal
e CT
Full Body
CT
Mem
brane
Mic
rosc
opy
Full Cel
l Mic
rosc
opy
Ocean
Sim
ulatio
n
SDSC Max
Sim
ulatio
n
Nu
mb
er
of
vo
xe
ls
Cross-over
• What about changing our data?
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
HistoryRenderingAuthoring
Volume Authoring…
HistoryRenderingAuthoring
• How might rendering issues affect how we create volumes?
• What might future authoring tools look like?
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
CPU vs. memory speedsHistoryRenderingAuthoring
• PC processors, excluding RDRAM & DDR SDRAM
• Main memory access (cache miss)
• Slower memory speed growth means memory references are getting more costly, in terms of cycles
0.0
1.0
2.0
3.0
4.0
5.0
6.0
7.0
8.0
9.0
10.0
1994 1995 1996 1997 1998 1999 2000 2001
CP
U M
Hz
/ Me
mo
ry M
Hz
Data source: SPEC benchmark database, www.spec.org
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
CPUs per SupercomputerHistoryRenderingAuthoring
99 117 128173
226 281230 281424
581 687 806
1
10
100
1000
10000
1994 1995 1996 1997 1998 1999 2000 2001
Nu
mb
er
of
pro
ce
ss
ors
Average of top 100
Average of top 500
• Adapt to slow memory by adding CPUs w/own memory• CPUs/Supercomputer grows by x2 every 3 years
Data source: Top 500 Supercomputers, www.top500.org
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Cycles & bandwidth
• Cycles are more available than memory bandwidth– “Easier” to add CPUs than increase bandwidth
– Parallel computing is an obvious industry trend• But CPU coordination and data sharing is still bandwidth limited
HistoryRenderingAuthoring
• Can we trade computation for memory accesses?– Data compression is clearly needed
• Texture compression nearly standard in graphics hardware
• Geometry compression becoming available
– Additional techniques to reduce memory access costs• Interleaved frame buffers, Z-buffer cache
• View frustum culling, occlusion culling
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Authoring with cycles
• “Shaders” compute parts of the scene as needed– Already common-place in software rendering
• “Data amplification” – small number of parameters generates large amount of scene content
– Programmable graphics pipelines arriving now
– For surface graphics: splines, procedural textures, particles
– For volume graphics: voxelization, procedural volumes
HistoryRenderingAuthoring
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Authoring with cycles
• Shaders aren’t just for shading any more– Procedural content creation
• Noise and turbulence functions
– Voxelization based upon geometry “parameters”• Voxelize on demand – don’t prevoxelize
• Authoring uses a mix of data and shaders– Import data sets (CT, MRI, cryosection, etc.)
– Sculpt & 3D paint volumes
– Program “shaders”
HistoryRenderingAuthoring
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Authoring with shaders
• Procedural turbulence creates “clouds”– Defines a color & opacity at each point in space
– Animate parameters to evolve the cloud
– Voxelize during ray-casting• No volumes created
• Very low memory use
HistoryRenderingAuthoring
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Authoring with shaders
• Manipulate shapes– Compute effects during ray-casting
• No additional volumes created
• For each point in space:– Compute shortest distance to surface
– Perturb the distance with turbulence
– Map distances to opacity
HistoryRenderingAuthoring
Compute distances Add turbulence Map to opacity Do at high resolution
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Authoring with shaders
• Constructing shapes– Constructive Solid
Geometry (CSG)
– Cut using primitive shapes• (or anything)
– Transform and “cut”during ray-casting
• No pre-cut volumes created
HistoryRenderingAuthoring
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Authoring with shaders
• Repeat to build volumetric scenes– Multiple data sets, geometry, shaders, …
– Evaluate some or all during ray-casting• Rendering is an active part of authoring
HistoryRenderingAuthoring
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Authoring scenes
• Several existing scene structure metaphors– CSG trees = combine primitives (most CAD apps.)
– Composite trees = combine images (Houdini, Shake)
– Shade trees = combine shaders (RenderMan)
– Scene graphs = lay out data sets (VRML, Java3D)
– Expression trees = compute scene (any programming lang.)
• “Compilation” prevoxelizes some, all, or none of the scene– Tune to minimize error, memory use, rendering time
HistoryRenderingAuthoring
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Visualizing the Orion nebula
• Fluorescing clouds of hydrogen, helium, oxygen, ...– Complex structure with pillars, swirls, ripples, and cavities
Eagle Nebula Lagoon Nebula Orion Nebula
HistoryRenderingAuthoring
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
• Build a surface for the ionization front– Derived from visual and infrared data
• Make it fuzzy– Perturb distance field with turbulence
• Project a color-corrected Hubble image through it– Jitter to reduce streaking
Visualizing the Orion nebulaHistoryRenderingAuthoring
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Visualizing the Orion nebula
• Add 85 proplyds (protostars), shock fronts, …– Each built in a similar way
• Fly around in it all
HistoryRenderingAuthoring
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Visualizing the Orion nebulaHistoryRenderingAuthoring
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Visualizing the Orion nebula
• 2112 shader nodes in the full scene– Surfaces, turbulence, image projection, transforms, …
• Prevoxelized as 86 volumes– 2 Gbytes of multi-resolution data
– 40 Gbytes if we didn’tvoxelized separately
• Ray-caster composited volumes and stars
HistoryRenderingAuthoring
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Concluding remarks
• What are some directions to explore?
HistoryRenderingAuthoring
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Data directions
• Volume registration– Align volumes for compositing
• Volume compression– Make it take less space
• Volume decimation– Express it well at a smaller size
“Multimodal Medical Volume Registration Based on Spherical Markers”
“Geometric Processing of Volumetric Objects”
“Exploiting Eigenvalues of the Hessian Matrix for Volume Decimation”
At WSCG 2001
“Towards Continuous Image Representations”
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Rendering directions
• Rendering from compressed data– Smaller data = less bandwidth needed
• Large (out-of-core) data rendering– I/O during rendering to get data
• Parallel rendering– More processors to get more bandwidth
• Hybrid volume + geometry rendering– Keep shapes in their smallest format
“A New Parallel Volume Rendering Algorithm”
At WSCG 2001
“Parallel Ray Tracing with 5D Adaptive Subdivision”
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Authoring directions
• Content creation “shaders”– Create using procedures
• Volume sculpting / 3D painting– Create from scratch in an artist-natural way
• Volume scene construction– Compose, composite, cut, …
• Volume scene compilation– Prevoxelize for best memory/CPU use
At WSCG 2001
“Hypertexturing Complex Volume Objects”
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Tool directions
• Work with users!– Put technology into user’s hands … make it a tool
– Help create that attention-grabbing event
?
SAN DIEGO SUPERCOMPUTER CENTERUniversity of California, San Diego
Acknowledgements
• USA National Science Foundation
• USA Department of Energy
• San Diego Supercomputer Center– Mike Bailey
– Steve Cutchin
– Alex DeCastro
– Eric Enquist
– Jon Genetti
– Mike Houston
– John Moreland