Download - Real-Time Volume Graphics
Klaus EngelSiemens Corporate ResearchPrinceton
Aaron E. LefohnCenter for Image Processing and Integrated ComputingUniversity of California, Davis
Markus HadwigerVR VIS Research CenterVienna, Austria
Christof Rezk SalamaComputer Graphics andMultimedia GroupUniversity of Siegen, Germany
Joe M. KnissScientific Computing and Imaging Insitute,University of Utah
Daniel WeiskopfVisualization and Interactive Systems Group,University of Stuttgart, Germany
Real-Time Volume Graphics
Klaus EngelSiemens Corporate ResearchPrinceton
Aaron E. LefohnCenter for Image Processing and Integrated ComputingUniversity of California, Davis
Markus HadwigerVR VIS Research CenterVienna, Austria
Christof Rezk SalamaComputer Graphics andMultimedia GroupUniversity of Siegen, Germany
Joe M. KnissScientific Computing and Imaging Insitute,University of Utah
Daniel WeiskopfVisualization and Interactive Systems Group,University of Stuttgart, Germany
Real Time Volume Graphics
Dealing With Large Volumes
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Large Volumes - MotivationLong-leg study 512x512x1747@16bit ~ 1GB
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Large Volumes - Motivation
Visible Female CT scan: 512x512x1724@16bit
~ 1GB
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Large Volumes - Motivation
Visible Male cryosection RGB data:2048x1216x1877@24bit ~ 14GB
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Large Volumes - Motivation
4D cardiac data: 512x512x240@16 bit, 20 frames ~ 2.5GB
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Large Volumes - Caches
Host memoryHost memory
Graphics card
memory
Graphics card
memoryAGP memoryAGP memory
Texture cacheTexture cache
~5 GB/sec
AGP8x: 2 GB/sec
4 GB
256 MB
Memory + Bandwidth!
160 MB/sec
> 30 GB/sec
Level 1 cache
Level 2 cacheLevel 3 cache
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Large Volumes - Introduction
Problems with large volumes on GPUsEach Voxel is accessed multiple times
FilteringGradient ComputationOversampling
=> Huge memory bandwidth required
Limited memory bandwidth: GPU: >30GB/sAGP8x: 2GB/sLimited GPU memory: typically 256 MB
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Large Volumes - Overview
BrickingMulti-Resolution techniquesCompression
Packing techniquesWaveletsVector quantization
Procedural description of detailOptimizations
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Large Volumes - Bricking
Subdivide volume into smaller blocksAllocate memory for one block on GPUCopy in GPU mem. and render one block at a timeOne voxel overlap for contiguous interpolation
Brick1 Brick2
No overlap
One voxel overlap
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Large Volumes - Bricking
Performance mainly limited by AGP transferSubsampled copy of data in GPU mem. for interactionBricking only for final quality image
Texture cacheTexture cache
AGP8x: 2 GB/sec
160 MB/sec
> 30 GB/sec
Level 1 cache
Level 2 cacheLevel 3 cache AGP memoryAGP memory
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Large Volumes - Multi-Resolution VRLaMar et al., Multi-Resolution techniques for interactive
texture-based volume visualization, IEEE Visualization’99
Octree-based decomposition of volume into bricks
Render bricks at differentresolution:
Distance to viewerFocus point
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Large Volumes - Multi-Resolution VRWeiler et al., Level-of-Detail volume rendering via 3D textures, In Volume Visualization and Graphics Sympsium 2000
Extension to fix boundaries
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Large Volumes - CompressionTexture Compression OpenGL extensions: 2D: EXT_texture_compression_s3tc.3D: NV_texture_compression_vtc
Hardware implementation in several graphics chips, e.g. NVIDIA GeForce series, ATI Radeon Series.
Disadvantages of S3TC:Moderate compression ratios,Block-artifacts, inappropriate for non-smooth dataFixed compression schemeOnly for RGB(A) data
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Large Volumes - Wavelets
Volume dataMostly smoothFine high-frequency detail incertain regions
WaveletsAnalyze data at differentresolutions and frequenciesMany coefficient very small=> compressionHierarchy of signals=> Multi-res reconstruction
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Large Volumes - WaveletsGuthe et al., Interactive Rendering of Large Volume Data Sets, Visualization’02
Hierarchical Wavelet RepresentationDivide data into blocks of (2k)3 voxels (k=16 good)Apply wavelet filters=> lowpass filtered block=> wavelet coefficientsGroup 8 blocksRepeat until 1 block left
2 encoding schemes
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Large Volumes - Wavelets
Decompression of blocks during rendering on CPURendering of block on the GPUCaching Strategy
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Large Volumes - Wavelets
Adjust resolution of data to screen resolution
Project voxel-space to screenRefine if above screen res.
View-dependent priority schedule
Interactive walkthrough of visible female/maleStill AGP bandwidth bound=> GPU decompression possible ?
Google “dwtgpu”
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Large Volumes - Packing
TWO LEVELS OF THE DATA REPRESENTATION:
Index data (upper level): Each cell/texel of a coarse gridcorresponds to one data block.Each cell/texel specifies coordinates and scaling factorsof the corresponding data block.
Packed data (lower level):All data blocks packed into oneuniform grid/texture.Kraus et al., Adaptive Texture Maps, Graphics Hardware Workshop’02
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Large Volumes - PackingTWO STEPS OF SAMPLING ADAPTIVE TEXTURES:
Read index data and calculate coordinates for the second step.
Read and interpolate actual texture data from packed data.
Decoding in fragment stage
(s,t)
(s’,t’)
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Large Volumes - PackingWei Li et al. – Texture Partitioning and Packing for
Accelerating Texture-based Volume Rendering, GI 2003Partition texture space with box-growing algorithmBased on similar densities and non-zero gradient magnitudes
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Large Volumes - Packing
Determine gradient sub-textures with non-zero magnitudePack sub-textures into a single smaller textureDecoding in vertex stage using texture coords.
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Large Volumes - VQ
Image formats with palettes specify for each pixel one index into a color palette (= codebook).1 1 1 2
0 0 1 1
0 0 0 1
0 0 0 0
C1
C1
C1
C2
C0
C0
C1
C1
C0
C0
C0
C1
C0
C0
C0
C0
index data
palette(codebook)
decoded data
C0
C1
C2
0 1 2
dependent texture read
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Large Volumes - VQSchneider/Westermann – Compression Domain Volume Rendering, IEEE
Visualization 2003
3 Level Hierarchical decomposition:Partition data into blocks of size 43
Downsample to 23, store difference vector (64 vector)
Downsample to 1, store difference vector (8 vector)
VQ 64 vector, VQ 8 vector
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Large Volumes - VQCodebooks: 256x64 + 256x8 images, dependent texture lookup
Codebook generation: modified LGB-Algorithm (Linde, Buzo and Gray)
Series of PCA-Splits (Principle component analysis) determine initial codebook
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Large Volumes - VQWith a codebook of length 256, a 10243 volume is reduced to 3*2563 bytes = 48 MBytes, i.e. it fits easily into texture memory.
Compression of 4D sequences: store complete sequence in GPU memory (shockwave sequence - original: 1.5 GB, compressed: 70 MB).
currently: only nearest-neighbor filtering=> decouple decompression and rendering
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Large Volumes – Procedural Approach
Ebert et al.: Texturing and Modeling:
A Procedural Approach, Academic Press, 1998
coarse volume for macrostructure
procedural noise for
microstructure
Images Courtesy of Joe Michael Kniss
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Large Volumes – Procedural ApproachKniss et al., Interactive Translucent Volume Rendering and
Procedural Modeling, Visulization’02:perturb data access (instead of data)good for distortion of boundariesimplemented using offset-textures
original texture multiple scaled
versions of perturbation texture
summed offset
vectors
result of
dependent fetch
Images Courtesy of Joe Michael Kniss
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Optimizations - Shading
Pre-computation of gradients impractical due to memory requirements
Calculate the gradient vector on-the-flyFetch neighbors and compute normal in fragment programrequires a lot of texture fetches (6 for central diff.)Only possible on newer graphics hardware(nvidia NV3x, ATI R3xx)
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Optimizations - Shading
Shade only where necessaryShade only if alpha is above a threshold
Two passes1st Pass: Render unshaded in first pass1st Pass: Set z/alpha where alpha is above threshold2nd Pass: Use early-z/stencil test2nd Pass: shade where z/alpha test succeeds
Requires early-z/stencil test (ATI R3xx)
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Optimizations – Space LeapingDon’t access memory that contains no data
Subdivide volume into blocksStore minimum/maximum value per blockCheck with transfer function and min/max if block is non-emptyRender block only if not empty
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Optimizations – Ray TerminationKrueger/Westermann – Acceleration Techniques for GPU-based
Volume Rendering, IEEE Visualization 2003
Terminate rays(pixels) that have accumulated maximum opacity
Termination is done in a separate passRender bounding box with framebuffer as textureCheck for each pixel if alpha above thresholdSet z/stencil value if above thresholdRequires early-z/stencil test (ATI R3xx)
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Conclusions
Large Volumes can be rendered with GPUs
Compression and adaptive Algorithms required for interactive Performance
Lots of Optimizations possibleEarly-Z/Stencil tests very useful
REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA
Questions?Thank you!
Acknowledgements:NVIDIA and ATI for the demo hardware
For some of the images and figures:Joe Michael KnissWei LiJens SchneiderMartin KrausStefan Guthe