direct volume rendering (dvr): ray-casting jian huang this set of slides references slides used by...
TRANSCRIPT
Direct Volume Rendering (DVR): Ray-casting
Jian Huang
This set of slides references slides used by Prof. Torsten Moeller (Simon Fraser), Prof. Han-Wei
Shen (Ohio State).
Papers
• Tuy and Tuy, 1984, IEEE CG & A (one of the earliest volume rendering techniques)
• Levoy, 1988 IEEE CG&A, and later improvements
• Drebin, Carpenter, Hanrahan, 1988, SIGGRAPH
Direct: No conversion to surface geometry
Basic Idea
Based on the idea of ray tracing• Trace from eat each pixel as a ray into object space• Compute color value along the ray• Assign the value to the pixel
Data Representation•3D volume data are represented by a finite number of cross sectional slices (hence a 3D raster)•On each volume element (voxel), stores a data value (if it uses only a single bit, then it is a binary data set. Normally, we see a gray value of 8 to 16 bits on each voxel.)
N x 2D arraies = 3D array
Data Representation (2)
What is a Voxel? – Two definitions
A voxel is a cubic cell, whichhas a single value cover the entire cubic region
A voxel is a data pointat a corner of the cubic cellThe value of a point inside the cell is determined by interpolation
Viewing
Ray Casting
• Where to position the volume and image plane • What is a ‘ray’ • How to march a ray
Viewing (1)
1. Position the volumeAssuming the volume dimensions is w x w x w
We position the center of the volume at the world origin
y
(0,0,0) x
z
Volume center = [w/2,w/2,w/2](local space)
Translate T(-w/2,-w/2,-w/2)
(data to world matrix? world to data matrix )
Viewing (2)
2. Position the image planeAssuming the distance between the image plane and the volume center is D, and initially the center of the image plane is (0,0,-D)
y
(0,0,0) x
z
Image plane
Viewing (3)
3. Rotate the image planeA new position of the image plane can be defined in termsof three rotation angle with respect to x,y,z axes
Assuming the original view vector is [0,0,1], then the newview vector g becomes:
cossincossing = [0,0,1] 0 1 0 0 cos sinsincos sin 0 cossincos
Viewing (4)
y
(0,0,0) x
z
uv
E
S
E0 u0v0
+ S0
B
B = [0,0,0]S0 = [0,0,-D]u0 = [1,0,0]v0 = [0,1,0]
Now,R: the rotation matrix S = B – D x g U = [1,0,0] x R V = [0,1,0] x R
Viewing (5)
R: the rotation matrix S = B – D x g U = [1,0,0] x R V = [0,1,0] x R
+ S
Image Plane: L x L pixels
E
Then
E = S – L/2 x u – L/2 x v
So Each pixel (i,j) has coordinates
P = E + i x u + j x v
u
v
We enumerate the pixels by changingi and j (0..L-1)
Viewing (6)
4. Cast raysRemember for each pixel on the image planeP = E + i x u + j x vand the view vector g = [0,0,1] x RSo the ray has the equation:
Q = P + k (d x g) d: the sampling distance at each step
xx
dx
x p
Q
K = 0,1,2,…
Early Methods• Before 1988• Did not consider transparency• did not consider sophisticated light
transportation theory• were concerned with quick solutions• hence more or less applied to binary data
non-binary data - require sophisticated
classification/compositing methods!
Ray Tracing -> Ray Casting
• “another” typical method from traditional graphics
• Typically we only deal with primary rays -hence: ray-casting
• a natural image-order technique• as opposed to surface graphics - how do we
calculate the ray/surface intersection???• Since we have no surfaces - we need to
carefully step through the volume
Ray Casting• Stepping through the volume: a ray is cast
into the volume, sampling the volume at certain intervals
• The sampling intervals are usually equi-distant, but don’t have to be (e.g. importance sampling)
• At each sampling location, a sample is interpolated / reconstructed from the grid voxels
• popular filters are: nearest neighbor (box), trilinear (tent), Gaussian, cubic spline
• Along the ray - what are we looking for?
Example: Using the nearest neighbor kernel
In tuys’ paperQ = P + K x V (v=dxg)
At each step k, Q is roundedoff to the nearest voxel(like the DDA algorithm)
Check if the voxel is on the boundary or not (compareagainst a threshold)
If yes, perform shading
Basic Idea of Ray-casting Pipeline
- Data are defined at the corners of each cell (voxel)
- The data value inside the voxel is determined using interpolation (e.g. tri-linear)
- Composite colors and opacities along the ray path
- Can use other ray-traversal schemes as well
c1
c2
c3
Ray Traversal - First
Depth
Intensity
First
• First: extracts iso-surfaces (again!)done by Tuy&Tuy ’84
Ray Traversal - MIP
Depth
IntensityMax
• Max: Maximum Intensity Projectionused for Magnetic Resonance Angiogram
Ray Traversal - Accumulate
Depth
Intensity
Accumulate
• Accumulate opacity while compositing colors: make transparent layers visible!Levoy ‘88
Raycasting
color c = c s s(1 - ) + c
opacity = s (1 - ) +
1.0
object (color, opacity)
volumetric compositingInterpolationkernel
Volume Rendering PipelineAcquired values
Data preparation
Prepared values
classificationshading
Voxel colors
Ray-tracing / resampling Ray-tracing / resampling
Sample colors
compositing
Voxel opacities
Sample opacities
Image Pixels
DCH - PipelineOriginal data
Material percentage volumes
Color volumeOpacity volumeDensity volume
Gradient Shaded volume
Transformed volumeFinal image
Classification
Normals
Shading
shears
compositing
Common Components of General Pipeline
• Interpolation/reconstruction
• Classification or transfer function
• Gradient/normal estimation for shading– Question: are normals also interpolated?
Levoy – Gradient/Normals
• Central difference• per voxel
2
2
2
1,,1,,
,1,,1,
,,1,,1
kjikjiz
kjikjiy
kjikjix
vvG
vvG
vvG
X+1y-1,
Y+1
z-1
Levoy - Shading
• Phong Shading + Depth Cueing
))()((21
nsd
pap HxNkLxNk
xdkk
CkCxC
• Cp = color of parallel light source
• ka / kd / ks = ambient / diffuse / specular light coefficient
• k1, k2 = fall-off constants
• d(x) = distance to picture plane• L = normalized vector to light• H = normalized vector for maximum highlight
• N(xi) = surface normal at voxel xi
Classification
Classification: Mapping from data to opacities
Region of interest: high opaicity (more opaque) Rest: translucent or transparent
The opacity function is typically specified by the user
Levoy came up with two formula to compute opacity1. Isosurface
2. Region boundary (e.g. between bone and fresh)
Classification/Transfer Function
• Maps raw voxel value into presentable entities: color, intensity, opacity, etc.
Raw-data material (R, G, B, , Ka, Kd, Ks, ...)
• May require probabilistic methods (Drebin). Derive material volume from input. Estimate % of each material in all voxels. Pre-computed. AKA segmentation.
• Often use look-up tables (LUT) to store the transfer function that are discovered
Levoy - Classification
• Usually not only interested in a particular iso-surface but also in regions of “change”
• Feature extraction - High value of opacity exists in regions of change
• Transfer function (Levoy) - Saliency• Surface “strength”
Levoy - Classification
• Chemistry Data– only iso-value - loose information of layers– iso-range - could be too narrow or too wide– thickness of region should be constant– hence linear fall off of opacity– wider fall off for larger gradient
• Medical Data– assume at most two tissues meet– linear transition between opacities of “neighboring”
tissues– reflects linear combination of tissues within one
voxel
Opacity function (1)
Goal: visualize voxels that have a selected threshold value fv - No intermediate geometry is extracted - The idea is to assign voxels that have value fv the maximum opacity (say ) - And then create a smooth transition for the surrounding area from 1 to 0 -Levoy wants to maintain a constant thickness for the transition area.
Opacity function (2)
Maintain a constant isosurface thickness
opacity = opacity = 0
Can we assign opacity based on function value instead of distance? (local operation: we don’t know wherethe isosurface is)
Yes – we can based on the value distance f – fv but we need to take into account the local gradient
Opacity function (3)
Assign opacity based on value difference (f-fv) and local gradient
gradient: the value fall-off rate grad = fsAssuming a region has a constant gradient and the isosurfacetransition has a thickness R
opacity = F = fv
opacity = 0F = fv – grad * R
thickness = R
F = f(x)Then we interpolate the opacity
opacity = – * ( fv-f(x))/ (grad * R)
Levoy - Classification A
Opacity (xi)
Acquired value f (xi)
Gradient magnitude |f (xi)| Opacity (xi)
Acquired value f (xi)
Gradient magnitude |f (xi)|
v
fv
i
ivvi xf
xff
rx
'
11
Levoy - Classification B
Opacity (xi)
Acquired value f (xi)
Gradient magnitude |f (xi)|
vc
vbva
fvafvb
fvc
nn
n
n
nn
n
n
vv
ivv
vv
vivii ff
xff
ff
fxfxfx
1
1
1
1'
DCH - Material Percentage V.
• Probabilistic classifier• probability that a voxel has intensity I:
• pi - percentage of material
• Pi(I) - prob. that material i has value I
• Pi(I) given through statistics/physics
• pi then given by:
n
iii IPpIP
1
n
jj
ii
IP
IPIp
1
DCH - Classification• Like Levoy - assumes only two materials per voxel• that will lead to material percentage volumes
• from them we conclude color/opacity:
– where Ci=(iRi, iGi, iBi, i)
n
iiiCpC
1
DCH- Classification
Air Fat TissueBone
CT Number
Air Fat Tissue Bone
histogramConstituent’s Distributions
Material Assignment%
CT
Levoy - Improvements
• Levoy 1990• front-to-back with early ray termination
= 0.95• hierarchical oct-tree data structure
– skip empty cells efficiently