ieee transactions on image processing, vol. 9, no. 7, …mwestenb/papers/ipwavfour.pdf · ieee...

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 9, NO. 7, JULY 2000 1249

Frequency Domain Volume Rendering by theWavelet X-ray Transform

Michel A. Westenberg, Student Member, IEEE and Jos B. T. M. Roerdink, Member, IEEE

Abstract— We describe a wavelet-based X-ray renderingmethod in the frequency domain with a smaller time com-plexity than wavelet splatting. Standard Fourier volumerendering is summarized and interpolation and accuracy is-sues are briefly discussed. We review the implementationof the fast wavelet transform in the frequency domain. Thewavelet X-ray transform is derived, and the correspondingFourier-wavelet volume rendering algorithm (FWVR) is in-troduced. FWVR uses Haar or B-spline wavelets and lin-ear or cubic spline interpolation. Various combinations aretested and compared with wavelet splatting (WS). We usemedical MR and CT scan data, as well as a 3-D analyticalphantom to assess the accuracy, time complexity, and mem-ory cost of both FWVR and WS. The differences betweenboth methods are enumerated.

Keywords—Fourier volume rendering, Fourier-wavelet vol-ume rendering, wavelet splatting, wavelet X-ray transform.

I. Introduction

VISUALIZATION and exploration of large three-dimensional (3-D) digital data volumes is becom-

ing increasingly popular. Volume rendering is prominentamong the techniques which have been developed for thispurpose, using advanced computer graphics techniquessuch as illumination, shading and color. The desire to ex-change volume data through systems such as the Internethas created a need for fast and efficient methods of transferand display. To relieve the demand on the server capacity,volume data may be stored on a central server, while (partof) the rendering is performed on client systems. Not allof these clients will have a high-bandwidth network con-nection, so that we need a mechanism to visualize dataincrementally as it arrives (‘progressive refinement’). Forthis purpose multiresolution models are developed, allow-ing systematic decomposition of the data into versions atdifferent levels of resolution. Another benefit of such ap-proaches is local level-of-detail (LOD), i.e., using a lowerresolution for small, distant or unimportant parts of thedata. Such a mechanism is provided by the wavelet tech-nique.

This paper is concerned with a direct volume renderingmethod [1] called X-ray volume rendering, which is basedupon integrating the 3-D data along the line of sight, yield-ing a two-dimensional (2-D) image in the view plane. Themethod supports shading and depth-cueing [2], [3], but noocclusion and perspective projection. However, it turns out

Manuscript received May 6, 1999; revised November 30, 1999. Theassociate editor coordinating the review of this manuscript and ap-proving it for publication was Prof. Kannan Ramchandran.

The authors are with the Institute for Mathematics and ComputingScience, University of Groningen, P.O. Box 800, 9700 AV Groningen,The Netherlands. E-mail: [email protected]; [email protected]

Publisher Item Identifier S 1057-7149(00)05316-1.

to be one of the preferred techniques for medical applica-tions, because physicians are well-trained in interpretingX-ray like images for diagnosis. The corresponding math-ematical concept is the X-ray transform, well-known fromcomputerized tomography [4]. An efficient way to com-pute this transform makes use of frequency domain tech-niques [5], [6]; in the following, this method is referred toas Fourier volume rendering, abbreviated as FVR. Afteran initial 3-D Fourier transform of the data, a view direc-tion θ is chosen and the values of the Fourier transformin a plane, called the slice plane, perpendicular to θ arecomputed. Interpolation in frequency space is necessary toobtain the values of the Fourier transform of the functionto be visualized at a regular pixel grid in the slice plane. Asubsequent inverse 2-D Fourier transform gives the desiredimage in the view plane. The time complexity of FVR isdominated by the 2-D inverse Fourier transform from theslice plane to the view plane, hence is O(N2 logN) for avolume data set of size N × N × N . Frequency domainvolume rendering algorithms have to deal with problemsrelated to high interpolation cost and high memory cost.However, since processing power has increased and mem-ory costs have dropped significantly, these problems are lessserious on modern hardware.

Another volume rendering method based on the X-raytransform is splatting [7]. This is an object order methodwhich reconstructs a continuous function from discrete databy convolution with a reconstruction filter, followed by amapping to the image plane by superposition of buildingblocks called “splats” or “footprints”. This method sup-ports occlusion, shading, and perspective projection, butsuffers from color bleeding or “popping” [8] and aliasing [9].Previously, wavelet splatting has been proposed by Lippertand Gross [10] as an extension of the splatting method, seealso [11], [3]. Wavelet splatting modifies the splatting al-gorithm by using wavelets as reconstruction filters, so thatdata can be visualized at different levels of detail. Just asthe original splatting method, the time complexity of thisalgorithm is O(N3).

The purpose of this paper is the derivation of a wavelet-based X-ray rendering method with a smaller time com-plexity than wavelet splatting. Since in ordinary volumerendering this goal can be achieved by frequency domaintechniques, a similar approach is followed in this paper, re-sulting in an algorithm with the same time complexity asordinary FVR, i.e. O(N2 logN).

To achieve this goal, we study the wavelet X-ray trans-form, as introduced in [12], which combines integrationalong the line of sight with a simultaneous 2-D wavelettransform in the plane perpendicular to this line. A closely

1250 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 9, NO. 7, JULY 2000

related transform was introduced in [13], [14], and combinesintegration over a line with a simultaneous 1-D wavelettransform along this line. We derive an efficient imple-mentation of the wavelet X-ray transform by using a fre-quency domain implementation of the wavelet transform[15], [16]. This is particularly efficient when the length ofthe wavelet decomposition and/or reconstruction filters islarge, as is the case for some of the basic wavelets (e.g.B-spline wavelets) used in this paper. This results in an al-gorithm whose initial step, i.e. computation of the Fouriertransform in a slice plane, is identical to that of ordinaryFVR. The additional step is a wavelet decomposition ofthe slice plane data in Fourier space to a given level ofdetail. Approximation images are then obtained by a par-tial wavelet reconstruction in Fourier space, followed by a2-D inverse Fourier transform. Since wavelet detail coeffi-cients are available in Fourier space, progressive refinementis straightforward. The Fourier space representation doesnot allow local level-of-detail.

The organization of this paper is as follows. Section IIsummarizes standard Fourier volume rendering. Interpola-tion and accuracy issues are briefly discussed. Section IIIintroduces some basic wavelet concepts, and reviews im-plementation of the wavelet transform in the frequency do-main. Section IV then introduces the wavelet X-ray trans-form and the corresponding Fourier-wavelet volume render-ing (FWVR) algorithm. A comparison of the new methodand wavelet splatting with respect to accuracy, time com-plexity and memory cost is presented in Section V. Sec-tion VI contains a summary and discussion of future work.

II. Fourier Volume Rendering

Fourier domain volume rendering methods [6], [5] pro-vide an implementation of X-ray volume rendering, wherethe volume data are integrated along the line of sight.The mathematical basis is the X-ray transform, well-knownfrom computerized tomography [17].

A. The X-ray Transform

Consider the line integrals of a continuous function f(x),x = (x, y, z) ∈ R3, along a direction vector θ. Let u and vbe two mutually orthogonal vectors perpendicular to θ, cf.Fig. 1. The X-ray transform Pθf of f is defined by

Pθf(u, v) =∫R3f(x)δ(x · u− u)δ(x · v − v) dx

=∫R

f(uu+ vv + tθ) dt, (1)

where a dot denotes the inner product in R3.The Fourier projection slice theorem [17] states that the

2-D Fourier transform of the X-ray transform Pθf equalsthe 3-D Fourier transform of f along a slice plane throughthe origin and perpendicular to θ:

F2Pθf(ωu, ωv) = F3f(ωuu+ ωvv), (2)

where Fnf denotes the n-dimensional Fourier transform ofa function f .

P (u,v)

y

θx

θ u

v

z

Fig. 1. View plane orthogonal to the direction vector θ.

B. FVR Algorithm

The Fourier slice theorem is the key to Fourier volumerendering. Assume that the volume data are samples on auniform grid of a band-limited function f , whose highestfrequency is determined by the sampling rate of the volumedata (non-uniform grids require resampling). The FVRalgorithm consists of the following steps:

• Preprocessing. Compute the 3-D discrete Fouriertransform of the volume data by FFT.• Actual volume rendering. For each direction θ, do:1. Interpolate the Fourier transformed data and re-

sample on a regular grid of points in the slice planeorthogonal to θ (‘slice extraction’).2. Compute the 2-D inverse Fourier transform, again

by FFT. This yields a discrete approximation to Pθf .The first step is just preprocessing: the 3-D Fourier trans-form is computed only once. The next two steps are re-peated for each viewing direction. The time complexity forcomputing one view depends both on the complexity of the2-D Fourier transform and on the interpolation cost. If theextracted slice is of size N by N , then the complexity ofthe Fourier transform is O(N2 logN). The complexity of3-D interpolation is O(K3N2), where K is the linear size ofthe interpolation filter with K much smaller than N . Al-though the Fourier transform is asymptotically dominant,in practice most of the running time is spent on interpola-tion.

C. Interpolation

Interpolation is the most critical step in Fourier render-ing, and good interpolation functions are needed to avoidartefacts such as aliasing and dishing. Dishing is a hill-shaped weighting artefact due to the shape of the Fouriertransform of the interpolation function, resulting in re-duced intensities away from the center of the image. Alias-ing is due to insufficient sampling. A set of discrete samplesin the frequency domain corresponds to an infinitely peri-odic signal in the spatial domain. If the original samplingstep in frequency space is F0, one has to resample with astep size of at least F0/

√3 in order to prevent aliasing [5].

In practice one usually takes a resampling step size of F0/2.A way to reduce aliasing is to pad the data in the spatialdomain with zeros before the initial 3-D Fourier transformis taken. This separates the replicas in the spatial domain,

WESTENBERG AND ROERDINK: FREQUENCY DOMAIN VOLUME RENDERING BY THE WAVELET X-RAY TRANSFORM 1251

and decreases the sampling distance in the frequency do-main. A disadvantage of zero-padding is increased memoryusage. In the case of volume data, zero-padding by a fac-tor of two increases the required amount of memory by afactor of eight.

Frequency domain interpolation with a filter F (k) corre-sponds to multiplication (windowing) of the original imageby the inverse Fourier transform of F (k). Since ideally thewindow function should be rectangular, the ideal resam-pling filter in Fourier space is a sinc-function. To reducethe computational demands, interpolation functions withsmaller support are used. Linear interpolation is compu-tationally attractive, but the side lobes of the linear in-terpolation function in the spatial domain are the sourceof severe aliasing, requiring a large amount of initial zero-padding of the volume data. The cubic spline functionis a better approximation of a rectangle than the linearinterpolation function; it has a higher computational com-plexity, but requires less zero-padding [6]. The 2-D filtersneeded for FVR are constructed as products of the 1-D fil-ters. Both interpolation functions have been compared byNapel et al. [6] for Fourier domain volume rendering. Keys[18] and Parker et al. [19] give a more extensive comparisonof interpolation filters in general.

Malzbender [5] introduced spatial premultiplication as apreprocessing operation to reduce dishing. Spatial premul-tiplication entails point-wise multiplication of the data inthe spatial domain by the reciprocal of the inverse Fouriertransform of the interpolation function. Although spatialpremultiplication reduces dishing, it increases the aliasingerror, because effectively all periodic copies of the datain the spatial domain are premultiplied by the reciprocalfunction.

D. Accuracy

In order to assess the quality of various volume renderingalgorithms we use a 3-D head phantom, as defined in Kakand Slaney [17], consisting of a collection of ellipsoids of dif-ferent density values. Because of the linearity of the X-raytransform, a projection of an object consisting of ellipsoidsis the sum of the projections of the individual ellipsoids.These projections can be computed analytically. We haveslightly adapted the phantom, originally devised to testthe accuracy of tomographic reconstruction algorithms, sothat the range of grey values is better suited for visualinspection (see the Appendix). Besides analytical projec-tions, we generated a volume data set of size 1283 from themathematical description. To reduce high frequencies inthe phantom volume data, the ellipsoids are smoothed bysupersampling by a factor of two and by weighting with afourth-order B-spline filter. This makes the phantom morerealistic, since data originating from a CT-scanner, for ex-ample, are low-pass filtered as well. Independent of theamount of zero padding, we use a doubling of the originalsampling step in frequency space, so that volume data ofsize N3 give rise to rendered images of size (2N)2. Fig. 2shows an analytical projection of the phantom.

Fig. 3 shows images of size 2562 of the phantom data

Fig. 2. Analytical projection image of the 3-D head phantom forθ = (1, 0, 0).

rendered by FVR without premultiplication, for both lin-ear and cubic spline interpolation, and with 20% and 100%zero-padding. The images are displayed with the greyvalues inverted, so that aliasing artefacts show up moreclearly. For 20% zero-padding, linear interpolation still suf-fers from severe aliasing. The first replica shows up clearly,and close inspection reveals that the second replica is alsovisible, though very faintly, between the image in the cen-ter and the first replica near the edge. For cubic splineinterpolation, aliasing is much less prominent. For 100%zero-padding, no visible aliasing occurs, but memory costsare increased by a factor of eight.

(a) (b)

(c) (d)

Fig. 3. Results of Fourier volume rendering with different inter-polation functions and amounts of zero-padding. The images aredisplayed with inverted grey values. (a) Linear interpolation, 20%zero-padding. (b) Linear interpolation, 100% zero-padding. (c)Cubic spline interpolation, 20% zero-padding. (d) Cubic splineinterpolation, 100% zero-padding. θ = (1, 0, 0).

Fig. 4 shows intensity profiles corresponding to the linex = 122 of the images shown in Fig. 3. Without pre-multiplication and 20% zero-padding, dishing is very se-vere for linear interpolation, but almost negligible for cu-bic spline interpolation. Premultiplication reduces dish-


ing, but amplifies aliasing. This explains the large valuesaround y = 128 for 20% zero-padding and linear interpola-tion in Fig. 4(b). For both interpolation functions, dishingdisappears completely with 100% zero-padding.

80 100 120 140 160 1801000

1500

2000

2500

3000

3500

reference20% ZP100% ZP

(a) without premultiplica-tion

80 100 120 140 160 1801000

1500

2000

2500

3000

3500


(b) with premultiplication

80 100 120 140 160 1801000

1500

2000

2500

3000

3500


(c) without premultiplica-tion

80 100 120 140 160 1801000

1500

2000

2500

3000

3500


(d) with premultiplication

Fig. 4. Profiles of FVR renderings of the head phantom along the linex = 122. (a) and (b) linear interpolation, (c) and (d) cubic splineinterpolation. The reference profiles are obtained by analyticalcomputation.

In conclusion, cubic spline interpolation with 20% zero-padding appears to offer a good compromise for FVR, re-sulting in small aliasing error and dishing artefact. Lin-ear interpolation with 100% zero-padding produces nearlythe same results as cubic spline interpolation, but memorycosts are increased by a factor of eight. Premultiplicationincreases the aliasing error, therefore, we do not use it inthis paper.

E. Reducing Memory Costs

The 3-D Fourier transform requires complex arithmeticand a floating point representation. Since the Fouriertransform of a real signal is hermitian, a factor of twocan be saved by dropping half of the Fourier transformeddata. Whereas some authors invoke the Hartley transformfor this purpose, we accomplish this by the use of a real-to-complex/complex-to-real FFT, such as provided by thepackage FFTW [20]. Values of coefficients which are notknown directly can always be computed by complex conju-gation of the corresponding coefficients in the known part ofthe Fourier transform. The slice extraction process makesuse of this fact to save a factor of two in the number ofcomputations.

Secondly, the original data usually has only 2 bytes pervoxel. We experimentally found that, instead of using 8bytes per voxel, it is possible to save another factor of twoby quantizing the floating point values to 2-byte shorts,

without seriously affecting the accuracy. This is done byscaling the floating point values linearly to the full rangeof shorts. During the slice extraction process, the floatingpoint value is reconstructed, which requires one multipli-cation and one addition.

III. Wavelets

In this section some basic facts about wavelet represen-tations are introduced. In particular, we discuss the fastwavelet transform and its Fourier domain implementation.

A. Wavelet Representation

A 1-D biorthogonal wavelet basis can be constructedfrom a scaling function φ with associated wavelet ψ, anddual scaling function φ with dual wavelet ψ. The cor-responding basis functions are {φj,k} and {ψj,k}, j, k ∈Z, where φj,k(x) = 2−j/2φ(2−jx − k) and ψj,k(x) =2−j/2ψ(2−jx−k); the dual basis functions are defined sim-ilarly. The parameters j and k denote scale and transla-tion, respectively. From the 1-D basis, one constructs a2-D separable wavelet basis (of the so-called nonstandardtype) with four basis functions, i.e. one scaling functionΦ0j,k,l(x, y) and three wavelet basis functions Ψτ

j,k,l(x, y),τ ∈ T = {1, 2, 3}, defined as follows:

Φ0j,k,l(x, y) = φj,k(x)φj,l(y)

Ψ1j,k,l(x, y) = φj,k(x)ψj,l(y)

Ψ2j,k,l(x, y) = ψj,k(x)φj,l(y)

Ψ3j,k,l(x, y) = ψj,k(x)ψj,l(y). (3)

An analogous definition holds for the dual scaling functionΦ0j,k,l(x, y) and wavelet basis functions Ψτ

j,k,l(x, y).The M -level wavelet representation of a 2-D function f

is then given by

f(x, y) =∑k,l

cMk,lΦ0M,k,l(x, y)

+M∑j=1

∑τ∈T

∑k,l

dj,τk,lΨτj,k,l(x, y). (4)

The approximation coefficients are cMk,l = 〈f, Φ0M,k,l〉 and

the detail coefficients are dj,τk,l = 〈f, Ψτj,k,l〉, where 〈·, ·〉 de-

notes the inner product in the space L2(R2) of square in-tegrable functions on R2.

In practice one deals with functions f of compact sup-port. In order to apply the wavelet representation, f has tobe extended to a function on the real line. The simplest ap-proach, and also the one we use in this paper, is to extend fby zero values outside the support. Note that, in the follow-ing, zero-padding is not done explicitly, because the inputto the wavelet transform has already been padded with ze-ros for slice resampling (c.f. Section II-D). The amount ofzero-padding is usually large enough to prevent artefactsdue to circular convolution, provided that the number ofdecomposition levels in the wavelet transform is kept small,and that the associated filters have a not too large support.


We use this simple approach, because other methods adaptthe analysis and synthesis filters near the boundaries, andare therefore difficult to use in the frequency domain.

B. The Fast Wavelet Transform

The fast wavelet transform computes the wavelet de-composition, i.e., the approximation and detail coefficients,with a subband filtering scheme called the pyramid algo-rithm [21]. For the 1-D case, the basis functions φ andψ are represented by discrete filters h = (hn)n∈Z andg = (gn)n∈Z, respectively. Furthermore, there exist dualfilters h and g; for the orthogonal case these are definedby hn = h−n and gn = g−n (here f denotes the complexconjugate of f). The filters h and g are used in the forwardwavelet transform, and are therefore called decompositionor analysis filters; the filters h and g are used for the in-verse wavelet transform, and are called reconstruction orsynthesis filters.

The 2-D basis (3) is represented by the four possible ten-sor products, hh, hg, gh and gg, of the 1-D filters h andg. [For example (hh)k,l = hk hl.] Let cj denote the 2-Dsequence (cjk,l)k,l∈Z, and similarly for dj,τ . The wavelet de-composition computes the sequences cj+1 and dj+1,τ fromcj by convolution followed by downsampling, as follows:

cj+1 =↓↓2 (hh ∗ cj) dj+1,1 =↓↓2 (hg ∗ cj)

dj+1,2 =↓↓2 (gh ∗ cj) dj+1,3 =↓↓2 (gg ∗ cj), (5)

for j = 0, . . . ,M − 1. Here ∗ denotes discrete 2-D convo-lution, and ↓↓2 denotes downsampling by a factor of twoin both dimensions. The filters hh, hg, gh, and gg are thedual filters. [For example (hh)k,l = hk hl.]

Wavelet reconstruction is performed recursively startingat level M by upsampling (denoted by ↑↑2) followed byconvolution:

cj = hh ∗ (↑↑2 cj+1) + hg ∗ (↑↑2 dj+1,1)

+ gh ∗ (↑↑2 dj+1,2) + gg ∗ (↑↑2 dj+1,3).(6)

C. The Fast Wavelet Transform in the fourier Domain

As motivated in the introduction, we need a way to com-pute the fast wavelet transform (FWT) in the frequency do-main. Especially when the decomposition/reconstructionfilters have large support, such a Fourier domain imple-mentation using the fast Fourier transform (FFT) is moreefficient than a direct computation, as shown in [15], [16]for the 1-D case. We now consider the extension of thismethod to 2-D signals.

C.1 Up- and Downsampling

Upsampling and downsampling can be expressed in thefrequency domain using Z-transforms. The Z-transform ofa 2-D discrete signal {x(n1, n2) : n1 = 0, 1, 2, . . . , N1 −1, n2 = 0, 1, 2, . . . , N2 − 1} is defined by

X(z1, z2) =N1−1∑n1=0

N2−1∑n2=0

x(n1, n2)z−n11 z−n2

2 , z1, z2 ∈ C.

(7)

On the unit circle, the Z-transform X(e2πikN1 , e

2πilN2 ) coin-

cides with the element Xk,l of the 2-D discrete Fouriertransform (DFT) of the signal x of length N1 by N2. Wecan split this Z-transform into contributions of the sam-ples with even and odd index, a classical technique knownas biphase decomposition in filter bank design [15], [16].Downsampling corresponds to taking the samples witheven index in both dimensions, leading to a signal withZ-transform given by

Xdown(z1, z2) =14

(X(z121 , z

122 ) +X(−z

121 , z

122 )

+X(z121 ,−z

122 ) +X(−z

121 ,−z

122 )).

The values Xdownk,l , k = 0, . . . , N1/2− 1, l = 0, . . . , N2/2− 1

of the 2-D DFT of the downsampled signal are given by

Xdownk,l = Xdown(e

2πikN1/2 , e

2πilN2/2 ) (8)

=14

(Xk,l +Xk−N1

2 ,l+X

k,l−N22

+Xk−N1

2 ,l−N22

).

So, for a DFT that stores the DC component in the firstarray position, downsampling is implemented by dividingthe 2-D array of DFT coefficients of x into two in bothdimensions, and averaging the resulting four subarrays.

Conversely, upsampling by a factor of two in the spatialdomain means inserting zeros between the samples in bothdimensions. The Z-transform of the upsampled signal isgiven by

Xup(z1, z2) = X(z21 , z

22).

So, if the array of DFT coefficients has dimensions N1 byN2, the DFT of the upsampled signal is a 2N1× 2N2 arrayobtained by replicating this array in both dimensions.

C.2 Wavelet Decomposition and Reconstruction

With the above results, it is possible to represent the2-D wavelet transform in the frequency domain. De-note the Z-transforms of the filters hh, hg, gh and gg

by HH, HG, GH and GG, respectively. For example,HH(z1, z2) = H(z1)H(z2), where H(z) :=

∑n hnz

−n isthe 1-D Z-transform of h, etc. In the same way, denotethe Z-transform of cj by Cj(z1, z2) and that of dj,τ byDj,τ (z1, z2). Since convolution in the spatial domain isequivalent to multiplication in the Fourier domain, we canrewrite (5) in terms of Z-transforms:

Cj+1(z1, z2) = [HH · Cj ]down(z1, z2)

Dj+1,1(z1, z2) = [HG · Cj ]down(z1, z2)

Dj+1,2(z1, z2) = [GH · Cj ]down(z1, z2)

Dj+1,3(z1, z2) = [GG · Cj ]down(z1, z2), (9)

where the dot denotes pointwise function multiplication.The reconstruction (6) becomes

Cj(z1, z2) = HH(z1, z2)Cj+1(z21 , z

22)

+ HG(z1, z2)Dj+1,1(z21 , z

22)

+ GH(z1, z2)Dj+1,2(z21 , z

22)

+ GG(z1, z2)Dj+1,3(z21 , z

22). (10)


In practice we deal with a finite 2-D input sequence c0,represented by an array of size N1 ×N2. In the frequencydomain, the result of an M -level decomposition then yieldsan approximation array CM of size 2−MN1× 2−MN2, anddetail arrays Dj,τ , j = M,M − 1, . . . , 1, τ = 1, 2, 3 of size2−jN1× 2−jN2. An inverse FFT of CM and of Dj,τ yieldsthe arrays cM , dj,τ of the wavelet decomposition of c0 in theoriginal domain. However, since reconstruction also can beperformed in frequency space, it is not necessary to carryout these inverse FFT’s. Instead, a reconstruction at adesired level K is first computed in the Fourier domain by(10) and the resulting approximation CK is then inverselyFourier transformed to give the desired approximation cK .

The DFT values of the filters HH, HG, GH, and GGare computed from H and G as follows. The length of thesignal in the decomposition (9) decreases with increasingscale level j. At level j, let the signal length in a givenspatial direction be 2−jN , larger than the length L of thefilter h. Then the required DFT values of H at level j are

Hjk = H(exp

[2πik2−jN

]) =

L−1∑n=0

hne− 2πink2j

N

= H0k2j , k = 0, 1, . . . , 2−jN − 1.

The DFT values of G are obtained in the same way.Clearly, it is sufficient to compute the DFT filters H0 andG0. The filters for the other scales are obtained by down-sampling the filters for the finest scale j = 0. The DFTvalues of the synthesis filters are obtained in an analogousway. Again, it suffices to compute the filters once for j = 0,the filters for the other scales being obtained by downsam-pling. Based on the 1-D filter coefficients, we define 2-Dfilter matrices Hj and Gj,τ , τ = 1, 2, 3, by

(Hj)k,l = HjkH

jl , (Gj,1)k,l= Hj

k Gjl ,

(Gj,2)k,l = GjkHjl , (Gj,3)k,l = GjkG

jl

and similarly for the dual filters. Also, Cj is the matrixwith elements Cjk,l := Cj(e

2πikN1 , e

2πilN2 ), with Dj,τ defined

analogously. Then the wavelet decomposition (9) has thematrix representation

Cj+1 = [Hj • Cj ]down, Dj+1,τ = [Gj,τ • Cj ]down, (11)

where A • B denotes pointwise multiplication of matricesA and B, and, for a matrix X with an even number of rowsand columns, Xdown is defined by [cf. (8)]

Xdown =14

(Xa + Xb + Xc + Xd) when X =(

Xa XbXc Xd

),

where Xa,Xb,Xc,Xd are the four submatrices obtained bydividing X into two along the row and column direction.

Wavelet reconstruction (10) has the matrix representa-tion

Cj = Hj • [Cj+1]up +3∑

τ=1

Gj,τ • [Dj+1,τ ]up, (12)

procedure fwd

{Input: N ×N matrix X,containing 2-D FFT of input image

Output: matrix W containing FWD}

{N -point 1-D FFT of decomposition filters}H = fft(h, N) ; G = fft(g, N)

{compute 2-D decomposition filters}H = HHT ; G1 = HGT

G2 = GHT ; G3 = GGT

{pyramid algorithm, M levels}for j = 1 to M do{filter and decimate in frequency space}C = [H •X]down ; D1 = [G1 •X]down

D2 = [G2 •X]down ; D3 = [G3 •X]down

{put detail coefficients in output matrix W}W (1 : N/2;N/2 + 1 : N) = D1W (N/2 + 1 : N ; 1 : N/2) = D2W (N/2 + 1 : N ;N/2 + 1 : N) = D3

{set up next iteration}X=C

{downsample filters}H = dsamp (H, 2) ; G1 = dsamp (G1, 2)

G2 = dsamp (G2, 2) ; G3 = dsamp (G3, 2)

{reduce size}N =N/2

end for{put lowest approximation coefficients in W}W (1 : N ; 1 : N) = C

Fig. 5. Two-dimensional wavelet decomposition in the Fourier do-main. h, g are column vectors of 1-D analysis filter coefficients.

where, for any matrix X of Fourier coefficients, Xup is thematrix twice its size, defined by

Xup =(

X XX X

)We will refer to (11) and (12) as Fourier-wavelet decompo-sition (FWD) and Fourier-wavelet reconstruction (FWR),respectively.

For a pseudocode of wavelet decomposition and recon-struction in the Fourier domain, the reader is referred toFigs. 5 and 6, respectively. In the pseudocode, AT denotesthe transpose of A, dsamp(A, k) denotes downsampling ofA by a factor of k in both dimensions, and A(p : q; r : s) isthe submatrix of A obtained by retaining only those rowsi and columns j for which p ≤ i ≤ q and r ≤ j ≤ s.

D. Time Complexity

Assume that the input image c0 is square and containsN2 elements, with N a power of two. In this case, the max-imal number of decomposition levels is M = log2N . Thefirst step is a 2-D FFT of c0 yielding an array C0 of Fouriercoefficients. We express the time complexity of the wavelettransform in the number of complex multiplications in thefrequency domain. The number of multiplications for thefirst decomposition level is 4N2, since there are four filters.For the second decomposition level, both the array size and


procedure fwr

{Input: N ×N matrix X containing FWDOutput: matrix W containing FWR. A 2-D

IFFT of W gives the inverse wavelettransform in the spatial domain}

{set initial blocksize}K = N/(2M ){N -point 1-D FFT of reconstruction filters}H = fft(h,N) ; G = fft(g,N){full-size 2-D reconstruction filters}Hfs = HHT ; G1

fs = HGT

G2fs = GHT ; G3

fs = GGT

{initialize W}W = X

{pyramid algorithm, M levels}for j = 1 to M do{downsample by factor dsf }dsf = N/(2K)H = dsamp (Hfs, dsf) ; G1 = dsamp (G1

fs, dsf)G2 = dsamp (G2

fs, dsf) ; G3 = dsamp (G3fs, dsf)

{approximation coefficients}C = W (1 : K; 1 : K){detail coefficients}D1 = W (1 : K;K + 1 : 2K)D2 = W (K + 1 : 2K; 1 : K)D3 = W (K + 1 : 2K;K + 1 : 2K){undecimate and filter}W (1 : 2K, 1 : 2K) = H • [C]up + G1 • [D1]up+

+G2 • [D2]up + G3 • [D3]up

{set up next iteration: double size}K =2K

end for

Fig. 6. Two-dimensional wavelet reconstruction in the Fourier do-main. h, g are column vectors of 1-D synthesis filter coefficients.

the filter lengths are reduced by a factor of two in both di-mensions, so that 4(N/2)2 multiplications are needed; etc.Altogether, the total number of multiplications for M de-composition levels is given by

4M−1∑j=0

(N

2j

)2

=163N2(1− 4−M )

M=log2 N

≤ 163

(N2 − 1).

Therefore, the time complexity of the computations in thefrequency domain is O(N2).

An approximation image in the spatial domain is ob-tained by inverse FFT of the corresponding array in theFourier domain. Since the complexity of the initial andfinal FFT is O(N2 log2N), we conclude that the overallcomplexity of a Fourier domain implementation of the 2-Dwavelet transform is O(N2 log2N). It can be shown in asimilar way that the reverse transform has the same timecomplexity.

IV. Wavelet X-ray Transform

In this section the wavelet X-ray transform, as intro-duced in [12], is studied and an efficient implementationis derived by computing the wavelet transform in the fre-quency domain. This results in an algorithm which startsby computation of the Fourier transform in a slice plane,as in ordinary FVR, followed by a wavelet decompositionof the slice plane image in Fourier space.

The wavelet X-ray transform is defined by expanding theX-ray transform Pθf of a function f in a 2-D wavelet series[cf. (4)]:

Pθf(u, v) =∑k,l

cMk,l(θ)Φ0M,k,l(u, v)

+M∑j=1

∑τ∈T

∑k,l

dj,τk,l (θ)Ψτj,k,l(u, v).

(13)

The coefficients cMk,l and dj,τk,l , τ ∈ T = {1, 2, 3}, now dependon the viewing direction θ.

This transform can be viewed as a close relative of thewavelet X-ray transform which combines integration overa line with a simultaneous 1-D wavelet transform alongthis line [13, 14]. The difference is, that we perform a 2-Dwavelet transform in the plane perpendicular to the line.

Now we can state the main result, which is an extensionof [12] to the biorthogonal case.

Theorem 1: The coefficients in the wavelet representa-tion (13) for the X-ray transform of f ∈ L2(R3) are

cMk,l(θ) = F−12

(F2Pθf · F2Φ′

0

M

)(2Mk, 2M l) (14)

and

dj,τk,l (θ) = F−12

(F2Pθf · F2Ψ′

τ

j

)(2jk, 2j l), (15)

where

Φ′0

M (u, v) = Φ0M,0,0(−u,−v)

Ψ′τ

j (u, v) = Ψτj,0,0(−u,−v).

Proof: We only prove (15), the result (14) followsanalogously. Using the Plancherel formula, we observe

dj,τk,l (θ) = 〈Pθf, Ψτj,k,l〉 = 〈F2Pθf,F2Ψτ

j,k,l〉 (16)

=∫∫F2Pθf(ωu, ωv)F2Ψτ

j,k,l(ωu, ωv) dωu dωv.

Now,

F2Ψτj,k,l(ωu, ωv) =

∫∫e2πi(ωuu+ωvv)Ψτ

j,k,l(u, v) du dv

=∫∫

e2πi(ωuu+ωvv)Ψ′τ

j (2jk − u, 2j l − v) du dv

= e2πi(2jk ωu+2j l ωv)

∫∫e−2πi(ωuu

′+ωvv′)Ψ′

τ

j (u′, v′) du′ dv′

= e2πi(2jk ωu+2j l ωv)F2Ψ′τ

j (ωu, ωv).

Using this in (16), we find

dj,τk,l (θ) =∫∫F2Pθf(ωu, ωv) e2πi(2jk ωu+2j l ωv)

· F2Ψ′τ

j (ωu, ωv) dωu dωv

= F−12

(F2Pθf · F2Ψ′

τ

j

)(2jk, 2j l).


By the Fourier slice theorem (2), F2Pθf(ωu, ωv) =F3f(ωuu + ωvv). Therefore, the wavelet coefficients atscale j in (13) can be computed by multiplying a slice ofthe 3-D Fourier transform of f by the 2-D Fourier trans-form of the scaling or wavelet function at scale j, followedby an inverse 2-D Fourier transform evaluated at the pointsof the form (2jk, 2j l) in the view plane.

A. Algorithm

The proposed wavelet extension of FVR requires only asmall modification of the standard algorithm. The imple-mentation is facilitated by the fact that wavelet decompo-sition and reconstruction can be performed in the Fourierdomain, see Section III-C. The algorithm, henceforth re-ferred to as Fourier-wavelet volume rendering (FWVR),can be summarized as follows.

• Preprocessing. Compute the 3-D Fourier transformof the volume data (size N3).• Actual volume rendering. For each direction θ, do

the following.1. Interpolate the Fourier transform on a regular grid

of size (2N)2 in the slice plane orthogonal to θ. Thisyields the array C0 to be used for initializing the wavelettransform.2. Perform a 2-D Fourier-wavelet decomposition

(FWD) of depth M , yielding approximation coefficientsCMk,l and detail coefficients Dj,τ

k,l , j = M,M−1, . . . , 1, re-spectively.3. Perform a partial Fourier-wavelet reconstruction

(FWR) from CM , putting all detail signals Dj,τ equalto zero, followed by a 2-D inverse Fourier transform toobtain an initial approximation c0,M [size (2N)2] in thespatial domain.4. Refine the approximation by partial FWR using the

detail signals Dj,τ with K < j ≤ M , followed by a 2-D inverse Fourier transform to obtain an approximationc0,K [size (2N)2] at a finer scaleK in the spatial domain.

This approach enables us to implement a client-server vi-sualization system. The server performs the initial 3-DFourier transform, as well as the slicing and FWD at eachview angle (steps 1 and 2), and sends the required approxi-mation and detail coefficients to the client. The client per-forms the FWR and inverse Fourier transform to obtain anapproximation image (steps 3 and 4). Below we describe indetail how to implement this efficiently. As long as a user isinteracting with the data, only the coarsest Fourier domainapproximation coefficients CMk,l are used. When interactionceases, the Fourier domain detail coefficients Dj,τ

k,l are takeninto account, so that the client can obtain reconstructionsat higher levels of detail. It is not necessary to send float-ing point representations (4 bytes) of the coefficients, butthe coefficients can be quantized to shorts (2 bytes). Thequantization error is in the order of 10−8, which results inno visible artefacts.

The progressive refinement inherent in the algorithm canimprove interaction with the data significantly, since theresponse time of the system drops. Table I provides anestimate of the time it takes to send full images of size

2562 and 5122 (in bytes), or approximation and detail co-efficients (in shorts) only. Times are given for an ISDNconnection with a bandwidth of 128 kbit/s and a 1 Mbit/sconnection, which can be considered a fast Internet connec-tion. The time for computing the wavelet decompositionis not included; also communication protocol overhead isignored.

TABLE I

Estimated time (in seconds) to send full sized images (bytes)

or approximation and detail coefficients (shorts) over an

ISDN connection and a 1 Mbit/s connection.

ISDN Internet(128 kbit) (1 Mbit)

256× 256full image 4.10 0.52approximation coefficients 0.51 0.07detail coefficients level 2 1.54 0.20detail coefficients level 1 6.14 0.79

512× 512full image 16.38 2.10approximation coefficients 0.51 0.07detail coefficients level 3 1.54 0.20detail coefficients level 2 6.14 0.79detail coefficients level 1 24.55 3.15

A.1 Progressive Refinement

For an M -level wavelet decomposition, progressive re-finement with the pyramid algorithm is done as follows.First, an approximate reconstruction at level M in Fourierspace is made by ignoring all the detail coefficients and ap-plying a full wavelet reconstruction to the approximationcoefficients. Then, a full wavelet reconstruction with thepyramid algorithm is computed with the detail coefficientsof level M only, and the result is added to the approxi-mation. This refines the approximation at level M to anapproximation at level M − 1. The process continues withthe detail coefficients of level M−1 to compute an approx-imation at level M − 2, etc. That is, all images at levelM,M − 1, . . . , 0 are successively computed.

It is possible to reduce the number of multiplicationsand additions by adopting the so-called nonpyramidal re-construction scheme. Write the FWR equation (12) in thefollowing form,

Cj = HjCj+1 +3∑

τ=1

Gj,τDj+1,τ ,

where Hj and Gj,τ are the operators defined by

HjCj+1 = Hj • [Cj+1]up, Gj,τDj+1 = Gj,τ • [Dj+1,τ ]up.

By iterating this equation, the full reconstruction C0 can


be written as follows:

C0 = C0,M +M−1∑j=0

D0,M−j

C0,M = H0H1 . . .HM−1CM

D0,M−j = H0H1 . . .HM−j−23∑

τ=1

GM−j−1,τDM−j,τ .

Here C0,M is the level M approximation reconstructedfrom CM by ignoring all the detail coefficients. Bysuccessively adding full resolution images D0,M−j recon-structed from DM−j,τ , we obtain approximations on levelM − 1,M − 2, etc.

Since pointwise multiplication is associative, successiveupsamplings implicit in the product of operators Hj canbe performed first and combined filtering at full resolutioncan be done afterwards. The advantage of this approachis that the filters needed for each level can be computedin advance. This avoids all multiplications for the levelsbetween the lowest resolution and the full resolution, need-ing N2 multiplications. For the detail contributions thereis an initial filtering step by GM−j−1,τ . Since the 2-D fil-ters are separable, only the 1-D filters for the incrementalreconstruction are precomputed, which are then applied torows and columns separately. Part of this process for the1-D case is illustrated in Fig. 7 for a three-level waveletdecomposition.

x

x xx

= point-wise multiplication

C3 C3 C3

C2

upsampling

H2

replace C2

C2 C2 C2

H1 H2 H2

H1

C3 C3 C3C3 C3 C3 C3C3upsampling

x

x combined filter

C1

xa)

b)

Fig. 7. (a) Computation of C2 from C3 via upsampling followed bymultiplication. (b) Computation of C1 from C3 directly. Thefilters H2 and H1 are combined into a new filter, and C3 isupsampled twice. Multiplication yields C1.

The time complexity of progressive refinement recon-struction to full resolution is estimated by counting thenumber of additions (adds) and multiplications (mults) foran M -level decomposition, see Table II (this concerns thepart of the computation in the Fourier domain, i.e. ex-cluding the final inverse 2-D FFT). From this table, it iseasily verified that indeed the non-pyramidal reconstruc-tion is more efficient than the pyramidal algorithm; equalefficiency obtains for M = 1. Of course the pyramidal algo-rithm is more efficient when only C0 is wanted, i.e. withoutthe intermediate approximations. Although M can be as

large as log2N , the number of decomposition levels is usu-ally fixed to a small number, like two or three, in orderfor the approximation image to be useful. Therefore, thecomplexity of progressive refinement is O(N2).

TABLE II

Computational complexity of progressive refinement in

Fourier space (image size N ×N , M-level reconstruction).

pyramidal non-pyramidal

mults 4N2M N2M + 4N2(1− 4−M )

adds 3N2M N2M + 83N2(1− 4−M )

B. Results

Experiments with phantom data were carried out for twobasic wavelets, the Haar wavelet and a second-order B-spline wavelet, which gives much smoother results at largecompression ratios. In the latter case, we need a biorthog-onal wavelet basis, defined by filters of unequal length fordecomposition and reconstruction (length 41 and length5, respectively, see [22, Appendix]). We used 20% zero-padding and cubic spline interpolation.

Fig. 8 shows the reconstruction from a three-level Haarwavelet decomposition (M = 3), using the approximationcoefficients C3 only, cf. Fig. 8(a), and with detail coeffi-cients added, cf. Fig. 8(b)-(d). Fig. 9 shows the recon-struction from a three-level second-order B-spline waveletdecomposition of the same volume data. Since the numberof dual coefficients of the B-spline wavelet is 41, at most 3decomposition levels are possible.

(a) level 3 (b) level 2

(c) level 1 (d) level 0

Fig. 8. FWVR rendering by a three-level Haar wavelet decompositionof phantom volume data.


(a) level 3 (b) level 2

(c) level 1 (d) level 0

Fig. 9. FWVR rendering by a three-level second-order B-splinewavelet decomposition of phantom volume data.

Table III shows rendering times of the Fourier-waveletrendering algorithm with progressive refinement. Linearand cubic spline interpolation with 20% zero-padding wasapplied. A Haar wavelet was used as a basic wavelet. Otherwavelets give only marginally different timing results, inagreement with the complexity estimates in Section III-D. Three data sets were used: phantom and CT data ofsize 1283, and an MR data set of size 2563. Resolution ofthe slice plane was (2N)2 for a dataset of size N3. Tim-ings were performed on a Silicon Graphics Onyx with a200 MHz R4400 processor. Results listed for computingapproximation images include the time used by the inverse2-D FFT. The computational complexity is only dependenton the size of the rendered image, which explains the iden-tical timings for phantom and CT data. In fact, results forthese volume data are only included to facilitate a compar-ison with wavelet splatting below. For the same reason,the decomposition depth for the MR data was set to threelevels, and for the other data sets only two levels were used.Observe that the timings increase approximately by a fac-tor of four when the size of the data set becomes eight timesas large, as expected from the theoretical complexity.

Application of progressive refinement in a client-serversystem is only useful if the total time needed for com-puting the approximation coefficients, transmitting themto the client, and performing a partial reconstruction, issmaller than the time required to send the full data overa transmission line. Referring to Table I we see that thiswill be the case for an ISDN line, but not for a 1 Mb/sconnection (assuming this bandwidth is really available).

C. Memory Usage

The fraction k of zero-padding of the volume data de-termines to a large extent memory usage of Fourier ren-

TABLE III

Cumulative rendering timings (in seconds) of

Fourier-wavelet volume rendering. A Haar wavelet was

used as a basic wavelet.

Phantom CT head MR head1283 1283 2563

Linear interpolationslice extraction 0.27 0.27 1.11FWD 0.57 0.57 2.38Level 3 approximation 2.86Level 2 approximation 0.68 0.68 3.64Level 1 approximation 0.86 0.86 4.49Full reconstruction 1.10 1.10 5.56

Cubic spline interpolationslice extraction 1.41 1.41 5.64FWD 1.71 1.71 6.91Level 3 approximation 7.40Level 2 approximation 1.82 1.82 8.18Level 1 approximation 2.00 2.00 9.02Full reconstruction 2.24 2.24 10.09

dering. Volume data are stored in a 3-D array of floatsof size (kN)3 + (kN)2, where the volume data have sizeN3 and k is the factor due to zero-padding. The extra(kN)2 elements are required by the 3-D real-to-complexFFT. The FWD uses two temporary 2-D arrays of floats ofsize (2N)2 + 2N . The precomputed filters for progressiverefinement are 1-D. The hierarchy of combined Hj filterscontains MN complex numbers, where M is the numberof decomposition levels. The hierarchy of filters to recon-struct detail coefficients contains 2N/2j complex numbersfor each decomposition level j.

V. Wavelet Splatting

Wavelet splatting (WS) [10], [11], [3] modifies the ba-sic splatting algorithm in two ways: i) it uses wavelets asreconstruction filters, and ii) it provides a mechanism tovisualize data at different levels of detail. First, the algo-rithm performs a 3-D wavelet decomposition of the volumedata. The 3-D separable wavelet basis with 8 basis func-tions is given by [cf. (3)]

Φ0j,k,l,m(x, y, z) = φj,k(x)φj,l(y)φj,m(z)

Ψ1j,k,l,m(x, y, z) = φj,k(x)φj,l(y)ψj,m(z)

Ψ2j,k,l,m(x, y, z) = φj,k(x)ψj,l(y)φj,m(z)

Ψ3j,k,l,m(x, y, z) = φj,k(x)ψj,l(y)ψj,m(z)

Ψ4j,k,l,m(x, y, z) = ψj,k(x)φj,l(y)φj,m(z)

Ψ5j,k,l,m(x, y, z) = ψj,k(x)φj,l(y)ψj,m(z)

Ψ6j,k,l,m(x, y, z) = ψj,k(x)ψj,l(y)φj,m(z)

Ψ7j,k,l,m(x, y, z) = ψj,k(x)ψj,l(y)ψj,m(z)

Substitution of the expansion of f on this basis in (1)


results in:

Pθf(u, v) =∑k,l,m

cMk,l,m

∫R

Φ0M,k,l,m(uu+ vv + tθ) dt

+M∑j=1

∑τ∈T

∑k,l,m

dj,τk,l,m

∫R

Ψτj,k,l,m(uu+ vv + tθ) dt, (17)

where T = {1, 2, . . . , 7}. This equation expresses Pθf(u, v)as a weighted summation of integrals along the line of sight.The integrals are 2-D functions on the view plane: the foot-prints. These have to be evaluated only once for a givenviewing direction at the coarsest scale j = M and transla-tion (k, l,m) = (0, 0, 0), yielding eight prototype footprints.The footprints for other scales and translations can be com-puted by rescaling and shifting. Prototype footprints canbe computed efficiently by slicing their 3-D Fourier trans-forms. When analytical expressions exist for the Fouriertransforms of the scaling function and wavelet, as is thecase for the Haar and cardinal B-spline wavelets, no inter-polation from discrete samples is necessary. For the caseof B-spline wavelets, the required formulas for the Fouriertransforms are somewhat involved, cf. [22, Ch. 6 & Ap-pendix]. By computing footprints analytically, we do nothave problems with aliasing and dishing as in (wavelet)Fourier rendering. Also, no zero-padding of the data isnecessary.

To summarize, the algorithm consists of the followingsteps:

1. Preprocessing. Perform a 3-D discrete wavelettransform (of depth M) of the volume data.2. Splatting. For each viewing direction θ do:• Compute prototype footprints at level M in the view

plane orthogonal to θ.• Compute footprints for lower levels by scaling and

downsampling.• Compute a low resolution image by summation of

scaled and translated footprints weighted by the ap-proximation coefficients cMk,l,m. Since, in general, thecoordinates of the center of the footprint are not inte-gers, use bilinear interpolation to convert to pixel coor-dinates.• Refine the image incrementally by adding footprints

corresponding to Ψτj,k,l,m, weighted by the detail coeffi-

cients dj,τk,l,m.

A. Memory Usage

After the 3-D wavelet transform, the coefficients for eachwavelet type and level are stored in 1-D sequences. Zerocoefficients are discarded, which makes it necessary to storethe spatial position of each coefficient as well. This positionis encoded with an integer (4 bytes), which allows volumedata of sizes up to 4 GB. Larger volumes require a differentencoding for the spatial position. Storage of the positiondoubles memory requirements if there are no coefficientswith value zero at all. In practice, a large number of datasets has 20–40% zero coefficients, which makes memoryrequirements of WS comparable to those of Fourier ren-dering. By storing the coefficients in sequences instead of

a 3-D array, the algorithm has a regular memory accesspattern. This increases rendering performance, since thememory cache is fully exploited. The amount of memoryused to store the footprints is dependent on the decomposi-tion depth M and the support s of the 1-D scaling functionand wavelet. The size (i.e. number of pixels) of a footprintat the coarsest scale is the first power of two larger thans22M . The size of the footprints decreases by a factor offour for each finer scale.

B. Results

Fig. 10 shows intensity profiles corresponding to the linex = 122 of rendered images at full resolution of the phan-tom data rendered with WS. We used a Haar wavelet anda second-order B-spline wavelet. A comparison with theplots in Fig. 4 shows that the accuracy of this method iscomparable to FVR with cubic spline interpolation. Fig-ure 11 shows renderings of the CT data, both by FWVRand WS, also with visually very similar results.

The WS algorithm has time complexity O(N3) for aN × N × N data set, as is obvious from the summationover k, l,m in (17). Timings are shown in Table IV. Datasets and image resolution are the same as for FWVR (Ta-ble III). The scaling of the timings for increasing data sizeis in agreement with the theoretical complexity. Due toits larger support, the B-spline wavelet is computationallymuch more expensive than the Haar wavelet. Furthermore,the rendering speed is very data dependent. In case of theHaar wavelet, rendering the phantom data takes almostfour times as long as rendering the CT data. The reasonis that the CT data are smoother than the phantom data.For the Haar wavelet, this results in a larger number ofnonzero wavelet coefficients and slower rendering, since ithas only one vanishing moment. A more sparse decompo-sition is obtained by using B-spline wavelets which have alarger number of vanishing moments.

Comparison with Table III makes it clear that WS isindeed computationally more demanding, and also moredata-dependent, than FWVR.

80 100 120 140 160 1801000

1500

2000

2500

3000

3500

referencewavelet splatting

(a) Haar wavelet

80 100 120 140 160 1801000

1500

2000

2500

3000

3500

referencewavelet splatting

(b) B-spline wavelet

Fig. 10. Profiles along the line x = 122 of rendering by waveletsplatting of phantom data.

VI. Discussion

In this paper, we have described an extension to Fouriervolume rendering (FVR) based on a wavelet decomposi-


TABLE IV

Cumulative rendering timings (in seconds) of wavelet

splatting.

Phantom CT head MR head1283 1283 2563

Haar waveletLevel 3 approximation 1.71Level 2 approximation 0.30 0.52 11.46Level 1 approximation 1.15 3.49 36.89Full reconstruction 2.59 12.62 109.81

B-spline waveletLevel 3 approximation 3.49Level 2 approximation 1.04 1.07 28.99Level 1 approximation 8.29 8.38 89.03Full reconstruction 25.86 30.24 231.39

(a) FWVR (b) WS

Fig. 11. Level 1 rendering of CT data (size 2563) using a Haarwavelet.

tion, which allows the data to be visualized at progressivelyhigher levels of detail, as can be useful for client-server sys-tems.

The wavelet X-ray transform was introduced, whichcombines integration along the line of sight with a simul-taneous 2-D wavelet transform in the plane perpendicularto this line. An efficient implementation was derived bycomputing the wavelet transform in the frequency domain.The initial step of this Fourier-wavelet volume renderingalgorithm (FWVR), i.e., computation of the Fourier trans-form in a slice plane, is identical to that of ordinary FVR,and requires interpolation. Aliasing can be prevented byinitial zero-padding of volume data and the use of accurateinterpolation filters. The additional step is a wavelet de-composition of the slice plane data in Fourier space to agiven level of detail. Approximation images are then ob-tained by partial wavelet reconstruction in Fourier space,followed by a 2-D inverse Fourier transform. We comparedthe new method with wavelet splatting (WS), which mod-ifies the basic splatting algorithm by using wavelets as re-construction filters.

A 3-D head phantom consisting of a collection of ellip-soids of different density values was defined, for which ana-lytical projections, as well as a volume data set of size 1283,were computed. In the experiments, three data sets wereused: phantom and CT data of size 1283, and an MR dataset of size 2563. For a dataset of size N3, the resolution ofthe slice plane was taken as (2N)2. For interpolation in fre-

quency space, we used linear and cubic spline interpolation.Experiments were carried out for two basic wavelets, theHaar wavelet and a second order B-spline wavelet. Tim-ings were performed on a Silicon Graphics Onyx with a 200MHz R4400 processor.

A. Differences Between FWVR and WS

In the following we enumerate the main differences be-tween Fourier-wavelet volume rendering and wavelet splat-ting, and summarize the conclusions of the experimentalinvestigations.1. FWVR works in the frequency domain, and is initializedby a 3-D Fourier transform. WS works in the spatial do-main (only the prototype footprints are obtained by Fourierdomain computation), and is initialized by a 3-D wavelettransform.2. In FWVR, a 2-D wavelet transform is computed for eachview plane, therefore, the wavelet coefficients depend onthe view direction. In WS, the 3-D wavelet coefficients arecomputed only once, and are independent of view direction.3. The time complexity of FWVR is the same as for ordi-nary FVR, i.e. O(N2 logN), whereas WS has complexityO(N3), just as the original splatting method.4. To prevent aliasing, FWVR requires zero-padding of theinput data. In contrast, no zero-padding is necessary inWS: footprints are computed analytically in the Fourierdomain.5. The rendering accuracy was assessed by using phantomdata and comparing intensity profiles of the rendered im-ages with analytically computed projections. Using cubicspline interpolation with 20% zero-padding we found thatFWVR results in accurate renderings with quality verysimilar to that of WS.6. Timings for FWVR were found to depend only on thesize of the input data. In contrast, the computation timeof WS is dependent on the basic wavelet used; e.g. theB-spline wavelet is computationally much more expensivethan the Haar wavelet. Furthermore, the rendering speedof WS is very data dependent, because the number ofnonzero wavelet coefficients depends on the smoothness ofbasic wavelet and volume data.7. Memory requirements of FWVR are comparable tothose of WS. FVWR needs a 3-D array of floats of size(kN)3 + (kN)2 (k is the zero-padding factor), and usestwo temporary 2-D arrays of floats of size (2N)2 + 2N tocompute the forward transform. In WS, the coefficientsfor each wavelet type and level are stored in 1-D sequencesto increase rendering performance, with the spatial posi-tion of each coefficient stored as an integer. The amountof memory used to store the footprints is dependent onthe decomposition depth M and the support s of the 1-Dscaling function and wavelet.

B. Final Remarks

The FWVR method can be straightforwardly extendedto include gradient-based shading and depth cueing [2], [3].Also, we would like to point out that the comparison of WSand FWVR made in this paper concerns non-optimized


implementations. For example, by compression of waveletcoefficients, through thresholding or more advanced tech-niques, wavelet splatting can be substantially accelerated.Among such techniques we mention the embedded zerotreewavelet algorithm [23], and the conversion of wavelet trans-formed data into a sequential bitstream [3]. However, bothmethods require advanced spatial data structures, whichare not necessary in FWVR.

A disadvantage of FWVR in its current form is that itrequires the interpolation of a slice in Fourier space at fullresolution in order to perform a 2-D wavelet decomposi-tion. In contrast, in WS progressive refinement is imme-diate, since the initial data is subjected to a 3-D wavelettransform. Therefore, we plan to investigate the possibil-ity of combining both methods to take advantage of thestrengths of each of them.

Appendix

Definition of the Head Phantom

The 3-D phantom used in this paper consists of a numberof ellipsoids of various densities. Initially, each ellipsoidis assumed to have its three axes aligned with the axesof the coordinate system. Subsequently, a rotation of theellipsoid around the z-axis is performed. Table V gives,for each ellipsoid, the center, lengths of the three axes ofthe ellipsoid, rotation angle β around the z-axis, and thedensity.

TABLE V

Parameters of the 3-D head phantom.

Center Axis lengths β Density(0, 0, 0) (0.69, 0.92, 0.9) 0 151.00(0, 0, 0) (0.6624, 0.874, 0.88) 0 -125.44(-0.22, 0, -0.25) (0.41, 0.16, 0.21) 108 -25.60(0.22, 0, -0.25) (0.31, 0.11, 0.22) 72 -25.60(0, 0.1, -0.25) (0.046, 0.046, 0.046) 0 25.60(-0.08, -0.605, -0.25) (0.046, 0.023, 0.02) 0 12.80(0.06, -0.605, -0.25) (0.046, 0.023, 0.02) 90 12.80(0.06, -0.105, 0.625) (0.056, 0.04, 0.1) 90 25.60(0, 0.1, 0.625) (0.056, 0.056, 0.1) 0 -25.60(0, 0.35, -0.25) (0.25, 0.21, 0.41) 90 25.60

References

[1] R. A. Drebin, L. Carpenter, and P. Hanrahan, “Volume render-ing,” Computer Graphics (SIGGRAPH ’88 proceedings), vol.22, no. 4, pp. 65–74, 1988.

[2] T. Totsuka and M. Levoy, “Frequency domain volume render-ing,” in Computer Graphics (SIGGRAPH ’93 Proceedings),J. T. Kajiya, Ed., 1993, vol. 27, pp. 271–278.

[3] L. Lippert, M. H. Gross, and C. Kurmann, “Compression do-main volume rendering for distributed environments,” in Proc.Eurographics’97, 1997, pp. 95–107.

[4] F. Natterer, The Mathematics of Computerized Tomography, B.G. Teubner & J. Wiley, 1986.

[5] T. Malzbender, “Fourier volume rendering,” ACM Transactionson Graphics, vol. 12, no. 3, pp. 233–250, 1993.

[6] S. Napel, S. Dunne, and B. K. Rutt, “Fast Fourier projectionfor MR angiography,” Magnetic Resonance in Medicine, vol. 19,pp. 393–405, 1991.

[7] L. A. Westover, “Footprint evaluation for volume rendering,”Computer Graphics, vol. 24, no. 4, pp. 367–376, 1990.

[8] K. Mueller and R. Crawfis, “Eliminating popping artifacts insheet buffer-based splatting,” in Visualization’98, 1998, pp. 239–245.

[9] K. Mueller, T. Moeller, J. E. Swan, R. Crawfis, N. Shareef, andR. Yagel, “Splatting errors and anti-aliasing,” IEEE Transac-tions on Visualization and Computer Graphics, vol. 4, no. 2, pp.178–191, 1998.

[10] L. Lippert and M. H. Gross, “Fast wavelet based volume render-ing by accumulation of transparent texture maps,” ComputerGraphics Forum, vol. 14, no. 3, pp. 431–443, 1995.

[11] L. Lippert, Wavelet-based Volume Rendering, Ph.D. thesis,Swiss Federal Institute of Technology Zurich, 1998, Diss. ETHNo. 12612.

[12] J. B. T. M. Roerdink and M. A. Westenberg, “Wavelet-basedvolume visualization,” Nieuw Archief voor Wiskunde, vol. 17,no. 2, pp. 149–158, 1999.

[13] A. L. Warrick and P. A. Delaney, “A wavelet localized Radontransform,” in Proc. SPIE Vol. 2569, Wavelet Applications inSignal and Image Processing III, Andrew F. Laine; Michael A.Unser; Mladen V. Wickerhauser; Eds., Sept. 1995, pp. 632–643.

[14] R. A. Zuidwijk, “The wavelet X-ray transform,” Tech. Rep.PNA-R9703, Centre for Mathematics and Computer Science,March 1997.

[15] O. Rioul and P. Duhamel, “Fast algorithms for discrete andcontinuous wavelet transforms,” IEEE Transactions on Infor-mation Theory, vol. 38, no. 2, pp. 569–586, 1992.

[16] M. Vetterli and C. Herley, “Wavelets and filter banks: Theoryand design,” IEEE Transactions on Signal Processing, vol. 40,no. 9, pp. 2207–2232, 1992.

[17] A. C. Kak and M. Slaney, Principles of Computerized Tomo-graphic Imaging, IEEE Press, New York, 1988.

[18] R. G. Keys, “Cubic convolution interpolation for digital im-age processing,” IEEE Transactions on Acoustics, Speech, andSignal Processing, vol. ASSP-29, no. 6, pp. 1153–1160, 1981.

[19] J. A. Parker, R. V. Kenyon, and D. E. Troxel, “Comparison ofinterpolation methods for image resampling,” IEEE Transac-tions on Medical Imaging, vol. MI-2, no. 1, pp. 31–39, 1983.

[20] M. Frigo and S. G. Johnson, “FFTW: An adaptive softwarearchitecture for the FFT,” in Proc. ICASSP, 1998, vol. 3, p.1381.

[21] S. G. Mallat, “A theory for multiresolution signal decomposi-tion: the wavelet representation,” IEEE Transactions on Pat-tern Analysis and Machine Intelligence, vol. 11, pp. 674–693,1989.

[22] C. K. Chui, An Introduction to Wavelets, Academic Press, 1992.[23] J. M. Shapiro, “Embedded image coding using zerotrees of

wavelet coefficients,” IEEE Transactions on Signal Processing,vol. 41, no. 12, pp. 3445–3462, 1993.

Michel A. Westenberg received the M.Sc.degree in computing science from the Univer-sity of Groningen, The Netherlands, in 1996.He is currently pursuing the Ph.D. degree incomputing science at the same university. Hisresearch interests include scientific visualiza-tion, wavelets, and biomedical image process-ing.

Jos B. T. M. Roerdink received the M.Sc.degree in theoretical physics from the Univer-sity of Nijmegen, The Netherlands, in 1979and the Ph.D. degree from the University ofUtrecht, The Netherlands, in 1983. He wasa Postdoctoral Fellow with the University ofCalifornia, San Diego, from 1983 to 1985. Hewas with the Centre for Mathematics and Com-puter Science, Amsterdam, The Netherlands,from 1986 to 1992, where he worked on im-age processing and tomographic reconstruc-

tion. He is currently an Associate Professor of computing science withthe University of Groningen, The Netherlands. His current researchinterests include mathematical morphology, wavelets, biomedical im-age processing and scientific visualization.

ieee transactions on image processing, vol. 9, no. 7, …mwestenb/papers/ipwavfour.pdf · ieee...

Documents