homomorphic filter

IEEE TRANSACTIONS ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, VOL. ASSP-27, NO. 6, DECEMBER 1979 62 5

Image Enhancement by Stochastic Homomorphic Filtering

ROBERT W. FRIES AND JAMES w. MODESTINO, MEMBER, IEEE

Abstract-The problem of image enhancement by nonlinear two- dimensional (2-D) homomorphic filtering is approached using stochastic models of the signal and degradations. Homomorphic filtering has been previously used for image enhancement, but the linear filtering operation has generally been chosen heuristically. In this paper stochastic image models previously described and analyzed by the authors are used to model the true image and interfering components (shadows and salt-and-pepper noise). The problem of designing the linear filter can then be formulated as one of linear least mean-squared error (wiener) filtering. Examples of processing of typical real-world images are included to indicate the obtainable results.

A I. INTRODUCTION

PROBLEM in representing data in any format is emphasizing the significant features while minimizing distract-

ing detail. Solutions to this problem when the data consists of images are called image enhancement techniques.

Elements in images which distract the viewer from relevant information are shadows and salt-and-pepper noise. This is the case for either photochemical or photoelectronic image sensing and display systems. For example, shadows frequently obscure detail in ordinary photographs; even when detail is present within the area darkened by the shadow, it is often subdued because the contrast in the darkened area is less than that in the surrounding bright regions.'

The same effect occurs in the recording and display .of images by photoelectronic means although here the problem is further aggravated by the limited dynamic range generally available. Many TV monitors and storage CRT displays, for example, are not capable of more than 32 distinct gray levels. The severity of this restriction is best appreciated in comparison to the capabilities afforded by a good quality photograph under subjective human evaluation. According to Zoethout [ 1 ] , the ratio of the smallest variation in intensity discernible by the eye to the overall intensity is generally considered to be 1/100. Using the standard definition [2] of the optical density D of a photographic emulsion

Manuscript received March 28, 1979; revised July 23, 1979. This work was supported in part by the Office of Naval Research under Contract N00014-754 0281.

R. W. Fries was with the Department of Electrical and Systems Engi- neering, Rensselaer Polytechnic Institute, Troy, NY 12181. He is now with the Pattern Analysis and Recognition Corporation, Rome, NY.

J. W. Modestino is with the Department of ElectIical and Systems Engineering, Rensselaer Polytechnic Institute, Troy, NY 12181.

'This is due to the common practice in modern photography of exposing film so that shadows fall on the toe of the D-log E curve

_ _ _ (Cf. [19], pp. 29-32).

D = log,, l /T (1)

where T is the transmittance of the emulsion, we conclude that the smallest discernible change in density is roughly log, lOl/lOO = 0.0043. Typical photographic emulsions (cf. [2, p. 691) cover a range of densities from 0 to 1.5. Hence, assuming grain noise is negligible, there are approximately 347 discernible density levels in a good quality black-and-whte photograph, far exceeding the capabilities of many photoelectronic displays.

The mechanism by which the eye achieves its wide dynamic range suggests techniques by which the dynamic range of photographic materials and electronic displays may be ex- tended. The sensitivity of the eye to local changes is described by Hochberg [3]. Within a broad range of illuminations, the apparent brightness of a region is a function of the ratio between its intensity and that of its surrounding region and not of its absolute intensity alone. It is as if the eye takes the logarithm of the incoming light intensities and then passes these signals through a spatial high-pass filter which attenuates the dc component. This contention is supported by Stockham [4] who carries it to the point of estimating the frequency response of the high-pass filter. Additional work along these lines has been described by Mannos and Sakrison [20] in the context of image coding.

One method, then, of extending the dynamic range of re- corded images is to take the logarithm of the original intensities, attenuate slowly varying components which contribute little information to the eye (and, indeed act as a source of degradation) while ' simultaneously enhancing rapidly varying components to which the eye is most sensitive, and finally, taking the exponential of the processed image. Photographers have, to a degree, accomplished this processing in recording images on film by using a fast-acting developer with little agitation; exhausted chemicals from heavily exposed areas diffuse into adjacent lightly exposed areas causing them to develop even less and thereby emphasizing the transitions from light to dark [2]. For images stored in digital form, this processing is readily accomplished by a digital computer.

This paper describes a specific digital implementation of a scheme for enhancing rapid local variations due to object boundaries and deemphasizing more gradual changes such as those which occur as a result of shadows while simultaneously controlling the degrading effects due to salt-and-pepper noise. In order to formulate this problem the observed image intensity so(x) will be considered to be the product of three components: the true reflectivity of the objects in the image ~(x), the

0096-3518/79/1200-0625$00.75 0 1979 IEEE

626 IEEE TRANSACTIONS ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, VOL. ASSP-27, NO. 6, DECEMBER 1979

nonconstant illumination function i(x), and a white noise component n (x). That is,

so (x) = r(x) . i(x) . n (x). (2)

We note that the noise here is modeled as a multiplicative process in the intensity domain or, equivalently, additive in the density domain. There is substantial evidence to justify this for either photochemical or photoelectronic sensing and display systems. The discussion in Andrews and Hunt [21] is particularly illuminating in this regard. In general, however, the noise process n(x) will depend upon the signal in a rather complicated fashion. We have chosen to disregard this dependence and assume a signal-independent noise process n ( ~ ) . Again this is common practice in digital image processing

Taking the logarithm of both sides of (2) results in the (cf. [21, pp. 20-231).

density image

g(x) e In so (x) = f,(x) + fi (x) + f n (x) (3)

where f,(x) & In r(x), fi(x) = In i(x), and finally &(x) = In n(x). If precise stochastic descriptions of the signals f,(x), fi(x), and f,(x) are available, the problem of extracting the true image from the illumination and noise can be formulated as one of Wiener or linear least mean-square filtering, where the signal is f,(x) and the noise isfi(x) +fn(x). Passing the result of this filtering through an exponentiation operation yields an estimate ?(x) of the true image as represented by the reflectance process r(xj. A block diagram of the indicated processing is provided in Fig. 1.

The technique of sandwiching a linear operation between two complimentary nonlinear operations (homomorphic filtering) [7] as applied to image enhancement is not new; what is unique about the approach described here is the use of explicit stochastic image models as a guide in the choice of a linear-filter transfer function. In particular, we develop explicit stochastic models for both the reflectance process f,(x) and the illumination process fi(x). The reflectance process f,(x) has been developed to exhibit predominant and pronounced edge structure, such as might be expected of real- world imagery. This process possesses considerably high spatial frequency components. The illumination processfi(x), on the other hand, has been specifically developed to possess relatively large low spatial frequency components typical of illumination fields.

These stochastic models are described in Sections 111 and IV, respectively. The noise processf,(x) is described in Section V while a description of the digital implementation of the Wiener

filter is provided in Section VI. Some typical results are illustrated in Section VI1 which is followed by a summary and conclusions in Section VIII. We begin with some preliminary discussions on random fields as image models.

11. PRELIMINARY DISCUSSION The approach described earlier is not complete until the

stochastic descriptions of f,(x), fi(x), and &(x) are specified. The models to be considered here will be a family of random variables cfx(o), x E R 2 } defined on the plane where o denotes the dependence on a fxed probability space (a, Q ,Pj. Wong [lo] refers to such a collection as a random field, but notes that it is also called a stochastic process with a several- dimensional time. In the discussion that follows, for convenience, the dependence upon the underlying probability space will not be explicitly denoted;f(x) will be written for &(a). Furthermore, consideration will be restricted to zero- mean fields of a second-order (variances exist). The second- order properties are described by the covariance function of the random field which is given by

Rff(X,.Y) = E Cf(x>f(u>); x,.Y E R 2 (4)

where E { .} represents the expectation operator over the probability space (a, Q , P). All of the random fields to be presented here have been constructed so as to possess a covariance function invariant under all Euclidean motions; according to the definitions of Wong [lo] , they are homogeneous and isotropic. For these fields, (4) can be written

E{f(x+u)f(x)}=Rff(IIu 11); uT=(u1,u2)ER2 (5)

where I( u 1 1 represents the Euclidean norm defined in terms of an inner product according to 1 1 u 11' = (u , u ) = u; + uz. Since the random fields are homogeneous, their power-spectral density is given by

S f f ( o ) = [ Rff (u ) exp {-i (w, u ) } du ( 6 )

where o = (a1, w 2 ) represents a spatial frequency vector and du is the differential volume element in R 2 . Furthermore, for isotropic fields, Rff (u ) = Rff(ll u 11) and (6) can be evaluated up to the functional form with the aid of a theorem of Bochner [ 1 1 ] yielding

R2

Sff(0) = Sff(LQ) = 27r hRff(h) Jo(hS2) d h L- (7)

where S2 4 11 o 11 = (a; t &)' I2 represents radial frequency

FRIES AND MODESTINO: IMAGE ENHANCEMENT BY STOCHASTIC HOMOMORPHIC FILTERING 621

and Jo( - ) denotes the ordinary Bessel function of the first kind of order zero. Thus, S f f ( * ) and R f f ( e ) are related through a Hankel transform [ 121 , [ 131 .

The enhancement problem as we have posed it consists of emphasizing fr(x), while minimizing the degrading effects of &(x) +f,(x); specifically, we will look for the linear least mean-squared estimate of f,.(x) based on observing g(x). If we have knowledge of the power-spectral densities2 Srr(o) , Sii(o), and &,(a) associated with fr(x) , &(x), and f,(x), respectively, and these three fields are uncorrelated, the two- dimensional linear least mean-square filter possesses the transfer function

which is simply an extension to two dimensions of the Wiener filter [SI.

We will now proceed to describe the random fields which are to be used as the reflectance, illuminance, and noise models, respectively.

111. REFLECTANCE PROCESS The process to be considered as a model of the reflectance

will be referred to as the “rectangular process.” This process divides the plane into regions whose boundaries form rectangles, and then assigns intensities to these regions such that the intensity within each region is constant and possesses specified correlation properties with intensities in contiguous regions. It was originally developed to satisfy the need for an edge model for the design of a linear filtering operation to be used in edge detection [15] . Roughly speaking, it is a two- dimensional extension of a generalization of the random telegraph wave [ 141 .

As a first step in describing this process, the method of partitioning the plane into rectangles will be detailed. A fundamental role in this stochastic image model will be played by the vector-valued random field N(x) which provides a two- dmensional generalization of a counting process. In particular, suppose the vector x“ is obtained from x according to ?=Ax where A is the matrix

A = [- cOse sin e cos ““1 e .defined for some 0 E [-n, n] . The transformation (9) then results in a rotation of the plane in terms of the Cartesian coordinate axes (xl, xz), as illustrated in Fig. 2. Consider now the vector-valued random field defined by

where 0 is uniformly distributed on [-n, n] and {Ni(t), t > 0}, i = 1, 2 are mutually independent one-dimensional counting processes, i.e., Ni(t) represents the number of events which have occurred in the interval [0, t] . Specifically, let

‘In general, the power-spectral density of a random fieldf(x) will be denoted by Sf-a). In situations where subscripts a-e-used, such as fr(x) , the corresponding power-spectral density will be distinguished by the associated subscript, i.e., S,,(O) in the case offr(x) .

Fig. 2. Rotation of Cartesian coordinate axes.

the probability density function p ( t ) of the distance between events be given by the exponential distribution function

p ( t ) = XrFArt; t 2 0. (1 1)

The quantity X, represents the average number of events per unit distance.

Consider now the random field {&(x), x E R 2 } , which un- dergoes transitions at the boundaries of the elementary rectangles defined by {N(x) , x ER2}. The gray level assumed throughout any elementary rectangle will be assigned so as to be zero-mean Gaussian3 with variance u,“ and correlated with the gray levels in contiguous rectangles. More specifically, assume that the random orientation 0 E [-n, 771 has been chosen and that k transitions have occurred between the two points x and x + u. It will be assumed that

E(f(x+u)f(x)Ie ,k}=u,2pyk; k = 0 , 1 ; * * , (12)

where I p r I < 1. The quantity pr is the correlation coefficient governing the spatial evolution of the random amplitude process. For example, let Xi,j represent the amplitude, or gray level, assigned after i transitions in the Tl direction and j transitions in the z2 direction. The sequence {Xi, j} can then be generated recursively according to x. a,] . = p r x. z - 1 , j + Prxi , j - l - dxi-1,j-I + W , j (13)

where {Wi, j } is a two-dimensional sequence of independent and identically distributed (i.i.d.) zero-mean Gaussian variates with common variance u$ = uz(1 - p:)“ The sequence defined by (13) has an alternative interpretation as the output of a recursive two-dimensional digital filter excited by a white noise field. It can be shown that

E {xi , jxi+ k, , j + k , } = 0, pr 2 Ik, I+ Ik, I , (1 4)

so that the condition expressed by (1 2) is indeed satisfied. Typical computer-generated realizations of the resulting

random field are illustrated in Fig. 3 for selected values of p r and X,. Note that X, controls the edge density, while p r indi- cates the degree of correlation of the intensities in adjacent regions. The displayed images here and throughout the re- mainder of this paper are square arrays consisting of 256 elements, or samples, on a side. Where a value of X, is specified, it is measured in normalized units of events per sample distance so that-t&re are, on average, 256Xr transitions along each of the orthogonalaxes.

3The Gaussian assumption here is not critical and is easily removed.


Fig. 3. Selected realizations of the reflectance process. (a) h, = 0.0125, P, = -0.9. (b) A,= 0.0125, P, = 0.0. (c) hr= 0.0125, p , = 0.5. (d) A, = 0.025, p, = -0.9. (e) h, = 0.025, P, = 0.0. (f) h, = 0.025, p, = 0.5. (8) A, 0.05, p r = -0.9. (h) A, = 0.05, p, = 0.0 (i) h, = 0.05, P, = 0.5.

In [I51 it has been shown that the power-spectral density of hand, the transition across an edge boundary is much more this process is given by gradual. These properties are easily related to any particular

1 real-world image. ] ‘ I 2 . (15) IV. ILLUMINATION PROCESS

Although realizations of this process may not resemble any particular real-world image, the general properties of regions of constant reflectivity and boundaries of regions being parallel are common to many real-world images. More specifically, we do not claim that the model proposed here is universally representative of real-world imagery except in a certain quali- tative sense. This model is completely defined, up to a scale factor, by the two parameters X, and p,. The parameter X, represents the “edge busyness” associated with an image, while p , is indicative of the degree of abruptness across an edge boundary. For p r large (in magnitude) and negative, there is an abrupt almost black-to-white or white-to-black transition across an edge boundary. For p > 0, on the other

In the previous section a two-dimensional random field which possesses an inherent rectangular structure was described and analyzed. While this random field is a reasonable model of objects, shadows do not generally exhibit this rectangular mosaic, but rather a much more random edge orientation. Here the construction and properties of a class of two-dimensional random fields, which provide a more appro- priate model for illumination in the sense that individual realizations do not exhibit this rectangular pattern will, be presented. Consider the random partition of the plane by a series of sensed (directed) lines which can be described as a marked-point process. Specifically, an arbitrary sensed line can be described in terms of a 3-tuple (r, 0, d ) where r repre-


\

L

I \

Fig. 4. Parameterization of a directed line segment.

sents the perpendicular distance, or radius, to the line in ques- tion, 6 E [-n, n] represents the orientation of the vector of length r drawn from the origin and perpendicular to this line, and d E {- 1 , l } specifies the sense of the line where Fig. 4 illustrates a case of d = 1 (pointing counterclockwise). As shown in Fig. 4, by virtue of the direction imposed on this line segment, the plane is partitioned into two distinct regions, R (right of line) and L (left of line), such that R U L = R 2 .

Now consider the set of lines represented by the sequence {(ri, B i , di)}. Here {ri} represents the event positions associated with a homogeneous Poisson process {N(r),v> O } , with intensity A eventslunit distance evolving according to the radial parameter Y. The sequence {ei} represents an (i.i.d.) sequence of random variables uniformly distributed on [-x, n] , while the sequence Idi} is likewise i.i.d., assuming the binary values +1 with equal probability. Finally, the sequences {ri}, (03 , and {d;} are assumed to be independent.

Now note that the set of lines generated by the process described above will divide the plane into disjoint polygonal regions. For any point x, define NLR (x) to be the number of left-to-right crossings of sensed lines incurred in moving along a straight path from the origin to the point x; similarly, define NRL(X) to be the number of right-to-left crossings. Observe that N L R ( x ~ ) = N L R ( x ~ ) and NRL(x1) = N R ~ ( x Z ) for any two points x1 and x2 which are the same region. Assign a label k to each region as k =NLR(x) - NRL (x) where x is a point in the region. This label can in turn be used to assign a gray level to each region by using it to index into a sequence {X,} of zero-mean Gaussian random variables. In particular, assume that the sequence {X,} is generated recursively according to

x k = p i x , - , +w,; k = 1 , 2 , . . ’

and (1 6)

X,=&&+, +w,; k = - l , - 2 ; . . ,

with {W, } an i.i.d. zero-mean Gaussian sequence with variance u$ = ( 1 - p f )u ; , and the initial value X , chosen to have zero-mean Gaussian distribution with variance u;. Note that this sequence is stationary with the covariance given by

Selected computer-generated realizations of the resulting random field are illustrated in Fig. 5 for various values of p i and Xi h/n. The quantity Xi can be shown to represent the average edge density along any randomly chosen line segment on the pline. From Fig. 5 observe that, for pi = 0.5, light

regions tend to be surrounded by light regions, while for p i = -0.9 there is a tendency for light regions to be surrounded by dark regions and vice versa. Indeed, in the latter case there is almost a black-to-white or white-to-black inversion across an edge boundary.

An expression for the autocorrelation function given by (4) can be obtained, as in [ 1 7 ] , with the result

(18)

where I,( e ) is the modified Bessel function of the first kind of order k. Similarly, the corresponding power-spectral density evaluated according to (7) is given by

which is illustrated in Fig. 6 for various values of pi. One noteable characteristic of this random field is that for a small compared to Ai, the power-spectral density behaves approximately as i.e., Sii(a) has a singularity at the origin except for p i = - 1. This high concentration of energy at low spatial frequencies is a direct result of the construction procedure which allows relatively large correlation between gray levels in regions relatively far apart. We feel that this characteristic is typical of random illumination processes and as a result it was purposely built into the construction procedure.

The effect on a real-world image of multiplying the intensity by the exponential of a realization of the polygonal process is illustrated in Fig. 7 . Note how the illumination process seems simply to have added extra shadows to the building. Although somewhat of an exaggeration, this figure has been included to illustrate the nature of effects which may be caused by the proposed random illumination process.

V. WHITE NOISE PROCESS In any photographic image that is scanned there is some

noise present that is independent from pixel-to-pixel; due to its appearance it is often referred to as “salt-and-pepper noise.” Here we will model this noise, which is represented by f , (x) , as white Gaussian noise. While the Gaussian assumption is not critical to the analysis which follows, it is interesting to note the observation of Selwyn (cf. [2, p. 1251) on noise due to film grains. He notes that if the spot area covered by a beam used to measure the density of a photographic emulsion is much larger than the projection of individual grains then the density distribution will be Gaussian if the grain noise is small, and presumably homogeneous. In fact, the following relationship between the size of the scanning aperture and variance of the measured densities has been found to hold

where u i is the variance of the density, G is a measure of the

630 IEEE TRANSACTIONS ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, VOL. ASSP-27, NO. 6 , DECEMBER 1979

Fig. 5 . Selected realizations of the illumination process. (a) hi= 0.0125, p i = -0.9. @) hi= 0.0125, p i = 0.0. (c) hi= 0.0125, p i = 0.5. (d) hi = 0.025, pi = -0.9. (e) hi = 0.025, p i = 0.0. (f) hi = 0.025, pi = 0.5. (g) hi = 0.05, pi = -0.9. (h) hi = 0.05, p i = 0.0 (i) hi = 0.05, p i = 0.5.

m

;;= IO

Normalized Spatial Frequency W 2 7 X i

Fig. 6. Power-spectral density of the illumination process.

granularity of the film, and a is the area of the scanning the grain noise were white, this modeling assumption is justi- aperture. If the scanning aperture is considered to be a spatial fied. The power-spectral density of this noise can be expressed low-pass filter whose bandwidth is proportional to 1 / 6 , then then as (20) implies that the noise power present in the density mea- surement is proportional to the low-pass filter bandwidth squared. Since this is the effect that would be observed if where 0; is the variance of the samples.

~ r Z n ( W = 0; (2 1)


( C)

Fig. 7. Effects of the illumination process. (a) Original. (b) Iflumina- tion. (c) Original with illumination.

VI. LINEAR FILTER DESCRIPTION AND IMPLEMENTATION

Using the models presented above, the linear least mean- squared error (Wiener) filter for estimating the reflectance process will now be derived. The system transfer function given by (8) can be rewritten as

which suggests defining the parameters yi and -yn as the ratios

of illumination and noise powers to reflectance power, respectively. These parameters provide a measure of the degree of degradation caused by shadows and salt-and-pepper noise. More specifically, we have

while

so that the filter is completely specified in terms of the quantities yi , -yn p r , h,., pi, and hi. These quantities can be defined, at least empirically, for a wide range of images. Finally, observe that, for the given power-spectral densities, the filter is radially symmetric, i.e., it is a function only of s2 = Ilwll.

Since the final objective of this process is to make the best use of the limited dynamic range available for display, it wiU prove expedient to include a gain factor A . chosen such that the power leaving the linear processor equals that entering. It follows then that do is given by

(25)

where Sgg(o) is the power-spectral density of the fiiter input g(x) in (2 j and is given by

The description of the filter provided above is for a con- tinuous spatial domain; we have access only to sampled data. Hence we seek a (2 - D) digital filter with system transfer function H o ( z l , z2) whose frequency response4 approximates Ko(!2). Specifically, the Wiener filter has been implemented as a two-dimensional infinite impulse response (IIR) [lS] digital filter whose point spread function exhibits quadrant symmetry. The choice of an IIR filter was based on the computational economies which result from the ability to implement the filter with a recursive structure. Consider first a filter with the folrowing recursive structure:

where5 D 2 { ( i , j ) : i = O , I ; * . , N , j = O , I ; * - , M } a n d D ' = D - {(O, 0) j . At this point we will assume that the coefficients of y in (32) have been chosen such that the resulting filter is stable when recursing from the upper left-hand corner. Ob- serve that the resulting filter will have a nonzero response only in the lower right quadrant. Since the filter described in (22) exhibits radial symmetry, it is desirable for the resulting digital filter to exhibit four-quadrant symmetry.

One method for achieving a point spread function with this inherent symmetry is to allow repeated application of the

4The frequency response of the fwo-dimensional digital f'iiter is simply Ho(z, , 22) evaluated for zi = elwi, i = 1, 2. We assume the spatial frequency variable wi is measured in units of radians per sample distance.

'The support D of the recursive two-dimensional filters taken here will be such as to force a quarter plane causality condition. That is, D consists of a finite number of points above and to the left of the pixel in position (i, j ) .


. .

. .

Fig. S. Point spread function of the composite filter.

TABLE I TYPICAL FILTER PARAMETERS FOR yn = -20 dB

Shadow

Characteristic Y i ( d B ) Dr h r P i h i

Soft -.@147 ,1831 -.1957 .2376 -.a240 ,7833 -.e650 3 .05 0.5 .0125 -6

Medium

-.0065 .41;58 -.4?18 .5109 -.0118 ,7959 -.a817 @ . @ 5 -.9 .0125 6 Harsh

-.0160 .2072 -.2216 .2681 -.2326 . 771.3 ' -.e503 0 .05 0 . 5 .I3125 -3

I I I 1 . 1

same filter recursing from each of the four corners. If {hi,j} represents the point spread function associated with a single application of the filter specified by (27), then the composite filter possesses point spread function as indicated in Fig. 8. The corresponding system transfer function of the composite filter is then

~ ( z 1 ~ z 2 ~ ~ H 0 ( z 1 ~ z 2 ) t H 0 ( z ~ ' ~ z 2 ~ t H 0 ~ z Z , ~ z ~ 1 ~

t Ho(z; l , z ; ' ) (28)

where Ho(zl , z2 ) is the transfer function corresponding to (27) and is given by

H o ( z l , z 2 ) = bj,jz:iz;/ [1 t ai,jz;iz;i- ,

( W E D (i, j ) E D ' 1 (29)

In choosing Ho(zl , z 2 ) to provide an approximation to Ho(CL) for zi = i = 1, 2, we have restricted attention to the case where Ho (zl , z 2 ) is a second-order section of the form

line 45" to the axis, i.e., h ; , ~ = hj,i. Similar properties extend, of course, to the composite filter represented by H(zl , z2 ) . A computer program has been written for determining the coefficients boo, bol , bl l , bo2, aol , a l l , and aO2 according to an iterative gradient procedure to result in a frequency response for H ( z l , z 2 ) which provides a least mean-squared error approximation to the desired response Ho(i2), The details of this program are described in [6]. In Table I we summarize the results of t h s procedure for selected values of yi, Xi, pi, A,, and p r , all with yn = -20 dB. Here we have found it convenient to classify the shadow characteristic as soft, medium, or harsh, depending upon the value of yi.

For the three cases described in Table I, the corresponding power-spectral densities Sr,(f2) and Si&?), together with the resulting Wiener filter response Ho(S2>, are plotted in Fig. 9 as a function of the radial frequency variable 52. Observe the sharp dc rejection in all cases. This is, of course, due to the relatively high concentration of energy at low spatial frequencies associated with the illumination process fi(x>, as described previously. The Wiener filter response Ho(52) exhibits the

with the restriction essential characteristics of the linear high-pass filter originally proposed by Stockham [4] on an ad hoc basis. It is of some

modeling assumptions made here. boo = -2b01 - bll - 2boz. (31) interest that this choice can be justified rigorously under the

This choice ensures zero frequency response at the origin and In Fig. 10 we illustrate three-dimensional plots of one symmetry of the corresponding point spread function about a quarter-plane of the frequency response of the filters asso-


-20 1 0 OD1 0302 003 004 005 0.06

Radial Frequency 0/2t

- 20 0 0.01 002 0.03 0.04 0.05 0.06

Rodial Frequency n/2r

(a) Fig. 9. Illustration of typical power-spectral densities and

ciated Wiener filter response. (a) Soft shadow model. (b) shadow model. (c) Harsh shadow model.

z -10 O r -20 '

o 0.01 0.02 0.03 0.04 a05 0.06 Radial Frequency 0/2r

( C)

the asso- Medium

(e) (f)

Fig. 10. Frequency responses of desired and actual filters. (a) Desired response for soft shadow. (b) Actual response for soft shadow. (c) Desired response for medium shadow. (d) Actual response for medium shadow. (e) Desired response for harsh shadow. (f) Actual response for harsh shadow.

ciated with the three cases described in Table I. The left- hand column shows the desired responses while the right- hand column illustrates the responses obtained by the digital filters. The frequencies displayed run from 0 at the front to R

radians per pixel at the back. Note that in all cases the complete attenuation of dc and rolloff at higher frequenceis has been preserved in the transition from the desired analog filters to the digital approximations.

VII. RESULTS The algorithm presented above has been applied to several

real-world images for selected values of the parameters yn, yi, p,, X,., p i , and Xi. The processed images shown here illus-

trate the effects of varying these parameters on the recon- structed images.

Parts (a) of each of the four sets of images comprising Figs. 11-14 are the original images, while parts (b), (c), and (d) are the corresponding processed versions.. All of the processed images have the salt-and-pepper noise 20 dB below the reflectance, while the reflectance has zero correlation of intensities in different regions ( p , = 0), and an edge density such that on the average there will be 12 transitions on a side of the image (X, = 0.05). The variations among (b), (c), and (d) are due to changes in the illumination intensity yi and the illumination correlation pi.

Part (b) of these figures has the edge density of the illumina-

6 34 ILEEE TRANSACTIONS ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, VOL. ASSP-27, NO. 6 , DECEMBER 1979

Fig. 11 . Typical results on the head-and-shoulder image. (a) Original. Fig. 12. Typical results on an outdoor scene. (a) Original. (b) Soft (b) Soft shadow. (c) Medium shadow. (d) Harsh shadow. shadow. (c) Medium shadow. (d) Harsh shadow.


Fig. 13. Typical results on a building image. (a) Original. (b) Soft Fig . 14. Typical results on an FLIR image. (a) Original. (b) Soft shadow. (c) Medium shadow. (d) Harsh shadow. shadow. (c) Medium shadow. (d) Harsh shadow.

636 IEEE TRANSACTIONS ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, VOL. ASSP-27, NO. 6 , DECEMBER 1979

( 4 (dl Fig. 15. Typical results on a tomograph. (a) Original. (b) Soft sha-

dow. (c) Medium shadow. (d) Harsh shadow.

tion process Xi = 0.0125 or, on the average, three transitions on a side of the image, pi = 0.5 (positive correlation of the illumination in adjacent regions) and illumination power 6 dB below reflectance power (y j = -6 dB), corresponding to a soft shadow.

Note how in part (b) local variations are much more evident; shadow and highlight detail have both been emphasized. Specifically, in Fig. l l(b) detail in the hair is much more evident than in Fig. 1 l(a). In Fig. 12(b) the light scaffolding blends in less with the clouds, while the dark slatting on the side of the tower is also more evident. Similarly, in Fig. 13(b) the bright regions in the center have been deemphasized and detail in the building is more apparent. Finally, for the forward looking infrared (FLIR) image in Fig. 14(b), the tank blends in less with its surroundings. This processing brings out detail in both dark and light areas.

Parts (c) (medium shadow) of Figs. 11-14 vary from parts (b) in the increase of the illumination power by 3 dB. The effects noted above have simply become more evident.

Part (d) (harsh shadow) deviates from (c) in both illumination power (which is increased by 9 dB) and the correlation of illuminances in different regions which is now -0.9 instead of 0.5. Part (d) of all the figures brings out detail in adjacent light and dark regions better than (c) as is expected from the shadow being modeled as having black-to-white transitions across boundaries between adjacent regions.

An application of this technique to a dynamic range reduc- tion problem is illustrated in Fig. 15. In the original

tomograph image little detail is evident in the dark area, while the light areas appear muddy. In Fig. 15(b) some detail is evident in the dark region, while local shading is more apparent in light regions. When harsher shadows are assumed, as in 15(c) and 15(d), local detail is emphasized.

In all cases the processing has increased the visibility of detail in such a way that less scrutiny need be applied when observing the processed images than is required when observing the originals. There are, however, differences in the characteristics of the processing that occurs. Increasing the shadow power and making the shadow more pronounced by having negative correlation (pi = -0.9) instead of positive correlation (pi = 0.5) has made local detail more visible by sup- pressing variations in intensity that occur over large regions.

VIII. SUMMARY AND CONCLUSIONS

A method for determining the linear filtering operation to be performed as part of homomorphic processing based on Wiener filtering concepts has been described. The resulting linear filter is determined by the parameters of the assumed stochastic models of the signal and degradation. These parameters are associated with salient characteristics of the signal and degradation. Although no claims can be made about the optimality of a particular filter for any given real-world image, it has been shown that the parameters in the models provide a useful means for specifying the desired effects of the processing.

The implementation of the linear operation has taken the

IEEE TRANSACTIONS ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, VOL. ASSP-27, NO. 6, DECEMBER 1979 637

form of two-dimensional recursive IIR digital filter whose P I order has been restricted to result in computational efficiency and whose coefficients have been chosen to result in a good [91

. approximation to the Wiener filter. The resulting nonlinear filtering operation has been applied [ lo]

to a number of real-world images to illustrate the relationship [111 between the parameters of the assumed stochastic models and the effects of the resulting processing.

[I21

ACKNOWLEDGMENT

The authors would like to thank the anonymous reviewers ~ 3 1

who helped to improve an earlier version of this paper. [ 141

REFERENCES

[ 11 W. D. Zoethout, Physiological Optics. Chicago, IL: The Profes- [ 161,

[2] P. Kowaliski, Applied Photographic Theory. New York: Wiley,

[3] J. E. Hochberg, Perception. Englewood Cliffs, NJ: Prentice-

[4] T. G. Stockham, “Image processing in the context of a visual [18]

[SI H. L. Van Trees, Detection, Estimation and Modulation Theory;

[6] R. W. Fries, “Theory and application of a class of two-dimen-

sional Press, 1939, p. 157.

1972. ~ 7 1

Hall, 1964.

mode1,”Proc. IEEE, vol. 60, pp. 828-842, July 1972.

Vol. 1. New York: Wiley, 1968. ~ 1 9 1

sional random fields,” Ph.D. dissertation, Rensselaer Polytech. [ 201 Inst., Troy, NY, in preparation.

“Nonlinear filtering of multiplied and convolved signals,” pfoc. [ 211 [7] A. V. Oppenheim, R. W. Schafer, and T. G. Stockham, Jr.,

ZEEE,vol. 56, pp. 1264-1291, Aug. 1968.

E. Wong, “Homogeneous Gauss-Markov random fields,” Ann. Math. Stat., vol. 40, pp. 1625-1634,1969. E. Wong, “Two-dimensional random fields and the representa- tion of images,” SIAM J. Appl. Math., vol. 16, pp. 756-710, 1968. E. Wong, Stochastic Processes in Information and Dynamical Systems. New York: McGraw-Hill, 1971, ch. 7. S . Bochner, “Lectures on Fourier Integrals,” Annals o f Mathe- matical Studies, Vol. 42. Princeton, NJ: Princeton Univ. Press,

A. Papoulis, “Optical systems, singularity functions, complex Hankel transforms,” J. Opt. Soc. Amer., vol. 57, pp. 207-213, 1967. A. Papoulis, Systems and Transforms with Applications in Optics. New York: McGraw-Hill, 1968. J. W. Modestino and R. W. Fries, “A generalization of the random telegraph wave,” submitted to ZEEE Trans. Inform. Theory.

filtering,” Computer Graphics and Image Processing, vol. 6,

J. W. Modestino, R. W. Fries, and D. G. Daut, “A generalization of the two-dimensional random checkerboard process,” to appear in J. Opt. SOC. Amer. J. W. Modestino and R. W. Fries, “Construction and properties of a useful two-dimensional random field,” submitted to IEEE Trans. Inform. Theory. L. R. Rabiner and B. Gold, Theory and Applications of Digital Signal Processing. Englewood Cliffs, NJ: Prentice-Hall, 1975, p. 5 1. C. Shipman, Understanding Photography. Tuscon, AZ: H. P. Books, 1974. J. Mannos and D. L. Sakrison, “The effects of a visual fidelity criterion on the encoding of images,” IEEE Trans. Inform. Theory, vol. IT-20, pp. 525-536,1974. H. C. Andrews and B. R. Hunt, Digital Image Restoration. Engle- wood Cliffs, NJ: Prentice-Hall, 1977.

1959, pp. 235-238.

- , “Edge detection in noisy images using recursive digital

pp. 409-433, Oct. 1977.

A New Approach for the Design of Digital Interpolating Filters

GEERD OETKEN

Aktmct-The paper presents a new class of interpolators character- ized by the property that the mean power of their error sequence, as a function of frequency, approximates zero in the Chebyshev sense. The design method, which is based on some observation and newly found general properties, will be described in some detail.

An example shows the special properties of these interpolators in comparison with former results.

Measurements done with a practical implementation are presented as well. A design chart for these filters is provided.

T I . INTRODUCTION

HE design of digital interpolators has been considered in quite a few recent papers (e.g., [ I ] - [ 6 ] ) . In this paper

we refer especially to [ 2 ] and [ 3 ] , where the investigation is

Manuscript received August 27, 1975; revised October 15, 1975,

The author is with the Institut fur Nachrichtentechnik, Universitat April 26,1976, and April 16,1979.

Erlangen-Niirnberg, West Germany.

based on an error function, the definition of which is repeated here for convenience (see Fig. 1).

The given sequence { ~ ( k ) } , to be interpolated with a rater, is considered as being a sampled version of the unknown, but bandlimited, sequence uo(k). For a convenient notation, we disregard causality for the time being and write the transfer function of the FIR system to be designed as

H(2) = h ( 2 ) z - k N

k = - N

where n = 2 N = 2Lr is the degree of the system, and 2 L the number of input samples the interpolation is based on. Ob- viously the output sequence is y(k) = h(k) * u(k). Now we apply a sinusoidal input sequence as a test signal, written as uo(k) = Uo(Q) ejkn with Vo(S2) = 0, Q 2 Qg = m / r , 0 < a < 1 , Using the error sequence Ay(k, Q) =y(k) - u,(k), an error function was defined in [2 ] and [ 3 ] as

0096-3518/79/1200-0637$00.75 0 1979 IEEE

homomorphic filter

Documents