image analysis via the general theory of moments*

Image analysis via the general theory of moments*MichaelReed Teague'

Air Force Weapons Laboratory, Beam Control Systems Branch, Kirtland AFB, Albuquerque, New Mexico 87117(Received 1 November 1979)

Two-dimensional image moments with respect to Zernike polynomials are defined, and it is shownhow to construct an arbitrarily large number of independent, algebraic combinations of Zernikemoments that are invariant to image translation, orientation, and size. This approach is contrastedwith the usual method of moments. The general problem of two-dimensional pattern recognition andthree-dimensional object recognition is discussed within this framework. A unique reconstruction ofan image in either real space or Fourier space is given in terms of a finite number of moments.Examples of applications of the method are given. A coding scheme for image storage and retrieval isdiscussed.

INTRODUCTION

The general problem to be considered is that of character-izing, evaluating, and manipulating visual information thatis present in the image plane or any other plane (e.g., theFourier plane) of an optical system. An example of such usagecould be the automatic identification of a pattern, size, andorientation measurements of an image, or the automatic ex-traction of the coordinates of an algorithm search point thatis known to be located somewhere in the image. In all suchcases the general method of moments furnishes a solution ofthe problem. Moreover, at present with dedicated digitalprocessors these operations may be performed in real time,for example, at TV frame rates. In addition, it appears thatthese methods have potential for coping with the difficultproblem of real time sensing of the time varying aberrationsin an optical system degraded by, for example, atmosphericturbulence or thermal blooming.'- 4 (This last problem,however, will not be discussed further in the present paper.)While moment methods are generally employed with video-digital processors, we will briefly indicate how moments couldbe extracted in a coherent optical processor that might beutilized in a high-speed film scanner that automaticallyidentifies biological patterns.

Historically, Hu5 published the first significant paper onthe use of image moment invariants for two-dimensionalpattern recognition applications; his approach is based on thework of the nineteenth century mathematicians Boole, Cayley,and Sylvester, on the theory of algebraic forms. More re-cently, Dudani et al. 6 used moment invariants in automaticaircraft identification algorithms. The well-known propertiesof second- and lower-order moments are summarized in Sec.I, while Sec. II treats the question of what is gained by in-cluding higher-order moments in the analysis of an image. InSec. III, rotation and reflection invariance is studied usingZernike moments. This approach is unusually simple and ina straightforward manner allows moment invariants to beconstructed to an arbitrarily high order and in addition fur-nishes an elementary procedure to determine whether a givenset of moment invariants is composed of functionally inde-pendent members.

If Zernike moments are related to the usual moments, theinvariants constructed in this paper are largely equivalent tothose of Hu; however, as may be seen in Appendices A and B,if it is necessary to use higher-order moment invariants, the

expressions based directly on Zernike moments are enor-mously simpler than those based on the usual central mo-ments. In the statistical image recognition problem, theweights assigned to the various moment invariants used aspattern identifiers are crucially important. It is interestingthat the weights that occur naturally in the present paperdiffer from those in Ref. 5. In addition, the present papertreats the general problem of object size or range normaliza-tion of the moments.

At the outset it should be stressed that the use of low-ordermoments, for example, centroid position and moments of in-ertia, is commonplace in many disciplines. On the other hand,it is well known7 that by using a sufficiently large number ofimage moments we in principle recapture all the image in-formation, but then we would merely have replaced one totaldescription (approximately infinite number of pixel intensi-ties) by an equivalent one (approximately infinite number ofmoments). The interesting question is how well the methodof moments works for a particular application if one considersmoments up through, e.g., fifth or tenth order. For example,a single video image may contain 104-108 bits of information,while a photographic image can contain orders of magnitudemore. Often, however, the quantitative information that isneeded from an image is more likely to be of the order of 10or 20 real numbers. Then the method of moments furnishesa general systematic procedure for extracting a set of quan-titative features from an image.

1. PROPERTIES OF LOW-ORDER MOMENTS

In all cases we consider a finite image plane and all integralsnot otherwise designated are over this finite image plane. Westart by recalling certain well-known 8'9 properties of thelowest-order moments. Denoting the image plane irradiancedistribution by f(x,y) we have the zero-order moment

g00 ff dxdYf(xy) (1)

representing the total image power. The first-order mo-ments

t10o = Jj'dxdyxf~xy),

=l fj dxdyyf(x,y)

(2)

(3)

920 J. Opt. Soc. Am., Vol. 70, No. 8, August 1980 0030-3941/80/080920-11$00.50 � 1980 Optical Society of America 920

FIG. 1. Image ellipse.

locate the centroid of the irradiance distribution, i.e., x= /lo/oo, y = Aol/0oo, and the second-order moments

A20 = fdx dyf (x,y) x 2, (4)

A = ff dxdyf(x,y)xy, (5)

/02 =fJf dxdyf(x,y)y 2 (6)

characterize the size and orientation of the image. In fact, ifonly moments up through second order are considered, theoriginal image is completely equivalent to a constant irra-diance ellipse having definite size, orientation, and eccentricityand centered at the image centroid. In this section we assumethe coordinate oriain has been chosen to coincide with theimage cehtroid.

The parameters of this "image ellipse" are (referring to Fig.1)

a (120 + /02 + [Q 2 0 -/o 2 )2 + 4/21]1/2 1/2

a _=A0 0 + R o20 - A0 (7)

1 + cos2 . 1 - cos2(pj, -A20___ _ - sin2(p - A20

/20 2 2

sin2(p sin25 IIA11 = 2 cos2o - 2 Ali ,

1 - cos2( . 1 + cos2)A02 -i2 sn2o 2 A02

(11)

where (p is the rotation angle. The fact that 24) appears is aconsequence of the ellipse being unchanged by a 1800 rota-tion.

There is an ambiguity in the tilt angle 0 of the ellipse whichwe resolve by choosing (p always to be the angle between thex axis and the semimajor axis, i.e., by definition

a > b.

Secondly, we pick the principal value of the function arctangent such that

-7r/2 < tan-lx < 7r/2.

With these conventions we arrive at the results for the tiltangle that are given in Table I.

Finally, we point out that one may show9 by an applicationof the classical Schwarz's inequality that the image ellipseexists with real a and b if and only if the function f(x,y) isnon-negative, which is true for irradiance distributions, butnot for amplitude distributions.

II. INVERSE MOMENT PROBLEM: OPTIMALRECONSTRUCTION OF AN IMAGE

We now look at higher-order moments which in the generalcase are defined by

Mjk = ffdxdyf(x,y)xjy h. (12)

To investigate the information content of the higher-ordermoments, we begin by considering the inverse problem:Given only some (finite) set of moments of an image, how wellcan we reconstruct the image? We will give two equivalentanswers to this question. The first is a direct approach andthe second, which is computationally simpler, uses orthogonalmoments based on Legendre polynomials. Both methodsfurnish our definition of the best reconstruction of an imagebased on a finite set of its moments, i.e., they produce a con-tinuous irradiance function whose moments match the given

b = (/20 + /02 - [(20 - /02)2 + 4A 1]1/2l/2

/1oo/2

for the semimajor and semiminor axes, respectively, and

0 = (1/2) tan-' ( 2/1A/20 - 02

(8) TABLE I. Ellipse Tilt Angle for Various Cases of the signs of the secondmoments.

(9)

for the ellipse tilt angle. The irradiance of the image ellipse

F = /oo/7rab (10)

inside the ellipse and zero outside.

These results follow9 from a consideration of the rotationtransformation properties of the second-order moments:

A20 - Y02 111 (

Zero Zero OZero Positive +45° A20 - A02Zero Negative -450Positive Zero 0Negative Zero -900Positive Positive (1/2)tah-1' (O < ( < 45)Positive Negative (1/2)tan-l' (-45° < (p < 0)Negative Positive (1/2)tan-l + 90' (450 < (h < 900)Negative Negative (1/2) tan-'l - 90° (-90° < 0 <-450)

921 J. Opt. Soc. Am., Vol. 70, No. 8, August 1980 Michael Reed Teague 921

set. The methods and outlook of this section lead naturallyto the approach of Sec. III where symmetry properties ofmoments are treated.

It should be mentioned that the inverse moment problemwe are considering in this section is a special case of a muchmore general, extensively studied, mathematical problemformally stated by Stieljes10 in the last century.

As we stated before, one of our primary interests is in de-termining how well an image may be characterized by a smallfinite set of its moments. Moreover, the infinite set of allmoments contains exactly the 'same information as the originalimage, the connection between the two descriptions beingmade via the characteristic function. However, we will pointout that the characteristic function is not useful for treatinga small set of moments.

Finally, we point out that since we only consider imageplanes of finite extent, we always choose the unit of lengthsuch that the maximum distance of any point in the imageplane from the center of the image plane is less than unity, e.g.,we can take the image plane over which we calculate momentsto be the region

X E (-1,+1), (13)y e -,+)

(For a rectangular format image plane, we would just requirethat one of the maximum distances be less than unity, e.g.,0.8.) With this choice, higher-order moments will, in general,have increasingly smaller numerical values. Later we will seethat the moments are related to the coefficients in an expan-sion that reconstructs f(xy), and we improve the convergenceproperties by arranging to have I x I and Ify I less than unity.

A. Characteristic functionConsider now the characteristic function" for the irradiance

function f (x,y):

FQM= ff dx dye-i27rx+nY)f(x,y). (14)

Since in cases of interest to us f(x,y) is at worst piecewisecontinuous, and the integration limits are finite, F(Q,-) is acontinuous function and may be expanded as a power seriesin #j. To see what the expansion is, note that

F(,7) = dxdy E (-i2Trx)i E (-i27.ny)kff_ a! k=O k!

E iT Mjk 07 , (15)j=O k=O jph!

where the interchange of order of summation and integrationis permissible for the reason given in the last sentence. Thuswe see that the moment MJk is essentially the expansioncoefficient of the (inJk term in a power series expansion of thecharacteristic function of the image irradiance function f (x,y).Given FQ,-q), f/(x,y) is given by the inversion formula

Now, however, the order of summation and integration in Eq.(48) cannot be interchanged, or one will immediately derivethe false conclusion that the original intensity function f (xy)is an infinite series-all of whose terms are derivatives of deltafunctions located at the origin. The conclusion is that thepower series expansion for F(Q,-q) cannot be integrated termby term. It must first be summed in closed form to obtain afunction that vanishes sufficiently fast as a2 +712 - -, andthen integrated. In particular, if one knows only a finite smallset of moments, one cannot use a truncated series in Eq. (47)and learn about the original function f (x,y). The same con-clusion follows if the moment generating function (which hasthe same relation to the characteristic function as the Laplacetransform has to the Fourier transform) is used.

B. Moment matching approachIn Sec. III we will see that one must, on the basis of rota-

tional invariance, consider all moment Mjk of a given orderN = i + k together on the same footing. Suppose now thatone knows all moments Mjk up to a given order Nmax. Wenow show that no matter what type of function f (x,y) gave riseto the given finite set of moments, it is possible to obtain acontinuous function g(x,y) whose moments exactly matchthose of f (xy) up to the given order Nmax. We assert that thegiven function may be expressed in the form

g(x,y) = g 0 0 + g90 X + g90 1y + g 2 0 X2 + g1l xy + g 0 2 y 2

+ g3 0 x3

± gi2 x2

y + g92 xy2+ g0 3 y

3+ (17)

where the constant coefficients gik are now to be determinedsuch that the moments of g(x,y) match those of f (xy); i.e.,

r+ +lf dx f dyg(x,y)xiyk -Mjh . (18)

As an example, we determine the coefficients such that g(x,y)has moments that match those of f(x,y) up through thirdorder. Applying Eq. (18) to Eq. (17) for the ten cases of j+ k < 3, we produce ten algebraic equations, which we writein matrix form:

1

1

3

1

3

1

3

1

5

1

1

3

151

9

1

5

1

71

15

MOo

(1/4) X M20,

M02

Mlo1

(1/4) X M30,

[M2

(20)

(21)(XY) = f dt 5 d7ei2 r(x+'7Y)F(,7)) = (9/4)M,.

= f dJ I d?7ei27rx+t7y)

X E E (-i2r)j+k M j!kj=O k=O Jlk!

(16)

[The three remaining equations are obtained by interchangingj = k in Eq. (20). Note in particular we did not get a 10 X 10coupled set of equations, but instead a number of smaller di-mensional coupled sets. We immediately display the answerfor g(x,y)!]

922 J. Opt. Soc. Am., Vol. 70, No. 8, August 1980

13 00

1 920

9

1 902

5

I- 9 I 09

1 9.90 =15

1 912

1

Michael Reed Teague 922

16g(x,y) = (14Moo - 15M 20 - 15Mo2 )+ (90M,0 - 105M 30 - 45M,2 )x

+ (90Moi - 105Mo3 - 45M2 1)y+ (-15Moo + 45M 2 0 )x 2 + 36M,,xy+ (-15Moo + 45MO2 )y 2 + (-105M,0 + 175M 3 0 )X3

+ (-45Mol + 135M 21 )x2

y + (-45M,0 + 135M12 )xy2

+ (-105Mol + 175Mo3)y3 . (22)

If we wanted g(x,y) to match the moments of f(x,y) onlythrough second order, Eq. (22) would still be correct if all thethird-order moments were set to zero on the right-hand sideand, in addition, all terms of degree three were dropped.Conversely, if, for example, fourth- and fifth-order momentswere being matched also, then the coefficients already givenin Eq. (22) would change and also there would be additionalterms of degree four and five.

In the general case, one immediately infers that by solvinga linear coupled set of equations one can always exhibit acontinuous function that produces moments matching a finiteset of given moments. In Sec. II C, it will be obvious that themethod recreates exactly the original function f (x,y) as in-creasingly more moments are matched, within the limitationsof the well-known Gibbs phenomenal2 at jump discontinuitiesof the original intensity pattern. The disadvantages of thisdirect approach are (i) the coefficients already determinedchange as more moments are included, and (ii) one must solveincreasingly large coupled sets of equations.

C. Method of orthogonal momentsIn the language of orthogonal functions, we notice that the

definition of the general moment

Mik = ff dxdyf(x,y)xjyk

has the form of the projection of the irradiance function f (x,y)onto the monomial xiyk. Unfortunately, the basis set lxjykhwhile complete (Weierstrass approximation theorem), is notorthogonal. We now show that by introducing the notion oforthogonal moments it is possible to make a "best" recon-struction of f (x,y) in the sense of Sec. B, given only a finite setof its moments. Such an approach is suggested by remarksin Ref. 13.

We need a set of basis functions, orthogonal over a finiteinterval and in the present case it is natural to choose theweighting function to be unity. This uniquely specifies theLegendre polynomials as the appropriate set. At this pointwe stress that we consider x and y to be dimensionless vari-ables, giving the location of a given pixel in the image space,and not dimensional distances. Thus we may freely add x andx2, for example. Then, since f(x,y) is assumed (piecewise)continuous over the basic interval, we can write

f(x,y) = Z Z XinnPm(X)Pn(y), (23)m=O n=O

where Pm (x) is the usual Legendre polynomial'4 with nor-malization p1 2

dX Pm (x)Pm' (x) = 1mm'. (24)f 1 2m + 1

The orthogonal (Legendre) moment of f(x,y) is definedby

(2m + 1)(2n + 1)XMM J 4 f dx dy f(x,y )Pm(x )Pn (y). (25)

We now show that the set IXm, I is simply related to the usualmoments IMmn, . The Legendre polynomial may be written

Pm(X) = Z CmjXJi (26)j=()

where the coefficients C,,j are given in Ref. 14.

Then, inserting Eq. (26) into Eq. (25), we get the desiredconnection

(2m + 1)(2n + 1) n nn = Z Z C,,jChkMjk. (27)

4 j=0 k=()

Thus, for example, Xoo = Moo/4, Xlo = (3/4)M10,Xoi = (3/4)Mol, X20 = (5/4)[3/2 M 20 -(1/2)Moo],

11 = (9/4)M11, X02 = (5/4)[(3/2)Mo2 - (1/2)Moo], etc.It is straightforward to show that the usual moments dependat most on the Legendre moments of the same order andlower, and conversely.

Now the infinite series expansion in Eq. (23) is basically asummation by infinite rows and in fact is never used. Instead,one sums along finite diagonals and, truncating at a given fi-nite order Nmax, we have

f(x,y) - XN-n,nPN-.(X)Pn(Y)- (28)N=O n=0

This is the basic equation of the method of Legendre mo-ments. The coefficients Xmn, (m = N - n), are final; if Nmaxis increased, the previously determined \,,, do not change.In addition, not only do the Legendre moments Xmn of theright-hand side of Eq. (28) match those of the original imagedistribution f(x,y) up to a given order Nmax, but so do theusual moments Mjk match to the same order. In fact, to agiven order, the right-hand side of Eq. (28) and the functiong(x,y) of Sec. II B are identical; the terms have just been re-grouped. The big advantage of the present approach is thatto get the Legendre moments, we need not solve any coupledalgebraic equations. We just calculate them from Eq. (27)using known Legendre polynomial coefficients.

A possibly unexpected feature is that, in general, the mo-ment matching function, such as g(x,y) above, is not neces-sarily positive definite, even if the original function f(x,y) was.If the lack of positive definiteness of the approximatingfunction g(x,y) is objectionable in certain applications, thereis a way to get around this feature. Since f (x,y) > 0, one canwork first with V/f (xy), which is real, and find g(x,y), whichmatches the moments of V/f(x,y). Then, one uses g 2(x,y) asa positive definite approximation to f (x,y).

In Figs. 2 and 3 we show examples of the reconstruction ofan image from its moments. In both cases the input f (x,y) isa binary-valued figure which for illustration purposes we havetaken simply as capitalized letters. For convenience we tookthe image plane to be a 21 X 21 pixel array (then the step sizeis 0.1). Following the input irradiance in each figure, we showthe reconstructed images obtained by including increasinglyhigher-order moments. Actually, the reconstruction functiong(x,y) is continuous across the image plane and what we haveshown in these figures is a thresholded version of g(x,y) tofacilitate viewing the results. As expected, the second-order


FIG. 2. Reconstruction of the letter E. Top row, left to right: original inputimage, reconstructed image with up through 2nd-order moments, up through3rd order, up through 4th order. Succeeding rows in left to right order showeffect of adding up through 5th-order through up through 16th-order mo-ments.

reconstruction is simply a quantized version of an ellipse. InFig. 2 the reason odd moments are nonvanishing is that theinput figure was intentionally not centered horizontally. Itis perhaps surprising that in both cases, the inclusion of third-,fourth-, and fifth-order moments has little effect on the"quality" of the reconstruction, and one begins to see the inputshapes emerging only with the inclusion of higher-order mo-ments. In Table II we have shown the average pixel error inthe reconstruction function, i.e., the average pixel error be-tween the continuous g(x,y) (not the thresholded version) andthe input function f(x,y). In both cases (Figs. 2 and 3) thethresholded version of the reconstruction does not changeappearance if increasingly more moments are included;however, the average pixel error steadily decreases as moremoments are added. The fact that the average pixel errorremains relatively large is simply a reflection of the fact thatwe are fitting a 441-pixel image with a much smaller numberof moments.

The point of this section and the examples just discussedis to show explicitly that moments may describe an image orany other set of two-dimensional data as well as is needed fora given application. In particular, while second-order mo-ments contain only the simplest estimates of size, shape, andorientation of the image data, a larger (but still relativelysmall) finite set of moments may characterize an image ex-tremely well.

In addition, we will see in Sec. III that moments havesymmetry properties that allow a very economical numericalencoding of the image, or alternatively furnish a basis forautomatic pattern recognition schemes.

III. SYMMETRY INVARIANTS AND ZERNIKEMOMENTS

When a symmetry operation is applied to an image, its ir-radiance function changes

F(x,y) - P(xy),

.... ... .... ..............~~.. .... .... .. .. .. ..

...... ....... .. ......

FIG. 3. Reconstruction of letter F. Description of Fig. 2 applies exceptthe effect of adding up through 2nd order through up through 15th order isshown and last entry includes all moments through 18th order. The imagefor 16th and 17th order were very similar to that for the 15th order.

and correspondingly the moments change

MJ, MIe.

However, for the simplest operations, namely, size changes,translations, rotations, or reflections, it is possible to "nor-malize" the set of moments so that they are invariant to theseoperations.

A. Translation and range invariance of momentsA transverse motion of the object with respect to the system

optic axis produces a displacement of the image in the xyplane, while object motion parallel to the optic axis producesa change in image size.

TABLE II. Average relative pixel errors in thefunctions of Fiqs. 2 and 3. Errors are in %.

reconstructed irradiance

Maximum order ofmoments included E F

2 133 1493 132 1494 103 1185 102 1196 90 1027 91 1018 87 989 83 93

10 81 9211 74 8112 67 7513 61 6514 54 6015 47 5616 46 5117 40 4818 36 4319 33 3920 27 33


........ :::::: ....

................................


In this section we recall how to define image moments thatare invariant to these two transformations.

The coordinates of the image centroid are

x = Mlo/Moo, (29)

y = Moi/Moo

Central moments jz, are defined with respect to the imagecentroid, and it is straightforward to show that the connectionbetween central and raw moments is

Pik = Es Y \I§ ()(r=o s=o r sThe gjh directly contains information on the size and shapeof the image pattern without having a lever arm folded intothe numbers, and one sees that a given central moment de-pends only on usual moments of the same order and lower. Itis obvious that central moments are invariant to imagetranslation. Also note that

Ilo = 0o1 = 0 (31)

always. Thus to normalize away translation effects, one workssimply with central moments. If the object changes range,the image changes size, and we shall assume the imaging sys-tem is such that the new irradiance function is

f/'(x,y) = f (X/X,Y/X), (32)

where X is a scale factor

X = RIR' (33)

equal to the ratio of object ranges. Under this transformationthe moments change

Ppq= ff dx dyf(x/Xy/X)xPy = X2 +P+qIpq, (34)

and it is clear that the normalized moments

Apq1(A00)(p+q+1)12 (35)

are invariant to size (range) changes. This normalizationcorresponds to having the total image power goo always equalto unity. Such a normalization for size change is, of course,not unique. An alternative choice corresponds to making the(unnormalized) radius of gyration (P20 + g02)1/2 equal to unity.Then, the general moment must be normalized by ,pA/(i20+ 1 02 )(2 +P+q)/4

B. Rotation properties of momentsTo arrive at a set of moments "normalized" with respect to

rotations is more involved. It is straightforward to show thatif the image rotates through an angle q5, the moments changeaccording to

YE (jl (kIr=O s=O r s/

X (sin0)k+r-s(i.j+k--r-s r+s), (36)

and we see that the set of moments Plik of given order N = j+ k transform into the set PAjk of the same order N = j' + k',as indicated in Eq. (45). This incidentally shows that allmoments of a given order must be treated on equal footing.

Equation (36) is the basis of the work of Hu5 and the ap-plications of Dudani et al.6 Using the nineteenth century

methods of algebraic invariants, Hu derived rotation invariantcombinations of moments. We now show a much morestraightforward approach to this problem.

As is mentioned in Sec. II, the moments uM( are basicallyprojections of the function f(x,y) onto the nonorthogonal basisset JXpyq , and the lack of a simple transformation law of theusual moments Apq is a reflection of the fact that the basis sethas no simple rotation properties. This suggests that newmoments should be defined with respect to a set of functionswith simple rotation properties. This will allow us to con-struct in a straightforward manner algebraic combinationsof moments that are rotationally invariant.

C. Definition of Zernike momentsA set of orthogonal functions with simple rotation proper-

ties and with weight function unity are the well-known Zernikepolynomials

V,1(x,y) = V, 1(psinQ, pcos6) = R, 1(p) exp(ilO). (37)

(We use the notation and normalization of Ref. 15.) Thesefunctions are complete over the unit circle and satisfy therelation

if dXdy[VnI(xy)*Vmk(xy) = 7/(n + 1)Wmnnkt, (38)

where now the integration is over the unit circle x 2 + y

2 < 1.

Moreover, we shall always consider only images that vanishoutside the unit circle, and it will be clear from contextwhether the integration region is over the unit circle or thesquare defined by Eq. (13), or whether in fact it does notmatter which region is chosen. Note also that 0 is the anglebetween radius vector p and the y axis, while + is the anglebetween p and the x axis.

Subject to the limitations discussed previously for Legendremoments, the image irradiance function may then be ap-proximated by

f (x,y) = Z Y A,21 VnI (p, f),fl I

(39)

where n = 0, 1, 2_ .. ., , and I takes on positive and negativeinteger values subject to the conditions:

n-ll iseven, III <n. (40)

The complex Zernike moment is by definition

Ant = [(n + 1)/1w] J dx dyf (x,y)[VnI (p, 0)]*

= (A,,, 1 )*. (41)

(The second equal sign holds because f is real and Rn,-

= Rnl- )

The function Rn1 (p) of Eq. (37) is real valued, and since theimage intensity function f (x,y) is also real, it is often conve-nient to work with real expansions and real-valued Zernikemoments. The real expansion corresponding to Eq. (39) is

f(x,y) = Y _ (C n cosl1O + Sn1 sinlO)Rni(p),n l

(42)

where I now takes on only positive integral values subject toEq. (40).


The real Zernike moments are

rf] = [(2n + 2)/7r] j*,fdx dyf (xsy)R.1 (p) coslo] (43)[Sf I sinlO ' 43

for I not zero and

C,2o = Ano = (1/7w) Jf dx dyf (xy )Rno(P) (44)

while

SIZO = 0.

[In the diffraction theory of aberrations, only the cosine termsin Eq. (42) occur for axially symmetric systems. We have inthe present paper need of the general case.]

The connection between real and complex Zernike momentsis (1 > 0)

Cl/ = 2 Re(An,),

Sl = -2 Im(A,,,),A,, = (C,,l - iSn,)/2 = (A,,,-,)*. (45)

D. Relation between Zernike and the usual momentsThe Zernike moments are related to the usual moments g,.

Thus once Ap,( have been calculated by integration (and nor-malized), we have the Zernike moments also, although inpractical applications it is much simpler to calculate theZernike moments directly from their definition Eq. (41).

The radial polynomials have the form

n

R,,, (p)= E B,, , p (46)Ie = 1

where n-k is even and the coefficients IB,,k I are given in Ref.15. Using Eq. (46) and Eq. (41), one gets

A,= [(n+ 1)/r] LLE( i) (I) B1h=1 j=o 2=o 0 m

X Ahk-2j-l+nz, 2j+1-tn, (47)

where q = (k - 1)/2. For reference purposes we quote thefollowing cases of the last equation:

Ao( = 1(o/7r= 1/7r,All = Al- = 0,A 22 = 3 (102-1120-2iu I l /7r,A 2 0 = 3(2A20 +

2AO2 - 1)/w,

A:I:i = 4[,11:3 - 3121 + i(A:1o - 3/]2)/7r,

A:, 1 = 12[po:j + 1121 - i( 1 3 () + 111 2)1/7r,A,,z = 5[f4() - 6122 + 11N4 + 4i(: 113 - pi)]/7r,

A 4 2 = 514 (yo,.(- 14)) + 3(1120 - P2)

-2i[4(11:11 + p11) - 3g,11]1/7r,A.1( = 5[6(Ato + 21122 + 104) - 6(1120 + 102) + 1]!

E. Behavior under rotationConsider a rotation of the image through the angle 0.

the passive viewpoint of symmetry operations, this wcorrespond to transforming to new coordinates:

x' = x cosOo - y singO,

yI = x sin0o + y cosO0.

Instead of this viewpoint we use only one set of image pcoordinates and rotate instead the image: the original inirradiance function f (p, 0) becomes after rotation


f'(pO) = (P,0 - 0o),

and the new Zernike moment is

(A,,,)' = [(n + 1)/7r] Jfpdpdff(p,) - 0(X)

(50)

x R,I,(p) exp(-ilf),

which after manipulation becomes

(A,,,)' = A,,, exp(-i10(). (51)

Thus the Zernike moments have simple rotational transfor-mation properties; each (complex) Zernike moment merelyacquires a phase factor on rotation. From Eq. (51) and Eq.(45), it is straightforward to find that the real-valued Zernikemoments transform as

(C,1,)' = C,,, cosltO(- Sn, sinl(O,

(SIl)' = C,,, sinlO( + S,/l coslOo.(52)

IThe simplicity of the Zernike moments's transformationproperty [Eq. (51) and Eq. (52)] under image rotation shouldbe contrasted with the behavior of the usual moments asshown in Eq. (36).1

F. Behavior under reflectionA true invariant (scalar) is unchanged by a reflection while

a pseudoinvariant (pseudoscalar) changes sign. This meansan object and its mirror image will produce the same true in-variant, but they may be distinguished if one looks at pseu-doinvariants.

Consider the general case of reflection across a line throughthe origin, rotated through positive angle Oo with respect tothe y axis. The image point (xy) then becomes the imagepoint (x',y') where

x' = -x cos2O( - y sin2O(, (53)y' =-x sin2O0 + y cos2O(.

Note that this transformation P is self-inversive, P2

= 1 andhas determinant =-1. The fact that this transformationdepends on 200 is a consequence of the reflection line havingno unique direction: it is unchanged by 1800 rotation.

The image intensity function f(x,y) is changed by the re-flection to

f'(x,y) = (X ,y'),

(48) and corresponding to Eq. (27) we have

p' = P.0' = 20o - 0.

(54)

(55)

The new Zernike moment is

(A01)' = [(n + 1)/7r]wJ fdxdyf(xy) Rnl(p)

X exp[-il(200 - 0)]

or

(A,,,)'= (A,,,)* exp(-i2lOO), (56)

and corresponding to Eq. (53) the real Zernike momentstransform as

(Cnl)' = C,,l cos21J0 + 5,,, sin210 0,

(Snl)' = +C,,l sin21O0 - S,, cos2 100.


G. Construction of invariants and pseudoinvariantsCombinations of image moments may be determined that

are independent of the object's size, lateral displacement, andorientation. The first two properties have already been takencare of by the use of normalized, central moments. We nowconstruct quantities invariant under rotation and reflec-tion.

We work with complex Zernike moments

A,,,= IAn1I exp(ioanl), (58)

and the basic results being applied are the rotation behavior[Eq. (51)] and the reflection behavior [Eq. (56)] of the Zernikemoments. (We also wish to make sure that the invariants weconstruct are real valued.)

The first two true invariants are Aoo and All A ,-I = IA a12,but these are trivial [same values for all images; see Eq. (48)]and will not be counted.

In second order we have the two true invariants

SI = A 20,

S2 = A22A2 ,-2 = IA2212,

which are unchanged under any orthogonal transformation,proper or improper.

In third order we have to work with the four moments A3:3,A:31,A:3,-1,AA:,-:3. The following two true invariants are im-mediately constructed:

[Taking instead a minus sign in the first line of Eq. (65) wouldgive another pseudo-scalar, but we avoid listing furtherpseudoscalars for now.]

At this point we note the important fact that 'given S3through S6 we cannot solve for the four third-order momentsbecause 022 also appears. There will always occur one toomany phase angles to allow one to solve for the moments, giventhe invariants. This, however, just corresponds to the onedegree of rotational freedom of the image plane. That is,while we are constructing valid invariants, we cannot actuallyinvert the invariant defining equations to find the momentsuntil we have specified the target orientation in the imageplane, which we can do, for example, by specifying 022. Theseremarks are crucial for determining whether a set of invariantsthat one has constructed are functionally independent. Forexample, instead of S 6, one could have used

(A 3 3) 2 [(A 2 2 )*] 3 + c.c. = 2IA 3 312IA 2 2I3 cos(2033 - 3022)

as a true invariant, but not (usefully) both this and S6, as itobviously is already determined by the preceding invari-

(59) ants.

(60)Now consider the fourth-order moments A44, A42, A40,

A4,-2, A4,- 4. Again we immediately have the invariants

S 7 = IA4412,

S8= IA4 2 12

,

S9 = A40.

(66)

(67)

(68)

S:3 = IA3 31 2,

S4 = 1A3112.

The maximum possible number of independent third-orderinvariants is obviously four because, at this point, given theinvariants, we could solve for the moments. So we should seektwo more third-order invariants. One might think a termsuch as

A:33(Ax,-i):1 = A3:3[(A 3 1)*]:

would serve as an additional invariant. It has two short-comings: While it is rotationally invariant, under an impropertransformation, it changes to

(A33:)* (A31)3.

In addition, both of these terms are complex valued. How-ever, a third true, real-valued, scalar is obviously the sum ofthese two:

Ss = A:3 4[(A:31)*]3 + c.c.

= 21A33: 1AA313cos(b 3 3 - 3031), (63)

where c.c. is the complex conjugate of the preceding term. Weget the first pseudoscalar by using the difference

P1 =-iJA3 3 [(A3 0)*1 3 -c.c.1

= 2IA33i IA3 1I3 sin(k 33 - 331)- (64)

No further independent invariants exist using only third-ordermoments. However, by combining third- and second-orderinvariants we do get another independent invariant:

S 6 = (A 3 1)2

(A 2 2)* + c.c.

= 2IA31 2A22 cos(2031 - 022). (65)

(61) The remaining invariant consisting only of fourth-order mo-ments is

(62)Sio = (A 44 )*(A4 2 )

2 + c.c.

= 21A 441 tA 4212 cos(0 44 - 2042), (69)

and as a fifth independent invariant we can choose

Si, = A42(A22)* + c.c. = 21A 421 IA221 COS(042 - 022). (70)

For the fifth-order moments A55 , A,53, A51, A5 ,- 1 , A5,- 3,A5 ,-5, we seek six invariants. We get three easily:

S 12 = IA5512,

S 13 = IA5312

,

S 14 = IA51 12

.

(71)

(72)

(73)

For the first time we can construct three more invariants usingonly the fifth-order moments (and these we might expectcomplete our set of six invariants). They are

(Ass)*(A 53 )5 + c.c. = 2IAss53IAssI5 cos(305ss -553)

(Ass)*(As5)5 + c.c. = 21A 551 IA51l 5cos(kO 55 - 5)51),

(As3)*(A 5 1)3 + c.c. 21 A5 31 IAsl13 cos(0 5 3 - 3051).

These are six true invariants, but any of the last three isfunctionally dependent on the other five. To see this, we tryto invert the six equations to find the six fifth-order moments.We immediately get three moduli from Eq. (71) through Eq.(73), leaving the last three equations to get the three phases.Inverting the cosines, we get equations of the form

355 -5053 = a,

055 - 5051 = b,

¢>53 - 3451 =C,


where a, b, c are known quantities. This set has zero deter-minant showing that one cannot solve for the phases and,hence, the six invariants were functionally dependent. Again,this is just a reflection of the fact that the image orientationhas not been specified, and one cannot find the phase of anyAnl (1 5z 0) until, e.g., 022 has been specified.

Note also that it is a feature of this method that while weuse nonlinear methods to construct invariants, in order tocheck for functional dependence of the invariants, one hasonly to consider a linear algebra problem in the phases as wasdone above. This is much more tractable than the generalmathematical requirement to consider a Jacobian determi-nant to check functional invertability.

To get the needed three remaining fifth-order invariants,we pick

S15 = (As1)*A31 + c.c., (74)

S16 = (A53 )*A3 3 + c.c., (75)

S17 = (A55)*(A 31)s + c.c. (76)

These choices are made for later convenience. As a generalrule, one should build invariants using the lowest possiblepowers of the Zernike moments. In Appendix A we expressthe preceding invariants in terms of the usual normalized,central moments Aph, and the resulting expressions are simplerif we avoid high powers of the Zernike moments.

Finally, it is obvious that wherever we constructed a true,real-valued invariant by taking a quantity plus its complexconjugate, we could have obtained the corresponding, real-valued, pseudoinvariant by taking -i times the quantityminus its complex conjugate.

The reader will now understand how to construct inde-pendent sets of Zernike moment invariants to arbitrarily highorder and between the text and Appendix B, we have listedresults up through the eighth order.

IV. APPLICATIONS OF MOMENT INVARIANTS

In Figs. 4(a)-4(e), we show the Zernike moment invariantsfor several simple input images. Again, a 21 X 21 pixel imagespace was used. The numbers plotted are actually statisticalaverages of the numerically calculated Zernike moment in-variants when the input image was rotated through 360° atfinite steps. If there were no quantization errors associatedwith rotating the image and no numerical errors associatedwith the numerical integrals necessary to obtain the Zernikemoments [which were calculated directly from their definingequation (41)], then the Zernike invariants would be trulyinvariant as the image rotates. The error sources mentionedproduce standard deviations of from a few percent to 10 or 20percent of the associated average values. In all Figs. 4(a)-4(e),we have calculated 43 quantities. The first 41 are the in-variants S, through S41, which include all Zernike invariantsthrough eighth order. The last two entries, the 42nd and 43rd,are the two pseudoinvariants Pi and P2 from Appendix A.While one pseudoinvariant is in principle sufficient to identifya mirror image, it is better to calculate several in case some arefortuitously numerically small and buried in noise.

It is obvious that for this set of calculations, an image isidentified as a point in a 41-dimensional space. (The last two

quantities which are pseudoscalars serve only to identifywhether one has an image or its mirror image and are notfunctionally independent of the remaining 41 true scalars.)However, for visual comparison purposes, we plot in Figs.4(a)-4(e) the numerical values of the components of the 41-dimension image vector. There is no significance to theconnecting lines, the Zernike invariants exist only at integervalues.

The first invariant S, measures the normalized size of theimage, i.e., the radius of gyration of the irradiance distributionwhen the total image power is normalized to unity. Thesecond moment S2 measures the skinniness of the distribu-tion. It is zero for a circle or a square. These two invariantsalone were used by Hu5 for alphabetical character recognitionschemes, while Dudani et al. 6 also included S:3-S 6 in aircraftrecognition schemes. (Actually, they used equivalent momentinvariants not based on Zernike moments.) The number ofsuch invariants that must be used for pattern recognitionpurposes obviously depends on the complexity of the class ofrecognition images. For example, the first two invariants maysuffice to recognize alphabetical characters. However, theresults of Sec. II indicate that if one wishes to reconstruct fromtheir moments images of the same general complexity as al-phabetical characters, then even including eighth-order mo-ments is not expected to be sufficient.

We now point out that the Zernike moment invariantsfurnish an optimal means to code the essential features of animage pattern, i.e., they contain information about the imagethat is independent of the image size, centroid position, andrelative angular orientation. The missing features, if needed,are supplied by the second- and lower-order moments.Therefore, given the Zernike invariants, one can solve for theZernike moments and, using the techniques of Sec. II, recon-struct the image to a given order of accuracy. If only Zernikeinvariants are used (and not the information in second- andlower-order moments), then one will get a normalized recon-struction, i.e., one with standard size and one which is centeredand unrotated.

The moment invariants furnish a natural method for therecognition of two-dimensional patterns and at this point therecognition problem becomes a statistical one; for example,how should one define a weighted metric in an n-dimensionalrecognition space-a difficult problem discussed in Ref. 6. Atpresent, if moment methods are used for three-dimensionalobject recognition, one must consider different views of a givenobject as different two-dimensional patterns to be recognized.Thus the recognition space is enlarged to contain informationabout all the relevant Euler angles of the set of recognitionobjects.

As mentioned earlier, image moments may be calculatedwith a coherent optical processor by using the Fourier trans-forming properties of a converging lens. 16 Moments are thendetermined as partial derivatives of the image Fouriertransform evaluated at the origin of Fourier space. The dif-ficulty here is that one cannot simply place a detector arrayin the Fourier plane since both the real and imaginary partsof the Fourier transform must be measured and not just itsmodulus. The details of an optical arrangement to performthe necessary calculation are being reported elsewhere.' 7

An alternative approach to calculating irradiance moments


In U , --

7:.. FIG. 4. Zernike moments invariants for: (a) a centered, constant irra-

diance, rectangle of 5 by 19 pixels; (b) a centered 5 by 13 pixel rectangle, . with a small 3 by 3 pixel square sitting on the left-hand end of the big rec-

tangle; (c) the letter E (same as Fig . 2); (d) the letter F (same as Fig. 3); (e)Lr, ,- - Athe letter C. For these figures the Zernike polynomials were normalized

. / \ Ato unity inside the unit circle rather than using the norm of Eq. (38).

z Fr l ,\ /;\, ,,I l,,l k > X , ,Xusing optical processing techniques has recently been de-scribed by Casasent and Psaltis.

- ACKNOWLEDGMENT

-3 . . aThe author expresses his appreciation to Lawrence Sher,(b) Paul Merritt, and Carey O'Bryan for their support, sugges-

dtions, and critical comments (regarding this paper.

3~~~wt a sml 3 by 3 pie sqar sitn on th lethn en of th bi rec- llllll llllll

APPENDIX A: ZERNIKE MOMENT INVARIANTS

2. . EXPRESSED IN TERMS OF USUAL MOMENTS

7: I

g o l p g t e hs r Second Order

- /\ ASI A20 =3[2(920 + A02)-1/

J S2 c A22r 2 s9[(s20n a 2)n 2 + 4(s1)2a/t7i2

tn a c commentsThird Order

Z 2 ,S A,3:312 =1[o-32)2 + (pro - 3y12)21/7r2,

ES4 NIA3 :2 Z144[N3 + N2A ( R30 + A12 NT

S5= (A33)* (A31)3 + C.C.

C 7 S:) 18 24 [(p 3 - 321)(10)3 + (21)

N S, =,,,,, IA,,12 =144[( X [(203 + ( 21 )2 - 3(,30 + 12/)2

-(130 - 31112)(130 + 1112)X [(o30 + A12)2 - 3(103 + A21) 2]1,

S6 = (A 3 1) 2A 2 2 * + c.c.

z864 22cz' /\ = - {(1o02 - 120)[(9103 + 121)2 - (130 + 112)21

Li / V + 41111(103 + 121)(1130 + 1 12)b

z A Fourth Order

-S7 1A4412

= 25[(A40 - 61122 + 1104)2 + 16(1131 -A13)2]/72,

(d- S 8 = IA4 2 12= 251[4(0o4 - 140) + 3(1120 - 102)12

+ 44(1131 + 113) - 3Aj1j12]/w2,


l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l lI I


S9 = A,,()= 5[6(p,(4 + 2122 + AN) -6(120) + 1102) + 11/7r,

SI( = (A4,1 )*(A,12)2 + c.c. = ((/1() - 6A22 + 1104)73

X 4(t(,.i - ,1.(1) + 3(1120 - /o 2 )]2 -4[4(j3u1 + t1i:i) -3,1 1]21- 16[4(,u(; - 1140) + 3(1U20 - 102)]

X [4(,: I + j 1:1) -3t II] (j:u -j1:3)),

S1 I = A 12(A,,)* + c.c.

3 2 [4(y() - 11.() + 3(920 - 102)1(102 - 120)

+ 4,u11[4(y:1 1 + y 1:j) -3,41

The following are pseudo-invariants:P1 =-i(A,* A:,, - c.c.)

= 13824 (ju:1( - 3A112)(1103 + 121) [(90:3 + /121)2

-3 (930 + 12 )2]- (10:3 - 3,21)(43:0o + 112)X [(1:30 + A112)2 -3 (A03 + A21I)2]/7rt,

P2 -i[(A:1 )2

(A 2 2 )* - c.c.]17281 1 1[( (y1:, + 1121)2 -(1:i3 + 112)2]

-(,o:3 + P21)(3:0O + /112)(K02 - 1 20))$/Wr:.

APPENDIX B: HIGHER-ORDER ZERNIKE MOMENTINVARIANTS

For reference purposes we list further independent momentinvariants. Between this appendix and the preceding text,results are given for all cases of the Zernike polynomial tablesof Ref. 15.

Sixth OrderS18 = IA 6 61

2,

Sig= JA64 12,

S 2 0 = IA6212,

S21 = A 6 0 ,S22 = (A6o)*(Aa3:3) 2

+ c.c.,

S2 3 = (A 6 ,)* A 4.1 + c.c.,

S 2 4 = (A 6 2 )* A 2 2 + c.c.

Seventh OrderS 25 -

S2 6 =

S2 7 =

S 2 8 =

S 2 9 =

S:10 =

S31 =S:32 =

IA 7 7 12,

A 7 12,

IA 7 1 12,

(A77)*(A:3 1)7 + c.c.,(A75)* A55 + c.c.,(A7 3)* A:1:1 + c.c.,(A71)* A:31 + c.c.

Eighth OrderS:3:3 = IA 8812,S:j.1 = IA86 1

2,S:35 = IA8412,S:36 = IA8212,S:37 = A 8 0 ,S ta = (A88) * (A 44)2 + c.c.,S:io = (A86)* A66, + c.c.,S.,o = (A84)* A441 + c.c.,S.1 I = (A82)* A 2 2 + c.c.

* Portions of this paper were presented at the 1979 Annual Meetingof the Optical Society of America.

t Present Address: The Charles Stark Draper Laboratory, Inc., MailStation 84 555 Technology Square, Cambridge, Mass. 02139.

'R. A. Gonsalves, "Phase retrieval from mnodulus data," ,J. Opt. Soc.Am. 66, 961-964 (1976).

2W. H. Southwell, "Wave-front analyzer using maximum likelihoodalgorithm," J. Opt. Soc. Am. 67, 396-399 (1977).

:'S. R. Robinson, "On the problem of' phase from intensity measure-ments," J. Opt. Soc. Am. 68, 87-92 (1978).

4A. ,J. Devaney and R. Childlaw, "On the uniqueness question in theproblem of phase retrieval from intensity measurements," eJ. Opt.Soc. Am. 68, 1352-1354 (1978).

5M. K. Hu, "Visual pattern recognition by moment invariants," IRETrans. Inf. Theory IT-8, 179-187 (1962).

6S. A. Dudani, K. J. Breeding, and R. B. McGhee, "Aircraft identifi-cation by moment invariants," IEEE Trans. Comput. C-26, 39-45(1977).

7 R. 0. Duda and P. E. Hart, Pattern Classification and SceneAnalysis (Wiley, New York, 1973).

8N. Bareket and W. L. Wolfe, "Image chopping, techniques for fastmeasurements of irradiance distribution parameters," Appl. Opt.18, 389-392 (1979).

9M. R. Teague, "Automatic image analysis via the method of mo-ments," Laser Digest, Summer 1979, p. 25-43, AFWL, KirtlandAFB, New Mexico (unpublished).

'0 See, for example, N. I. Akhiezer, The Classical Moment Problemand Some Related Question in Analysis (Hafner, New York,1965).

1 A. Papoulis, Probability, Random Variables, and Stochastic Pro-cesses (McGraw-Hill, New York, 1965).

'2A. Sommerfeld, Partial Differential Equations in Physics (Aca-demic, New York, 1949).

':A. D. Myskis, Aduanced Mathematics for Engineers (MIR, Moscow,1975).

"'1R. Courant and D. Hilbert, Methods of Mathematical Physics, Vol.I (Interscience, New York, 1953).

15M. Born and E. Wolf, Principles of Optics (Pergamon, New York,1975).

16J. W. Goodman, Introductiort to Fourier Optics (McGraw-Hill, SanFrancisco, 1968).

17M. R. Teague, "Optical calculation of image moments," Appl. Opt.19, 1353-1356, (1980).

18D. Casasent and D. Psaltis, "Optical pattern recognition usingnormalized invariant moments," SPIE Proc. (to be published).


image analysis via the general theory of moments*

Documents