Proceedings of the 2007 International Conference on Information AcquisitionJuly 9-11, 2007, Jeju City, Korea

A Fusion Algorithm of Remote Sensing Images Based onNormalized Correlation Moment of Characteristic Values

Linling Zhang and Lizhong Xu Fengchen HuangCollege ofComputer and Information Engineering College ofComputer and Information Engineering

Hohai University Hohai UniversityNanjing, Jiangsu Province, China Nanjing, Jiangsu Province, China

lzhxu@hhu.edu.cn snowball_dancinggyahoo.com.cn

Abstract - Now multi-spectral images and panchromatic of the original multi-spectral image. The other disadvantage ofimages are widely used in remote sensing. In this paper we this technique is that the number of bands participating inproposed an algorithm using normalized correlation moment of fusion of the multi-spectral image must be 3. The PCAcoefficient characteristic values based on principal component transform can fuse all bands of the multi-spectral imageanalysis and wavelet transform. The basic idea of this algorithm together with panchromatic image and preserve spectralis to transform the multi-spectral image into three principal . -components at first, then perform wavelet decomposition of the proprmties bettr, out orithmcaue the loss pectralfirst component image and the panchromatic image separately. information on account of replacing the first principalUsing normalized correlation moment fusion formula to select component of multi-spectral image by high-resolution image.the wavelet coefficient of high frequency component in different The algorithm needs to calculate Eigen value and Eigenresolutions. Finally, by performing inverse PCA transform to get vector, which will cause more workload and poor real-timethe fused image. We compared the fused image using this property. The wavelet multi-scale decomposition preservesalgorithm with the results of PCA, DWT and other familiar spectral properties better in low decomposition level, but itmethods, computer simulation shows that the fused image using will cause blocks in the fused image. The blocks willthis algorithm obtain better results in terms of both preserving disappear with the number of decomposition levelsspectral information and improving spatial resolution of the graduallOriginal multi-spectral images than the others. increasing, but that will cause more and more serious losing of

spectral information at the same time. How to make full use ofIndex Terms -Remote Sensing, Image Fusion, PCA all the advantages of popular methods, some specialists and

Transform, Wavelet Transform, Normalized Correlation Moment. scholars have proposed lots of improved methods in recentyears, for example, the IHS transform combined with wavelet

I. INTRODUCTION transform, the PCA transform combined with wavelettransform [5], and so on.Along with the constant development of remote sensing the detaild usions

techniques, both the spectral and spatial resolutions of remote Tr m combined with w e transorma as ollowssesnimge hav bencnieal.mpoe.Btmg transform combined with wavelet transform are as follows:

sensingprovimagesd bya lesen nsorider itablyimprot.But, img. Firstly, the multi-spectral image and the panchromaticdataproidedbysinle ensr invitbly as imittios, mage should be registered to each other, to an error smallerUsually, TM images have rich spectral information and thane pixel.relatively lower spatial resolution (30m); however, SPOTimages have high spatial resolution and lower spectral Secondly, execute PCA transform on the multi-spectralresolution (10m). To effectively utilize the high spectral image to get the Eigen values and their Eigen vectors, and

resolution of multi-spectral images and the high spatial then obtain all the principal components.resolutionofpanchromatic images, image fusion techniqThirdly the PAN image and the first principal componentresolution of iacrmh mages, image fusion technique y, g

has been proposed in remote sensing image processing. The image should be matched the histogram.goal of image fusion is to match data provided from different Fourthly, the high-resolution image and the low resolutiontypes of sensors above the same area, combine the image are decomposed individually using the Mallatcomplementary information of this image data, and eventually algorithm, then each one's low frequency approximatecreate a new image [1]. components and high frequency detailed components will be

The IHS (Intensity-Hue-Saturation) transform [2], the obtained.PCA (Principle Component Analysis) transform [2][3] and the Fifthly, the detailed components of PAN substitutes those

W rein of the first principal component image. And retain the lowWavelet transform [4] are three popular fusion techniques frqec prxmaecmoet f h atroeimage fusion at present. However, all of them havedisadvantages. Method based on IHS transform usually results Sixthly, the wavelet coefficients of the multi-spectral

image are executed for inverse wavelet transform, and then

directly. That iS the color Of the fused image differs from that te nw picpl cmoet wl e cetd

1-4244-1220-X/07/$25.OO ©C2007 IEEE 465

Finally, the multi-spectral image getting from the resolution image. So, a novel fusion algorithm usingprevious step is executed for inverse wavelet transform, and normalized correlation moment of high frequency coefficientgets the final fused result. characteristic values based on the principal component

An outstanding problem is how to select fusion rules to analysis and wavelet transform is proposed in this paper.fuse for wavelet coefficients after decomposition when using Fig.1 shows the flow chat of the novel algorithmwavelet multi-resolution analysis on image fusion. This rule proposed in this paper. The detailed steps of the noveldecides the quality of the fused image, so selecting coefficient algorithm are similar with those of the PCA+DWT method,features according to which fusion rule is the key technique. but the selection rule of the high frequency coefficients in theThe normalized correlation moment proposed by Y. -F. Gu fifth step shows as follows[6] is defined as k if Mk >Mk

k =k ~~~~~~~~PC()-1S(,j 2Mi~~~~ "~~~ rj~ ~ ~ ~ ~ F(i,j) ~~CpcOSP(I,J)±OPC(IJ) i Mk(Q

mkL k F(iij)jSP(i,j)'Ti Where SP is the PAN image, PC I is the first principal

Where 'j iS the coefficient value of the No. i pixel in the component of TM image, F is the fused image, k = '2 3 isthe high frequency coefficient symbol of horizontal direction,

No. j window of wavelet image, Ais the mean value of No. I vertical direction and diagonal direction, , is the wavelet

window, i is the standard deviation of wavelet coefficients coefficient.

of all the pixels in No. j window, k =1,2,3 is the high III. EVALUATION STANDARDfrequency coefficient symbol of horizontal direction, vertical There is no uniform performance measure for evaluatingdirection and diagonal direction. The fusion rule based on image fusion in the recent literature. The fusion results arenormalized correlation moment proposed in reference [6] mostly evaluated subjectively or objectively. Subjectivetakes in account not only the whole coefficient property of evaluation means visual effects analysis. Objective evaluationlocal area, but also that of each pixel's in the window. So means to judge by statistical parameters of image. Entropythat's the reason why a novel fusion Algorithm using the and correlated coefficient are chose for evaluating in thisnormalized correlation moment of coefficient characteristic paper.values based on the principal component analysis and wavelet A. Entropytransform is proposed in this paper. Image entropy is one of the important indexes to weigh

II. THE FUSION ALGORITHM USING NORMALIZED image information abundance. It can be described asCORRELATION MOMENT OF COEFFICIENT CHARACTERISTIC L-1VALUES BASED ON THE PRINCIPAL COMPONENT ANALYSIS H=-Epi In pi. (3)

AND WAVELET TRANSFORM i. OWhere p1 is the ratio between the number of pixels whose grey

Accordingly, spectral distortions caused by the PCA values equal and the total number of pixels in the image. Thetransform are smaller than that of the IHS transform. Since the larger entropy, the more image information is increased in thefirst principal component determined by all bands fused image.participating the PCA transform, comparing with Intensity B. Correlation Coefficientcomponent, its other bands of spectral information reflect to -, f)x(g -,g)]corresponding components uniquely. But the spectral property giof the first principal component is not coincident with that of C= . (4)the SPOT. Replacing the first principal component image with I,j(f 1f)2]XJ(g/J kg)2]SPOT image will cause missing some spectral properties of iji'jTM, and result in color distortion. So in the improved Correlation coefficient between two images can be usedalgorithm of the PCA transform, we change the quomodo of to measure the degree of relevance. By comparing thesubstituting the first principal component of TM with SPOT correlation coefficient between the fused and the originalPAN by choosing appropriate method and merge the two image, we can see the change of the spectral information andimages together. Thereby the new first principal component of the spatial information of the original image. The smaller theTM contains both detailed information of TM and SPOT, difference, the more information is extracted from the originalwhich can improve the integrated property of the final fused image. The correlation coefficient can be used to moreimage in preserving spectral information and enhancing accurately determine the merits of fusion. The correlationspatial information [7]. Using the method based on PCA coefficient values are objective statistical parameters totransform combined with wavelet transform to fuse TM and measure the fusion effect.SPOT PAN, it is easy to result in ringing phenomenon due todirectly abandoning the low frequency components in the high

466

TM432 ( 30mn) SPOT PAN ( I0m)

Geometric Correction

TM432 ( tOrn) SPOT ( tOmf)

The PCA Transform

Th e othertbfph scom ponent s Histogrm mth pon SPO t Hm

The wavelet transform The wavelet transform

Low freqtency High freqtuency High frequency Low frequetncycomponents components components components

Merge by the normAlized correlation moment method

Ieverse wavelet transform

ThePAtf4ffffiffi§6dlainge

Fig. 1 Flow chart of the novel algorithm used in this paper

moment. DWT is discrete wavelet transform. PCA+DWT isPCA transform combined with wavelet transform.

To testify the effectiveness of our fusion scheme, we didsimulation experiments on TM and SPOT images of ERDAS8.7 software. All the fused results using IHS transform, PCAtransform, wavelet transform, normalized correlation momentmethod and PCA±DWT method is compared with each otherin Fig.2-Fig.9 as follows. The original TM image and SPOTimage have been strictly registered before be merged. The sizeof

.

orgia imge is 5 .15pie ThR, G,I-B bands of

Fig.2 TM image (Bands of 4,3,2). Fig.3 SPOT PAN image.

467

TABLE 11COMPARISON OF CORRELATION COEFFICIENT ABOUT FUSED RESULTS

AcCORDING To TM IMAGE

-U' ~ ~~~~~~~~~~~ ~~~~~~Band[ Fig.4 Fig.5 Fig.6 Fig.7 Fig.8 Fig.9 Ideal valueR 0.6334 0.7514 0.6407 0.8573 0.8604 0.8763 1

G0.8467 0.7546 0.7951 0.8551 0.8616 0.8778

0.7527 0.8063 0.6796 0.7962 0.8868 0.8929 1A. Entropy Analysis

From Tab. 1, we can see that all entropy values ofFig.4The IHS fused image. Fig.5 The PCA fused image. different methods fused images are higher than that of the

original TM image. Entropy values of Fig.9 are 6.8811 inband R, 6.9386 in band, G, 6.7849 in band B. Entropy valuesof the three bands are bigger than all the others. That meansmore information is increased in Fig.9. Fig.10-Fig.12 showsthe comparison of entropy values of various bands.

B. Correlation CoefficientTab. lI shows the comparison of correlation coefficients

of various bands, whose values are calculated from the fusedresults and the original multi-spectral images. The greater the

Figo6 The DWT fused image. Fig7 The NCM fused image. value, the more spectral information is reserved from the TMimage. The ideal value of correlation coefficient is 1.Correlation coefficient values of Fig.9 are 0.8763 in band R,0.8778 in band G, 0.8929 in band B. Fig.13-Fig.15 show thecomparison of correlation coefficient values of various bands.

In a word, the novel method proposed in this paperperform better effect on preserving spectral information andenhancing spatial information of the original TM image. Thefusion algorithm of remote sensing images based on thenormalized correlation moment of characteristic values isbetter than IHS, PCA, DWT, PCA+DWT, and NCM

Fig.8 The PCA±DWT fused image. Fig.9 The new algorithm fused transforms.image.

The IHS transform and the PCA transform enhanced the V. CONCLUSIONSperformance capability of spatial information, but causedseriously colour distortion for visual opinion. For example, Aoetonwavlgorth usefiieng cacthenrmsialz sbaedcorltonmountain vegetation is vague in northeast region outside the momeprntcofpavleomoefient charyisactritiwvaluesbransedr oncity. That is, white takes on green in Fig.4and Fig.5. Spectral tepropsdincipal compoent analysisuandwvavleatio tandsformistiaadisPCA,o buchefnaausionbyeffecltofan DWTis worsatie thanIH analysis of experimental result compared with other fusedand CA,butthefinl fsio effct f D T i wose han results using common approaches confirmed that the newIHS and PCA in lower level. Fig.9 has the better spectra[l algorithm represents almost the same colour as the originalinformation preserving and spatial resolution improvement multi-spectral images and the same spatial details as thethan Fig.7 and Fig.8. To further illustrate the efficiency of the orinal panchromahc images. The technique proposed in thismethod used in this paper, we objectively evaluate the results original pnromatcuiage. Thetechni spropsed intfue by th vaiu mehd in th enrp an the paper is more powerful on preserving spatial detail andcorrelation coefficin fetures. The erfrmac aramte reducing significantly the colour distortion of the fusedcorrlaton oeficint eatres.Theperonnnceparmetrs mages, which usually occurred when using the PCA approachof each fused result, as shown in the tables below. or PCA+DWT approach.

Now we analyse the statistical data in TABLE I andTABLE II, and draw these conclusions VI. ACKNOWLEDGMENT

TABLE I National Natural Science Foundation of ChinaCOMPARISON OF ENTROPY VALUES OF FUSED RESULTS (60374033), Science and Technology Research Projects of

Band TM Fig.4 Fig.5 Fig.6 Fig.7 Fig.8 Fig.9 Ministry of Education (107057) and High TechnologyIR I5.4716 5.8509 6.5123 6.4150 |5.71 13 |6.7136 |6.8811 Projects of Jiangsu Province of China (BG2006003) support|G |6.2715 |6.3630 |6.5866 |6.7109 |6.5131 |6.9476 |6.9386 this paper.

|B |6.1843 |6.0889 |6.6889 |6.6812 |6.4433 |6.6533 |6.7849|

468

8 7

6.836

6.65

4 6.4

6.22

6

0 5.3TM Fig.4 Fig.5 Fig.6 Fig.7 Fig.8 Fig.9 TM Fig.4 Fig.5 Fig.6 Fig.7 Fig.3 Fig.9

Fig. 10 Comparison of entropy values of band R Fig. 11 Comparison of entropy values of band G7 1

6 .8 090.8

6.6 0.7

6.4 0.60.5

6.20.

6 03.0.2

5.30.1

5.6 0

TM Fig.4 Fig.5 Fig.6 Fig.7 Fig.8 Fig.9 Fig.4 Fig.5 Fig.6 Fig.7 Fig.8 Fig.9Fig. 12 Comparison of entropy values of band B Fig. 13 Comparison of correlation coefficients of band R

0 .9 1U .88 0.9-----U0.86 0.gU0.84 0.70 .82 0.6U .8 0.5U0.78040.76 0.40.74 0.30.72 0.20.7 0.1

0.63 0Fig.4 Fig.5 Fig.6 Fig.7 Fig.8 Fig.9 Fig.4 Fig.5 Fig.6 Fig.7 Fig.8 Fig.9Fig. 14 Comparison of correlation coefficients of band G Fig. 15 Comparison of correlation coefficients of band B

[4] H. Li, B. -S. Manjunath and S. -K. Mitra, "Multi-sensor image fusionREFERENCES using the wavelet transform," IEEE International Conference on Image

[1] G. -J. He, K. -L. Li and D. -Y. Hu, "Information fusion of multi-sensor Processing, pp. 51-55, 1994.satellite remote sensing data: theory, methodology and experiment," [5] D. -C. Tseng, Y. -L. Chen and M. -S. Liu, "Wavelet -based multi-spectralJournal ofImage and Graphics, vol. 4, 1999. image fusion," IEEE Int'l Conf on IGARSS, vol.4, pp. 1956-1958, 2001.

[2] P. -5. Chavez, S. -C. Sides and J. -A. Anderson, "Comparison of threedifferent methods to merge multi-resolution and multi-spectral data: image fusion based on the normalized correlation moment," Journal of

LansatTM nd POTpanhroati," holgrameticEngineering Image and Graphics, vol. 7, pp. 1220-1224, 2002.LandsamteT aendigSPoT panchomati,"pp Photogrammetric[7] B. -C. Li, J. -S. Luo, "Wavelet analysis and applications," Electric

[3] A. -X. Mei, W. Peng and Q. -M. Qin, "Remote sensing theory," High Indu-XieQ ubiang, ng-X ompegandy .-.Wn, "RmoesesigimgEducation Publishing Company, 2001. [8] Z

fusion based on local deviation of wavelet transform," Hua Zhong Univ.ofSci. & Tech. (Nature Science Edition), vol. 3, pp. 89-9b1, 2003.

saelie eot enig at:thoy,mthdloyan epr1et, maeuson"EEIt'Cng nIASSvo.,p.146998201