Image Quality Assessment Based on the Correlation coefficient and the 2-D Discrete Wavelet Transform*

Junfeng Li, Wenzhan Dai Department of Automatic control

Zhejiang Sci-Tech University HangZhou, Zhejiang Province, China

e-mail: ljf2003@zstu.edu.cn

*This work is partially supported by NSFC Grant #60804010 to R. D. Zhang and ZJNSF Grant #Y607556 to J. F. Li.

Abstract – Image quality assessment is a complex problem due to subjective nature of human visual perception. In this paper, a novel image quality assessment is proposed based on the discrete 2-D wavelet transform and the correlation coefficient. Firstly, the reference image and the distorted images are decomposed into several levels by means of the wavelet transform respectively. Secondly, the approximation and detail coefficients of the reference image are as the reference sequences and the approximation and detail coefficients of the distorted images are as the comparative sequences respectively. And the correlation coefficients are calculated between the reference sequences and the comparative sequences respectively. Moreover, the image quality assessment matrix of every distorted image can be constructed based on the correlation coefficients and the image quality can be assessed. The algorithm makes full use of perfect integral comparison mechanism of the correlation coefficient and the well matching of discrete wavelet transform with multi-channel model of human visual system. The experimental results show that the proposed algorithm can not only evaluate the integral and detail quality of image fidelity accurately but also bears more consistency with the human visual system than the traditional method PSNR.

Index Terms - Image quality assessment.The correlation coefficient. The 2-D discrete wavelet transform

I. INTRODUCTION

Digital images may be corrupted by degradation such as linear frequency distortion, noise, and blocking artifacts. These sources of degradation may arise during image acquisition, processing, compression, storage, transmission and reproduction, any of which have a direct bearing on visual quality. For applications in which images are ultimately to be viewed by human beings, the only method is through quantifying visual image quality. Image quality assessment process of Human Vision System (HVS) is composed of complex psychological activities and the result of image quality assessment will be affected by many factors. The result of image quality assessment will be improved if these factors are extracted and applied to the objective image quality assessment. These factors are mainly as follows: (1) brightness, correct brightness is the basic conditions to view an image. The image quality will be declined when the environment brightness is too dark or too light; (2) definition, HVS always describes the image quality by the words such as blur and sharp. In the view

of the frequency domain, an image will become sharp only when the proportion between the high frequency and the low frequency is propriety. (3) correlation, which is the related to measure similarith among the images[1].

Image quality assessment can be classified as subjective and objective image quality assessment[2]. Subjective testing methods[3] require human assessors to judge the overall quality of a set of images. Those approaches are time-consuming, and very expensive. Moreover, they give an empirical, consistent representation of the perceptual phenomenon, but do not lead to a deterministic model of the evaluation process. Conversely, objective assessment methods aim at the automated estimation of perceived quality by using numerical features extracted from images. These approaches bypass human assessors but, to be effective, they must provide results consistent with data obtained from subjective experiments. Objective methods are usually classified with respect to the amount of information available from the original, unprocessed image. No-reference approaches[4-6] assess perceived quality in the absence of the original image. Full reference methods [7-11] assume to have both the original and the processed image available, and relate perceived quality to the difference between these two. Typically, these approaches are extended by including analytical models of the human visual system (HVS) in order to improve their accuracy and reliability [12-15]. Finally, reduced-reference schemes only use a limited set of features from the original image; they can access the quality of noise-affected images.

However, Vision models which treat visible distortions equally, regardless of their location in the image, may not be powerful enough to accurately predict picture quality in such cases. This is because we are known to be more sensitive to distortions in areas of the image to which we are paying attention than to errors in peripheral areas. In this paper, a novel image quality assessment based on the characteristics of wavelet coefficients of images and the correlation coefficient is proposed. The algorithm makes full use of perfect integral comparison mechanism of the correlation coefficient and the well matching of discrete wavelet transform with multi-channel model of human visual system. Experimental results show that the proposed algorithm can not only evaluate the integral and detail quality of image fidelity accurately but also bears more consistency with the human visual system than the traditional method PSNR.

II. THE 2-D DISCRETE WAVELET TRANSFORM

789

Proceedings of the IEEE

International Conference on Automation and Logistics Shenyang, China August 2009

978-1-4244-4795-4/09/$25.00 © 2009 IEEE

Perhaps the most well-know mathematical technique for analyzing signal is the Fourier analysis which breaks down a signal into constituent sinusoids of different frequencies. For many signals, Fourier analysis is extremely useful but Fourier analysis has serious drawback if signal has non-stationary characteristics. So another efficient analyzing technique is needed. Wavelet were developed independently in the fields of mathematics, quantum physics, electrical engineering, seismic geology, and image processing. Interchanges between these fields during the last ten years have led to many new wavelet applications. Wavelet are mathematical functions that cut up data into different frequency components and then study each component with resolution matched to its scale. They have advantage over traditional Fourier methods in analyzing physical situations where the signal contains discontinuities and sharp spikes.

The discrete wavelet transform(DWT) of image signals produces a non-redundant image representation, which provides better spatial and spectral localization of image formation, compared with other multi scale representations such as Gaussian and Laplacian pyramid. The DWT can be interpreted as signal decomposition in a set of independent, spatially oriented frequency channels. The signal S is passed through two complementary filters and emerges as two signals, approximation and details. This is called decomposition or analysis. The components can be assembled back into the original signal without loss of information. This process is called reconstruction or synthesis. The mathematical manipulation, which implies analysis and synthesis, is called discrete wavelet transform and inverse discrete wavelet transform. An image can be decomposed into a sequence of different spatial resolution images using DWT. In case of a 2-D image, an N level decomposition can be performed resulting in 3N+1 different frequency bands namely, LL, LH, HL and HH as shown in Fig. 1. These are also known by other names, the sub-bands may be respectively called a1 or the first average image, h1 called horizontal fluctuation, v1 called vertical fluctuation and d1

called the first diagonal fluctuation. The sub-image a1 is formed by computing the trends along rows of the image followed by computing trends along its columns. In the same manner, fluctuations are also created by computing trends along rows followed by trends along columns. the next level of wavelet transform is applied to the low frequency sub band image LL only.

III. IMAGE QUALITY ASSESSMENT BASED ON THE CORRELATION COEFFICIENT AND THE 2-D DISCRETE WAVELET

TRANSFORM

3.1 the Correlation Coefficient

Fig.1 2-D DWT with 3-level decomposition

The correlation coefficient is also to measure relational degree among things and factors. The bigger the correlation coefficient is, the more similar the referenced sequence and the comparative sequence are. The definition of the correlation coefficient against time sequence can be gotten.

Definition Supposed },,,{ 002010 nxxxX = is a referenced

sequence; },,,{ 21 iniii xxxX = is a comparative sequence.

So iX ’s correlation coefficient with 0X is:

∑ ∑

∑

= =

=

−−

−−=

n

k

n

kiikk

iik

n

kk

i

xxxx

xxxxR

1 1

2200

100

0

)()(

))(( (1)

Where 0x and ix are the Means of 0x and ix respectively.

Because the dimension of the behavior gene sequence is likely to be inconsistent, the referenced sequence and comparative sequence should be unified before calculating the correlativity index. Three unified means are as follows:

“The better is, the bigger is”

'

'

max iji

ijij x

xx = (2)

“The better is, the smaller is”

'

'

min

min

iji

ijiij x

xx = (3)

“Propriety”, as“the better it is, the more it is close to the

standard ir ”

iiji

iij

ijrx

rxx

−

−−=

'

'

max1 (4)

3.2 the New Algorithm of Image Quality Assessment The steps of the assessment method are as follow.

Step1. Decompose the reference image and the

),,2,1( niith = distorted image into m levels by means of

wavelet transform respectively, such as “db”, “harr”, “sym”, “coif” and so on. The approximation and detail coefficients of reference image which are the referenced sequences are as follow

790

01

01

01

01

01

01

0000

,,,,

,,,,,,

cDcVcHcD

cVcHcDcVcHcA

m

mmmmmm

−

−−

The approximation and detail coefficients of the thi distorted image which are the comparative sequences are as follow

iiiim

im

im

im

im

im

im

cDcVcHcD

cVcHcDcVcHcA

1111

11

,,,,

,,,,,,

−

−−

Where ni ,,2,1= . The superscript 0 denotes the wavelet

sequences of the normal image and the superscript i denotes

the wavelet sequences of the thi image assessed image. The

subscript m~1 denotes the wavelet coefficients at m~1 levels respectively. Step2. Calculate the correlation coefficient values between the

referenced sequences and the thi comparative sequences and get the following image quality assessment matrix.

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

=

im

ii

im

ii

im

ii

i

i

qqqqqqqqq

q

Q

44241

33231

22221

11 00

Suppose the number of the thj wavelet coefficients is

),,2,1( mjPj = and calculate the elements of the image

quality assessment matrix as follow.

∑ ∑

∑

= =

=

−−

−−=

m m

m

P

k

P

k

im

immm

im

im

P

kmm

i

cAkcAcAkcA

cAkcAcAkcAq

1 1

2200

1

00

11

))(())((

))()()((,

∑ ∑

∑

= =

=

−−

−−=

j j

j

j P

k

P

k

ij

ijmm

ij

ij

P

kmm

i

cHkcHcHkcH

cHkcHcHkcHq

1 1

2200

1

00

))(())((

))()()((

2,

∑ ∑

∑

= =

=

−−

−−=

j j

j

j P

k

P

k

ij

ijmm

ij

ij

P

kmm

i

cVkcVcVkcV

cVkcVcVkcVq

1 1

2200

1

00

))(())((

))()()((

3,

∑ ∑

∑

= =

=

−−

−−=

j j

j

j P

k

P

k

ij

ijmm

ij

ij

P

kmm

i

cDkcDcDkcD

cDkcDcDkcDq

1 1

2200

1

00

))(())((

))()()((

4

Where 10 <≤ ikjq , ni ,,2,1= ; mj ,,2,1= ;

4,3,2,1=k . The thj column of the image quality

assessment matrix represents the correlation coefficient values

between the referenced sequences and the thi comparative

sequences at thj level wavelet decomposition. The thk row

of the image quality assessment matrix represents the correlation coefficient values of approximation coefficients, horizontal detail coefficients, vertical detail coefficients and diagonal detail coefficients between the referenced sequences

and the thi comparative sequences respectively. Step3. Assess image quality. The first row of the image quality assessment matrix reflects the general picture quality between the referenced image and the comparative image. The other rows reflect horizontal detail quality, vertical detail quality and diagonal detail quality between the referenced image and the comparative image respectively. Every column reflects the image quality at different scale. The smaller the elements of the image quality assessment matrix are, the better the comparative image quality at the corresponding direction and scale is.

IV. EXPERIMENT AND ANALYSIS

In order to validate the validity of this method, the four quality assessment experiments will be implemented. The experiment image choices the cameraman image which size is

256256× and the level of wavelet decomposition is three. The function of wavelet decomposition choices the “coif3”. 4.1 the Quality Assessment of the Compression Image

Fig.2 is the normal image and Fig.3 are the JPG compression images whose quality is 33%, 67% and 92% respectively. TABLE 1, TABLE 2 and TABLE 3 are the corresponding quality assessment matrixes of the JPG compression images respectively. The peak signals to noise ratio of the JPG compression images are 30.1286, 33.3114 and 43.6513 respectively.

Fig. 2 The original image

(A) the image quality is 0.33 (B) the image quality is 0.67

791

(C) the image quality is 0.92

Fig. 3 The JPG compression images TABLE 1

THE ASSESSMENT MATRIX OF FIG. 3 (A) 0.99987 1 1

0.9972 0.98538 0.86474

0.99864 0.99063 0.89834

0.99269 0.92909 0.54845

TABLE 2 THE ASSESSMENT MATRIX OF FIG. 3 (B) 0.99997 1 1

0.99926 0.99527 0.893293

0.9997 0.99682 0.96034

0.99753 0.97185 0.74464

TABLE 3 THE ASSESSMENT MATRIX OF FIG. 3 (C)

1 1 1

0.99989 0.99911 0.98482

0.99995 0.99947 0.99243

0.99958 0.99446 0.9434

4.2 the Quality Assessment of the Noise Image Fig.4 are the salt & pepper noise images whose noise

density is 0.01, 0.1 and 0.5 respectively. TABLE 4, TABLE 5 and TABLE 6 are the corresponding quality assessment matrixes of the salt & pepper noise images respectively. The peak signals to noise ratio of the salt & pepper noise images are 43.6513, 15.1075 and 10.3246 respectively.

(A) the noise density is 0.01 (B) the noise density is 0.1

(C) the noise density is 0.5

Fig. 4 The salt & pepper noise images TABLE 4

THE ASSESSMENT MATRIX OF FIG. 4 (A) 0.99948 1 1

0.98261 0.92683 0.72954

0.99082 0.9576 0.8117

0.95269 0.81011 0.51899

TABLE 5 THE ASSESSMENT MATRIX OF FIG. 4 (B)

0.9944 1 1

0.83298 0.57424 0.30513

0.90883 0.70577 0.38539

0.67484 0.37263 0.17613

TABLE 6 THE ASSESSMENT MATRIX OF FIG. 4 (C) 0.92365 1 1

0.36784 0.17799 0.07683

0.50459 0.25967 0.11409

0.29385 0.1082 0.04089

4.3 the Quality Assessment of the Gaussian Blur Image Fig.5 are the gaussian blur images whose range is 0.8, 1.3

and 2.1 respectively. TABLE 7, TABLE 8 and TABLE 9 are the corresponding quality assessment matrixes of the gaussian blur images respectively. The peak signals to noise ratio of the gaussian blur images are 27.5734, 23.7496 and 21.6400 respectively.

(A) the range is 0.8 (B) the range is 1.3

(C) the range is 2.1

Fig. 5 The gaussian blur images TABLE 7

THE ASSESSMENT MATRIX OF FIG. 5 (A) 0.99984 1 1

0.99548 0.9727 0.83904

0.99622 0.96575 0.83249

0.98509 0.91139 0.59653

TABLE 8 THE ASSESSMENT MATRIX OF FIG. 5 (B) 0.99908 1 1

0.97357 0.86921 0.586

0.97434 0.82423 0.58212

0.92754 0.62906 0.23666

TABLE 9 THE ASSESSMENT MATRIX OF FIG. 5 (C) 0.99604 1 1

0.90275 0.70382 0.40441

0.89927 0.62266 0.40404

0.74011 0.34494 0.09456

4.4 the Quality Assessment of the Vignette Image

Fig.6 are the vignette images whose range is 1.0, 2.0 and 3.0 respectively. TABLE 10, TABLE 11 and TABLE 12 are is the corresponding quality assessment matrixes of the vignette images respectively. The peak signals to noise ratio of the

792

vignette images are 27.4292, 19.8500 and 15.7344 respectively.

(A) the range is 1.0 (B) the range is 2.0

(C) the range is 3.0

Fig. 6 The vignette images TABLE 10

THE ASSESSMENT MATRIX OF FIG. 6 (A) 0.99959 1 1

0.99427 0.98544 0.96826

0.99516 0.98569 0.97334

0.99116 0.98111 0.94605

TABLE 11 THE ASSESSMENT MATRIX OF FIG. 6 (B) 0.999593 1 1

0.96159 0.9313 0.90156

0.96351 0.94547 0.91134

0.95958 0.93281 0.85629

TABLE 12 THE ASSESSMENT MATRIX OF FIG. 6 (C) 0.98859 1 1

0.90137 0.84121 0.81146

0.92446 0.89967 0.82032

0.911 0.84283 0.74458

It is easy to see that the assessment result of every experiment is consistent with the peak signals to noise ratio’s from TABLE 1 to TABLE 12. Of course, the results are accordant with human visual system. Moreover, the order of the image quality among Fig. 3(a), Fig. 4(a), Fig. 5(a) and Fig. 6(a) is Fig. 4(a), Fig. 3(a), Fig. 5(a) and Fig. 6(a) based on peak signals to noise ratio. But we can get different results based on the textual method. The image quality can be assessed from the general picture quality, horizontal detail quality, vertical detail quality and diagonal detail quality in the textual method. It is easy to see that the assessment results based on the textual method are more accordant with human visual system from Fig. 3(a), Fig. 6(a), Fig. 5(a) and Fig. 4(a).

V. CONCLUSION

Image quality assessment is one of the key technologies in various image processing applications. The image quality assessment process of HVS is composed of complex psychological activities and the result of image quality assessment will be affected by many factors. The image quality assessment algorithm of this paper makes full use of

the property of multi-resolution analysis of the wavelet transform to analyze image in frequency domain. And the algorithm makes full use of perfect integral comparison mechanism of the correlation coefficient and the well matching of discrete wavelet transform with multi-channel model of human visual system. Experimental results show that the proposed algorithm can not only evaluate the integral and detail quality of image fidelity accurately but also bears more consistency with the human visual system than the traditional method PSNR.

REFERENCES [1] Y. H. Yi, X. Q. Yu, “Image quality assessment based on structural

distortion and image definition,” 2008 International Conference on Computer Science and Software Engineering, pp. 253-256, December 2008.

[2] J. X. Pang, R. Zhang, L. Lu, Z. K. Liu, “Image Quality Assessment Based on Energy of Structural Distortion,” Lecture Notes in Computer Science, vol. 4810, pp. 785–793, 2007.

[3] L. Zhai, X. M. Tang, “Fuzzy comprehensive evaluation method and its application in subjective quality assessment for compressed remote sensing images,” Proceedings - Fourth International Conference on Fuzzy Systems and Knowledge Discovery, vol. 1, pp. 145–148, 2007.

[4] D. V. Niranjan, B. L. Brian, “Image quality assessment based on a degradation model,” IEEE Transactions on Image Processing, vol. 9, no. 4, pp. 636-649, April 2000.

[5] P. Gastaldo, G. Parodi, J. Redi, R. Zunino, “No-Reference Quality Assessment of JPEG Images by Using CBP Neural Networks,” Lecture Notes in Computer Science, Artificial Neural Networks – ICANN 2007, vol. 4669, pp. 564–572, 2007.

[6] Y. W. Yu, Z. D. Lu, H. F. Ling, F. H. Zou, “Lecture Notes in Computer Science,” Advances in Neural Networks - ISNN 2006, vol. 3972, pp. 638–645, 2006.

[7] Z. Wang, M. F. Sabir, A. C. Bovik, “A statistical evaluation of recent full reference image quality assessment algorithm,” IEEE Trans. Image Processing, vol. 15, pp. 3441–3452, 2006.

[8] H. R. Sheikh, M. F. Sabir, A. C. Bovik, “A statistical evaluation of recent full reference image quality assessment algorithms,” IEEE Transactions on Image Processing, vol. 15, no. 11, pp. 3440–3451, November 2006.

[9] F. Russo, A. A. De, P. Carbone, “A vector approach to quality assessment of color images,” 2008 IEEE International Instrumentation and Measurement Technology Conference Proceedings, pp. 814–818, 2008.

[10] W. F. Zhao, H. B. Shen, X. L. Yan, “Structural similarity measurement based on dual-tree complex wavelet,” Journal of Zhejiang University (Engineering Science), vol. 42, no. 8, pp. 1385–1388, August 2008.

[11] Z. Wang, H. R. Sheikh, “Image quality assessment : from error visibility yo structural similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 1-14 , April 2004.

[12] J. Starck, J. Kilner, A. Hilton, “Objective quality assessment in free-viewpoint video production,” 2008 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video, pp. 225–228, 2008.

[13] B. Yang, L. Lei, J. L. Yang, “HVS-based structural image quality assessment model,” Proceedings of the 7th World Congress on Intelligent Control and Automation, WCICA'08, pp. 8492–8496, 2008.

[14] W. Wei, “An image quality assessment method based on HVS,” Proceedings 2007 41st Annual IEEE International Carnahan Conference on Security Technology, ICCST, pp. 320–324, 2007.

[15] Y. P. Kong, P. Zeng, N. Jiang, “Image quality assessment and visualization of color difference for color inverse halftoning using HVS,” Journal of Huazhong University of Science and Technology (Natural Science Edition), vol. 36, no. 8, pp. 21–23, August 2008.

793