Journal of ELECTRICAL ENGINEERING, VOL. 54, NO. 9-10, 2003, 237–243

IN–SERVICE IMAGE MONITORING USINGPERCEPTUAL OBJECTIVE QUALITY METRICS

Tubagus Maulana Kusuma — Hans-Jurgen Zepernick∗

User-oriented image quality assessment has become a key factor in multimedia communications as a means of monitoring

perceptual service quality. However, existing image quality metrics such as Peak Signal-to-Noise Ratio (PSNR) are inappro-

priate for in-service quality monitoring since they require the original image to be available at the receiver. Although PSNRand others are objective metrics, they are not based on human visual perception and are typically designed to measure the

fidelity. On the other hand, the human visual system (HVS) is more sensitive to perceptual quality than fidelity. In order

to overcome these problems, we propose a novel objective reduced-reference hybrid image quality metric (RR-HIQM) that

accounts for the human visual perception and does not require a reference image at the receiver. This metric is based on a

combined approach of various but specific image artifact measures. The result is a single number, which represents an overall

image quality.

K e y w o r d s: objective image metric, perceptual image quality assessment, in-service quality monitoring, reduced-

reference system, JPEG

1 INTRODUCTION

Transmission of multimedia services such as image andvideo over a wireless communication link can be expectedto grow rapidly with the deployment of third and futuregeneration mobile radio systems [1]. These mobile radiosystems also aim at offering higher data rates, improvedreliability and spectrum efficiency. This will allow for a va-riety of wireless multimedia services to be delivered overa hostile radio channel while maintaining satisfactory ser-vice quality. However, experiments have shown that theconventional quality measures, such as Bit Error Rate(BER) or Frame Error Rate (FER) are not adequate forquality assessment related to image and video transmis-sion. For example, the effects of only a few bit errorsor even a single bit error in the source-compressed im-age or video stream can propagate through a substantialportion of the decompressed data and may cause a se-vere degradation in the user-perceived quality. Therefore,user oriented quality metrics that incorporate human vi-sual perception have become of great interest in imageand video delivery services [2,3]. Although the best andtruest judge of quality is human through subjective tests,continuous monitoring of communication systems qual-ity by subjective methods is tedious, expensive and notpractical in a real-time environment. Therefore, objectivequality measurement methods which closely approximatethe subjective test results are sought after.

Another incentive for the search after user orientedquality metrics is the fact that the commonly used im-age fidelity and quality metrics like Mean-Squared Error(MSE) and Peak Signal-to-Noise Ratio (PSNR) are in-appropriate for in-service quality monitoring because the

reference image is unavailable at the receiver. These ex-isting metrics fall into the category of full-reference (FR)metrics [3]. In order to overcome this problem, the pro-posed novel reduced-reference (RR) objective perceptualimage quality metric does not require a reference imageand is based on the human visual system (HVS).

This paper is organized as follows. Different types ofimage artifacts and metrics are outlined in Section II.The proposed Hybrid Image Quality Metric (HIQM) isexplained in Section III. In Section IV, application ofHIQM for quality monitoring is discussed, followed byexperimental results in Section V. Finally, concluding re-marks are given in Section VI.

2 IMAGE ARTIFACTS AND METRICS

2.1 Image Artifacts

Image artifacts are caused by impairments such astransmission errors and depend on the image compressionscheme used. A received data packet may have header in-formation and/or data segments corrupted. In some im-age formats a single corrupted bit might lead to an incom-plete or even undecodable image. In case of Joint Photo-graphic Experts Group (JPEG) images, for example, thebit error location can have a significant impact on thelevel of image distortion. A bit error that occurs at amarker segment position can severely degrade the imagequality. An image may be even completely lost becausethe decoder fails to recognize the compressed image.

In this paper, we consider five types of artifacts thathave been observed during our simulations. These arti-

∗ Western Australian Telecommunications Research Institute, 39 Fairway, Nedlands, WA 6907, Australia, E-mails: mkusuma@watri.org.au,

hans@watri.org.au

First version of this paper was presented at SympoTIC´ 03, 26-28 October 2003, Bratislava, Slovakia

ISSN 1335-3632 c© 2003 FEI STU

238 T. M. Kusuma — H.–J. Zepernick: IN-SERVICE IMAGE MONITORING USING PERCEPTUAL OBJECTIVE QUALITY METRICS

Fig. 1. Samples of image artifacts: (a) Smoothness, (b) Blocking, (c) Ringing/Shifting, (d) Masking, (e) Lost block.

facts are smoothness, blocking, ringing, masking, and lostblock and will be briefly described below [4,5,6]:

• Smoothness which appears as edge smoothness or tex-ture blur, is due to the loss of high frequency compo-nents when compared with the original image. Blur-ring means that the received image is smoother thanthe original, see Fig. 1(a).

• Blocking appears in all block-based compression tech-niques and is due to coarse quantization of frequencycomponents. It can be observed as surface discontinu-ity (edge) at block boundaries. These edges are per-ceived as abnormal high frequency components in thespectrum, see Fig. 1(b)

• Ringing is observed as periodic pseudoedges aroundoriginal edges. It is due to improper truncation of highfrequency components, see Fig. 1(c).

• Masking is the reduction in the visibility of one imagecomponent (the target) due to the presence of another(the masker). There are two kinds of masking effects.The first is called luminance masking, also known aslight adaptation. The second is texture masking, whichoccurs when maskers are complex textures or maskerand target have similar frequencies and orientations,see Fig. 1(d).

• Lost block is an alteration of a pixel value, so that itdoes not match with its neighbours pixel value. In com-mon operation of still image compression standardslike JPEG, the encoder tiles the image into blocks ofn×n pixels, calculates a 2-D transform, quantizes thetransform coefficients and encodes them using Huff-man coding. In common wireless scenario, the image istransmitted over wireless channel block-by-block. Dueto severe fading, entire image blocks can be lost, seeFig. 1(e).

2.2 Image Metrics

Image metrics may be divided into two categories:

• Image fidelity metrics indicate image differences bymeasuring pixel-wise closeness between images. MSEand PSNR fall into this category.

• Image quality metrics define quality based on indi-vidual image features; these metrics also incorporateHVS. Much research is being carried out on the image

quality metrics, but they mostly concentrate on singleartifacts [7,8,9,10].

There are three approaches of measuring image fidelityand image quality as follows:

• Full-Reference (FR) is a method that compares a dis-torted image with its undistorted original.

• Reduced-Reference (RR) is a method that does not re-quire to store the entire original image by extractingimportant features from the distorted image and com-paring them with corresponding stored features of theoriginal image.

• No-Reference (NR) techniques do not require priorknowledge about the original image but perform as-sessment to the distorted image to search for the pres-ence of known artifacts.

3 HYBRID IMAGE QUALITY METRIC

The proposed Hybrid Image Quality Metric (HIQM)employs several quality measurement techniques and iscalculated as weighted sum of respective quality metrics.It is designed to detect and to measure different imageartifacts. The result is a single number that correlateswell with perceived image quality. HIQM does not requirea reference image at the receiver to measure the qualityof a target image. The proposed RR approach considersfive different artifact measurements relating to blocking,blur, image activity, and intensity masking detection.

The blocking measurement is based on the algorithmproposed by Wang et al. [7]. This algorithm extracts theaverage difference across block boundaries, averages ab-solute differences between in-block image samples, andcalculates the zero-crossing rate. The system has beentrained using subjective test results in order to complywith human visual perception. The final blocking mea-sure is calculated using statistical non-linear curve fittingtechniques. This metric is classified as an NR type be-cause only the received image is needed to measure theblocking. In our approach, this metric is also used to de-tect and to measure lost blocks.

The blur measurement algorithm is based on the workof Marziliano et al. [8]. This algorithm accounts for thesmoothing effect of blur on edges by measuring the dis-tance between edges from local maximum and local min-imum, the so-called local blur. A Sobel filter is used to

Journal of ELECTRICAL ENGINEERING VOL. 54, NO. 9-10, 2003 239

Fig. 2. Sample original image and its edge representation

Fig. 3. Sample blur image and its edge representation

detect the edges. Once the edges are detected, the dis-

tance between local maximum and local minimum can

be measured for both horizontal and vertical directions.

The final blur measure is obtained by averaging the lo-

cal blur values over all edge locations in both directions.

Fig. 2 shows an original image and its edge representation

whereas Fig. 3 shows the blurred version of this image.

Another important characteristic of an image relates

to the activity measure that indicates the ’busyness’ of

the image. The active regions of an image are defined as

those with strong edges and textures. Due to distortion,

a received image normally has more activity compared

to the original image. The technique used by HIQM is

based on Saha and Vemuri’s algorithm [10]. We use this

metric to detect and to measure ringing and lost blocks.

Especially, two types of Image Activity Measures (IAM)

are deployed, edge and gradient-based IAM. For an M ×

N binary image, the edge activity measure is given by

[10]:

IAMedge =

[

1

M N

M N∑

i=1

B(i)

]

× 100 (1)

where B(i) denotes the value of the detected edge at pixel

location i , M is the number of image rows and N is the

number of image columns.

The gradient-based activity measure for an M × N

image is given by [10]:

IAMgrad =1

M N

[

M−1∑

i=1

N∑

j=1

|I(i, j) − I(i + 1, j)|

−

M∑

i=1

N−1∑

j=1

|I(i, j) − I(i, j + 1)|

]

(2)

where I(i, j) denotes the intensity value at pixel location(i, j) and | · | denotes absolute value.

Finally, the intensity masking detection is based onthe standard deviation of the first-order image histogramwhich indicates the distribution of the image data values.The shape of the histogram provides many insights intothe character of an image [11]. The histogram of an M×N

image is defined as the percentage of pixels within theimage at a given gray level. For a 256 gray level image,the histogram hi is given by:

hi =ni

M N, for 0 ≤ i ≤ 255 (3)

where ni denotes the number of pixels at gray level i

and M N denotes the total number of pixels within theimage.

240 T. M. Kusuma — H.–J. Zepernick: IN-SERVICE IMAGE MONITORING USING PERCEPTUAL OBJECTIVE QUALITY METRICS

(a) (b) (c)

0 50 100 150 200 250

0

500

1000

1500

2000

2500

3000

Intensity Level

Number of Pixels

Original Image 'Lena'

0 50 100 150 200 250

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

Intensity Level

Number of Pixels

Image 'Lena' with White Mask

0 50 100 150 200 250

010002000300040005000600070008000900010000

Intensity Level

Number of Pixels

Image 'Lena' with Black Mask

Fig. 4. Samples for test image “Lena” with histogram information: (a) Original Image, (b) Image with White Mask, (c) Image withBlack Mask

Blocking Measure

Blur Measure

Image Activity Measureedge

Image Activity Measuregrad

Intensity MaskingDetection

HIQMS

w2

w3

w5

w1

w4

Fig. 5. Formation of the combined metric HIQM

From the histogram information, we can measure theimage perceived brightness by calculating the averagegray level that is given by:

brightness =255∑

i=0

i hi (4)

where hi denotes the image histogram at gray level i .

A low average value means dark image, whereas largeaverage implies a bright image. The image contrast cannow be measured by estimating the average gray levelvariation within the image (standard deviation) that isgiven by:

contrast =

√

√

√

√

255∑

i=0

i2 hi − brightness2 (5)

A small standard deviation indicates a low contrast im-age, while large value implies a high contrast image. Theintensity masking (maskint ) detection is based on con-trast measurement. Therefore, the same expression asmentioned in (5) is applied. Fig. 4 shows the exampleof images and their respective histogram information.

The proposed overall quality measure is a weightedsum of all the aforementioned metrics. The weight allo-cation for individual metrics was based on the impactof the metric on the overall perceptibility of images byhuman vision. The fine-tuning of the weights was doneempirically and was justified by requesting opinion froma group of unbiased test persons.

Initially, all the metrics were given the same weightw ∈ [0 . . . 1] and were then adjusted based on the con-

Journal of ELECTRICAL ENGINEERING VOL. 54, NO. 9-10, 2003 241

Received

MeasureHIQMQuality

Baseline

Insert Baseline

into PayloadEncoder Modulator

Decoder Demodulator

OriginalImage

ExtractBaseline from

PayloadBaseline

Add DegradationTolerance Factor

to Baseline

Image

MeasureHIQM

Compare HIQM

with Reference

Quality

Image

HIQM

ReferenceQuality

Quality

Accepted / Rejected

Channel

Fig. 6. Reduced reference in-service quality monitoring using the proposed HIQM

Sign1 bit

[0/1]

Quality(integer)8 bits[0-255]

Qualityst(1 dec.)4 bits[0-15]

Image File /bitstream

Quality

(2 dec.)nd

4 bits[0-15]

Fig. 7. RR-HIQM packets format

tribution of each metric to the perceptibility of image byhuman eye. In JPEG, for example, blocking is given ahigher weight compared to other metrics because it is themost frequently observed artifact in this particular imageformat and can be easily perceived by human vision. Theoverall quality is given by:

HIQM = w1 × blockMetric + w2 × blurMetric+

+w3 × IAMedge + w4 × IAMgrad + w5 × maskint

(6)

where wi denotes the weight of a particular metric.

4 QUALITY MONITORING USING HIQM

In image transmission systems, HIQM can be used forcontinuous in-service quality monitoring since it does notrequire a reference image to measure the quality of a re-ceived image. However, different original images differ inactivity and other characteristics. Therefore, each imagehas its individual HIQM value, which we will refer to asquality baseline.

To obtain a proper measure, we need to normalize orcalibrate the measurement system to the quality baseline.Therefore, the quality baseline of an original image needsto be communicated to the receiving end of the systemunder test. Obviously, this constitutes an RR approach

and may replace conventional FR quality techniques. Ascorrect reception of the quality baseline is vital for imagequality assessment at the receiver, error control coding isrecommended to protect this important parameter.

The basic steps of in-service quality monitoring usingHIQM can be summarized as follows (Fig. 6):

• At the transmitter, measure the quality baseline of theoriginal image in terms of its HIQM value.

• Concatenate quality baseline and image file to form theoverall packet format (see Fig. 7) before transmissionof respective packets.

• At the receiver, provide a reference quality by extract-ing quality baseline from received packet and addinga tolerable degradation value to it.

• Measure HIQM of the received image and compare itwith the reference quality.

The total length of the RR-HIQM related quality valuehas been chosen as 17 bits. It consists of sign information(1 bit), quality (8 bits for the integer, 4 bits for each

the 1st and the 2nd decimal). The HIQM of -0.57, forexample, will be concatenated to the header part of theimage file as 000000000010101112 .

In practice, there is a number of possible applicationsfor RR-HIQM, which include system testing, continuousin-service quality monitoring and performance character-ization of image transmission system.

242 T. M. Kusuma — H.–J. Zepernick: IN-SERVICE IMAGE MONITORING USING PERCEPTUAL OBJECTIVE QUALITY METRICS

0 5 10 15 20 25 30 35 40

0

5

10

15

20

25

30

35

40

45

HIQM

PSNR (dB)

Image quality sample

Fitting curve

Confidence level (95%)

Fig. 8. Metric comparison for image “Lena”

0 2 4 6 8 10

5

10

15

20

25

30

35

40

45

50

55

HIQM

PSNR (dB)

Image quality sample

Fitting curve

Confidence level (95%)

Fig. 9. Metric comparison for image “Tiffany”

5 10 15 20 25 30

5

10

15

20

25

30

35

40

45

50

HIQM

PSNR (dB)

Image quality sample

Fitting curve

Confidence level (95%)

Fig. 10. Metric comparison for image “Goldhill”

5 EXPERIMENTAL RESULTS

In this section, we provide experimental results for testimages “Lena” (Quality baseline = −1.76), “Goldhill”(Quality baseline = 1.34) and “Tiffany” (Quality base-line = −0.57). The test scenario for these standard testimages was chosen as a Rayleigh flat fading channel. Asimple (31, 21) BCH code was applied for error protec-tion purposes along with a soft-combining scheme usinga maximum of two retransmissions. The average Signal-to-Noise Ratio (SNR) was set to 5dB . We did not use aJPEG restart marker in this experiment in order to gainextreme artifacts.

For this test scenarios, the weights of the various met-rics were finally obtained from involving a group of testpersons as: w1 = 1, w2 = 0.5, w3 = 1, w4 = 0.5, andw5 = 0.3. From the experimental results, it can be con-cluded that HIQM inversely relates to PSNR (see Figs. 8,9 and 10). In other words, the lower the HIQM value, thebetter the quality whereas the lower the PSNR value, theworse the quality (see Figs. 8, 9 and 10).

Fig. 11. Quality samples for test image “Lena”: (a) PSNR =9.76dB and HIQM = 9.28 , (b) PSNR = 9.78dB and HIQM =

11.99

Fig. 12. Quality samples for test image “Tiffany”: (a) PSNR =23.17dB and HIQM = 3.30 , (b) PSNR = 24.15dB and HIQM =

7.78

Figs. 8, 9 and 10 show some outliers which are due todisagreement of HIQM with PSNR. These disagreementsare mostly due to the misjudgement of the PSNR in rela-tion to the opinion of the human subjects. Two samplesof misjudgement are presented in Figs. 11, 12. For justi-fication, we interviewed 10 persons to give their opinionabout the quality when disagreements occurred betweenHIQM and PSNR. The average opinion of these peoplewas that HIQM provides a better judgement than PSNR.Fig. 13 shows an example where HIQM and PSNR agreeon the image quality.

Journal of ELECTRICAL ENGINEERING VOL. 54, NO. 9-10, 2003 243

Fig. 13. Quality samples for test image “Goldhill”: (a) PSNR = 8.28dB and HIQM = 34.09 , (b) PSNR = 22.05dB and HIQM =

6.95 , (c) PSNR = 45.70dB and HIQM = 1.43

6 CONCLUSIONS

A novel reduced-reference image quality measurementtechnique was presented which takes the user perceivedquality into account. It is designed to detect and to mea-sure different image artifacts along with the calculationof a weighted sum of respective quality metrics. It wasshown by way of experiment that the proposed HIQMoutperforms PSNR with respect to quantifying user per-ceived quality. The introduced HIQM may be used forreduced-reference in-service image quality monitoring. Itdoes not require a reference image to be present at thereceiver and is therefore well suited to real-time applica-tions.

Acknowledgements

This work was partly supported by the Commonwealthof Australia through the Australian TelecommunicationsCooperative Research Centre, Perth, Australia.

References

[1] WEERACKODY, V.—PODILCHUK, C.—ESTRELLA, A. :

Transmission of JPEG-Coded Images over Wireless Channels,Bell Laboratories Technical Journal, Special Issue on Wireless,Autumn 1996.

[2] KNEE, M. : The Picture Appraisal Rating (PAR) - A Sin-gle-ended Picture Quality Measure for MPEG-2, Technical Re-

port, Snell and Wilcox, Jan. 2000.

[3] WEBSTER, A. : Objective and Subjective Evaluation forTelecommunications Services and Video Quality, National Tele-

communications and Information Administration (NTIA)/Insti-tute for Telecommunication Science (ITS), Rapporteur Q21/9,

2002.

[4] WINKLER, S. : Vision Models and Quality Metrics for Im-

age Processing Applications, PhD Thesis, Ecole PolytechniqueFederale De Laussane (EPFL), Dec. 2000.

[5] JAKULIN, S. : Baseline JPEG and JPEG2000 Artifacts Il-

lustrated, <http://ai.fri.uni-lj.si/∼aleks/jpeg/artifacts.htm> ,accessed on 8 May 2003.

[6] RANE, S. D.—REMUS, J.—SAPIRO, G. : Wavelet-Domain

Reconstruction of Lost Blocks in Wireless Image Transmissionand Packet-Switched Networks, Proc. IEEE Int. Conf. on Image

Processing,New York, USA, Sept. 2002, 309-312.

[7] WANG, Z.—SHEIKH, H. R.—BOVIK, A. C. : No-Reference

Perceptual Quality Assessment of JPEG Compressed Images,

Proc. IEEE Int. Conf. on Image Processing, New York, USA,

Sept. 2002, 477-480.

[8] WANG, Z.—BOVIK, A. C.—EVANS, B. L. : Blind Measure-

ment of Blocking Artifacts in Images, Proc. IEEE Int. Conf. on

Image Processing, Vancouver, Canada, Sept. 2000, 981-984.

[9] MARZILIANO, P.—DUFAUX, F.—WINKLER, S.—EBRA-

HIMI, T. : A No-Reference Perceptual Blur Metric, Proc. IEEE

Int. Conf. on Image Processing, New York, USA, Sept. 2002,

57-60.

[10] SAHA, S.—VEMURI, R. : An Analysis on the Effect of Im-

age Activity on Lossy Coding Performance, Proc. IEEE Int.

Symp. on Circuits and Systems, Geneva, Switzerland, May 2000,

295-298.

[11] WEEKS, A. R. : Fundamentals of Electronic Image Processing,

SPIE/IEEE Series on Imaging Science and Engineering, 1998.

[12] KUSUMA, T. M.—ZEPERNICK, H.-J. : A Reduced-Reference

Perceptual Quality Metric for In-Service Image Quality Assess-

ment, Proc. IEEE Symp. on Trends in Commun., Bratislava,

Slovakia, Oct. 2003, 71-74.

Received 13 November 2003

Maulana Kusuma received BE degree from GunadarmaUniversity, Jakarta, Indonesia, in 1994 and MESc degree fromCurtin University of Technology, Perth, Western Australia,in 1998. He is an academic member of the Department ofElectrical and Computer Engineering Gunadarma University,Jakarta, Indonesia. He is currently pursuing PhD degree inelectrical engineering at the Western Australian Telecommu-nications Research Institute (WATRI), Perth, Australia. Hisresearch interest include image processing, digital video, im-age and video transmission, and video communications.

Hans-Jurgen Zepernick received the Dipl-Ing degreefrom the University of Siegen, Germany, in 1987, and the Dr-Ing degree from the University of Hagen, Germany, in 1994,both in electrical engineering. From 1988 to 1989, he was aResearch Engineer with the Radio and Radar Departmentat Siemens AG, Munich, Germany. Currently, Dr. Zepernickholds the positions of a Professor in Wireless Communica-tions at Curtin University of Technology, Director of the Wire-less Communications Laboratory of the Western AustralianTelecommunications Research Institute (WATRI), and Asso-ciate Director of the Australian Telecommunications Cooper-ative Research Centre (ATcrc), Perth, Australia. His researchinterests include radio channel characterization and modelling,coding and modulation, equalization, spread-spectrum sys-tems, wireless multimedia communications, and future gen-eration wireless systems.