fuzzy homogeneity and scale-space approach to color image segmentation

Pattern Recognition 36 (2003) 1545–1562www.elsevier.com/locate/patcog

Fuzzy homogeneity and scale-space approach to color imagesegmentation

H.D. Cheng∗, J. LiDepartment of Computer Science, Utah State University, 401B Old Main Hall, Logan, UT 84322-4205, USA

Received 26 June 2002; received in revised form 21 August 2002; accepted 21 August 2002

Abstract

Image segmentation is the procedure in which the original image is partitioned into homogeneous regions, and has manyapplications. In this paper, a fuzzy homogeneity and scale-space approach to color image segmentation is proposed. A colorimage is transformed into fuzzy domain with maximum fuzzy entropy principle. The fuzzy homogeneity histogram is employed,and both global and local informations are considered when we process fuzzy homogeneity histogram. The scale-space 2lter isutilized for analyzing the fuzzy homogeneity histogram to 2nd the appropriate segments of the homogeneity histogram boundedby the local extrema of the derivatives. A fuzzy region merging process is then implemented based on color di3erence andcluster sizes to avoid over-segmentation. The proposed method is compared with the space domain approach, and experimentalresults demonstrate the e3ectiveness of the proposed approach.? 2003 Pattern Recognition Society. Published by Elsevier Science Ltd. All rights reserved.

Keywords: Fuzzy logic; Maximum entropy principle; S-function; Homogeneity; Scale-space 2lter; Region merge; Color image segmentation

1. Introduction

Image segmentation is a key step in image analysis andpattern recognition. It refers to the decomposition of a sceneinto its components [1,2], and has been widely applied inmany areas such as the skin tumor feature detection [3,4],the road following for robot vehicles [5], etc.

Color information is very useful for image processing.While human eyes can detect only in the neighborhood of afew dozen gray levels at any point in a complex image dueto brightness adaptation, they have the ability to distinguishthousands of color shades and intensities. There are threeperceptual attributes of color, namely brightness, hue, andsaturation. Basically, color image segmentation is the com-bination of the segmentation results of three components ofthe color space. Gray level segmentation methods are usedfor processing each component of a color space. Color imagesegmentation approaches can be roughly classi2ed into four

∗ Corresponding author. Tel.: +1-435-797-2054; fax: +1-435-797-3265.

E-mail address: [email protected] (H.D. Cheng).

types: histogram-based approaches, neighborhood-based ap-proaches, clustering-based approaches, and physical-basedapproaches [6]. A combination of these approaches is oftenutilized for color image segmentation.

The histogram-based approach utilizes the statistics ofpixels, which indicates the homogeneous objects, to obtainthe information of clusters. Color image segmentation canbe considered as a three-dimensional histogram segmenta-tion which is very time-consuming. However, threshold val-ues can be acquired from three color components separately,and the clustering information can be combined to constructthe 2nal results. The selection of the threshold values is im-portant for further processing. Ohlander et al. [7] select thepeak of the histogram by using a priority list of nine 2xedconditions, which is collected from RGB, HSI, and YIQcolor spaces, while Holla [8] 2nds peaks and valleys using2D histogram of opponent color pairs, which separate thesignal into luminance and chrominance. In both of these twopapers, color features are extracted and applied to histogramsegmentation. Recently, many applications use scale-space1lter (SSF) to analyze histograms in three color compo-nents of the color space [6,9,10]. The advantage of SSF

0031-3203/03/$30.00 ? 2003 Pattern Recognition Society. Published by Elsevier Science Ltd. All rights reserved.PII: S0031 -3203(02)00293 -5

mailto:[email protected]

1546 H.D. Cheng, J. Li / Pattern Recognition 36 (2003) 1545–1562

histogram segmentation method is that the local extrema ofthe histogram’s derivative cannot disappear from coarse to2ne scale. The drawback of the histogram method is that itdoes not consider the local information among pixels.

The neighborhood-based approach usually uses unifor-mity criteria to segment the regions in the image. Regionmerge method, which can be divided into two categories:region growing and split-merge, is used in many applica-tions. Ismaili and Gillies [11] use regression analysis anduniform hue seeds to merge the regions. The correlation rand regression line slope �1 on RG plane is used for classi-2cation if uniformity criterion is satis2ed. Further growingis performed using the neighborhood seeds in a quadtree.Chung-Lin Huang et al. [9] utilize Markov random 2eld(MRF) with region labeling to classify the regions. TheMRF model processes a label array with the same size asthe original image, and the iterated conditional modes areselected as the relaxation algorithm to minimize the energyfunction. The idea of MRF is also discussed in Ref. [6]. Theregion merge method is relatively time-consuming when itmerges the seeds to the neighborhoods.

The clustering-based approach usually uses fuzzy logicto de2ne the membership of the pixels and the regions arecreated by inspecting the membership values of pixels usingthe partition method, e.g. the fuzzy c-means (FCM) algo-rithm. Huntsberger et al. [12] de2ne the region if the mem-bership value is above “�-cuto: ”. Lim and Lee [10] assignthe unclassi2ed pixels produced after the coarse segmenta-tion to the closest classes using FCM method. Lim and Lee[10] mentioned that the number of classes and their centervectors for FCM can be determined by the coarse segmen-tation. The cluster number is relatively large if the scene iscomplicate since coarse segmentation, which is SSF, onlyroughly determines the intervals. For clustering-based ap-proaches, the fuzzy model of the image is a crucial factorfor the successful implementation.

In the multi-spectral images, the noise often a3ects thesegmentation results. Also some e3ects like shadow andhighlights are diJcult for processing by the method men-tioned above. The physical-based approach basically dealswith these situations. However, the problem of this methodis that the properties of the changes in materials should beknown and modeled properly [13].

In this paper, a color image segmentation algorithm basedupon the fuzzy homogeneity is proposed. Each color com-ponent is transformed into corresponding fuzzy domain toobtain the image model. To improve the histogram segmen-tation results, the homogeneity histogram, which is based onthe homogeneity values of the pixels, is de2ned. Both globaland local informations are taken into account while calcu-lating the homogeneity features of the fuzzi2ed image. Onlythe homogeneous objects will contribute to homogeneity his-togram analysis. The SSF is used to separate the fuzzy homo-geneity histogram into several intervals that are boundedby the local extrema of its derivatives. Finally, we mergethe sub-regions with similar colors by utilizing CIE color

di3erence criterion and cluster sizes to prevent over-segmentation.

2. Proposed algorithm

2.1. Fuzzy maximum entropy principle

Ambiguity and uncertainty caused by data manipulation,such as image projection from 3D to 2D and image dis-cretization from analog signal, are very common in imageprocessing. Fuzzy set theory is useful to deal with the un-certainty and fuzziness of these situations. While fuzzy settheory is commonly applied to model monochrome image,color image can also be described as fuzzy models withthree components. The R; G; B color system is used in ourapplication, and each component associates with a fuzzymodel describing the brightness of an image. The standardS-function is used for the membership function to representthe degree of brightness. It is de2ned [14,15] as

�bright(g) = S(g; a; b; c) =

0; g6 a;

(g− a)2

(b− a)(c − a); a6g6b;

1− (g−c)2(c−b)(c−a) ; b6g6c;

1 g¿ c;(1)

where g is a variable representing the gray level, a, b,and c are the parameters which determine the shape of theS-function. The corresponding fuzzy entropy is de2ned as

H (F) =1

MN ln 2

M∑x=1

N∑y=1

Sn(�bright(g(x; y))); (2)

where Sn(�) is Shannon’s entropy function. For simplicity,we use � to represent �bright(g(x; y)) so that Sn(�) can bewritten as

Sn(�) =−� ln � − (1− �) ln(1− �): (3)

In this paper, three components of the RGB color spaceare processed separately. Three fuzzy models should be con-structed based on the maximum entropy principle for eachcomponent. For each fuzzy model, a and c are chosen as thedarkest and brightest gray values in the corresponding colorcomponent, and b changes in the range of [a; c]. The b valuewhich corresponds to the maximum entropy, as shown inEq. (2), is selected.

2.2. Fuzzy homogeneity histogram method

The histogram-based segmentation does not need a pri-ori information of the image, however, it su3ers from thelack of local spatial information. Homogeneity deals withthe local information of the image with the assumption that

H.D. Cheng, J. Li / Pattern Recognition 36 (2003) 1545–1562 1547

objects can be represented by several homogeneous regionswith some fuzzy uncertainty. Two characteristics, standarddeviation and discontinuity of the intensities, are derivedfrom each of the three fuzzy domains, i.e. R, G, and B. ForanM×N color image, g�ij ∈ [0; G�] with �∈{R; G; B} is theintensity of the corresponding color component of a pixel atthe location (i; j). X × Y is a window centered at (i; j). Foreach color component, fuzzy homogeneity value is de2nedas

��h(i; j) = {‘�X×Y (��s (g�; i; j)) ∨ #�X×Y (�

�s (g�; i; j))

|g�(i; j)∈ [0; G�] 16 i6M;

16 j6N}; (4)

where ��h(i; j) is the fuzzy homogeneity value at the position(i; j), ��s (g; i; j) is the S-function, and G� is the maximumintensity of the color component. ‘�(•) and #�(•) are Lapla-cian operator and standard deviation operator, respectively,which are de2ned as [1,16]

‘�X×Y (��; i; j) =

{(��(i; j)− 1

4

∑��(k; l))

∣∣∣∣(k; l)∈ ((k = i; l∈ (j ± 1));

(l= j; k ∈ (i ± 1)))}

(5)

and

#�X×Y (��; i; j) =

{√1

X · Y∑

(��(k; l)− L��(i; j))2

∣∣∣∣ (k; l)∈(i ±

⌊X2

⌋; j ±

⌊Y2

⌋)}(6)

with

L��(i; j) ={ ∑

��(k; l)X · Y

∣∣∣∣ (k; l)∈(i ±

⌊X2

⌋; j ±

⌊Y2

⌋)}:

(7)

‘�(•) represents the discontinuity, i.e., the abrupt changesof the corresponding color component, and #�(•) representsthe standard deviation which describes the contrast of thecolor component within a local region. The composition rule[17] is used to 2nd the fuzzy homogeneity of the pixel. Thestandard deviation and discontinuity values are normalizedin order to achieve computational consistency.

The fuzzy homogeneity value of individual color com-ponent at each pixel of the image is in [0; 1]. The moreuniform the local region surrounding a pixel is, the largerthe homogeneity value the pixel has. That is to say, theobjects with the same intensity value in the moving win-dow will have the largest fuzzy homogeneity value. Thehomogeneity histogram, which describes the distributionof the homogeneity across corresponding color compo-nent, is obtained by adding up the fuzzy homogeneity

values of the corresponding gray levels of each colorcomponent.

2.3. Scale space 1lter

The idea of using Gaussian 2lters across a continuumof scales demonstrates an important role in constructingsymbolic signal description. Babaud et al. [18] proved thatGaussian kernel is the only linear 2lter which guaran-tees that extrema not be deleted from coarse to 2nescale.

For a continuous signal & : {R → R}, the scale-spaceimage is de2ned by Witkin [19] as a function ' : {R×R′ →R} with '(x; 0) = &(x) and convolution with the Gaussiankernel g : {R× R+ → R}:

'(x; () = g(x; () ∗ &(x)

=∫ ∞

−∞&(x − �)

1√2)(

exp[−�2

2(2

]d�; (8)

where “∗” denotes convolution, R and R+ are the set ofreal number and real positive number, respectively, andR′ = {R+ ∪ {0}}. (x; (), with (2 varying from a small toa large value, determines the scale space from 2ne scaleto coarse scale. Fingerprints are obtained by 2nding thelocation of zero-crossings of the derivatives of the signalwith multi-scales in the scale space. Using 2ngerprints, theinterval tree is built up by detecting the zero-crossing atcoarse scales and localized by tracking contours in scalespace down to 2ne scales [19], as shown in Fig. 1(c). Ac-tive node, which is a node in the interval tree with heightgreater than the mean value of its o3spring, determines thecluster of the continuous signal. In this paper, we start fromthe half of the top scale, 2nd the zero-crossing in the 2n-gerprint and then trace down to the coarse scale at the valueof 10, to build the interval tree. For example, the top scalevalue in Fig. 1 is 46, therefore, we search the zero-crossingof scale number 23 in the 2ngerprint. SSF can be used forhistogram analysis [6,9,20], and the locations of the secondderivative zero-crossings can be found using the followingequation:

'xx(x; () =929x2 ['(x; ()]: (9)

The scale-space extrema at a certain scale ( partitionthe gray levels of the histogram into several intervals thatassociate with the peaks and valleys in the correspondinghistogram, as shown in Fig. 1. Hence, we can determine thenumber and locations of the signi2cant peaks and valleys inthe histogram. Once the locations of peaks and valleys forthe homogeneity histogram of each color component are de-termined, the threshold values can be used for the segmen-tation in the corresponding color component. The algorithmfor thresholding of three color components is described as


0

0.002

0.004

0.006

0.008

0.01

0.012

0 50 100 150 200 250

P(r

)

Gray Level -->

0

5

10

15

20

25

30

35

40

45

50

0 50 100 150 200 2500

5

10

15

20

25

30

35

40

45

50

0 50 100 150 200 250

(a)

(b) (c)

Fig. 1. Scale-space analysis of the histogram: (a) histogram of the image, (b) 2ngerprints of the histogram, and (c) the interval tree.

follows:

=∗ Color Image Homogeneity Histogram Thresholding ∗=BeginEnumerate i∈{R; G; B};Calculate three homogeneity histograms hi;De2ne three arrays Si for storing threshold values of color

components;FOR i := 1 TO 3 indicating three color components{

Convolve hi with the second order Gaussian kernelto build Li(&);

Find 2ngerprint Li(&) = 0 and build up the interval tree;Segment the homogeneity histogram and write threshold

values into Si;}=∗ end FOR ∗=

FOR each pixel P(i; j) in the color image{IF RGB components of P(i; j) belong to any

valid cluster THEN{

Assign the cluster index to P(i; j);Increase the counter of the cluster by 1;Add pixel value to Rtotal, Gtotal, Btotal ofthe cluster;

}=∗ end IF ∗=}=∗ end FOR ∗=Calculate average values LR; LG; LB for each cluster;Use LR; LG; LB to represent the color of P(i; j);End

After using the method described above, valid classesare determined from the fuzzy homogeneity histogram. This


0

0.02

0.04

0.06

0.08

0.1

0.12

0 0.2 0.4 0.6 0.8 1

P(r

)

Fuzzy Membership -->

Fuzzy Homogeneity Histogram

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0 0.2 0.4 0.6 0.8 1

P(r

)



0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

0 0.2 0.4 0.6 0.8 1

P(r

)



(d)

(f)(e)

(c)

(b)(a)

Fig. 2. Results of the proposed approach: (a) original image “colors2” with RGB description, (b) result after color pixel merging methodwith 22 colors, (c) result after color di3erence merging method with 2ve colors, and (d)–(f) red, green, and blue color components offuzzy homogeneity histogram.

procedure reduces the computational burden required for 3Dhistogram segmentation.

2.4. Color region merging

At the previous stage, a coarse segmentation of the im-age is obtained. Color region merge technique is needed

in order to re2ne the segmentation results. There are threecases that require color region merge. First, regions withsmall number of pixels should be merged. Second, homo-geneous regions with narrow color transition might be splitas separate regions having small color di3erence. Third,there might be some clusters that are quite close to eachother in color distance.


0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0 0.2 0.4 0.6 0.8 1

P(r

)



0

0.005

0.01

0.015

0.02

0.025

0.03

0 0.2 0.4 0.6 0.8 1

P(r

)



0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0 0.2 0.4 0.6 0.8 1

P(r

)



(d)

(e) (f)

(a) (b)

(c)

Fig. 3. Results of the proposed approach: (a) original image “2re” with RGB description, (b) result after color pixel merging method with23 colors, (c) result after color di3erence merging method with 2ve colors, and (d)–(f) red, green, and blue color components of fuzzyhomogeneity histogram.

2.4.1. CIE color di:erence descriptionQuantitative measurement of color di3erence between

any two colors is needed. Small di3erences in color aredescribed on observations of just noticeable di:erences(JNDs). Here, CIE L∗; u∗; v∗ formula is employed in

order to avoid large complexity [1]. The criterion is de2nedas

L∗ = 25(100YY0

)1=3

− 16;


0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0 0.2 0.4 0.6 0.8 1

P(r

)



0

0.005

0.01

0.015

0.02

0.025

0 0.2 0.4 0.6 0.8 1

P(r

)



0

0.02

0.04

0.06

0.08

0.1

0.12

0 0.2 0.4 0.6 0.8 1

P(r

)



(f )(e)

(d)

(a) (b)

(c)

Fig. 4. Results of the proposed approach: (a) original image “rose” with RGB description, (b) result after color pixel merging method with21 colors, (c) result after color di3erence merging method with 2ve colors, and (d)–(f) red, green, and blue color components of fuzzyhomogeneity histogram.

u∗ = 13L∗(u′ − u0);

v∗ = 13L∗(v′ − v0);

u′ = u;

v′ = 1:5v =9Y

X + 15Y + 3Z;

(Ps)2 = (PL∗)2 + (Pu∗)2 + (Pv∗)2; (10)

where Ps is the color di3erence, (u0; v0; Y0) refers to refer-ence white and XYZ color system is transformed from RGB


(a) (b)

(d)(c)

(e) (f)

Fig. 5. Results of the proposed approach: (a) original image “football” with RGB description, (b) resulting image with 12 colors, (c) originalimage “airplane” with RGB description, (d) resulting image with 11 colors, (e) original image “peppers” with RGB description, and(f) resulting image with 2ve colors.

system using the following equation:

X

Y

Z

=

0:490 0:310 0:200

0:177 0:813 0:011

0:000 0:010 0:990

R

G

B

: (11)

CIE space has metric color di3erence sensitivity to a goodapproximation and is very convenient to measure small colordi3erence, while the RGB space does not [21].

2.4.2. Region merging methodAt the previous stage, we build up the segmentation al-

gorithm based on fuzzy homogeneity values of the images.


(a) (b)

(d)(c)

Fig. 6. Results of the proposed approach: (a) original image “monument” with RGB description, (b) resulting image with 10 colors,(c) original image “sail” with RGB description, and (d) resulting image with seven colors.

That is to say, due to composition operation in the fuzzydomain, the homogeneity pixels shade has a dominant inQu-ence on the histogram segmentation than the ordinary his-togram method which does not use homogeneity statistics.After we segment the original image with 3D clusters, whichis obtained by SSF method as mentioned before, pixel num-ber and color di3erence become important criteria in region

merge since we need to merge small clusters into its closesthomogeneity clusters with small color distance. In order toobtain the desired e3ects, two consecutive region mergingmethods are employed.

2.4.2.1. Region merging by number of pixels. The clustercontaining a small number of pixels should be merged. In


(a)

(b) (c)

(d) (e)

Fig. 7. Comparison of fuzzy homogeneity histogram and homogeneity histogram: (a) original image “peach” with RGB description, (b)result after SSF of the proposed method, (c) 2nal result of the proposed method, (d) result after SSF of the non-fuzzy approach, (e) 2nalresult of the non-fuzzy approach, (f) (h) and (j) red, green, and blue color components of fuzzy homogeneity histogram, (g) (i) and (k)red, green, and blue color components of homogeneity histogram.

order to maintain the color continuity of the objects in theimage, we merge the region iteratively according to the sizeof clusters and merge smaller one to its closest larger cluster,

which has the least color di3erence value with the previousone among all clusters. This procedure should be executediteratively until the pixel number in each cluster is greater


0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

0 0.2 0.4 0.6 0.8 1

P(r

)



0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0 32 64 96 128 160 192 224

P(r

)

Gray Level -->

Homogeneity Histogram

0

0.002

0.004

0.006

0.008

0.01

0.012

0.014

0 0.2 0.4 0.6 0.8 1

P(r

)



0

0.002

0.004

0.006

0.008

0.01

0.012

0 32 64 96 128 160 192 224

P(r

)

Gray Level -->


0

0.002

0.004

0.006

0.008

0.01

0.012

0 0.2 0.4 0.6 0.8 1

P(r

)



0

0.002

0.004

0.006

0.008

0.01

0.012

0 32 64 96 128 160 192 224

P(r

)

Gray Level -->


(f ) (g)

(h) (i)

(j) (k)

Fig. 7. (Continued).

than the pre-speci2ed threshold, which is 1000 in this paper,determined by experiments. The algorithm is described asfollows:

=∗ Region merge using color di3erence and pixel number */BeginDe2ne minPixNumInCluster := minimum number of

pixels in clusters preferred;De2ne numOfCluster := number of clusters we have;De2ne numOfPixInCluster := number of pixels

in a cluster;Sort clusters according to the number of pixels in

increasing order;Find the minimum color distance dxy for each clusterCx: dxy = min(d(Cx; Cy)); y¿ x

Record dxy and the index of cluster y;FOR i := 0 To numOfCluster DO{IF the pixel number in any cluster less than

minPixNumInClusterTHEN Merge the cluster into the cluster y;ELSE Stop;

}=∗ end FOR ∗=End


2.4.2.2. Region merging by color distance. After usingthe above algorithm, the process stops when all cluster sizesare greater than the pre-speci2ed threshold value. Thereare cases when both the regions both have large sizes butwith similar colors. We should combine these regions. Twothreshold values, one is based on color di3erence distribu-tion and the other is based on the perceptual uniform con-cept, which is related to human eye sensitivity to the colordi3erence, are proposed.

Suppose we have M regions generated from the previ-ous stages, there are totally M colors, each standing forone region. Each region has a structure which records thenumber of pixels of the region, and the color di3erence iscalculated by CIE L∗; u∗; v∗ formula. The color di3erenceis computed for any two of these M regions and totallythere are N =M (M −1)=2 color di3erences produced. Eachcolor di3erence associates with a pixel number, which is thesum of pixels in the two corresponding clusters with whichthe color di3erence is derived. The standard deviation iscalculated as

d4 =

√∑((di − d�)2 ∗ ni)∑

ni

∣∣∣∣∣∣ i∈ [0; N ]

(12)

with

d� =

∑Ni=1 di ∗ ni∑N

i=1 ni; (13)

where d� and d4 are the mean and standard deviation, re-spectively, N denotes the total number of color di3erences,di is the color di3erence, and ni is the number of pixels as-sociated with the color di3erence as mentioned above. Thethreshold for the color merging based on color di3erencedistribution is Td1 = d� − d4.

A perceptual color di3erence threshold Td2 is determinedby experiments according to the concept of JND. If the colordi3erence is less than the perceptual value, it can be consid-ered that two regions are similar in color and can be merged.In order to determine the constant value Td2, we make exper-iments in our image database by merging the regions withthe color di3erence less than Td2 to see the color distortion.Td2 is 2nally selected as 1.2 according to the results. The 2-nal threshold value Td is determined by Td=max{Td1; Td2}.Any regions with the color di3erence less than the thresh-old value Td should be merged. The value of Td will a3ectthe number of segments, and larger Td may result fewersegments.

2.5. Defuzzi1cation

Defuzzi2cation algorithm is necessary to map the pro-cessed color image from the fuzzy domain to the spacedomain. Since the previous mapping method from the

space domain to the fuzzy domain has already determinedthe parameters of the S-function, we should apply the in-verse S-function to defuzzify the processed image, which is

(a)

(b)

(c)

Fig. 8. Comparison of fuzzy homogeneity histogram and homo-geneity histogram: (a) original image “golf” with RGB description,(b) result of the proposed method, (c) result of the non-fuzzy ap-proach, (d) (f) and (h) red, green, and blue color components offuzzy homogeneity histogram, (e) (g) and (i) red, green, and bluecolor components of homogeneity histogram.


0

0.005

0.01

0.015

0.02

0.025

0 0.2 0.4 0.6 0.8 1

P(r

)



0

0.005

0.01

0.015

0.02

0.025

0 32 64 96 128 160 192 224

P(r

)

Gray Level -->


0

0.005

0.01

0.015

0.02

0.025

0.03

0 0.2 0.4 0.6 0.8 1

P(r

)



0

0.005

0.01

0.015

0.02

0.025

0.03

0 32 64 96 128 160 192 224

P(r

)

Gray Level -->


0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0 0.2 0.4 0.6 0.8 1

P(r

)



0

0.005

0.01

0.015

0.02

0.025

0 32 64 96 128 160 192 224

P(r

)

Gray Level -->


(d) (e)

(f) (g)

(h) (i)

Fig. 8. (Continued).

de2ned as

g�(��) = S−1(��; a�; b�; c�)

=

a�; �� = 0;√��(b�−a�)(c�−a�)+a�; 06��6

b�−a�c�−a� ;

c�−√

(1−��)(c�−b�)(c�−a�); b�−a�c�−a�6�

�61;

c�; �� = 1;(14)

where �∈{R; G; B}.

3. Experiments and discussions

The algorithms have been applied to a variety of colorimages in RGB format. The intensity value for each colorcomponent of the test images is from 0 to 255. The programis executed on Sun Sparc Ultra-1. The characteristics of theimages, the number of colors, the CPU time, etc. are listedin Table 1.

The criteria we adopt to evaluate the processing resultsin color image segmentation are: (1) the main feature of theobjects should be preserved; (2) no apparently distortion orcolor change; and (3) the “detail” information of the image


Table 1Results of the proposed approach in RGB color space

Image name Size CPU # of clusterstime(s) Original After After merge After merge

color SSF phase I phase II

Peach 243 × 278 26.01 243 143 24 8Golf 353 × 268 24.65 254 65 19 10Duck 256 × 256 12.16 248 41 12 7Salad 265 × 187 10.81 255 40 10 8Colors2 255 × 255 17.15 255 97 22 5Fire 256 × 256 14.65 255 52 23 6Airplane 256 × 256 15.49 216 80 21 11Football 331 × 283 33.76 255 127 21 12Monument 206 × 339 15.90 254 63 20 10Peppers 256 × 256 13.05 225 72 24 5Rose 256 × 256 12.11 253 39 21 5Sail 245 × 358 16.42 188 54 17 7

Table 2Results of the non-fuzzy approach in RGB color space

Image name Size CPU # of clusterstime(s) Original After After merge After merge

color SSF phase I phase II

Peach 243 × 278 16.11 243 66 22 7Golf 353 × 268 9.43 254 26 15 5Duck 256 × 256 12.49 248 50 14 3Salad 265 × 187 7.56 255 30 8 3Colors2 255 × 255 22.72 255 137 19 6Fire 256 × 256 12.14 255 43 29 5Airplane 256 × 256 14.62 216 51 17 6Football 331 × 283 16.61 255 40 23 5Monument 206 × 339 7.53 254 28 10 6Peppers 256 × 256 15.13 225 56 26 6Rose 256 × 256 13.15 253 81 15 8Sail 245 × 358 21.30 188 90 19 9

should be kept. The experimental results are illustrated toevaluate the performance of the algorithm.

First, some of the segmentation results using the proposedmethod are shown in Figs. 2–4. It is obvious that the re-sulting images can preserve the main features of the objectswith much smaller number of colors. Fig. 2 can represent thesegmented image “colors2” which mainly consists of twoobjects: the Qower and the leaves, with 2ve colors. In Fig. 3,the clouds, the ground, and the sky are well segmented. Wenotice that before merging, the segmentation number is 52and after the 2rst step of the merging method, the resultingimage has 23 colors, and after the second step of the merg-ing method using color di3erence, the image has only 2vecolors. The contours of the objects are well formed. Since

the scale-space method is a rough segmentation method,the number of clusters after SSF is relatively large and themerging method is important to obtain a well-segmentedimage. In Fig. 4, the resulting image “rose” is segmentedinto 2ve colors and the result is very good. In Figs. 5 and 6,we apply the proposed approach to many di3erent kinds ofimages. The experimental results show that the fuzzy homo-geneity method is very adaptive and e3ective. Table 1 liststhe color number for each segmentation step of the proposedmethod.

We have compared the experimental results of the pro-posed approach with those of non-fuzzy homogeneity colorsegmentation method, and used the same criteria of merging(Section 4.2) for both approaches. In Fig. 7, we notice that


(e) (f)

(d)(c)

(a) (b)

Fig. 9. Results of the proposed approach: (a) original image “salad” with RGB description, (b) original image “duck” with RGB description,(c) result of the proposed approach with eight colors, (d) result of the proposed approach with seven colors, (e) result of the non-fuzzyapproach with three colors, and (f) result of the non-fuzzy approach with three colors.

after using SSF method of non-fuzzy homogeneity method,the green color in upper left-hand side and lower right-handside of the image was mis-segmented as black, as shown inFigs. 7(d) and (e). The background information even is lostafter region merging using non-fuzzy approach (Table 2).In Fig. 8, the image “golf” mainly consists of 2ve objects:the men, the tree, the lake, the grassland and the hill. Theoriginal color image has 254 di3erent colors. After apply-ing the proposed approach, the resulting image has only 10colors which clearly display the objects in the original im-age. Comparing the proposed approach with the non-fuzzyapproach, we 2nd that some important information, suchas the background, the lake and the hill, has been lost ifnon-fuzzy approach is used. This can be explained by an-alyzing the histograms of both methods. From Figs. 8(d)–(i), we can 2nd that there exist peaks at the bright and darkpart of fuzzy homogeneity histogram. These peaks indicate

the lake and the tree that disappeared in non-fuzzy homo-geneity histogram. In Fig. 9, the glasses and plates are wellsegmented using the proposed method while they cannot bedistinguished by non-fuzzy approach. The main features ofthe duck are clearly segmented by the proposed approach,in contrast, the non-fuzzy homogeneity-based method canonly exhibit the contour of the duck.

We also compared the results of the proposed approachwith those using the approaches in Refs. [22] and [23], re-spectively. For the “peach” image, the method in Ref. [22]produces 24 colors, the one in [23] produces 43 colors,and the proposed approach produces eight colors. For the“rose” image, the method in Ref. [22] produces 39 colors,the method in Ref. [23] produces 16 colors, and the pro-posed approach produces six colors (Fig. 10). The majordi3erences of these methods are the criteria for clusteringand merging.


(c)

(a) (b)

(d)

(f)(e)

Fig. 10. (a) Result of “peach” image using the approach in Ref. [22] with 24 colors, (b) result of “rose’ image using the approach inRef. [22] with 39 colors, (c) result of “peach” image using the approach in Ref. [23] with 43 colors, (d) result of “rose” image using theapproach in Ref. [23] with 16 colors, (e) result of “peach” image using the proposed approach with eight colors, and (f) result of “rose”image using the proposed approach with six colors.

From the histogram analysis, we 2nd that the proposedfuzzy approach is only interested in the area of the fuzzy re-gion (a; b; c) and the width of the gray levels S1 is mappedto the membership region S2. The range S2 is much widerthan that of S1. This is determined by the non-linear charac-teristic of the S-function, as shown in Fig. 11. For example,the histogram in Fig. 11(b) will be mapped to the fuzzy do-main, as shown in Fig. 11(c). Usually, the fuzzy approachgenerates more clusters because the peaks which cannot beclearly distinguished in the original histogram may be seenclearly in the fuzzy domain after the histogram mapping,as shown in Table 1. The information in the extra clusters,which cannot be obtained by the non-fuzzy method, could bevery useful for the merging method, which can re-organizethe information, and eventually produce the better segmen-tation results.

4. Conclusions

In this paper, we proposed a fuzzy homogeneity histo-gram and scale-space approach to color image segmentation.The proposed algorithm uses fuzzy set theory and maximumfuzzy entropy principle to map the color image from spacedomain to fuzzy domain, which will keep the maximum in-formation. The fuzzy homogeneity histogram takes care ofboth global and local informations. SSF algorithm is usedto analyze the homogeneity histogram to obtain the clus-tering informations. Color region merge is conducted basedon both pixel number and color distance. The 2nal result istransformed from fuzzy domain to space domain using in-verse S-function. The experimental results demonstrate thatthe proposed approach can segment color images e3ectivelyand accurately, and it may have many applications in image


0 32 64 96 128 160 192 2240

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Gray Level >

Fuz

zy M

embe

rshi

p

S2

S1

0

0.002

0.004

0.006

0.008

0.01

0.012

0.014

0.016

0.018

0.02

0 32 64 96 128 160 192 224

P(r

)

Gray Level -->

0

0.002

0.004

0.006

0.008

0.01

0.012

0.014

0.016

0.018

0.02

0 0.2 0.4 0.6 0.8 1

P(r

)


(a)

(b) (c)

Fig. 11. S-function fuzzy mapping analysis: (a) S-function, (b) histogram of Gaussian distribution with � = 127 and 4 = 20, and(c) histogram after fuzzy mapping using S-function.

processing such as image understanding, image compres-sion, image retrieval, etc.

Readers may download the corresponding color imagesfrom http://www.cs.usu.edu/cheng to understand the abovediscussions better.

References

[1] A.K. Jain, Fundamentals of Digital Image Processing,Prentice-Hall, Englewood Cli3s, NJ, 1989.

[2] K.S. Fu, J.K. Mui, A survey on image segmentation, PatternRecognition 13 (1981) 3–16.

[3] G.A. Hance, S.E. Umbaugh, R.H. Moss, W.V. Stoecher,Unsupervised color image segmentation with application toskin tumor borders, IEEE Eng. Med. Biol. 15 (1996) 104–110.

[4] S.E. Umbaugh, R.H. Moss, W.V. Stoecker, G.A. Hance,Automatic color segmentation algorithms, IEEE Eng. Med.Biol. 12 (3) (1993) 75–82.

[5] X. Lin, S. Chen, Color image segmentation using modi2edHSI system for road following, Proceedings of the IEEEConference on Robotics and Automation, 1998–2003,Sacramento, CA, April 1991.

[6] Jianqing Liu, Yee-Hong Yang, Multiresolution color imagesegmentation, IEEE Trans. Pattern Anal. Mach. Intell. 16 (7)(1994) 689–700.

[7] R. Ohlander, K. Price, D.R. Reddy, Picture segmentationusing a recursive region splitting method, Comput. GraphicsImage Process. 8 (1978) 313–333.

[8] K. Holla, “Opponent colors as a 2-dimensional featurewithin a model of the 2rst stages of the human visualsystem, Proceedings of the 6th International Conference

http://www.cs.usu.edu/cheng


on Pattern Recognition, Germany, October 1982,pp. 561–563.

[9] Chung-Lin Huang, Tai-Yuen Cheng, Chaur-Chin Chen,Color images segmentation using scale space 2lter andMarkov random 2eld, Pattern Recognition 23 (10) (1992)1217–1229.

[10] Y.W. Lim, S.U. Lee, On the color image segmentationalgorithm based on the thresholding and the fuzzy c-meanstechnique, Pattern Recognition 23 (9) (1990) 935–952.

[11] I.A. Ismaili, D.F. Gillies, Color image segmentation usingregression analysis in RGB space, Mach. Graphics Vision3 (1) (1994) 373–384.

[12] T.L. Huntsberger, C.L. Jacobs, R.L. Cannon, Iterative fuzzyimage segmentation, Pattern Recognition 18 (2) (1985)131–138.

[13] W. Skarbek, A. Koschan, Color image segmentation—asurvey, Technical Report, Technical University of Berlin,October 1994.

[14] H.D. Cheng, J.R. Chen, J. Li, Threshold selection basedon fuzzy C-partition entropy approach, Pattern Recognition31 (7) (1998) 857–862.

[15] H.D. Cheng, J.R. Chen, Automatically determine themembership function based on the maximum entropyprinciple, Inf. Sci. 96 (3/4) (1997) 163–182.

[16] R.C. Gonzalez, P. Wintz, Digital Image Processing,Addison-Wesley Company, Reading, MA, 1987.

[17] T.J. Ross, Fuzzy Logic with Engineering Applications,McGraw-Hill, Inc., New York, 1995.

[18] J. Babaud, A.P. Witkin, M. Baudin, R.O. Duda, Uniquenessof the Gaussian kernel for scale-space 2ltering, IEEE Trans.Pattern Anal. Mach. Intell. PAMI-8 (1) (1986) 26–33.

[19] A.P. Witkin, Scale-space 2ltering: a new approach tomulti-scale description, in: S. Ullman, W. Richards (Eds.),Image Understanding, Ablex Publishing, Norwood, NJ, 1984,pp. 79–95.

[20] M.J. Carlotto, Histogram analysis using a scale-spaceapproach, IEEE Trans. Pattern Anal. Mach. Intell. PAMI-9(1) (1987) 121–129.

[21] G. Robinson, Color edge detection, Opt. Eng. 16 (5) (1977)479–484.

[22] H.D. Cheng, X.H. Jiang, J.L. Wang, Color image segmentationbased on homogram thresholding and region merging, PatternRecognition 35 (2002) 373–393.

[23] H.D. Cheng, Y. Sun, A hierarchical approach to color imagesegmentation using homogeneity, IEEE Trans. Image Process.9 (12) (2000) 2071–2081.

About the Author—HENG-DA CHENG received Ph.D. in Electrical Engineering from Purdue University, West Lafayette, IN, in 1985(supervisor: K.S. Fu). Currently, he is a Full Professor at Department of Computer Science, and Adjunct Full Professor at Department ofElectrical and Computer Engineering, Utah State University. Dr. Cheng is an Adjunct Professor and Doctorial Supervisor of Harbin Instituteof Technology (HIT), a Guest professor of Remote Sensing Application Institute, Chinese Academy of Sciences, a Guest professor ofWuhan University, a Guest professor of Shantou University, and a Visiting professor of Northern Jiaotung University.

Dr. Cheng has published more than 200 technical papers, is the co-editor of the book, Pattern Recognition: Algorithms, Architectures andApplications, and the editor of three conference proceedings. His research interests include: arti2cial intelligence, computer vision, patternrecognition & image processing, medical information processing, fuzzy logic, neural networks and genetic algorithms, parallel processing,parallel algorithms, and VLSI algorithms and architectures.

Dr. Cheng is the General Chair and Program Chair of the Fifth International Conference on Computer Vision, Pattern Recognition &Image Processing (CVPRIP2003), 2003. He was the General Chair and Program Chair of the Fourth International Conference on ComputerVision, Pattern Recognition & Image processing (CVPRIP2002), 2002, and General Chair and Program Chair of the Third InternationalConference on Computer Vision, Pattern Recognition & Image Processing (CVPRIP2000) 2000. He was the General Chair and ProgramChair of the First International Workshop on Computer Vision, Pattern Recognition & Image Processing (CVPRIP98), 1998, and was theProgram Co-Chair of Vision Interface ’90, 1990. He served as a program Committee Member and Session Chair for many conferences, andas a reviewer for many scienti2c journals and conferences.

Dr. Cheng has been listed in Who’s Who in the World, Who’s Who in America, Who’s Who in Communications and Media, Who’sWho in Science and Engineering, Who’s Who in Finance and Industry, Men of Achievement, 2000 Notable American Men, InternationalLeaders in Achievement, Five Hundred Leaders of InBuence, International Dictionary of Distinguished Leadership, etc. He has beenappointed as Member of the International Biographical Center Advisory Council, The International Biographical Center, England, and aMember of the Board of Advisors, the American Biographical Institute, USA.

Dr. Cheng is a Senior Member of IEEE Society, and a Member of the Association of Computing Machinery. Dr. Cheng is also anAssociate Editor of Pattern Recognition and an Associate Editor of Information Sciences.

About the Author—JIGUANG LI ([email protected]) received the Bachelor of Engineering degree from the Department ofElectrical Engineering, Zhejiang University, China, in 1993, the Master of Engineering degree from the Department of Electrical andComputer Engineering, Utah State University, UT, 1999.

From 1993 to 1996, he was an engineer working on advanced medical diagnostic system installation and maintenance in AnalogicScienti2c, Inc. in China. Since 2000, he has been with the Software Engineering Department at Embarcadero Technologies, Inc. as a softwareengineer. His research interests are image processing, control system and database tools design and implementation.

fuzzy homogeneity and scale-space approach to color image segmentation

Documents