pattern recognition using type-ii fuzzy sets

Information Sciences 170 (2005) 409–418

www.elsevier.com/locate/ins

Pattern recognition using type-II fuzzy sets

H.B. Mitchell *

ELTA Systems Ltd, Image Intelligence Exploitation Department (section 6174),

Image Intelligence and Radar Division, Ashdod, Israel

Received 30 July 2003; received in revised form 8 December 2003; accepted 5 February 2004

Abstract

Type II fuzzy sets are a generalization of the ordinary fuzzy sets in which the

membership value for each member of the set is itself a fuzzy set in ½0; 1�. We introduce a

similarity measure for measuring the similarity, or compatibility, between two type-II

fuzzy sets. With this new similarity measure we show that type-II fuzzy sets provide us

with a natural language for formulating classification problems in pattern recognition.

� 2004 Elsevier Inc. All rights reserved.

Keywords: Type-II fuzzy sets; Intuitionistic fuzzy sets; Type-II similarity measures;

Pattern recognition; Radiographic testing

1. Introduction

Pattern recognition problems typically involve the classification of an un-

known pattern Q given a set of K prototypes Pk, k 2 f1; 2; . . . ;Kg [1–6]. Each

prototype Pk belongs to a given class Cm, m 2 f1; 2; . . . ;Mg, which is specifiedby the indicator function Ak:

* Te

E-m

0020-0

doi:10.

Ak ¼ Cm if Pk belongs to the mth class Cm: ð1Þ

Let SðQ;PkÞ be a similarity measure which measures the degree of similarity,

or compatibility, between the unknown pattern Q and the kth prototype Pk.

l.: +972-8-8572-851; fax: +972-8-8572-844.

ail address: [email protected] (H.B. Mitchell).

255/$ - see front matter � 2004 Elsevier Inc. All rights reserved.

1016/j.ins.2004.02.027

mailto:[email protected]

410 H.B. Mitchell / Information Sciences 170 (2005) 409–418

Then, formally, we may write the process of classifying, or assigning, the un-

known pattern Q to the class C ¼ Ak , where

k ¼ argmaxk

ðSðQ;PkÞÞ: ð2Þ

Very often we apply the techniques of pattern recognition to situations

which are inherently vague and uncertain [7,8]. Such situations arise when theinformation regarding the prototypes is ‘‘linguistic’’ and is based on the

opinions and judgements of human experts. Examples of such situations are:

handwritten character recognition, fingerprint recognition, human face rec-

ognition, classification of X-ray images, classification of remotely sensed data.

One way of handling such situations is to make the information precise and to

give it a mathematically well-defined form by using the concepts and tech-

niques of fuzzy logic.

Let U denote the feature space. Then, mathematically, we represent theunknown pattern Q, and the prototypes, Pk, k 2 f1; 2; . . . ;Kg, with (ordinary)

fuzzy sets Q QðuÞ and Pk PkðuÞ, where u 2 U . The result is the following

optimization problem:

k ¼ argmaxk

ðSðQðuÞ;PkðuÞÞÞ; ð3Þ

where S denotes an appropriate similarity, or compatibility, measure [9].

Unfortunately, (3) is sometimes too precise in the sense that no uncertainty

whatsoever is allowed in specifying the fuzzy sets QðuÞ and PkðuÞ.Dengfeng and Chuntian [10] suggested that one way to introduce a con-

trolled amount of uncertainty into (3), is to replace the ordinary fuzzy sets QðuÞand PkðuÞ, with intuitionistic fuzzy sets [11], and to replace S with an appro-priate intuitionistic similarity measure [10,12–14]. This is correct if we model

the uncertainty with a uniform distribution. However, in many pattern rec-

ognition problems, it is more accurate to model the uncertainty with a non-

uniform distribution. In this case, we use type-II fuzzy sets eQðuÞ and ePkðuÞ and(3) becomes

k ¼ argmaxk

ðeSð eQðuÞ; ePkðuÞÞÞ; ð4Þ

where eS denotes a type-II similarity measure.

The main advantages of using a type-II framework are twofold:

• By using type-II fuzzy sets we transform a vague pattern classification

problem into a precise, well-defined, optimization problem.

• Type-II fuzzy sets, unlike ordinary fuzzy sets, retain a controlled degree of

uncertainty.

H.B. Mitchell / Information Sciences 170 (2005) 409–418 411

The main disadvantage to using a type-II formulation is

• The relatively high computational complexity.

The article is organized as follows. In Section 2 we discuss the concept of

type-II fuzzy sets. In Section 3 we describe a new type-II similarity measure eSwhich is specifically designed to measure the compatibility of two type-II fuzzy

sets. In Section 4 we show how we may reformulate the problem of pattern

classification using type-II fuzzy sets. In Section 5 we analyze a real-left pattern

classification problem and show how we may solve it, and similar classification

problems, using the new approach. Finally the article concludes with a brief

summary in Section 6.

2. Type-II fuzzy sets

The concept of a type-II fuzzy set was introduced by Zadeh [15] as a gen-

eralization of an ordinary fuzzy set. Type-II fuzzy sets are characterized by a

fuzzy membership function, i.e. the membership value for each element of theset, is itself a fuzzy set in ½0; 1�. 1 Thus, given the feature space U , a type-II

fuzzy set is defined as an object eA which has the following form:

1 Th

eA fhu; v; nAðu; vÞig; ð5Þ
where nAðu; vÞ represent the degree of membership of the element
ðu; vÞ; u 2 U ; v 2 ½0; 1� in eA. We find it convenient to write nAðu; vÞ as the

product lAðuÞmAðu; vÞ, where
lAðuÞ ¼ max
vðnAðu; vÞÞ; ð6Þ

mAðu; vÞ ¼nAðu;vÞlAðuÞ

if lAðuÞ > 0

1 otherwise

�ð7Þ

are known, respectively, as the primary and the secondary membership func-

tions. We delineate the region where nAðu; vÞ > 0, by means of a low mem-

bership function wAðuÞ, and a high membership function /AðuÞ, where
nAðu; vÞ > 0 if wAðuÞ6 v6/AðuÞ: ð8Þ
Physically the difference j/AðuÞ � wAðuÞj represents the uncertainty in specify-

ing the primary membership value lAðuÞ.We find it convenient to interpret eA from a statistical viewpoint [16–20]:

Suppose the feature space U is sampled at L points ul, l 2 f1; 2; . . . ; Lg, then we

e membership value for ordinary fuzzy sets is a crisp number in ½0; 1�.


may interpret eA as an ensemble of M ordinary, or embedded, membership

functions hðmÞA ðulÞ, m 2 f1; 2; . . . ;Mg:

hðmÞA ðulÞ ¼ rmðulÞ � ð/AðulÞ � wAðulÞÞ þ wAðulÞ; ð9Þ

where rmðulÞ is a random number chosen uniformly in the interval ½0; 1�.We associate a weight kðmÞ

A with each function hðmÞA ðulÞ, m 2 f1; 2; . . . ;Mg.

This is defined as the t-norm of the individual secondary membership grades

mAðul; hðmÞA ðulÞÞ, l 2 f1; 2; . . . ; Lg. Thus

kðmÞA ¼ tðmAðu1; hðmÞ

A ðu1ÞÞ; mAðu2; hðmÞA ðu2ÞÞ; . . . ; mAðuL; hðmÞ

A ðuLÞÞÞ; ð10Þ

where t denotes a given t-norm operator.

Further details concerning the statistical interpretation of type-II fuzzy sets

are given in [16–20].

3. Type-II similarity measure

We define a type-II similarity measure eS as follows. Given two type-II fuzzy

sets eA, eB, we suppose we have generated the corresponding embedded mem-

bership functions hðmÞA ðulÞ; hðnÞ

B ðulÞ, l 2 f1; 2; . . . ; Lg;m 2 f1; 2; . . . ;Mg; n 2f1; 2; . . . ;Ng. Let

Smn SðhðmÞA ðuÞ; hðnÞ

B ðuÞÞ ð11Þ

denote the similarity between the embedded functions hðmÞA ðuÞ and hðnÞ

B ðuÞ, whereSð�; �Þ denotes any ordinary similarity measure. Then we define the type-II

similarity measure between eA and eB as the weighted average

eSðeA; eBÞ ¼XMm¼1

XNn¼1

SmnKmn; ð12Þ

where the normalized weights

Kmn ¼tðkðmÞ

A ; kðnÞB ÞPM

i¼1

PNj¼1 tðk

ðiÞA ; kðjÞ

B Þð13Þ

satisfy the inequalities

06Kmn 6 1; ð14ÞXMm¼1

XNn¼1

Kmn ¼ 1: ð15Þ


4. Pattern recognition using type-II fuzzy sets

In order to solve (4) we require a procedure for calculating the appropriate

type-II fuzzy sets for the unknown pattern Q and for the prototypes Pk,k 2 f1; 2; . . . ;Kg.

4.1. Unknown pattern Q

We represent the unknown pattern Q with a type-II fuzzy set eQ which has a

primary membership function lQðulÞ, l 2 f1; 2; . . . ; Lg. At each ul, the mea-

surement of Q is subject to random measurement noise. This represents thephysical origin of the uncertainty in eQ. If we model the noise as a normal

distribution, Nð0; r2QÞ, with zero mean and standard deviation rQ, then the

corresponding low and high membership functions, wQðuÞ and /QðuÞ, are
wQðuÞ ¼ maxðlQðuÞ � arQðuÞ; 0Þ; ð16Þ/QðuÞ ¼ minðlQðuÞ þ arQðuÞ; 1Þ; ð17Þ
where a is a parameter chosen so that the probability of obtaining a mea-surement which lies outside the interval ½wQðuÞ;/QðuÞ� is very low. Typically, weuse a ¼ 2.

Mathematically, the individual embedded functions are limited to the

interval ½wQðulÞ;/QðulÞ� and are given by

hðmÞQ ðulÞ ¼ rmðulÞrQðulÞ þ lQðulÞ; ð18Þ

where rmðulÞ is a random number chosen from a normal distribution, Nð0; 1Þ,with zero mean and unit standard deviation. The embedded function hQðulÞ hasa weight

kðmÞQ ¼ t mQðu1; hðmÞ

Q ðu1ÞÞ; mQðu2; hðmÞQ ðu2ÞÞ; . . . ; mQðuL; hðmÞ

Q ðuLÞÞ� �

ð19Þ

associated with it, where

mQðul; hðmÞQ ðulÞÞ / exp� 1

2

hðmÞQ ðulÞ � lQðulÞ

rQðulÞ

!2

: ð20Þ

4.2. Prototypes Pk

We represent each prototype Pk, k 2 f1; 2; . . . ;Kg, with a type-II fuzzy setePk, or equivalently, an ensemble of N embedded membership functions hðnÞk ðulÞ,

n 2 f1; 2; . . . ;Ng, with weights kðnÞk .

The prototypes themselves are obtained by clustering a set of training pat-

terns: each prototypePk, k 2 f1; 2; . . . ;Kg, represents a cluster, or ensemble, of


training patterns: fhð1Þk ðulÞ; hð2Þ

k ðulÞ; . . . ; hðsÞk ðulÞ; . . .g, where hðsÞ

k ðulÞ denotes the

sth training pattern associated with the kth prototype. Then, at each ul, thereis a spread in the fhðsÞðulÞg and this is the physical origin of the uncertainty

in ePk.2

We regard all of the training patterns to be uniformly distributed between

the low and high membership functions, wkðulÞ and /kðulÞ, where

2 Th

feature

rkðul Þ

wkðulÞ ¼ maxh

ðhðhÞk ðulÞÞ; ð21Þ

/kðulÞ ¼ minhðhðhÞ

k ðulÞÞ: ð22Þ

The uncertainty associated with Pk is defined in terms of wkðulÞ and /kðulÞ:

rkðulÞ ¼1

2ð/kðulÞ � wkðulÞÞ; ð23Þ

and the embedded functions hðnÞk , n 2 f1; 2; . . . ;Ng, are assigned a unit weight:

kðnÞk ¼ 1: ð24Þ

5. Results

We illustrate the new approach by applying it to the problem of automatic

evaluation of welded structures using radiographic testing [21,22]. This is animportant topic in many manufacturing applications, both from the viewpoint

of cost control and from the viewpoint of safety [23].

Radiographic testing involves three stages:

• Production of a radiographic image of the welded structure by illuminating

it with an X-ray or gamma-ray source.

• Extraction of several features from the image.

• Classification of the structure as belonging to the ‘‘weld’’ class or to the‘‘nonweld’’ class.

Recently Liao and Li [22] collected 80 different radiographic images which

they divided into a training set consisting of 36 images (Table 1) and a test set

consisting of 44 images (Table 2). Three features ul, l 2 f1; 2; 3g, were extractedfrom each image which was classified as being ‘‘weld’’ or ‘‘nonweld’’.

Inspection of the training set (Table 1) shows that we may group the ‘‘weld’’

structures into five clusters, which we represent with the prototypes Pk,

is is, ofcourse, only true if the feature ul is required to identify Pk . Sometimes, a given

, say ul is not required to identify Pk . In this case, we set wkðul Þ ¼ 0, /kðul Þ ¼ 1 and

¼ 12.

Table 2

Feature values hQðulÞ, l 2 f1; 2; 3g, for each image in the test set

Image # u1 u2 u3 Image # u1 u2 u3

1 0.6471 0.0341 0.3490 23 0.2941 0.0238 0.1451

2 0.8824 0.0279 0.3020 24 0.5294 0.0109 0.0431

3 0.7647 0.0124 0.2588 25 0.4118 0.0687 0.1255

4 0.7647 0.0145 0.4784 26 0.2941 1.0000 1.0000

5 1.0000 0.0238 0.0394 27 0.5294 0.0372 0.1412

6 0.5294 0.0362 0.1608 28 0.7647 0.0770 1.0000

7 0.6471 0.0305 0.0235 29 0.5294 0.3409 0.1216

8 1.0000 0.0211 0.1922 30 0.2941 0.0176 0.0235

9 1.0000 0.0176 0.1882 31 0.2941 0.0537 0.0157

10 0.7647 0.0238 0.2000 32 0.4118 0.0692 0.1059

11 0.8824 0.0217 0.3059 33 0.5294 0.0150 0.1725

12 0.7647 0.0279 0.2902 34 0.4118 0.0573 0.1020

13 0.7647 0.0222 0.1882 35 0.2941 0.0465 0.0314

14 0.6471 0.0356 0.3255 36 0.7647 0.0439 0.0314

15 1.0000 0.0217 0.3333 37 0.5294 0.0475 0.0353

16 0.6471 0.0372 0.2267 38 0.2941 0.0233 0.1255

17 0.6471 0.0170 0.2078 39 0.2941 0.0274 0.1177

18 0.5294 0.0181 0.0353 40 0.4118 0.2764 1.0000

19 0.7641 0.0150 0.4745 41 0.6471 0.1545 1.0000

20 1.0000 0.0109 0.4588 42 0.5294 0.0129 0.2118

21 0.8824 0.0114 0.5098 43 0.2941 0.0269 0.1059

22 0.8824 0.0134 0.4314 44 0.5294 0.0212 0.1804

The structure in images 1–22 and 23–44 are, respectively, ‘‘weld’’ and ‘‘nonweld’’.

Table 1

Feature values hðsÞk ðulÞ, l 2 f1; 2; 3g, for each image in the training set

Image # u1 u2 u3 Image # u1 u2 u3

1 0.6471 0.0171 0.2039 19 0.4118 0.0689 0.1255

2 0.6471 0.0156 0.3059 20 0.2941 0.1068 0.0863

3 0.8824 0.0279 0.3020 21 0.7647 0.0374 0.0667

4 0.7647 0.0169 0.1843 22 1.0000 0.0139 0.5843

5 0.7647 0.0179 0.1961 23 0.2941 1.0000 0.3569

6 0.5294 0.0238 0.0353 24 0.2941 0.5581 0.6980

7 1.0000 0.0195 0.0392 25 0.7647 0.0476 0.6353

8 0.6471 0.0104 0.1490 26 1.0000 0.0157 0.6353

9 0.8824 0.0229 0.3059 27 0.2941 0.0174 0.0235

10 0.8824 0.0147 0.1490 28 0.2941 0.0539 0.0157

11 0.7647 0.0111 0.1569 29 0.2941 0.0441 0.1216

12 0.8824 0.0180 0.2941 30 0.5294 0.0164 0.1137

13 0.7647 0.0155 0.1373 31 0.4118 0.0459 0.0314

14 0.6471 0.0122 0.1686 32 0.5294 0.0662 0.1216

15 0.8824 0.0121 0.4784 33 0.2941 0.0468 0.0314

16 1.0000 0.0169 0.3098 34 0.2941 0.3766 1.0000

17 0.7647 0.0133 0.3020 35 0.2941 1.0000 1.0000

18 1.0000 0.0210 0.2314 36 0.5294 0.0110 0.0431

The structures in images 1–18 and 19–36 are, respectively, ‘‘weld’’ and ‘‘nonweld’’.


Table 3

Low and high membership functions, wkðulÞ and /kðulÞ, for features u1, u2 and u3 for the prototypesPk , k 2 f1; 2; . . . ; 11g

k wkðu1Þ /kðu1Þ wkðu2Þ /kðu2Þ wkðu3Þ /kðu3Þ1 0.48 0.58 0.00 1.00 0.00 0.04

2 0.60 0.70 0.00 0.10 0.00 0.50

3 0.71 0.81 0.00 0.03 0.00 1.00

4 0.83 0.93 0.00 0.50 0.00 1.00

5 0.95 1.00 0.00 1.00 0.00 0.50

6 0.25 0.35 0.00 1.00 0.00 1.00

7 0.36 0.46 0.00 1.00 0.00 1.00

8 0.48 0.58 0.00 1.00 0.04 1.00

9 0.71 0.81 0.03 0.10 0.00 1.00

10 0.95 1.00 0.00 1.00 0.50 1.00

11 0.60 0.70 0.10 1.00 0.50 1.00


k 2 f1; 2; . . . ; 5g, and that we may group the ‘‘nonweld’’ structures into six

clusters which we represent with the prototypes Pk, k 2 f6; 7; . . . ; 11g. Thecorresponding wkðulÞ and /kðulÞ are given in Table 3 (see also Fig. 1).

Each structure in the test set (Table 2) is classified using (4) with the fol-

lowing similarity measure:

1 2 30

0.5

1

1 2 3 1 2 3 1 2 3 1 2 3

1 2 30

0.5

1

1 2 3 1 2 3 1 2 3 1 2 3 1 2 3

Fig. 1. Shows the low membership functions wk and the high membership functions /k versus ul,l 2 f1; 2; 3g, for the prototypes Pk , k 2 f1; 2; . . . ; 11g, where P1–P5 represent the weld training

data (top row) and prototypes P6–P11 represent the nonweld training data (bottom row). Open

circles denote that a given feature ul is required to identify Pk . Features which are not required to

identify Pk are denoted by full circles.

Table 4

Percentage of correct classification Pc of training images using the new algorithm and the Liao–Li

algorithm

Algorithm Pc (%)

New algorithm 95

Liao–Li algorithm 93.2


eSð eQðuÞ; ePkðuÞÞ ¼Xm

Xn

SmnKmn; ð25Þ

where

Smn ¼ minl

max 1

�

jhðmÞQ ðulÞ � hðnÞ

k ðulÞjminðrQðulÞ; rkðulÞÞ

; 0

!!; ð26Þ

Kmn ¼minðkðmÞ

Q ; kðnÞk ÞP

i

Pj minðkðiÞ

Q ; kðjÞk Þ

; ð27Þ

and rQðulÞ ¼ 0:002 and a ¼ 2.The percentage of correctly classified structures in the test data is given in

Table 4 along with the original results of Liao and Li. We see the new tech-

nique provides a very high rate of correct classification (95%) and this com-

pares very favourably with the results of Liao and Li (�93%).

6. Conclusion

In this article we showed that by defining an appropriate similarity measure,

type-II fuzzy sets provide us with a natural, sufficiently rich language for

formulating classification problems which are inherently vague. We illustrated

the use of the new similarity measure on a real classification problem takenfrom the field of nondestructive testing.

References

[1] K. Fukunaga, Introduction to Statistical Pattern Recognition, Academic Press, San Diego,

CA, 1990.

[2] R.O. Duda, P.E. Hart, Pattern Classification and Scene Analysis, John Wiley, New York,

2001.

[3] C.M. Bishop, Neural Networks for Pattern Recognition, Oxford University Press, Oxford,

England, 1995.

[4] P.A. Devijer, J. Kittler, Pattern Recognition, A Statistical Approach, Prentice-Hall, London,

England, 1982.


[5] R. Schalkoff, Pattern Recognition. Statistical, Structural and Neural, John Wiley, New York,

1992.

[6] H.B. Mitchell, P.A. Schaefer, A ‘‘soft’’ k-nearest neighbour voting scheme, International

Journal Intelligent Systems 16 (2001) 459–468.

[7] J.C. Bezdek, S.K. Pal (Eds.), Fuzzy Models for Pattern Recognition, Institute of Electrical and

Electronics Engineers, New York, NY, 1992, pp. 1017–2394.

[8] G.J. Klir, B. Yuan, Fuzzy Sets and Fuzzy Logic, Prentice-Hall, Upper Saddle River, 1995.

[9] V.V. Cross, T.A. Sudkamp, Similarity and Compatibility in Fuzzy Set Theory, Physica-Verlag,

Heidelberg, Germany, 2002.

[10] L. Dengfeng, C. Chuntian, New similarity measures of intuitionistic fuzzy sets and application

to pattern recognitions, Pattern Recognition Letters 23 (2002) 221–225.

[11] K.T. Atanassov, Intuitionistic Fuzzy Sets, Physica-Verlag, Heidelberg, 1999.

[12] H.B. Mitchell, On the Dengfeng–Chuntian similarity measure and its application to pattern

recognition, Pattern Recognition Letters 24 (2003) 3101–3104.

[13] H.B. Mitchell, A correlation coefficient for intuitionistic fuzzy sets, International Journal of

Intelligent Systems 19 (2004) 483–490.

[14] Z. Liang, P. Shi, Similarity measures for intuitionistic fuzzy sets, Pattern Recognition Letters

24 (2003) 2687–2693.

[15] L.A. Zadeh, The concept of a linguistic variable and its application to approximate reasoning-

I, Information Science 8 (1975) 199–249.

[16] J.M. Mendel, R.I.B. John, Type-2 fuzzy sets made simple, IEEE Transactions on Fuzzy

Systems 10 (2002) 117–127.

[17] H. Wu, J.M. Mendel, Uncertainty bounds and their use in the design of interval type-2 fuzzy

logic systems, IEEE Transactions on Fuzzy Systems 10 (2002) 622–639.

[18] N.N. Karnik, J.M. Mendel, Interval Type-2 fuzzy logic systems: theory and design, IEEE

Transactions on Fuzzy Systems 8 (2000) 535–550.

[19] J.M. Mendel, Uncertainty, fuzzy logic and signal processing, Signal Processing 80 (2000) 913–

933.

[20] N.N. Karnik, J.M. Mendel, Centroid of a type-2 fuzzy set, Information Sciences 132 (2001)

195–220.

[21] A. Gayer, A. Saya, A. Shiloh, Automatic recognition of welding defects in real-time

radiography, NDT International 23 (1990) 131–136.

[22] T.W. Liao, D. Li, Two manufacturing applications of the fuzzy K–NN algorithm, Fuzzy Sets

and Systems 92 (1997) 289–303.

[23] C.H. Chen, Pattern recognition in nondestructive evaluation of materials, in: Handbook of

Pattern Recognition and Computer Vision, 1993, pp. 493–509 (Chapter 3.1).

pattern recognition using type-ii fuzzy sets

Documents