pattern recognition using type-ii fuzzy sets
Post on 26-Jun-2016
217 Views
Preview:
TRANSCRIPT
Information Sciences 170 (2005) 409–418
www.elsevier.com/locate/ins
Pattern recognition using type-II fuzzy sets
H.B. Mitchell *
ELTA Systems Ltd, Image Intelligence Exploitation Department (section 6174),
Image Intelligence and Radar Division, Ashdod, Israel
Received 30 July 2003; received in revised form 8 December 2003; accepted 5 February 2004
Abstract
Type II fuzzy sets are a generalization of the ordinary fuzzy sets in which the
membership value for each member of the set is itself a fuzzy set in ½0; 1�. We introduce a
similarity measure for measuring the similarity, or compatibility, between two type-II
fuzzy sets. With this new similarity measure we show that type-II fuzzy sets provide us
with a natural language for formulating classification problems in pattern recognition.
� 2004 Elsevier Inc. All rights reserved.
Keywords: Type-II fuzzy sets; Intuitionistic fuzzy sets; Type-II similarity measures;
Pattern recognition; Radiographic testing
1. Introduction
Pattern recognition problems typically involve the classification of an un-
known pattern Q given a set of K prototypes Pk, k 2 f1; 2; . . . ;Kg [1–6]. Each
prototype Pk belongs to a given class Cm, m 2 f1; 2; . . . ;Mg, which is specifiedby the indicator function Ak:
* Te
E-m
0020-0
doi:10.
Ak ¼ Cm if Pk belongs to the mth class Cm: ð1Þ
Let SðQ;PkÞ be a similarity measure which measures the degree of similarity,
or compatibility, between the unknown pattern Q and the kth prototype Pk.
l.: +972-8-8572-851; fax: +972-8-8572-844.
ail address: hmitchell@elta.co.il (H.B. Mitchell).
255/$ - see front matter � 2004 Elsevier Inc. All rights reserved.
1016/j.ins.2004.02.027
410 H.B. Mitchell / Information Sciences 170 (2005) 409–418
Then, formally, we may write the process of classifying, or assigning, the un-
known pattern Q to the class C ¼ Ak , where
k ¼ argmaxk
ðSðQ;PkÞÞ: ð2Þ
Very often we apply the techniques of pattern recognition to situations
which are inherently vague and uncertain [7,8]. Such situations arise when theinformation regarding the prototypes is ‘‘linguistic’’ and is based on the
opinions and judgements of human experts. Examples of such situations are:
handwritten character recognition, fingerprint recognition, human face rec-
ognition, classification of X-ray images, classification of remotely sensed data.
One way of handling such situations is to make the information precise and to
give it a mathematically well-defined form by using the concepts and tech-
niques of fuzzy logic.
Let U denote the feature space. Then, mathematically, we represent theunknown pattern Q, and the prototypes, Pk, k 2 f1; 2; . . . ;Kg, with (ordinary)
fuzzy sets Q QðuÞ and Pk PkðuÞ, where u 2 U . The result is the following
optimization problem:
k ¼ argmaxk
ðSðQðuÞ;PkðuÞÞÞ; ð3Þ
where S denotes an appropriate similarity, or compatibility, measure [9].
Unfortunately, (3) is sometimes too precise in the sense that no uncertainty
whatsoever is allowed in specifying the fuzzy sets QðuÞ and PkðuÞ.Dengfeng and Chuntian [10] suggested that one way to introduce a con-
trolled amount of uncertainty into (3), is to replace the ordinary fuzzy sets QðuÞand PkðuÞ, with intuitionistic fuzzy sets [11], and to replace S with an appro-priate intuitionistic similarity measure [10,12–14]. This is correct if we model
the uncertainty with a uniform distribution. However, in many pattern rec-
ognition problems, it is more accurate to model the uncertainty with a non-
uniform distribution. In this case, we use type-II fuzzy sets eQðuÞ and ePkðuÞ and(3) becomes
k ¼ argmaxk
ðeSð eQðuÞ; ePkðuÞÞÞ; ð4Þ
where eS denotes a type-II similarity measure.
The main advantages of using a type-II framework are twofold:
• By using type-II fuzzy sets we transform a vague pattern classification
problem into a precise, well-defined, optimization problem.
• Type-II fuzzy sets, unlike ordinary fuzzy sets, retain a controlled degree of
uncertainty.
H.B. Mitchell / Information Sciences 170 (2005) 409–418 411
The main disadvantage to using a type-II formulation is
• The relatively high computational complexity.
The article is organized as follows. In Section 2 we discuss the concept of
type-II fuzzy sets. In Section 3 we describe a new type-II similarity measure eSwhich is specifically designed to measure the compatibility of two type-II fuzzy
sets. In Section 4 we show how we may reformulate the problem of pattern
classification using type-II fuzzy sets. In Section 5 we analyze a real-left pattern
classification problem and show how we may solve it, and similar classification
problems, using the new approach. Finally the article concludes with a brief
summary in Section 6.
2. Type-II fuzzy sets
The concept of a type-II fuzzy set was introduced by Zadeh [15] as a gen-
eralization of an ordinary fuzzy set. Type-II fuzzy sets are characterized by a
fuzzy membership function, i.e. the membership value for each element of theset, is itself a fuzzy set in ½0; 1�. 1 Thus, given the feature space U , a type-II
fuzzy set is defined as an object eA which has the following form:
1 Th
eA fhu; v; nAðu; vÞig; ð5Þ
where nAðu; vÞ represent the degree of membership of the elementðu; vÞ; u 2 U ; v 2 ½0; 1� in eA. We find it convenient to write nAðu; vÞ as the
product lAðuÞmAðu; vÞ, where
lAðuÞ ¼ maxvðnAðu; vÞÞ; ð6Þ
mAðu; vÞ ¼nAðu;vÞlAðuÞ
if lAðuÞ > 0
1 otherwise
�ð7Þ
are known, respectively, as the primary and the secondary membership func-
tions. We delineate the region where nAðu; vÞ > 0, by means of a low mem-
bership function wAðuÞ, and a high membership function /AðuÞ, where
nAðu; vÞ > 0 if wAðuÞ6 v6/AðuÞ: ð8ÞPhysically the difference j/AðuÞ � wAðuÞj represents the uncertainty in specify-
ing the primary membership value lAðuÞ.We find it convenient to interpret eA from a statistical viewpoint [16–20]:
Suppose the feature space U is sampled at L points ul, l 2 f1; 2; . . . ; Lg, then we
e membership value for ordinary fuzzy sets is a crisp number in ½0; 1�.
412 H.B. Mitchell / Information Sciences 170 (2005) 409–418
may interpret eA as an ensemble of M ordinary, or embedded, membership
functions hðmÞA ðulÞ, m 2 f1; 2; . . . ;Mg:
hðmÞA ðulÞ ¼ rmðulÞ � ð/AðulÞ � wAðulÞÞ þ wAðulÞ; ð9Þ
where rmðulÞ is a random number chosen uniformly in the interval ½0; 1�.We associate a weight kðmÞ
A with each function hðmÞA ðulÞ, m 2 f1; 2; . . . ;Mg.
This is defined as the t-norm of the individual secondary membership grades
mAðul; hðmÞA ðulÞÞ, l 2 f1; 2; . . . ; Lg. Thus
kðmÞA ¼ tðmAðu1; hðmÞ
A ðu1ÞÞ; mAðu2; hðmÞA ðu2ÞÞ; . . . ; mAðuL; hðmÞ
A ðuLÞÞÞ; ð10Þ
where t denotes a given t-norm operator.
Further details concerning the statistical interpretation of type-II fuzzy sets
are given in [16–20].
3. Type-II similarity measure
We define a type-II similarity measure eS as follows. Given two type-II fuzzy
sets eA, eB, we suppose we have generated the corresponding embedded mem-
bership functions hðmÞA ðulÞ; hðnÞ
B ðulÞ, l 2 f1; 2; . . . ; Lg;m 2 f1; 2; . . . ;Mg; n 2f1; 2; . . . ;Ng. Let
Smn SðhðmÞA ðuÞ; hðnÞ
B ðuÞÞ ð11Þ
denote the similarity between the embedded functions hðmÞA ðuÞ and hðnÞ
B ðuÞ, whereSð�; �Þ denotes any ordinary similarity measure. Then we define the type-II
similarity measure between eA and eB as the weighted average
eSðeA; eBÞ ¼XMm¼1
XNn¼1
SmnKmn; ð12Þ
where the normalized weights
Kmn ¼tðkðmÞ
A ; kðnÞB ÞPM
i¼1
PNj¼1 tðk
ðiÞA ; kðjÞ
B Þð13Þ
satisfy the inequalities
06Kmn 6 1; ð14ÞXMm¼1
XNn¼1
Kmn ¼ 1: ð15Þ
H.B. Mitchell / Information Sciences 170 (2005) 409–418 413
4. Pattern recognition using type-II fuzzy sets
In order to solve (4) we require a procedure for calculating the appropriate
type-II fuzzy sets for the unknown pattern Q and for the prototypes Pk,k 2 f1; 2; . . . ;Kg.
4.1. Unknown pattern Q
We represent the unknown pattern Q with a type-II fuzzy set eQ which has a
primary membership function lQðulÞ, l 2 f1; 2; . . . ; Lg. At each ul, the mea-
surement of Q is subject to random measurement noise. This represents thephysical origin of the uncertainty in eQ. If we model the noise as a normal
distribution, Nð0; r2QÞ, with zero mean and standard deviation rQ, then the
corresponding low and high membership functions, wQðuÞ and /QðuÞ, are
wQðuÞ ¼ maxðlQðuÞ � arQðuÞ; 0Þ; ð16Þ/QðuÞ ¼ minðlQðuÞ þ arQðuÞ; 1Þ; ð17Þwhere a is a parameter chosen so that the probability of obtaining a mea-surement which lies outside the interval ½wQðuÞ;/QðuÞ� is very low. Typically, weuse a ¼ 2.
Mathematically, the individual embedded functions are limited to the
interval ½wQðulÞ;/QðulÞ� and are given by
hðmÞQ ðulÞ ¼ rmðulÞrQðulÞ þ lQðulÞ; ð18Þ
where rmðulÞ is a random number chosen from a normal distribution, Nð0; 1Þ,with zero mean and unit standard deviation. The embedded function hQðulÞ hasa weight
kðmÞQ ¼ t mQðu1; hðmÞ
Q ðu1ÞÞ; mQðu2; hðmÞQ ðu2ÞÞ; . . . ; mQðuL; hðmÞ
Q ðuLÞÞ� �
ð19Þ
associated with it, where
mQðul; hðmÞQ ðulÞÞ / exp� 1
2
hðmÞQ ðulÞ � lQðulÞ
rQðulÞ
!2
: ð20Þ
4.2. Prototypes Pk
We represent each prototype Pk, k 2 f1; 2; . . . ;Kg, with a type-II fuzzy setePk, or equivalently, an ensemble of N embedded membership functions hðnÞk ðulÞ,
n 2 f1; 2; . . . ;Ng, with weights kðnÞk .
The prototypes themselves are obtained by clustering a set of training pat-
terns: each prototypePk, k 2 f1; 2; . . . ;Kg, represents a cluster, or ensemble, of
414 H.B. Mitchell / Information Sciences 170 (2005) 409–418
training patterns: fhð1Þk ðulÞ; hð2Þ
k ðulÞ; . . . ; hðsÞk ðulÞ; . . .g, where hðsÞ
k ðulÞ denotes the
sth training pattern associated with the kth prototype. Then, at each ul, thereis a spread in the fhðsÞðulÞg and this is the physical origin of the uncertainty
in ePk.2
We regard all of the training patterns to be uniformly distributed between
the low and high membership functions, wkðulÞ and /kðulÞ, where
2 Th
feature
rkðul Þ
wkðulÞ ¼ maxh
ðhðhÞk ðulÞÞ; ð21Þ
/kðulÞ ¼ minhðhðhÞ
k ðulÞÞ: ð22Þ
The uncertainty associated with Pk is defined in terms of wkðulÞ and /kðulÞ:
rkðulÞ ¼1
2ð/kðulÞ � wkðulÞÞ; ð23Þ
and the embedded functions hðnÞk , n 2 f1; 2; . . . ;Ng, are assigned a unit weight:
kðnÞk ¼ 1: ð24Þ
5. Results
We illustrate the new approach by applying it to the problem of automatic
evaluation of welded structures using radiographic testing [21,22]. This is animportant topic in many manufacturing applications, both from the viewpoint
of cost control and from the viewpoint of safety [23].
Radiographic testing involves three stages:
• Production of a radiographic image of the welded structure by illuminating
it with an X-ray or gamma-ray source.
• Extraction of several features from the image.
• Classification of the structure as belonging to the ‘‘weld’’ class or to the‘‘nonweld’’ class.
Recently Liao and Li [22] collected 80 different radiographic images which
they divided into a training set consisting of 36 images (Table 1) and a test set
consisting of 44 images (Table 2). Three features ul, l 2 f1; 2; 3g, were extractedfrom each image which was classified as being ‘‘weld’’ or ‘‘nonweld’’.
Inspection of the training set (Table 1) shows that we may group the ‘‘weld’’
structures into five clusters, which we represent with the prototypes Pk,
is is, ofcourse, only true if the feature ul is required to identify Pk . Sometimes, a given
, say ul is not required to identify Pk . In this case, we set wkðul Þ ¼ 0, /kðul Þ ¼ 1 and
¼ 12.
Table 2
Feature values hQðulÞ, l 2 f1; 2; 3g, for each image in the test set
Image # u1 u2 u3 Image # u1 u2 u3
1 0.6471 0.0341 0.3490 23 0.2941 0.0238 0.1451
2 0.8824 0.0279 0.3020 24 0.5294 0.0109 0.0431
3 0.7647 0.0124 0.2588 25 0.4118 0.0687 0.1255
4 0.7647 0.0145 0.4784 26 0.2941 1.0000 1.0000
5 1.0000 0.0238 0.0394 27 0.5294 0.0372 0.1412
6 0.5294 0.0362 0.1608 28 0.7647 0.0770 1.0000
7 0.6471 0.0305 0.0235 29 0.5294 0.3409 0.1216
8 1.0000 0.0211 0.1922 30 0.2941 0.0176 0.0235
9 1.0000 0.0176 0.1882 31 0.2941 0.0537 0.0157
10 0.7647 0.0238 0.2000 32 0.4118 0.0692 0.1059
11 0.8824 0.0217 0.3059 33 0.5294 0.0150 0.1725
12 0.7647 0.0279 0.2902 34 0.4118 0.0573 0.1020
13 0.7647 0.0222 0.1882 35 0.2941 0.0465 0.0314
14 0.6471 0.0356 0.3255 36 0.7647 0.0439 0.0314
15 1.0000 0.0217 0.3333 37 0.5294 0.0475 0.0353
16 0.6471 0.0372 0.2267 38 0.2941 0.0233 0.1255
17 0.6471 0.0170 0.2078 39 0.2941 0.0274 0.1177
18 0.5294 0.0181 0.0353 40 0.4118 0.2764 1.0000
19 0.7641 0.0150 0.4745 41 0.6471 0.1545 1.0000
20 1.0000 0.0109 0.4588 42 0.5294 0.0129 0.2118
21 0.8824 0.0114 0.5098 43 0.2941 0.0269 0.1059
22 0.8824 0.0134 0.4314 44 0.5294 0.0212 0.1804
The structure in images 1–22 and 23–44 are, respectively, ‘‘weld’’ and ‘‘nonweld’’.
Table 1
Feature values hðsÞk ðulÞ, l 2 f1; 2; 3g, for each image in the training set
Image # u1 u2 u3 Image # u1 u2 u3
1 0.6471 0.0171 0.2039 19 0.4118 0.0689 0.1255
2 0.6471 0.0156 0.3059 20 0.2941 0.1068 0.0863
3 0.8824 0.0279 0.3020 21 0.7647 0.0374 0.0667
4 0.7647 0.0169 0.1843 22 1.0000 0.0139 0.5843
5 0.7647 0.0179 0.1961 23 0.2941 1.0000 0.3569
6 0.5294 0.0238 0.0353 24 0.2941 0.5581 0.6980
7 1.0000 0.0195 0.0392 25 0.7647 0.0476 0.6353
8 0.6471 0.0104 0.1490 26 1.0000 0.0157 0.6353
9 0.8824 0.0229 0.3059 27 0.2941 0.0174 0.0235
10 0.8824 0.0147 0.1490 28 0.2941 0.0539 0.0157
11 0.7647 0.0111 0.1569 29 0.2941 0.0441 0.1216
12 0.8824 0.0180 0.2941 30 0.5294 0.0164 0.1137
13 0.7647 0.0155 0.1373 31 0.4118 0.0459 0.0314
14 0.6471 0.0122 0.1686 32 0.5294 0.0662 0.1216
15 0.8824 0.0121 0.4784 33 0.2941 0.0468 0.0314
16 1.0000 0.0169 0.3098 34 0.2941 0.3766 1.0000
17 0.7647 0.0133 0.3020 35 0.2941 1.0000 1.0000
18 1.0000 0.0210 0.2314 36 0.5294 0.0110 0.0431
The structures in images 1–18 and 19–36 are, respectively, ‘‘weld’’ and ‘‘nonweld’’.
H.B. Mitchell / Information Sciences 170 (2005) 409–418 415
Table 3
Low and high membership functions, wkðulÞ and /kðulÞ, for features u1, u2 and u3 for the prototypesPk , k 2 f1; 2; . . . ; 11g
k wkðu1Þ /kðu1Þ wkðu2Þ /kðu2Þ wkðu3Þ /kðu3Þ1 0.48 0.58 0.00 1.00 0.00 0.04
2 0.60 0.70 0.00 0.10 0.00 0.50
3 0.71 0.81 0.00 0.03 0.00 1.00
4 0.83 0.93 0.00 0.50 0.00 1.00
5 0.95 1.00 0.00 1.00 0.00 0.50
6 0.25 0.35 0.00 1.00 0.00 1.00
7 0.36 0.46 0.00 1.00 0.00 1.00
8 0.48 0.58 0.00 1.00 0.04 1.00
9 0.71 0.81 0.03 0.10 0.00 1.00
10 0.95 1.00 0.00 1.00 0.50 1.00
11 0.60 0.70 0.10 1.00 0.50 1.00
416 H.B. Mitchell / Information Sciences 170 (2005) 409–418
k 2 f1; 2; . . . ; 5g, and that we may group the ‘‘nonweld’’ structures into six
clusters which we represent with the prototypes Pk, k 2 f6; 7; . . . ; 11g. Thecorresponding wkðulÞ and /kðulÞ are given in Table 3 (see also Fig. 1).
Each structure in the test set (Table 2) is classified using (4) with the fol-
lowing similarity measure:
1 2 30
0.5
1
1 2 3 1 2 3 1 2 3 1 2 3
1 2 30
0.5
1
1 2 3 1 2 3 1 2 3 1 2 3 1 2 3
Fig. 1. Shows the low membership functions wk and the high membership functions /k versus ul,l 2 f1; 2; 3g, for the prototypes Pk , k 2 f1; 2; . . . ; 11g, where P1–P5 represent the weld training
data (top row) and prototypes P6–P11 represent the nonweld training data (bottom row). Open
circles denote that a given feature ul is required to identify Pk . Features which are not required to
identify Pk are denoted by full circles.
Table 4
Percentage of correct classification Pc of training images using the new algorithm and the Liao–Li
algorithm
Algorithm Pc (%)
New algorithm 95
Liao–Li algorithm 93.2
H.B. Mitchell / Information Sciences 170 (2005) 409–418 417
eSð eQðuÞ; ePkðuÞÞ ¼Xm
Xn
SmnKmn; ð25Þ
where
Smn ¼ minl
max 1
�
jhðmÞQ ðulÞ � hðnÞ
k ðulÞjminðrQðulÞ; rkðulÞÞ
; 0
!!; ð26Þ
Kmn ¼minðkðmÞ
Q ; kðnÞk ÞP
i
Pj minðkðiÞ
Q ; kðjÞk Þ
; ð27Þ
and rQðulÞ ¼ 0:002 and a ¼ 2.The percentage of correctly classified structures in the test data is given in
Table 4 along with the original results of Liao and Li. We see the new tech-
nique provides a very high rate of correct classification (95%) and this com-
pares very favourably with the results of Liao and Li (�93%).
6. Conclusion
In this article we showed that by defining an appropriate similarity measure,
type-II fuzzy sets provide us with a natural, sufficiently rich language for
formulating classification problems which are inherently vague. We illustrated
the use of the new similarity measure on a real classification problem takenfrom the field of nondestructive testing.
References
[1] K. Fukunaga, Introduction to Statistical Pattern Recognition, Academic Press, San Diego,
CA, 1990.
[2] R.O. Duda, P.E. Hart, Pattern Classification and Scene Analysis, John Wiley, New York,
2001.
[3] C.M. Bishop, Neural Networks for Pattern Recognition, Oxford University Press, Oxford,
England, 1995.
[4] P.A. Devijer, J. Kittler, Pattern Recognition, A Statistical Approach, Prentice-Hall, London,
England, 1982.
418 H.B. Mitchell / Information Sciences 170 (2005) 409–418
[5] R. Schalkoff, Pattern Recognition. Statistical, Structural and Neural, John Wiley, New York,
1992.
[6] H.B. Mitchell, P.A. Schaefer, A ‘‘soft’’ k-nearest neighbour voting scheme, International
Journal Intelligent Systems 16 (2001) 459–468.
[7] J.C. Bezdek, S.K. Pal (Eds.), Fuzzy Models for Pattern Recognition, Institute of Electrical and
Electronics Engineers, New York, NY, 1992, pp. 1017–2394.
[8] G.J. Klir, B. Yuan, Fuzzy Sets and Fuzzy Logic, Prentice-Hall, Upper Saddle River, 1995.
[9] V.V. Cross, T.A. Sudkamp, Similarity and Compatibility in Fuzzy Set Theory, Physica-Verlag,
Heidelberg, Germany, 2002.
[10] L. Dengfeng, C. Chuntian, New similarity measures of intuitionistic fuzzy sets and application
to pattern recognitions, Pattern Recognition Letters 23 (2002) 221–225.
[11] K.T. Atanassov, Intuitionistic Fuzzy Sets, Physica-Verlag, Heidelberg, 1999.
[12] H.B. Mitchell, On the Dengfeng–Chuntian similarity measure and its application to pattern
recognition, Pattern Recognition Letters 24 (2003) 3101–3104.
[13] H.B. Mitchell, A correlation coefficient for intuitionistic fuzzy sets, International Journal of
Intelligent Systems 19 (2004) 483–490.
[14] Z. Liang, P. Shi, Similarity measures for intuitionistic fuzzy sets, Pattern Recognition Letters
24 (2003) 2687–2693.
[15] L.A. Zadeh, The concept of a linguistic variable and its application to approximate reasoning-
I, Information Science 8 (1975) 199–249.
[16] J.M. Mendel, R.I.B. John, Type-2 fuzzy sets made simple, IEEE Transactions on Fuzzy
Systems 10 (2002) 117–127.
[17] H. Wu, J.M. Mendel, Uncertainty bounds and their use in the design of interval type-2 fuzzy
logic systems, IEEE Transactions on Fuzzy Systems 10 (2002) 622–639.
[18] N.N. Karnik, J.M. Mendel, Interval Type-2 fuzzy logic systems: theory and design, IEEE
Transactions on Fuzzy Systems 8 (2000) 535–550.
[19] J.M. Mendel, Uncertainty, fuzzy logic and signal processing, Signal Processing 80 (2000) 913–
933.
[20] N.N. Karnik, J.M. Mendel, Centroid of a type-2 fuzzy set, Information Sciences 132 (2001)
195–220.
[21] A. Gayer, A. Saya, A. Shiloh, Automatic recognition of welding defects in real-time
radiography, NDT International 23 (1990) 131–136.
[22] T.W. Liao, D. Li, Two manufacturing applications of the fuzzy K–NN algorithm, Fuzzy Sets
and Systems 92 (1997) 289–303.
[23] C.H. Chen, Pattern recognition in nondestructive evaluation of materials, in: Handbook of
Pattern Recognition and Computer Vision, 1993, pp. 493–509 (Chapter 3.1).
top related