a new fuzzy regression algorithm

A New Fuzzy Regression Algorithm

Horia F. Pop† and Costel Sarbu*,‡

Faculty of Mathematics and Computer Science and Faculty of Chemistry, Babes-Bolyai University,RO-3400 Cluj-Napoca, Romania

A new fuzzy regression algorithm is described and com-pared with conventional ordinary and weighted least-squares and robust regression methods. The applicationof these different methods to relevant data sets proves thatthe performance of the procedure described in this paperexceeds that of the ordinary least-squares method andequals and often exceeds that of weighted or robustmethods, including the two fuzzy methods proposedpreviously (Otto, M.; Bandemer, H., Chemom. Intell.Lab. Syst. 1986, 1, 71. Hu, Y.; Smeyers-Verbeke, J.;Massart, D. L. Chemom. Intell. Lab. Syst. 1990, 8,143). Moreover, we emphasize the effectiveness and thegenerality of the two new criteria proposed in this paperfor diagnosing the linearity of calibration lines in analyticalchemistry.

Quantitative analytical chemistry is based on experimental dataobtained with accurate measurements of various physical quanti-ties. The treatment of data is done mainly on the basis of simplestoichiometric relations and chemical equilibrium constants. Ininstrumental methods of analysis, the amount of the componentis computed from measurement of a physical property which isrelated to the mass or concentration of the component.

Relating, correlating, or modeling a measured response basedon the concentration of the analyte is known as calibration.Calibration of the instrumental response is a fundamental require-ment for all instrumental analysis techniques. In statistical terms,calibration refers to the establishment of a predictive relationbetween the controlled or independent variable (e.g., the concen-tration of a standard) and the instrumental response. Thecommon approach to this problem is to use the linear least-squaresmethod.1-5 The ordinary least-squares regression (LS) is basedon the assumption of an independent and normal error distributionwith uniform variance (homoscedastic). Much more common inpractice, however, are the heteroscedastic results, where they-direction error is dependent on the concentration and/or thepresence of outliers.

In practice, the actual shape of the error distribution functionand its variance are usually unknown, so we must investigate theconsequences if the conditions stated above are not met. Gener-ally, the least-squares method does not lead to the maximumlikelihood estimate. Despite the fact that the least-squares method

is not optimal, there is a justification for using it in the cases wherethe conditions are only approximately met.

In particular, the Gauss-Markov theorem states that if theerrors are random and uncorrelated, the least-squares methodgives the best linear unbiased estimate of the parameters, meaningthat out of all the functions for which each parameter is a linearfunction, the least-squares method is that for which the variancesof the parameters are the smallest.6

Nevertheless, if the tails of the experimental error distributioncontain a substantially larger proportion of the total area than thetails of a Gaussian distribution, the “best linear” estimate may notbe very good, and there will usually be a procedure in which theparameters are not linear functions of the data that gives lowervariances for the parameter estimates than the least-squaresmethod does; that is, the robust and resistant method.

A procedure is said to be robust if it gives parameter estimateswith variances close to the minimum variance for a wide range oferror distributions. The least-squares method is very sensitiveto the effect of large residuals, so the results are distorted if thereare large differences between the observed data and the modelpredictions given by a Gaussian distribution. The least-squaresmethod is therefore not robust.

A procedure is said to be resistant if it is insensitive to thepresence or absence of any small subset of data. In practice, itapplies particularly to small numbers of data points that are wildlydiscrepant with respect to the body of the datasthe so-calledoutliers. There are several reasons why data may be discrepant,a gross measurement error being the most obvious one. Anotheris the fact that certain data points may be particularly sensitive tosome unmodeled (or unadequately modeled) parameter or, fromanother point of view, particularly sensitive to some systematicerror that has not been accounted for in the experiment.7

In recent years, a great deal of work has been done ondetermining what properties a robust and resistant procedureshould have.8-15 Obviously, if the error distribution is Gaussianor very close to Gaussian, the procedure should give results veryclose to those given by the least-squares method. This suggeststhat the procedure should be much like the least-squares methodfor small values of the residuals. Because the weakness of the

† Faculty of Mathematics and Computer Science.‡ Faculty of Chemistry.

(1) Massart, D. L.; Vandeginste, B. M.; Deming, S. N.; Michotte, Y.; Kaufman,L. Chemometrics: A Textbook; Elsevier: Amsterdam, 1988; p 75.

(2) Miller, J. C.; Miller, J. N. Statistics for Analytical Chemistry, 2nd ed.; EllisHorwood: Chichester, 1988; p 101.

(3) Agterdenbos, J. Anal. Chim. Acta 1979, 108, 315.(4) Agterdenbos, J. Anal. Chim. Acta 1981, 132, 127.(5) Miller, J. N. Analyst 1991, 116, 3.

(6) Klimov, G. Probability Theory and Mathematical Statistics; Mir Publishers:Moscow, 1986; p 272.

(7) Rajko, R. Anal. Lett. 1994, 27, 215.(8) Thompson, M. Analyst 1994, 119, 127N.(9) Phillips, G. R.; Eyring, E. M. Anal. Chem. 1983, 55, 1134.

(10) Schwartz, L. M. Anal. Chem. 1977, 49, 2062.(11) Schwartz, L. M. Anal. Chem. 1979, 51, 723.(12) Rutan, R. C.; Carr, P. W. Anal. Chim. Acta 1988, 215, 131.(13) Hu, Y.; Smeyers-Verbeke, J.; Massart, D. L. J. Anal. At. Spectrom. 1989, 4,

605.(14) Wolters, R.; Kateman, G. J. Chemom. 1989, 3, 329.(15) Vankeerberghen, P.; Vandenbosch, C.; Smeyers-Verbeke, J.; Massart, D.

L. Chemom. Intell. Lab. Syst. 1991, 12, 3.

Anal. Chem. 1996, 68, 771-778

0003-2700/96/0368-0771$12.00/0 © 1996 American Chemical Society Analytical Chemistry, Vol. 68, No. 5, March 1, 1996 771

least-squares method lies in its overemphasis on large values ofresiduals, these should be deemphasized (downweighted) orperhaps even ignored. For intermediate values of the residuals,the procedure should connect the treatments of the small residualsand the extreme residuals in a smooth or fuzzy fashion.16-18

The purpose of the present study is to investigate theperformance of the new fuzzy regression algorithm, which appearsto be more efficient to overcome the difficulties discussed above.The results were applied to relevant calibration data discussed inanalytical literature.

THEORYThe Fuzzy 1-Lines Algorithm. Let us consider a data set X

) {x1, ..., xp} ⊂ Rs. Let us suppose that the set X does not havea clear clustering structure or that the clustering structurecorresponds to a single fuzzy set. Let us admit that this fuzzyset, denoted A, may be characterized by a linear prototype,denoted L ) (v,u), where v is the center of the class and u, with|u| ) 1, is its main direction. We raise the problem of findingthe fuzzy set that is the most suitable for the given data set. Wepropose to do this by minimizing a criterion function similar tothose presented in refs 19-21, 23, 24.

Let us consider the scalar product in Rs given by

where M ∈Ms(R) is a symmetrical, positive definite square matrix.If M is the unit matrix, then the scalar product is the usual one,⟨x,y⟩ ) xTy. We used the unit matrix for all the computationsperformed in this paper.

Let us consider the norm in Rs induced by the scalar product,

and the distance d in Rs induced by this norm,

To obtain the criterion function, we keep in mind that we wishto determine a fuzzy partition {A,Ah}. The fuzzy set A ischaracterized by the prototype L. In relation to the complemen-tary fuzzy set, Ah , we will consider that the dissimilarity betweenits hypotetical prototype and the point xj is constant and equal toR/(1-R), where R is a constant from [0,1], with a role to be seenlater in this paper.

Let us denote the dissimilarity between the point xj and theprototype L by

where

is the distance from the point xj to the line L ) (v,u) in the spaceRs.

The inadequacy between the fuzzy set A and its prototype Lwill be

and the inadequacy between the complementary fuzzy set Ah andits hypothetical prototype will be

Thus, the criterion function J, F(X)Rd f R+, becomes

where R ∈ [0,1] is a fixed constant.With respect to the minimization of the criterion function J,

the following two results are valid.Let us consider the fuzzy set A of X. The prototype L ) (v,u)

that minimizes the function J(A,ncd) is given by

u is the eigenvector corresponding to the maximal eigenvalue ofthe matrix

where M is the matrix from the definition of the scalar product(eq 1).

Similarly, let us consider a certain prototype L. The fuzzy setA that minimizes the function J(‚,L) is given by

The optimal fuzzy set will be determined by using an iterativemethod where J is succesively minimized with respect to A andL. The proposed algorithm will be called Fuzzy 1-Lines:

S1. We chose the constant R ∈ [0,1]. We initialize A(x) ) 1,for every x ∈ X, and l ) 0. Let us denote by A(l) the fuzzy set Adetermined at the lth iteration.

(16) Otto, M.; Bandemer, H. Chemom. Intell. Lab. Syst. 1986, 1, 71.(17) Hu, Y.; Smeyers-Verbeke, J.; Massart, D. L. Chemom. Intell. Lab. Syst. 1990,

8, 143.(18) Shimasaka, T.; Kitamori, T.; Harata, A.; Sawada, T. Anal. Sci. 1991, 7, 1389.(19) Dumitrescu, D.; Sarbu, C.; Pop, H. F. Anal. Lett. 1994, 27, 1031.(20) Pop, H. F.; Dumitrescu, D.; Sarbu, C. Anal. Chim. Acta 1995, 310, 269.(21) Dumitrescu, D.; Pop, H. F.; Sarbu, C. J. Chem. Inf. Comput. Sci. 1995, 35,

851.(22) Dumitrescu, D. Fuzzy Sets Syst. 1992, 47, 193.(23) Bezdek, J. C.; Coray, C.; Gunderson, R.; Watson, J. SIAM J. Appl. Math.

1981, 40, 339.(24) Bezdek, J. C.; Coray, C.; Gunderson, R.; Watson, J. SIAM J. Appl. Math.

1981, 40, 358.

⟨x,y⟩ ) xTMy, for every x, y ∈ Rs (1)

|x| ) ⟨x,x⟩1/2, for every x ∈ Rs

d(x,y) ) |x - y|, for every x,y ∈ Rs

D(xj,L) ) d2(xj,L)

d(xj,L) ) (|xj - v|2 - ⟨xj - v,u⟩2)1/2

I(A,L) ) ∑j)1

p

[A(xj,L)]2D(xj,L)

∑j)1

p

[Ah(xj)]2 R

1-R

J(A,L;R) ) ∑j)1

p

[A(xj)]2d2(xj,L) + ∑j)1

p

[Ah(xj)]2 R

1-R

v ) ∑j)1

p

[A(xj)]2xj/∑j)1

p

[A(xj)]2 (2)

S ) M ∑j)1

p

[A(xj)]2(xj-v)(xj-v)TM (3)

A(xj) )R/(1-R)

[R/(1-R)] + d2(xj,L)(4)

772 Analytical Chemistry, Vol. 68, No. 5, March 1, 1996

S2. We compute the prototype L ) (v,u) of the fuzzy set A(l)

using the relations 2 and 3.S3. We determine the new fuzzy set A(l+1) using the relation

4.S4. If the fuzzy sets A(l+1) and A(l) are close enough, i.e., if

where ε has a predefined value, then stop, else increase l by 1and go to step S2.

We noticed that a good value of ε with respect to the similarityof A(l) and A(l+1) would be ε ) 10-5. We used this value for all thecomputations performed in this paper.

To avoid the dependency of the memberships of the scale, inpractice we will use in the relation 4, instead of the distance d,the normalized distance dr given by

Thus, the relation used to determine the memberships is

The algorithm modified in this way will be called Modified Fuzzy1-Lines:

S1. We chose the constant R ∈ [0,1]. We initialize A(x) ) 1,for every x ∈ X, and l ) 0. Let us denote by A(l) the fuzzy set Adetermined at the lth iteration.

S2. We compute the prototype L ) (v,u) of the fuzzy set A(l)

using the relations 2 and 3.S3. We determine the new fuzzy set A(l+1) using the relation

5.S4. If the fuzzy sets A(l+1) and A(l) are close enough, i.e., if

where ε has a predefined value (10-5), then stop, else increase lby 1 and go to step S2.

Properties of the Produced Fuzzy Set. In this section wewill study the properties of the fuzzy set obtained via the algorithmpresented above.

Let X be a given data set and let A and L be the fuzzy set andits prototype, respectively, as they were determined by the

Table 1. First Example: Calibration Data Measured byICP-AES

signals for elements

concn, ppm Mo Cr Co Pb Nia Nib

0 8.19 -23.40 -1.83 13.46 6.40 28.330 16.05 -19.40 -2.45 -6.40 7.47 30.560.25 171.90 210.90 261.30 112.00 223.70 220.800.25 180.60 213.70 260.10 119.20 215.60 218.600.5 406.00 420.40 430.80 217.00 437.90 410.200.5 414.50 423.20 431.40 207.70 430.70 407.901 810.70 843.20 860.30 419.60 897.30 828.301 818.20 840.60 859.60 428.70 886.80 826.10

a Measured at wavelength 221.6 nm. b Measured at wavelength 231.6nm.

Table 2. First Example: Values of the EstimatedParameters

data set

algorithm parameter Mo Cr Co Pb Nia Nib

LS a0 -3.07 -11.9 17.2 9.84 0.47 21.9a1 814.5 858.5 846.1 413.0 886.3 798.7

LSA a0 8.19 -2.4 1.3 13.5 2.67 19.0a1 802.5 845.6 859.0 407.1 884.1 807.1

LMA a0 -6.29 -12.8 21.7 8.7 -3.37 19.8a1 802.1 864.0 862.1 412.7 889.8 797.7

IRLS6 a0 11.1 -11.7 -1.45 10.0 0.85 22.2a1 802.3 858.0 862.1 412.9 885.8 799.1

IRLS9 a0 -1.88 -11.8 15.7 9.91 0.63 22.0a1 813.6 858.3 847.2 412.9 886.1 798.8

MFV a0 9.36 1.31 -1.96 10.3 6.4 22.9a1 803.9 840.8 862.1 412.1 880.4 801.3

SM a0 5.25 -9.15 0.39 9.65 5.1 26.7a1 808.4 854.6 860.2 412.9 880.0 782.7

RM a0 2.01 -1.55 2.25 10.4 6.2 28.9a1 813.5 844.5 857.7 410.8 870.92 765.2

LMS a0 12.1 2.4 -1.34 11.8 8.98 29.4a1 802.5 839.6 862.0 407.1 848.6 761.1

FR a0 10.63 2.06 -1.35 6.36 6.31 17.35(R ) 0.05) a1 802.8 839.8 862.0 422.1 880.1 809.7XWLS a0 12.1 -21.4 -2.1 9.9 6.9 29.4

a1 751.7 894.8 926.6 413.9 863.6 772.7WLS a0 -19.6 -7.8 5.0 10.3 6.7 22.0

a1 836.2 852.9 857.0 412.7 872.1 779.0


|A(l+1) - A(l)| < ε

Table 3. First Example: Estimated ConcentrationsCorresponding to Signal Values of 100 and 600

data set

method Mo Cr Co Pb Nia Nib

y ) 100LS 0.123 0.123 0.098 0.218 0.112 0.098LSA 0.114 0.121 0.115 0.213 0.110 0.100LMA 0.133 0.131 0.91 0.221 0.116 0.101IRLS6 0.111 0.130 0.118 0.218 0.112 0.097IRLS9 0.125 0.130 0.100 0.218 0.112 0.098MFV 0.113 0.117 0.118 0.218 0.106 0.096SM 0.117 0.128 0.117 0.219 0.108 0.093RM 0.121 0.120 0.114 0.218 0.108 0.093LMS 0.110 0.116 0.118 0.217 0.107 0.093FR (R ) 0.05) 0.111 0.116 0.118 0.222 0.106 0.114XWLS 0.117 0.136 0.110 0.218 0.108 0.091WLS 0.143 0.138 0.111 0.217 0.107 0.098

y ) 600

LS 0.740 0.713 0.689 1.429 0.676 0.724LSA 0.737 0.712 0.697 1.441 0.676 0.719LMA 0.756 0.709 0.671 1.433 0.678 0.727IRLS6 0.734 0.713 0.698 1.429 0.676 0.723IRLS9 0.740 0.713 0.690 1.429 0.676 0.724MFV 0.735 0.712 0.698 1.430 0.674 0.720SM 0.736 0.713 0.697 1.435 0.676 0.733RM 0.735 0.712 0.697 1.435 0.682 0.746LMS 0.733 0.712 0.698 1.445 0.696 0.750FR 0.734 0.712 0.698 1.406 0.675 0.720XWLS 0.782 0.700 0.690 1.436 0.687 0.739WLS 0.741 0.724 0.694 1.429 0.680 0.723


dr(xj,L) )

d(xj,L)maxx∈X d(x,L)

A(xj) )R/(1-R)

[(R/1-R)] + dr2(xj,L)

(5)

|A(l+1) - A(l)| < ε

Analytical Chemistry, Vol. 68, No. 5, March 1, 1996 773

algorithm Modified Fuzzy 1-Lines. The following relations arevalid (the notations are those used previously):

A(x) ) 1 S d(x,L) ) 0 (i)

A(x) ) R S dr(x,L) ) 1 (ii)

A(x) ) ∈ [R,1], for every x ∈ X (iii)

R ) 0 S A(x) ) 0, for every x ∈ X (iv)

R ) 1 S A(x) ) 1, for every x ∈ X (v)

A(xi) < A(xj) S d(xj,L) < d(xi,L) (vi)

A(xi) ) A(xj) S d(xj,L) ) d(xi,L) (vii)

The algorithm presented here converges toward a localminimum. Normally, the results of algorithms of this type areinfluenced by the initial partition considered.25,26 In this case, theinitial fuzzy set considered being X, the obtained optimal fuzzyset is the one situated in the vicinity of X, and this makes thealgorithm even more attractive.

Let us remark that the role of the constant R is to affect thepolarization of the partition {A,Ah}. Also, now it is clear why Rwas chosen to be in [0,1] and the values 0 and 1 were avoided.

By using the relative dissimilarities, Dr, this method isindependent of the linear transformations of the space.

Having in mind the properties i-vii and the remarks above,the fuzzy set A determined here may be called “fuzzy setassociated with the classical set X and the membership thresholdR”.

On the model of the theory presented above, we may build atheory to determine the fuzzy set A represented by a pointprototype, or by any geometrical prototype.

As seen above, the (Modified) Fuzzy 1-Lines algorithmproduces the fuzzy set associated to the classical set X and to themembership threshold R, together with its linear representation.This particular procedure may also be Fuzzy Regression.

RESULTS AND DISCUSSIONSome relevant cases of calibration in the most popular books

and well-cited papers are considered to compare the advantagesof the fuzzy regression method described above.

Illustrative Example 1. To illustrate the characteristics ofperformance of the fuzzy regression algorithm (FR) in the caseof small deviations from homoscedasticity or in the presence ofoutliers, we refer to the data discussed by Rajko,7 concerning thedetermination by ICP-AES of Mo, Cr, Co, Pb, and Ni in subsurfaceand drinking water (Table 1).

Considering the results obtained (Table 2) using eight differentcallibration methods, namely ordinary least squares (LS), leastsum of absolute residuals (LSA), least maximum absolute residu-als (LMA), iteratively reweighted least squares with tuningconstants 6 and 9 (IRLS6 and IRLS9), most frequent values (MFV),single median (SM), repeated median (RM) and least median ofsquares (LMS), Rajko concluded that the best results were

(25) Dumitrescu, D. Classification Theory (Romanian); Babes-Bolyai UniversityPress: Cluj-Napoca, 1991.

(26) Bezdek, J. C. Pattern Recognition with Fuzzy Objective Functions Algorithms;Plenum Press: New York, 1981.

Table 4. First Example: Values of New QualityCoefficients

data set

algorithm parameter Mo Cr Co Pb Nia Nib

LS QC5 1.50 1.87 1.67 1.56 1.62 1.70NQC5 0.27 0.48 0.37 0.31 0.33 0.38QC6 3.51 3.06 3.42 3.31 3.33 3.21NQC6 0.13 0.04 0.11 0.09 0.09 0.07

LSA QC5 1.30 1.32 1.39 1.62 1.52 1.66NQC5 0.16 0.17 0.21 0.34 0.28 0.36QC6 4.34 4.44 5.19 3.69 3.55 3.73NQC6 0.29 0.31 0.45 0.16 0.14 0.17

LMA QC5 2.28 2.14 2.71 1.90 2.04 2.20NQC5 0.70 0.62 0.93 0.49 0.57 0.66QC6 2.90 3.07 2.83 3.40 3.27 3.17NQC6 0.01 0.04 0.00 0.11 0.08 0.06

IRLS6 QC5 1.29 1.84 1.39 1.54 1.60 1.64NQC5 0.16 0.46 0.21 0.29 0.33 0.35QC6 4.37 3.07 5.33 3.31 3.35 3.23NQC6 0.29 0.04 0.48 0.09 0.10 0.07

IRLS9 QC5 1.45 1.86 1.61 1.56 1.61 1.68NQC5 0.25 0.47 0.33 0.30 0.33 0.37QC6 3.59 3.06 3.51 3.31 3.34 3.22NQC6 0.14 0.04 0.13 0.09 0.09 0.07

MFV QC5 1.29 1.31 1.39 1.56 1.49 1.51NQC5 0.16 0.17 0.21 0.31 0.27 0.27QC6 4.34 4.83 5.35 3.32 3.81 3.43NQC6 0.29 0.38 0.48 0.09 0.19 0.11

SM QC5 1.31 1.55 1.39 1.61 1.56 1.52NQC5 0.17 0.30 0.21 0.33 0.31 0.28QC6 4.06 3.25 5.23 3.32 3.60 3.58NQC6 0.23 0.08 0.46 0.09 0.15 0.14

RM QC5 1.34 1.31 1.40 1.70 1.31 1.37NQC5 0.19 0.17 0.22 0.38 0.17 0.20QC6 3.84 4.55 5.10 3.37 3.89 4.98NQC6 0.19 0.33 0.43 0.10 0.20 0.41

LMS QC5 1.29 1.31 1.39 1.52 1.26 1.37NQC5 0.16 0.17 0.21 0.28 0.14 0.20QC6 4.40 4.86 5.32 3.51 4.53 5.19NQC6 0.30 0.39 0.48 0.13 0.32 0.45

FR QC5 1.29 1.31 1.39 1.71 1.50 1.77(R ) 0.05) NC5 0.16 0.17 0.21 0.38 0.27 0.42

QC6 4.36 4.85 5.32 3.94 3.78 3.71NC6 0.29 0.39 0.48 0.21 0.18 0.17

XWLS QC5 1.57 1.44 1.69 1.49 1.23 1.41NQC5 0.31 0.24 0.37 0.26 0.13 0.22QC6 3.41 3.96 3.48 3.34 4.34 4.14NQC6 0.11 0.22 0.12 0.09 0.29 0.25

WLS QC5 1.47 1.46 1.41 1.51 1.38 1.42NQC5 0.25 0.25 0.22 0.28 0.21 0.23QC6 3.49 3.41 4.49 3.32 3.76 3.99NQC6 0.12 0.11 0.32 0.09 0.18 0.22


Table 5. Second Example: Relevant HeteroscedasticData

case 11 case 12 case 21 case 22 case 31 case 32 case 4

x y x y x y x y x y x y x y

1 1 1 0.9 0.002 0.12 0 0.009 1 1.1 5.1 0.1083 0.564 1.4333 3 1 1.0 0.003 0.14 2 0.158 2 2.0 10.1 0.1977 1.107 3.006 5 1 1.1 0.005 0.27 4 0.301 3 3.1 20.2 0.4278 1.715 4.90

10 11 3 2.9 0.008 0.40 6 0.472 4 3.8 30.2 0.5916 2.27 6.1030 34 3 3.2 0.012 0.52 8 0.577 5 6.5 46.5 0.9050 2.28 7.8560 54 3 3.5 10 0.739 3.44 9.51

100 92 10 8.5 4.61 12.7810 9.5 5.16 13.8910 10.5 5.73 15.97


produced by LMS. MFV yielded appropriate results for measure-ments of Mo, Cr, Pb, Co, and Ni (221.6 nm). The calibration linecomputed by RM was acceptable for Cr, Pb, and Ni measured atboth wavelengths, and that computed by LSA only for Mo. IRLS9and LS gave nearly the same results. LMA gave very biasedparameters; in fact, it was the most sensitive to the outliers.

Table 2 also includes the results obtained by the weighted least-squares method using 1/x2 as a weighting factor (XWLS)27 andthe ordinary least-squares method using the reciprocal of variancesas weighting factors (WLS). The variances were computed onlyfor two replicates available in the data of Rajko. Concerning theXWLS and WLS methods, we note good results, especially in thecases of heteroscedasticity. XWLS appears to be closer to LMS.

Comparing the results obtained by computation of the fuzzyregression method (FR) discussed above, using a constant R )0.05, presented also in Table 2, it is easy to notice that FR is closerto LMS, in some cases being the best.

In addition, for a more realistic comparison of the 10 regressionmethods presented in Table 2, we computed the estimates ofsample concentrations considering two values of the signal y, 100and 600. A careful examination of the results presented in Table3 illustrates that the performance of FR equals that of LMS insome cases and exceeds the majority of the other methods.

To evalute the linearity of the methods studied, this is a goodopportunity to compare the different quality coefficients (QC) usedso far in analytical chemistry for judging the goodness of fit of aregression line. Thus we present in Table 4 the values obtainedfor QC1, defined as28

where yi and yi are the responses measured at each datum andthose predicted by the model in Table 2, respectively, and N isthe number of all data points.

We also show the values of QC2, constructed similarly to QC1,with the difference that the measured responses yi replace theestimates, yi, in the denominator,29 and also QC3 and QC4,respectively referring to the mean of estimated signal, yj, and themean signal, yj, rather than to the signal itself.30

The main conclusion drawn from the results also presentedin ref 27, by comparison to the statements of Rajko, is that QC2

and QC1 are more suitable for evaluating the goodness of fit. Thequality coefficients QC3 and QC4 appear to be a better solution

(27) Sarbu, C. Anal. Lett. 1995, 28, 2077.(28) Knegt, K.; Stork, G. Fresnius’ Z. Anal. Chem. 1974, 270, 97.(29) Koscielniak, P. Anal. Chim. Acta 1993, 278, 177.(30) Xiaoning, W.; Smeyers-Verbeke, J.; Massart, D. L. Analusis 1992, 20, 209.

Table 6. Second Example: Values of the Estimated Parameters

data set

algorithm parameter case 11 case 12 case 21 case 22 case 31 case 32 case 4

FR a0 -0.030 0.078 0.039 0.01001 0.1852 0.0082 -0.112(R ) 0.05) a1 0.9174 0.9434 40.0461 0.0728 0.9238 0.00193 2.795OLS a0 1.217 0.2134 0.0400 0.0132 -0.4800 0.0141 0.257

a1 0.9118 0.9328 41.6666 0.0725 1.2600 0.0193 2.718WLS a0 0.15 0.0212 0.0277 0.0091 0.0057 -0.099

a1 0.95 0.9954 43.935 0.0738 0.0199 2.782GWLS a0 0.0146 0.0212 0.0272 0.00899 -0.0390 0.0073 -0.1634

a1 0.9803 0.9954 44.0390 0.0737 1.0945 0.0196 2.896IRLS a0 0.19

a1 0.92ELS a0 0.0072

a1 0.0197

Table 7. Second Example: Values of New QualityCoefficients

data set

algorithmparam-

etercase11

case12

case21

case22

case31

case32

case4

FR QC5 1.05 1.54 1.34 1.20 1.00 1.01 1.04(R ) 0.05) NQC5 0.03 0.27 0.27 0.13 0.00 0.01 0.02

QC6 4.60 4.51 2.98 3.97 4.34 4.00 6.02NQC6 0.45 0.25 0.27 0.43 0.76 0.64 0.50

OLS QC5 1.15 1.48 1.76 1.22 1.43 1.17 1.11NQC5 0.09 0.24 0.61 0.15 0.34 0.14 0.05QC6 3.56 4.24 2.43 3.69 2.72 2.73 4.68NQC6 0.21 0.20 0.07 0.35 0.17 0.17 0.28

WLS QC5 1.33 1.17 1.45 1.39 1.43 1.04NQC5 0.20 0.08 0.36 0.27 0.35 0.02QC6 3.57 4.46 2.47 3.40 2.61 5.80NQC6 0.21 0.24 0.08 0.26 0.13 0.46

GWLS QC5 1.50 1.17 1.42 1.43 1.13 1.22 1.29NQC5 0.30 0.08 0.34 0.29 0.11 0.18 0.14QC6 3.63 4.46 2.47 3.45 3.12 2.76 4.17NQC6 0.22 0.24 0.08 0.28 0.32 0.19 0.19

IRLS QC5 1.00NQC5 0.00QC6 4.36NQC6 0.76

ELS QC5 1.41NQC5 0.33QC6 2.64NQC6 0.14

Table 8. Third Example: Data from Atomic AbsorptionSpectrometry

signals for elementsconcn,mg L-1 Al Al Mn Mn Cu Cu Pb Pb

0 0.000 0.000 0.000 0.000 0.000 0.000 0.00010 0.030 0.046 0.110 0.000 0.025 0.06120 0.056 0.084 0.089 0.226 0.097 0.088 0.055 0.13030 0.078 0.18340 0.133 0.194 0.171 0.430 0.182 0.162 0.097 0.23060 0.177 0.245 0.256 0.637 0.256 0.222 0.156 0.35880 0.330 0.320 0.765 0.331 0.285 0.194 0.433

100 0.372 0.870 0.417 0.344

QC1 )x∑i)1

N

((yi - yi)/yi)2

N - 1× 100% (6)


when comparing methods based on the same algorithm, i.e., leastsquares.

However, taking into account the contradictory values ofdifferent quality coefficients and the diversity of methods concern-ing their algorithm, we have introduced two new quality coef-ficients, QC5 and QC6.

The first coefficient, QC5, refers to the maximum of absoluteresiduals,

and QC6 refers to the mean of absolute residuals,

It is very easy to prove that the possible values for QC5 lie inthe range [1,N1/2], and the possible values for QC6 lie in the range[N1/2,N].

The results obtained by computing QC5 and QC6 are given inTable 4. Moreover, in the same table we introduce the values ofthe normalized QC5 and QC6, namely NQC5 (9) and NQC6 (10),which take values within the range [0,1] and thus appear to bemore practical:

and

Table 9. Third Example: Values of the EstimatedParameters

data set

algorithm parameter Al Al Mn Mn Cua Cua Pb Pb

FR a0 (×10-3) -0.15 -0.54 8.36 12.12 16.78 40.11 -0.11 1.99(R ) 0.05) a1 (×10-3) 2.94 4.10 3.91 9.44 3.99 3.04 2.60 5.93LS a0 (×10-3) 0.03 6.40 11.42 38.83 19.90 29.70 2.31 11.08

a1 (×10-3) 3.04 4.10 3.79 8.92 3.95 3.18 2.45 5.49FUZR a0 (×10-3) -1.11 8.84 1.37 8.74 23.66 35.32 9.13 4.72

a1 (×10-3) 2.97 3.94 4.10 10.40 3.87 3.09 2.32 5.89FUZL a0 (×10-3) -0.04 0.48 0.00 4.66 16.81 40.48 0.23 1.98

a1 (×10-3) 2.95 4.10 4.28 10.55 4.00 3.04 2.57 5.93SM a0 (×10-3) 0.00 1.00 6.00 14.38 19.25 35.25 0.75 4.00

a1 (×10-3) 2.98 4.11 4.00 9.56 3.95 3.11 2.43 5.70RM a0 (×10-3) 0.00 0.00 4.62 7.25 24.25 36.00 0.40 7.33

a1 (×10-3) 2.97 4.12 4.14 10.28 3.86 3.10 2.46 5.57LMSb a0 (×10-3) 0.00 0.00 4.00 4.60 33.00 40.67 0.00 29.00

a1 (×10-3) 2.95 4.13 4.20 10.54 3.73 3.03 2.43 5.05LMSc a0 (×10-3) 0.25 0.75 0.38 4.60 32.75 40.33 0.38 29.75

a1 (×10-3) 2.95 4.13 4.20 10.54 3.73 3.03 2.43 5.05

a Data sets do not include the zero point. b Obtained with the methodof Massart et al. c Obtained with Progress by Rousseeuw and Leroy.

Table 10. Third Example: Values of New QualityCoefficients

data set

algorithm parameter Al Al Mn Mn Cua Cub Pb Pb

FR QC5 1.01 1.00 1.16 1.33 1.35 1.00 1.13 1.04(R ) 0.05) NQC5 0.01 0.00 0.09 0.20 0.28 0.00 0.08 0.02

QC6 4.05 4.31 3.76 3.53 3.20 4.16 4.44 4.72NQC6 0.66 0.75 0.25 0.20 0.35 0.69 0.41 0.47

LS QC5 1.18 1.11 1.60 1.65 1.43 1.58 1.50 1.70NQC5 0.15 0.09 0.36 0.40 0.35 0.47 0.30 0.42QC6 3.05 2.84 3.15 3.09 2.53 2.55 2.89 3.07NQC6 0.29 0.22 0.11 0.10 0.10 0.11 0.05 0.09

FUZR QC5 1.01 1.08 1.07 1.08 1.35 1.09 1.63 1.05NQC5 0.01 0.06 0.04 0.05 0.29 0.08 0.38 0.03QC6 3.85 3.21 4.02 4.78 2.63 3.07 3.35 4.52NQC6 0.58 0.35 0.31 0.49 0.14 0.30 0.16 0.43

FUZL QC5 1.01 1.00 1.08 1.09 1.34 1.00 1.16 1.04NQC5 0.01 0.00 0.04 0.05 0.28 0.00 0.10 0.02QC6 4.11 4.29 4.92 4.96 3.21 4.17 3.93 4.71NQC6 0.68 0.74 0.52 0.53 0.35 0.70 0.29 0.47

SM QC5 1.04 1.00 1.08 1.19 1.48 1.06 1.26 1.22NQC5 0.03 0.00 0.05 0.11 0.39 0.05 0.16 0.13QC6 3.70 4.32 4.03 3.62 2.66 3.31 3.74 3.57NQC6 0.53 0.75 0.31 0.22 0.15 0.38 0.25 0.21

RM QC5 1.03 1.00 1.06 1.08 1.33 1.04 1.37 1.52NQC5 0.02 0.00 0.03 0.05 0.26 0.03 0.22 0.32QC6 3.83 4.39 4.79 4.55 2.68 3.48 3.41 3.24NQC6 0.57 0.77 0.49 0.43 0.16 0.45 0.17 0.13

LMSb QC5 1.02 1.00 1.07 1.09 1.39 1.01 1.28 1.48NQC5 0.01 0.00 0.04 0.05 0.31 0.00 0.17 0.29QC6 4.13 4.33 5.01 4.97 3.32 4.16 3.98 3.92NQC6 0.68 0.76 0.54 0.53 0.39 0.69 0.30 0.29

LMSc QC5 1.02 1.00 1.06 1.09 1.35 1.01 1.27 1.46NQC5 0.01 0.00 0.04 0.05 0.29 0.01 0.16 0.28QC6 4.02 4.14 4.53 4.97 3.42 3.99 3.86 3.85NQC6 0.64 0.69 0.43 0.53 0.42 0.63 0.28 0.27

a Data sets do not include the zero point. b Obtained with the methodof Massart et al. c Obtained with Progress by Rousseeuw and Leroy.

Figure 1. First example: variation of NQC6 as a function of R.

Figure 2. Second example: variation of NQC6 as a function of R.

QC5 )x∑i)1

N

(ri/max|ri|)2 (7)

QC6 )x∑i)1

N

(ri/rj)2 (8)

NQC5 )QC5 - 1

xN - 1(9)


Examining Table 4, it is easy to notice a very good agreementwith Rajko statements, especially for QC6. The fuzzy regressionmethod appears to be the best in some cases and much closer toLMS in others.

Overall, it may be stated that these new quality coefficientsconcerning the goodness of fit proposed in this paper confirmour main conclusions and are in good agreement with thestatements in the analytical literature (see the following examples).

Illustrative Example 2. Let us consider the relevant het-eroscedasticity examples discussed by Garden et al.,31 which usedthe inverse of variance as a weighting factor (wi ) si

-2) (cases 11and 12), and those of Caulcutt and Boddy32 (case 21) and Millerand Miller2 (case 22), which used wi ) si

-2(∑si-2/N). The data

are presented in Table 5, and the computed results are presentedin Table 6.

Case 31, also shown in Table 5, refers to the data computedby Phillips and Eyring,9 in order to illustrate the insensitivity ofthe technique of iteratively reweighted least squares (IRLS) toerroneous observations. Also, we compare here the results givenby FR using R ) 0.05, with the results reported by Aarons33 usingOLS, WLS, and an extended squares method (ELS) for studyingthe reproducibility for ibuprofen assay (case 32).

The last example in Table 5 (case 4) considers the paper ofSchwartz10 concerning the gas chromatographic determination ofbenzene34 under conditions of nonuniform variance. In this paper,Schwartz observed that the weighting factor must reflect thereplication number N and also the variance at the location of thepoint, so that wi ) ni/si

2. In all these cases, we included thevalues obtained using XWLS, which produces better results thanWLS.

Considering the parameters shown in Table 6, we computedthe normalized quality coefficients NQC5 and NQC6 (see Table7). Again, the FR method seems to be the best.

Illustrative Example 3. The last example reported in thispaper refers to data analyzed by Massart et al.13 concerning thedetermination of Al, Mn, Cu, and Pb in biological samples by AAS.Data listed in Table 8 were selected to cover a wide range of outliertypes.

Considering the results obtained (see Tables 9 and 10) usingseven different calibration methods, namely LS, SM, RM, LMS,two fuzzy methods (FUZR16 and FUZL17), and a modified lastmedian squares (LMSg), Massart et al. concluded that the bestresults were given by LMS. With respect to SM and RM, it wasnoted that RM is not only more robust but is also faster than SM.

In the method FUZR by Otto and Bandemer,16 a circle acts asthe support of the fuzzy observations. Relative to the methodFUZL by Massart et al.,17 as support of the fuzzy observations, aline appeared to be more suitable.

By a carefully examination of the data in Table 10, we noticeonce again a very good agreement between the conclusionsexpressed by Massart et al. and the values evaluated for NQC5

and NQC6.

By comparing the three fuzzy methods, we may conclude thatthe FUZL method17 seems to be the best and FUZR16 seems tobe the poorest. FR is much closer to FUZL when a value of R )0.05 is used.

CONCLUSIONSA new fuzzy regression algorithm has been described in this

paper. It was compared with conventional ordinary and weightedleast-squares and robust regression methods. The application ofthese different methods to relevant data sets proved that theperformance of the procedure described in this paper exceedsthat of the ordinary least-squares method and equals and oftenexceeds that of weighted or robust methods, including the twofuzzy methods proposed in refs 16 and 17.

(31) Garden, J. S.; Mitchell, D. S.; Mills, W. N. Anal. Chem. 1980, 52, 2310.(32) Caulcutt, R.; Boddy, R. Statistics for Analytical Chemists; Chapmann & Hall:

London, 1983; p 104.(33) Aarons, L. J. Pharm. Biomed. Anal. 1984, 2, 395.(34) Bocek, P.; Novak, J. J. Chromatogr. 1970, 51, 375.

NQC6 )QC5 - xN

N - xN(10)

Figure 3. Third example: variation in NQC6 as a function of R.

Figure 4. Third example: informational energy as a function of R.

Figure 5. Third example: informational entropy as a function of R.


Moreover, we underline the effectiveness and the generalityof the two new criteria proposed in this paper for diagnosing thelinearity of calibration lines in analytical chemistry.

The effectiveness of these criteria allowed us to select anoptimal value for the R constant, as we may see in Figures 1-3.

In addition, we emphasize that the fuzzy regression methoddiscussed above includes not only the estimates of the parametersbut also additional information in the form of membership degrees.In this way, the fuzzy regression algorithm gives a new aspect toregression methods. Using the membership degrees of each point

to the regression line, we may compute the informational energy(E ) (1/N)∑[A(xj)]2)35 and/or the informational entropy, as iswell illustrated in Figure 4 and 5. These quantities allow us toappreciate the presence of outliers and the linearity of theregression line.

Received for review June 7, 1995. Accepted October 31,1995.X

AC950549U

(35) Sarbu, C. Anal. Chim. Acta 1993, 271, 269. X Abstract published in Advance ACS Abstracts, January 1, 1996.


a new fuzzy regression algorithm

Documents