Proceedings of the 8th

World Congress on Intelligent Control and Automation July 6-9 2010, Jinan, China

978-1-4244-6712-9/10/$26.00 ©2010 IEEE

Online Approach of Fault Diagnosis Based on Lifting Wavelets and Moving Window PCA

Qing Yang 1, 2 Feng Tian1 Dazhi Wang2, 3 1) School of Information Science 2) College of Optical and Electronical Information 3) College of Information Science Shenyang Ligong University Changchun University of Science and Technology Northeast University

Shenyang 110159,China Changchun 130022,China Shenyang 110004,China

yangqingxp@126.com nicholas232@163.com

Abstract –To online monitor process, a combined approach of fault detection and diagnosis based on Lifting Wavelets and Moving Window PCA (LW-MWPCA) was presented. Firstly the data were pre-processed to remove noise and spikes through lifting scheme wavelets, and then MWPCA was used to diagnose faults. To validate the performance and effectiveness of the proposed scheme, LW-MWPCA was applied to diagnose the faults in TE Process. The results were given to show the effectiveness of these improvements for fault diagnosis performance in terms of low computational cost and high fault diagnosis rate.

Index Terms - Lifting wavelets; Online fault detection and diagnosis; LW-MWPCA; TE process.

I. INTRODUCTION

Fault detection and diagnosis (FDD) is very important for ensuring plant safety and product quality. Now that many methods, especially combined methods which have more advantages over single methods, of fault detection and diagnosis have been put forward [1-7]. But most of these approaches are not online.

In recent years, a few online methods were put forward, such as recursive PCA [8], MVS [9], BDPCA [10] and MPCA [11]. However all these methods paid little attention to online de-noise in the data which affected the quality of FDD.

Now that the traditional wavelet transform methods have been applied in many fields, such as signal and image processing. Wim Sweldens[12-13] proposed another wavelet transform based on lifting steps. It is a flexible wavelet construction method by using linear or nonlinear spatial prediction and operators to implement the wavelet transform and to make it reversible. The lifting scheme has the following features: (1) is independent of the Fourier transform; (2) is a computationally fast algorithm; (3) is convenient for reverse transform: and (4) makes non-linear wavelet transforms possi-ble, e.g. an integer to integer wavelet transforms.

For above reasons, a combined online FDD approach based on lifting scheme wavelets and moving window PCA was presented in this paper.

II. LIFTING WAVELETS AND MOVING WINDOW PCA

A. Lifting scheme wavelets Lifting scheme was first proposed by Wim Sweldens.

The original motivation for developing lifting was to build second generation wavelets, i.e., wavelets adapted to situations that did not allow translation and dilation like non-Euclidean spaces.

(1). The original signals ][nX are split into two non-intersecting subsets ][nc and ][nd .The greater the correlation between them, the better the split effect is. Usually a signal sequence is split into odd and even sequences, the even indexed samples ][nX e , and the odd indexed samples ][nX o .

(2). Using the similarity of data, we can predict ][nd from ][nc by using a predict operator P which is independent of the dataset. Of course, the predictor need not be exact, so we need to record the difference or detail d :

])[(][][ 0 nXPnXnd e−= (1) Given the detail d and the odd, we can immediately recover the odd as

][])[(][ ndnXPnX eo += (2) (3). In particular, the running average of the ][nX e is not

the same as that of the original samples ][nX .To correct this, a second lifting step is proposed , which replaces the evens with smoothed values ][nc with the use of an update operatorU applied to the details:

])[(][][ ndUnXnc e += (3)

][nX e

][nX o

][nc

][nd][nX o

][nX e

][nX ][nXP− U U P−

Fig. 1 The forward wavelet transform and inverse using lifting

From this we see that if P is a good predictor, and then

d approximately will be a sparse set; in other words, the first order entropy is expected to be smaller for d than for oX .

The de-noising procedure using the wavelet trans-form of discrete signals can be divided into three steps: wavelet decomposition, wavelet coefficient reduction (truncating the noise portion), and reducing the composition of wavelet coefficients. Currently the most popular de-noising methods are the soft-threshold method and the hard-threshold method.

B. Moving Window Principal Components Analysis

2909

1). PCA Algorithm: The PCA technique determines combi-nations of variables that describe major trends, or variation, in a data set. For a data matrix, X , with n rows (of measurements or samples) and m columns (of variables), the covariance matrix of X is defined as

1)(

−=

nXXXCOV

T

(4)

where the columns of X have been centered to have a mean of zero and scaled to a standard deviation of one. PCA decomposes this data matrix, X , as the sum of the outer product of vectors, it and ip plus a residual error matrix E

EptptptETPX Tkk

TTT ++⋅⋅⋅++=+= 2211 (5)

It is now possible to generate a new data set based on the number of principal components retained ( k ). In general, k will normally be much smaller than the number of variables in the original data.

2). MWPCA Algorithm: The details of MWPCA algorithm in this paper are shown in Figure 2 for a window size L :

IIIMatrix

mLLk

Lk

k

IIMatrix

mLLk

k

IMatrix

mLLk

k

k

x

x

x

x

x

x

x

x

×+

−+

+

×−−+

+

×−+

+

⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

⇒⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜

⎝

⎛

⇒

⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

0

01

01

)1(0

1

01

01

01

0

Fig. 2 Two-step adaptation to construct new data window

The three matrices in Fig.2 [14] represent the data in the previous window (Matrix I), the result of removing the oldest sample 0

kx (Matrix II), and the current window of selected data

(Matrix III) produced by adding the new sample 0lkx + to Matrix

II. Mean of Matrix I kμ :

∑−+

==

101 Lk

kiik x

Lμ (6)

Because of

01

1

0kk

Lk

kii xLx −=∑

−+

+=μ (7)

So that mean of Matrix II μ~ :

)(1

11

1~ 01

1

0kk

Lk

kii xL

Lx

L−

−=

−= ∑

−+

+=μμ (8)

Difference between means: μμμ ~~ −=Δ k (9)

Scale the discarded sample:

)(1 0kkk xx μ−

Σ= (10)

Bridge over Matrix I and Matrix III: Tkkk

Tkk xx

LRR

11~~ 11*

−−ΣΔΔΣ−= −− μμ (11)

Mean of Matrix III:

]~)1[(1 01 Lkk xL

L ++ +−= μμ (12)

Difference between means of Matrix III and Matrix II: μμμ ~

11 −=Δ ++ kk (13) Standard deviation of Matrix III:

1)]()([)]()([

))(~())(())(())((202

10

221

221

−−−−+

Δ−Δ+=

++

++

Liixiix

iiii

kkkLk

kkk

μμμμσσ

(14)

)}()1({ 111 mdiag kkk +++ =Σ σσ (15) Scale the new sample:

)(11

0

1++

++ −

Σ= kLk

kLk xx μ (16)

Correlation matrix of Matrix III:

TlkLk

kT

kkkkkkkk

xxL

RR

++

−+++

−+

−+

−++

−+

ΣΔΔΣ+ΣΣΣΣ=

11

1111

11

11

*111 μμ

(17)

C. LW-MWPCA

1). De-noise by lifting scheme wavelet The de-noising of process data using the lifting scheme

wavelet transform is described in this section. Choose complementary filter ),( gh first, then from z-transform we can get )(hg , )(zg . According to polyphase representation of filter get the polyphase components )(zhe , )(zho , )(zge , )(zgo and polyphase matrix )(zP . Then there exist Laurent polynomials

)(zsi and )(zti for mi ≤≤1 and a non-zero constant K so that

⎥⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡= ∏

= KK

ztzs

zPi

im

i 10

010

)(1

1)(

01

)(1

(18)

In this paper we use the soft-threshold method: ττ −= )())(()( ndndsignnd

⎪⎩

⎪⎨

⎧

<−>−

≤

=ττττ

τ

xndxnd

x

,)(,)(

,0 (19)

)(6745.01

log2 )(

dMed

Ne

=

=

σ

στ

where τ is the computation threshold, σ is the standard deviation of the noise estimation, and )(⋅Med is the median function. In the de-noising procedure, we first perform the lifting wavelet transform for the process data and then process the detail signals at each level using the soft-threshold method. As a result, the noise part in the wavelet coefficient is reduced.

2). Fault detection Fault detection using PCA or its variants is usually

performed by monitoring the squared prediction error ( SPE ) or Hotelling’s 2T statistic. The process is considered normal if )(αQSPE < where )(αQ denotes the upper control limit for confidence level α−1 based on a standard normal

2910

distribution. An upper control limit )(2 αT similar to )(αQ

can also be derived for the 2T statistic. The development of adaptive confidence limits, as discussed by [8] is first considered. Here, the 2T statistic for the thk sample are defined as

kT

kkkTkk XPPXT 12 −Λ= (20)

The %100α control limit for 2T is obtained using the F-distribution

),()(

)1)(1()(2 ααα

ααα −

−+−= LF

LLLLkT (21)

SPE is calculated as follows

kT

kkTkk XPPIXSPE )( −= (22)

The )%100( α− control limit for SPE is

0

1

21

002

1

21

202

1 ])1(1)2([)( hhhhCkSPEθ

θθθθ α

α−++= (23)

where ),( ααα −LF is the upper 100a% critical point of the F-distribution with α and α−L degrees of freedom, L is the

number of selected samples, ∑+=

=m

lj

ij

1λθ , 2

2

310 3

21θθθ−=h and

αC is the confidence limits for the α−1 percentile in a normal distribution.

3).Fault diagnosis Contribution method based on PCA model is a kind of

simple fault diagnosis method. It reflects the change of system variables influence the stability of statistical models, so as to realize the fault isolation. The steps below:

(1). Check the normalized scores 2)( iit σ for the obser-vation x and determine the kr ≤ scores responsible for the out-of-control status. Recall that it is the score of the

observation projected onto the thi loading vector, and iσ is the corresponding singular value.

(2). Calculate the contribution of each variable jx to the out-of-control scores it

)(,2, jjjii

ji xptcont μσ

−= (24)

where jip , is the thji ),( element of the loading matrix P . (3). When jicont , is negative, set it equal to zero.

(4). Calculate the total contribution of the thj process variable, jx ,

∑=

=r

ijij contCONT

1, )( (25)

(5). Plot jCONT for all m process variables, jx on a single graph.

The sketch of the proposed algorithm is given in Fig. 3.

Fig. 3 The sketch of LWMPCA

IV. CASE STUDY

In this section, we will demonstrate the use of LW-MWPCA for online fault detection and diagnosis purposes first with respect to a simple multivariate process as well as with the much more complex and realistic Tennessee Eastman process.

A. Multivariate process

Consider the following process [15]:

)1(4

231

)1(264.0

266.0847.0118.0

)( −⎥⎦

⎤−⎢

⎣

⎡+−⎥

⎦

⎤−⎢⎣

⎡= kukxkx (26)

)()()( kvkxky += (27)

where u is the correlated input:

)1(749.0

689.0320.0

193.0)1(

415.0266.0

477.0811.0

)( −⎥⎦

⎤−⎢

⎣

⎡−

+−⎥⎦

⎤−⎢⎣

⎡= kwkuku (28)

the input w is a random noise with zero mean and variance 1. The output y is equal to x plus the random noise, )(kv , with zero mean and variance 0.1. Both input u and output y are measured but x and w are not. This data set contains a total number of 1000 samples. The first 200 samples are selected as the normal data and the remaining 800 samples serve as fault data. A disturbance with unit step change of 1w is introduced at sample 200. Note that this

disturbance is not measured but affects the output. The 2T and SPE data collected during the occurrence of the disturbance are plotted in Fig.4 and Fig. 5. The SPE plot clearly indicates that the process is out of control after sample 200. The 2T plot also indicates that the process is out of control.

2911

Fig. 4 T2 statistics for fault

Fig. 5 SPE statistics for fault

B. Tennessee Eastman process As a more challenging case study we now examine

another example of the application of the proposed strategy. TE process [16] is a benchmark problem in process engineering. Downs and Vogel presented this particular process at an AIChE meeting in 1990 as a plant-wide control problem.

The simulator of the Tennessee Eastman process consists of five major unit operations: a reactor, a product condenser, a vapor-liquid separator, a recycle compressor, and a product stripper. Two products are produced by two simultaneous gas-liquid exothermic reactions, and a by product is generated by

Fig. 6 Control system of the Tennessee Eastman process

two additional exothermic reactions. The process has 12 manipulated variables, 22 continuous process measurements, and 19 compositions. The simulator can generate 21 types of different faults, listed in Table I. Once the fault enters the process, it affects almost all state variables in the process.

Fig.7 and Fig.8 show the 2T and SPE monitoring charts of LW-MWPCA monitoring method for fault 5. The LW-MWPCA based monitoring approach was shown to be more effective and faster. That is, under the similar false alarm rate to the other methods, LW-MWPCA gave not only low missing alarms but also little Computing time. More importantly, when the process data has lots of noise MWPCA may not detect a specific fault, whereas LW-MWPCA can detect it. To sum up, as shown through the application studies, the proposed LW-MWPCA based monitoring method had better performance than some normal methods for various faults.

Fig. 7 T2 statistics for fault 5

Fig. 8 SPE statistics for fault 5

Fig. 9 Contribution figure for fault 5

2912

V. CONCLUSIONS

The study of online fault detection and diagnosis in industrial process is full of challenge. In this paper, a combined method based on lifting scheme wavelet and moving window PCA was presented to diagnosis the faults in a multivariate process and TE process. The simulation results showed that the LW-MWPCA can provides higher diagnosis rate and more accurate than normal PCA and its extension.

REFERENCES [1] Manabu Kano, Yoshiaki Nakag, “Data-based process monitoring, process

control, and quality improvement: recent developments and applications in steel industry,” Computers and Chemical Engineering, vol.32, pp. 12-24, 2008.

[2] L. H. Chiang, F. L. Russell, and R. D. Braatz, Fault detection and diagnosis in industrial systems, London: Springer, 2001.

[3] B. R. Bakshi, “Multiscale PCA with application to multivariate statistical process monitoring,” American Institute of Chemical Engineering Journal, vol. 44, pp. 1596-1610, 1998.

[4] V. Venkatasubramanian, et al, “A review of process fault detection and diagnosis: Part III: Process history based methods,” Computers & Chemical Engineering, vol. 27, pp. 327-346, 2003.

[5] A. Maulud, D. Wang, and J. A. Romagnoli, “A multi-scale orthogonal nonlinear strategy for multi-variate statistical process monitoring,” Journal of process Control, vol. 16, pp. 671-683, 2006.

[6] Q. Yang, et al, “Fault diagnosis approach based on probabilistic neural network and wavelet analysis,” Proceedings of the 7th WCICA'08, pp.1796-1799, 2008.

[7] Y. Power and P.A.Bahri, “Integration techniques in intelligent operational management: a review,” Knowledge-Based Systems, vol.18, pp.89-97, 2005.

[8] W. Li, et al, “Recursive PCA for adaptive process monitoring,” Journal of Process Control, vol.10, no.5, pp. 471-486, 2000.

[9] I. Miletic, et al, “An industrial perspective on implementing on-line applications of multivariate statistics,” Journal of Process Control, vol.14, no.8, pp. 821-836, 2004.

[10] J. Chen and K. Liu, “ On-line batch process monitoring using dynamic PCA and dynamic PLS models,” Chemical Engineering Science, vol. 57, no.1, pp. 63–75, 2002.

[11] X. Liu, et al, “Moving window kernel PCA for adaptive monitoring of nonlinear processes,” Chemometrics and Intelligent Laboratory Systems, vol.96, no.2, pp.132-143, 2009.

[12] W. Sweldens, Construction and Applications of Wavelets in Numerical Analysis, Ph.D.thesis, Department of Computer Science, Katholieke Universiteit Leuven, Belgium, 1994.

[13] W. Sweldens, “The lifting scheme: A construction of second generation wavelets,” SIAM J. Math. Anal, vol.29, pp.511-546, 1998.

[14] X. Wang, U. Kruger, and G. W. Irwin, “Process Monitoring Approach Using Fast Moving Window PCA” Ind. Eng. Chem. Res, vol. 44, pp. 5691-5702, 2005

[15] W. Ku, “Disturbance detection and isolation by dynamic principal component analysis,”Chemometrics and Intelligent Laboratory Systems, vol. 30, pp. 179-196, 1995.

[16] J. J. Downs, E. F. Vogel, “A plant-wide industrial-process control problem,” Computers& Chemical Engineering, Vol.17, no.3, pp. 245-255, 1993.

2913