research article a denoising based autoassociative...
TRANSCRIPT
Research ArticleA Denoising Based Autoassociative Model for Robust SensorMonitoring in Nuclear Power Plants
Ahmad Shaheryar1 Xu-Cheng Yin1 Hong-Wei Hao12 Hazrat Ali3 and Khalid Iqbal4
1University of Science and Technology Beijing (USTB) 30 Xueyuan Road Haidian District Beijing 100083 China2Institute of Automation Chinese Academy of Sciences University of Science and Technology Beijing 95 Zhongguancun East RoadBeijing 100190 China3Department of Electrical Engineering COMSATS Institute of Information Technology University Road Abbottabad 22060 Pakistan4COMSATS Institute of Information Technology near Officers Colony Kamra Road Attock 43600 Pakistan
Correspondence should be addressed to Ahmad Shaheryar shaheryar011yahoocomand Xu-Cheng Yin xuchengyinustbeducn
Received 18 November 2015 Accepted 19 January 2016
Academic Editor Eugenijus Uspuras
Copyright copy 2016 Ahmad Shaheryar et al This is an open access article distributed under the Creative Commons AttributionLicense which permits unrestricted use distribution and reproduction in any medium provided the original work is properlycited
Sensors health monitoring is essentially important for reliable functioning of safety-critical chemical and nuclear power plantsAutoassociative neural network (AANN) based empirical sensor models have widely been reported for sensor calibrationmonitoring However such ill-posed data driven models may result in poor generalization and robustness To address above-mentioned issues several regularization heuristics such as training with jitter weight decay and cross-validation are suggestedin literature Apart from these regularization heuristics traditional error gradient based supervised learning algorithms formultilayered AANNmodels are highly susceptible of being trapped in local optimum In order to address poor regularization androbust learning issues here we propose a denoised autoassociative sensor model (DAASM) based on deep learning frameworkProposed DAASM model comprises multiple hidden layers which are pretrained greedily in an unsupervised fashion underdenoising autoencoder architecture In order to improve robustness dropout heuristic and domain specific data corruptionprocesses are exercised during unsupervised pretraining phase The proposed sensor model is trained and tested on sensor datafrom a PWR type nuclear power plant Accuracy autosensitivity spillover and sequential probability ratio test (SPRT) based faultdetectability metrics are used for performance assessment and comparison with extensively reported five-layer AANN model byKramer
1 Introduction
From safety and reliability stand point sensors are one of thecritical infrastructures in modern day automatic controllednuclear power plants [1] Decision for a control actioneither by operator or by automatic controller depends oncorrect plant state reflected by its sensors ldquoDefense in depthrdquo(defense in depth safety concept requires mission criticalsystems to be redundant and diverse in implementation toavoid single mode failure scenarios) safety concept for suchmission critical processes essentially requires a sensor healthmonitoring system Such sensor health monitoring systemhasmultifaceted benefits which are just not limited to processsafety reliability and availability but also in context of cost
benefits from condition based maintenance approach [2 3]A typical sensor health monitoring system may include tasksof sensor fault detection isolation and value estimation[4] Basic sensor monitoring architecture comprises twomodules as depicted in Figure 1The first module implementsa correlated sensormodel which provides analytical estimatesfor monitored sensorrsquos values Residuals values are evaluatedby differencing the observed and estimated sensor values andare supplied to residual analysis module for fault hypothesistesting These correlated sensor models are based on eitherthe first principles models (eg energy conservation andmaterial balance) or history based data driven models [5]However sensor modeling using empirical techniques from
Hindawi Publishing CorporationScience and Technology of Nuclear InstallationsVolume 2016 Article ID 9746948 17 pageshttpdxdoiorg10115520169746948
2 Science and Technology of Nuclear Installations
Sensornetwork
Signal reconstruction
model
ResidualFault
detection
Fault
Nofault
Residual analysis
SObserved
SEstimate
Figure 1 Integrated sensor estimation and fault detection architecture
statistics and artificial intelligence are an active area ofresearch [6 7]
In order to model complex nonlinearity in physicalprocess sensors autoassociative neural network based sensormodels had widely been used and reported for calibrationmonitoring in chemical processes [8ndash11] and nuclear powerplants [12ndash15] Data driven training procedures for suchneural network based sensor models discover the underlyingstatistical regularities among input sensors from history dataand try to model them by adjusting network parametersFive-layer AANN is one of the earliest autoassociative archi-tectures proposed for sensor and process modeling [8]
In contrast to shallow single layered architectures thesemultilayered neural architectures have flexibility for model-ing complex nonlinear functions [16 17] However harness-ing the complexity offered by these deep NN models with-out overfitting requires effective regularization techniquesSeveral heuristics based standard regularization methodsare suggested and exercised in literature [18 19] such astraining with jitter (noise) Levenberg-Marquardt trainingweight decay neuron pruning cross validation and Bayesianregularization Despite all these regularization heuristics thejoint learning of multiple hidden layers via backpropagationof error gradient inherently suffers from gradient vanishingproblem at the earlier layers [20] This gradient instabilityproblem restricts the very first hidden layer (closer to input)from fully exploiting the underlying structure in original datadistribution Result is the poor generalization and predictioninconsistency Problem gets evenmore complex and hard dueto inherent noise and colinearity in sensor data
Considering the complexity and training difficulty dueto gradient instability in five-layer AANN topology Tanand Mayrovouniotis proposed a shallow network topologyof three layers known as input trained neural network(ITN-network) [21] However the modeling flexibility getscompromised by shallow architecture of ITN-network
The regularization and robustness issues associated withthese traditional learning procedures motivate the need forcomplementary approaches Contrary to shallow architectureapproach by Tan andMayrovouniotis [21] here we are inter-ested in preserving the modeling flexibility offered by manylayered architectures without being compromised on gener-alization and robustness of the sensormodel Recent researchon greedy layerwise learning approaches [22 23] has beenfound successful for efficient learning in deep multilayered
neural architectures for image speech and natural languageprocessing [24] So for a multilayered DAASM we pro-posed to address poor regularization through deep learningframework Contrary to joint multilayer learning methodsfor traditional AANN models the deep learning frameworkemploys greedy layerwise pretraining approach Followingthe deep learning framework each layer in the proposedDAASM is regularized individually through unsupervisedpretraining under denoising based learning objective Thisdenoising based learning is commenced under autoencoderarchitectures as elaborated in Section 3 It essentially servesseveral purposes
(1) Helps deep models in capturing robust statisticalregularities among input sensors
(2) Initializes network parameters in basin of attractionwith good generalization properties [17 25]
(3) Implicitly addresses modelrsquos robustness by learninghidden layer mappings which are stable and invariantto perturbation caused by failed sensor states
Moreover robustness to failed sensor states is not anautomatic property of AANN based sensor models but isprimarily essential for fault detection Consequently tradi-tional AANN based sensor model requires explicit treatmentfor robustness against failed sensor states However forthe case of DAASM an explicit data corruption process isexercised during denoising based unsupervised pretrainingphase The proposed corruption process is derived fromdrift additive and gross type failure scenarios as elaboratedin Section 41 Robustness to faulty sensor conditions is animplicit process of denoising based unsupervised pretrain-ing phase Robustness of the proposed DAASM againstdifferent sensor failure scenarios is rigorously studied anddemonstrated through invariance measurement at multiplehidden layers in theDAASMnetwork (see Section 7)The fullDAASM architecture and layerwise pretraining is detailedin Section 4 We will compare the proposed DAASM basedsensor model with an extensively reported five-layer AANNbased sensor model by Kramer Both sensor models aretrained on sensor data sampled from full power steadyoperation of a pressurizedwater reactor Finally performanceassessment with respect to accuracy autosensitivity cross-sensitivity and fault detectability metrics is conducted underSection 8
Science and Technology of Nuclear Installations 3
2 Problem Formulation
In context of sensor fault detection application the purposeof a typical sensor reconstruction model is to estimatecorrect sensor value from its corrupted observation Theobjective is to model relationships among input sensorswhich are invariant and robust against sensor faults Soempirical learning for robust sensor relationships can beformulated as sensor denoising problem However contraryto the superimposed channelacquisition noise the termldquodenoisingrdquo specifically corresponds to the corruption causedby gross offset and drift type sensor failures Under suchdenoising based learning objective the empirical sensormodel can be forced to learn a function that captures therobust relationships among correlated sensors and is capableof restoring true sensor value from a corrupted version of it
Let 119878True and Obs be the normal and corrupted sensorstates related by some corruption process 120593(sdot) as follows
Obs = 120593 (119878True) (1)where 120593 119877
119899
rarr 119877119899 is a stochastic corruption caused by
an arbitrary type sensor failure The learning objective fordenoising task can be formulated as
119891 = arg min119891
119864119878True
10038171003817100381710038171003817119891 (Obs) minus 119878True
10038171003817100381710038171003817
2
2
(2)
Under minimization of above formulation the objective ofempirical learning is to search for 119891 that best approximates120593minus1 Furthermore we will formulate and learn such sensor
value estimation and restoration function under neural net-work based autoassociative model driven by deep learningframe work
21 Basic Deep Learning Framework Neural networkresearch suggests that the composition of several levels ofnonlinearity is key to the efficient modeling of complexfunctions However optimization of deep architecture withtraditional gradient based supervised learning methods hasresulted in suboptimal solutions with poor generalizationJoint learning of multiple hidden layers via backpropagationof error gradient inherently suffers from gradient vanishingproblem at the earlier layers and hence constrains the hiddenlayers from fully exploiting the underlying structure inoriginal data distribution In 2006 Hinton in his pioneeringwork proposed a systematic greedy layer by layer trainingof a deep network The idea is to divide the training ofsuccessive layers of a deep network in the form of smallsubnetworks and use unsupervised learning to minimizeinput reconstruction error This technique successfullyeliminates the shortcomings of the gradient based learningby averting the local minima Deep learning frameworkemploys a systematic three-step training approach as follows
(1) Pretraining one layer at a time in a greedy way(2) Using unsupervised learning at each layer in a way
that preserves information from the input and disen-tangles factors of variation
(3) Fine-tuning the whole network with respect to theultimate criterion of interest
x
x
g120579998400 (f120579(x))
120578SD S | S))
Figure 2 Suppose training data ( ) concentrate near a low-dimensional manifold Corrupted examples ( ) obtained by apply-ing corruption process 120578
119878
119863( | 119878) will lie farther from the manifold
The model learns 1198921205791015840 (119891120579()) with 119875(119883 | ) to ldquoproject them backrdquo
onto the manifold
Input layer
x
f120579h
x
Hidden layer
g120579998400
Outputlayer
x
L(x x)
Figure 3 Basic denoising autoencoder (DAE) scheme An emptycircle depicts a single neuron A filled circle depicts corrupted unitsin input vector
3 Building Block for DAASM
In relation to empirical modeling approach as formulatedin Section 2 denoising autoencoder (DAE) [26] is the mostpromising building block for pretraining and compositionof deep autoassociative sensor model DAE is a variant ofthe traditional autoencoder neural network where learningobjective is to reconstruct the original uncorrupted input119909 from partially corrupted or missing inputs Undertraining criterion of reconstruction errorminimization DAEis forced to conserve information details about the inputat its hidden layer mappings The regularization effect ofdenoising based learning objective pushes the DAE networktowards true manifold underlying the high dimension inputdata as depicted in Figure 2 Hence implicitly captures theunderlying data generating distribution by exploring robuststatistical regularities in input data A typical DAE architec-ture as depicted in Figure 3 comprises an input output anda hidden layer An empty circle depicts a neuron unit Theinput layer acts as a proxy layer to the original clean inputMeanwhile the red filed units in input layer are proxies toclean input units which are randomly selected for corruptionunder some artificial noise process 119871(119909 ) is an empiricalloss function to be optimized during training process
Let 119909119894be the original data vector with 119894 = 1 2 119873
elements while 119894represents the partially corrupted version
obtained through corruption process 120578119863 The encoder and
4 Science and Technology of Nuclear Installations
decoder functions corresponding to DAE in Figure 3 aredefined as
ℎ (119894) = 119891120579() = 120590 (119882
119894+ 119887)
(119894) = 1198921205791015840 (ℎ) = 120590 (119882
1015840
ℎ (119894) + 1198871015840
)
(3)
The encoder function 119891120579() transforms input data to ℎ(
119894)
mapping through a sigmoid type activation function 120590(119909) =
(1 + expminus119909)minus1 at hidden layer neurons (119894) is an approxi-
mate reconstruction of 119909 obtained through decoder function1198921205791015840(ℎ) through reverse mapping followed by sigmoid activa-
tion at output layer Meanwhile 120579 = 120579 1205791015840
= 119882 1198871198821015840
1198871015840
are the weight and bias parameters corresponding to theseencoder and decoder functions
In relation to sensor reconstruction model as formulatedin Section 2 the above-described DAE can be reinterpretedas follows
119909 sim 119878 = 119904119894119899
119894=1
sim = 120578119863(( | 119904))
sim = 120578minus1
119863()
(4)
119878 are the input sensor values under fault free steady stateoperation is a partially corrupted input which is generatedthrough an artificial corruption process 120578
119863on selected subset
in input sensor set 119904119894 are the estimated sensor values
by reconstruction function learnt on clean and corruptedinputs 119878 and Network parameters 120579 for DAE can belearned in an unsupervised setting through minimization ofthe reconstruction loss in
119871 (119909 120579) sim 119871 (119878 120579) = arg min120579
119873
sum
119894=1
10038171003817100381710038171003817119878 minus
10038171003817100381710038171003817
2
2
(5)
4 DAASM Architecture and Regularization
In order to capture complex nonlinear relationships amonginput sensors a multilayered architecture is proposed fordenoised autoassociative sensormodel (DAASM) Individuallayers in network hierarchy are pretrained successively frombottom to top For a well regularized sensor model thestructure and optimization objective in greedy layerwisepretraining play a crucial role Two heuristics are applied forrobust learning in DAASM as follows
(1) Each successive layer in multilayered DAASMassembly is pretrained in an unsupervised fashionunder denoising autoencoder (DAE) as elaborated inSection 3
(2) To address robustness data corruption processes fordenoising based pretraining task are incorporatedwith domain specific failure scenarios which arederived from different types of sensor faults Theseheuristics serve several purposes
(i) Forcing the DAE output to match the orig-inal uncorrupted input data acts as a strong
regularizer It helps avoid the trivial identitylearning especially under overcomplete hiddenlayer setting
(ii) Denoising procedure during pretraining leadsto latent representations that are robust to inputperturbations
(iii) Addition of corrupted data set increases train-ing set size and thus is useful in alleviatingoverfitting problem
Full DAASM is learnt in two stages (1) an unsupervisedpretraining phase and (2) a supervised fine-tuning phase Asshown in Figure 4 the pretraining phase follows a hierarchallearning process in which successive DAEs in the stackhierarchy are defined and trained in an unsupervised fashionon the preceding hidden layer activations Full sensor modelis constructed by stacking hidden layers from unsupervisedpretrained DAEs followed by a supervised fine-tuning phaseFor each DAE in the stack hierarchy the optimizationobjective for unsupervised pretraining will remain the sameas in relation (5) However weight decay regularization termis added to the loss function which constrains networkcomplexity by penalizing large weight values In relation (6)119882119882
1015840
are the network weight parameters corresponding toencoder and decoder function while 120582 is the weight decayhyperparameter
119871 (119878 120579) =1
119873
119873
sum
119894=1
10038171003817100381710038171003817119878 minus
10038171003817100381710038171003817
2
2
+120582
2(119882
2
+10038171003817100381710038171003817119882101584010038171003817100381710038171003817
2
) (6)
In a typical DAE architecture a number of input andoutput layer neurons are fixed corresponding to input datadimension 119889 however middle layer neuron counts 119889
1015840 canbe adjusted according to problem complexity Literature indeep learning suggests that under complete middle layer(1198891015840
lt 119889) for DAE architecture results in dense compressedrepresentation at the middle layer Such compressed repre-sentation has tendency to entangle information (change in asingle aspect of the input translates into significant changesin all components of the hidden representation) [27] Thisentangling tendency directly affects the cross-sensitivity ofsensor reconstruction model especially for the case of grosstype sensor failure Considering that here we choose foran overcomplete hidden layer setting (119889
1015840
gt 119889) Underovercomplete setting denoising based optimization objectiveacts as a strong regularizer and inherently prevents DAE fromlearning identity function
Anticlockwise flow in Figure 4 shows architecture andgreedy layer by layer unsupervised pretraining procedure forall hidden layers in DAASM stack For each hidden layer ℎ
119897
a DAE block is shown in which an encoder function 119891119897
120579(sdot)
and a decoder function 119892119897
120579(sdot) are learnt by minimizing the
loss function corresponding to fault free reconstruction of theinputs as in relation (6) For the case of first hidden layer ℎ
1
the corresponding DAE-1 is trained directly on sensor datausing 119871(119878 120579) loss function in (6) However hidden layersℎ2through ℎ
3are learnt on data from preceding hidden layer
activations using recursive relation in (7) So the loss function
Science and Technology of Nuclear Installations 5
DAE-2 for 2nd-layerpretraining
DAE-1 for 1st-layerpretraining
DAE-3 for 3rd-layerpretraining
Supervised fine-tuning
Randomlydropped out
f(1)120579
f(2)120579
f(3)120579
S
S
S S
h1
h1
h2
h3
h1
h2
h3
h3
h2
h2
h1
middot middot middot
g(2)120579998400
g(1)120579998400
g(3)120579998400
Si
middot middot middot
middot middot middotmiddot middot middot
middot middot middot
middot middot middot
h2f(2)120579
f(1)120579
h2
S
middot middot middot
middot middot middot
middot middot middot
middot middot middot
middot middot middot
h1f(1)120579
h1
S
h1(S)
middot middot middot
120578h3D h2 | h2(s)))
120578h2D h1 | h1(s)))
f(2)120579
h1W2)) g(2)120579998400(h2W998400
2)L(h1 h1 120579)
g(3)120579998400(h3W998400
3)L(h2 h2 120579)f(3)
120579h2W3)(
L(S S 120579)
g(1)120579998400(h1W998400
1)L(S S 120579)
f(1)120579
SW1))
120578SD S | S))
Figure 4 DAASM architecture and greedy learning procedure Greedy layerwise pretraining procedure is depicted by counterclockwise flowin the figure
corresponding to DAE-1 and DAE-2 can be represented as119871(ℎ119897 ℎ119897 120579) where ℎ is an approximate reconstruction of ℎ
ℎ119897
= 119891119897
120579(ℎ119897minus1
119882119897) = sigm (119882
119897
ℎ119897minus1
+ 119887119897
)
1 le 119897 le 119871 = 3
(7)
⟨119882⟩ are the network weights corresponding to encoder partin DAE
The noise process 120578119878
119863( | 119878) for DAE-1 corresponds to
a salt-and-pepper (SPN) type corruption process in whicha fraction of the input sensor set 119878 (chosen at random foreach example) is set to minimum ormaximum possible value(typically 0 or 1)The selected noise processmodels gross typefailure scenarios and drives the DAE-1 network to learninginvariance against such type of sensor failures The noisefunctions 120578
ℎ2
119863(ℎ1
| ℎ1()) employ a corruption process in
which ℎ1(119878) and ℎ
1() from pretrained DAE-1 will be used as
the clean and noisy input for DAE-2 pretraining Finally anadditive Gaussian type corruption process (AGN) | 119909 sim
119873(119909 1205902
) is used for DAE-3 noise function 120578ℎ3
119863(ℎ2| ℎ2()) We
will further mathematically formulate and discuss all thesecorruption processes in detail in Section 41
These pretrained layers will initialize the DAASM net-work parameters in basin of attractions which have goodgeneralization and robustness property In order to generate asensormodel that is fairly dependent on all inputs ldquoDropoutrdquo[28] heuristic is applied on ℎ
3hidden units during DAE-
3 pretraining Random dropouts make it hard for latentrepresentations at ℎ
3to get specialized on particular sensors
in the input set Finally pretrained DAEs are unfolded into adeep autoassociator network with 119871 number of encoder and119871 minus 1 decoder cascade as shown in unsupervised fine-tuning
6 Science and Technology of Nuclear Installations
phase in Figure 3 The final network comprises one inputlayer one output and 2119871 minus 1 hidden layers The input sensorvalues flow through encoder cascade 119891 = 119891
119897
120579119900119891119897minus1
120579119900 sdot sdot sdot 119891
1
120579
using recursive expression in (7) and a decoder cascade 119892 =
1198921
1205791015840119900119892119897+1
1205791015840 119900 sdot sdot sdot 119892
119871minus1
1205791015840 using the following equations
ℎ119871
= ℎ119871
ℎ119897
= 119892119897
1205791015840 (ℎ119897+1
1198821015840
119897) = sigm(119882
1015840
119897ℎ119897+1
+ 1198871015840
119897)
1 le 119897 le 119871 minus 1 = 2
= 1198920
(ℎ1
) = 1198821015840
0ℎ1
+ 1198871015840
0
(8)
where ⟨1198821015840
1198871015840
⟩ are network weights and biases of the decoderpart in DAE The entire network is fine-tuned using asemiheuristic based ldquoAugmented Efficient BackpropagationAlgorithmrdquo proposed by Embrechts et al [29] with followingminimization objective
119871 (119878 120579) =1
119873
119873
sum
119894=1
10038171003817100381710038171003817119878 minus
10038171003817100381710038171003817
2
2
+120582
2
2119871
sum
119896=1
10038171003817100381710038171198821198961003817100381710038171003817
2
2 (9)
A 119871minus2weight decay term is added to the above loss functionfor network regularization purpose during fine-tuning phaseTo circumvent the overfitting an early stopping procedurewhich uses validation error as proxy for the generalizationperformance is used during fine-tuning phase
41 Corruption Process 120578(sdot) for Invariance For the case of cal-ibration monitoring an ideal DAASM should learn encoderand decoder functions which are invariant to failed sensorstates So during DAE based pretraining phase engineeredtransformations from prior knowledge about the involved
failure types are imposed on clean input Different datacorruption processes 120578(sdot) are devised for learning of eachsuccessive hidden layer Denoising based learning objectivedrives the hidden layermappings to get invariant against suchengineered transformations on input data It is important tounderstand that denoising based learning approach does notcorrect the faulty signal explicitly rather it seeks to extractstatistical structure among input signals which is stable andinvariant under faults and hence implicitly estimates correctvalue for faulty signal Two failure types are identified anddefined as follows
(i) Gross sensor failure it includes catastrophic sensorfailures Salt-and-pepper type corruption process inwhich a fraction ] of the input sensor set 119878 (chosenat random for each example) is set to minimum ormaximum possible value (typically 0 or 1) is selectedfor modeling gross type failure scenarios
(ii) Miscalibration sensor failure it includes drift multi-plicative and outlier type sensor failures and is mod-eled through isotropic Gaussian noise (GS) | 119909 sim
119873(119909 1205902
) Instead of selecting an arbitrarily simplenoise distribution we estimated the distribution ofsensorrsquos natural noise and exaggerated it to generatenoisy training data
We propose to distribute the denoising based invariancelearning task across multiple hidden layers in the DAASMnetwork Both gross and miscalibration noise types areequally likely to occur in the input space Gaussian typecorruption process is not suitable for input data space 119878
because of its low denoising efficiency against gross typesensor failures Contrarily salt-and-pepper type corruptionprocess covers two extremes of sensors failure range andhence provides an upper bound on perturbation due tominor offset and miscalibration type sensor failures So salt-and-pepper type corruption process is devised for DAE-1pretraining as follows
120578119878
119863( | 119878) = SPN
119894= 119904119894 119894 notin 119895
119894= 119896 119894 isin 119895 119896 = 0 1 with
Pr (0) =1
2
Pr (1) =1
2
119894=V119899
⋃
119894=1
119895119894sdotsdotsdot119873=V119899 119895
119894= [1 119899] = rand ( ) ] = input fraction 119899 = input dimension
(10)
Gross type sensor failures usually have high impact on cross-sensitivity and can trigger false alarms in other sensorsSuch high cross-sensitivity effect may affect isolation ofmiscalibration type secondary failures in other sensors Inorder tominimize the effect a corruption procedure in whichℎ1(119878) and ℎ
1() from pretrained DAE-1 are proposed as the
clean and noisy input for DAE-2 pretrainingThis corruptionmethod is more natural since it causes next hidden layermappings to get invariant against cross-sensitivity effects
and network aberrations from previous layerThe corruptionprocess is supposed to improve invariance in ℎ
2layer map-
pings against cross-sensitivity effects from gross type sensorfailures
120578ℎ2
119863(ℎ1| ℎ1()) = ℎ
119894
1= ℎ119894
1()
where = 120578119878
119863( | 119878) = SPN
(11)
Science and Technology of Nuclear Installations 7
Table 1 List of NPP sensors
Transmitter ID Transmitter name Service Units Low range High rangeFF1 Feed flow 1 Feedwater flow KGS 0 600FF2 Feed flow 2 Feedwater flow KGS 0 600SF1 STM flow 1 Steam flow KGS 0 600SF2 STM flow 2 Steam flow KGS 0 600SP1 STM PSR 1 Steam pressure BARG 0 100SP2 STM PSR 2 Steam pressure BARG 0 100SP3 STM PSR 3 Steam pressure BARG 0 100PP1 PZR PSR 1 Pressurizer pressure BARG 116 170PP2 PZR PSR 2 Pressurizer pressure BARG 116 170PL1 PZR LVL 1 Pressurizer level 0 100PL2 PZR LVL 2 Pressurizer level 0 100SGL1 SG LVL NR 1 range Steam generator level narrow 0 100SGL2 SG LVL NR 2 range Steam generator level narrow 0 100
Here ℎ119894
1(119904) corresponds to hidden layer activations against
clean sensors at the input layer while ℎ119894
1() corresponds to
hidden layer activations against partially faulted sensor setFinally to add robustness against small offset and
miscalibration type sensor failures an isotropic Gaussian
type corruption process is devised for DAE-3 pretrain-ing The corruption procedure corrupts the ℎ
2hidden
layer mappings against clean sensors at the input layeras ℎ2(ℎ1(119878)) by employing an isotropic Gaussian noise as
follows
120578ℎ3
119863(ℎ2| ℎ2(119878)) = AGN
ℎ119894
2= ℎ119894
2(ℎ119894
1(119878)) 119894 notin 119895
ℎ119894
2| ℎ119894
2sim 119873(119904 120590
2
119868) 119894 isin 119895
119894=V119899
⋃
119894=1
119895119894sdotsdotsdot119873=V119899 119895
119894= [1 119899] = rand ( ) ] = input fraction 119899 = total inputs
(12)
Finally clean input is used for the supervised fine-tuningphase in Figure 4
5 Data Set Description
Intentionally for study purposes we limited the modelingscope of DAASM to full power steady operational state It isthe common state in which NPP operates from one refuelingto the next However in practice it is not possible for NPPsystems to be in perfect steady state Reactivity inducedpower perturbations natural process fluctuations sensor andcontroller noises and so forth are some of the evident causesforNPP parameter fluctuations and are responsible for steadystate dynamics Considering that the collected data set shouldbe fairly representative of all possible steady state dynamicsand noise the selected sensors are sampled during differenttime spans of one complete operating cycleThe training dataset consists of 6104 samples collected during the first twomonths of full power reactor operations after refueling cycleMeanwhile 3260 and 2616 samples are reserved for validationand test data sets respectively Five test data sets are used formodelrsquos performance evaluation Each test data set consistsof 4360 samples collected during eight-month period afterrefueling operation In order to account for fault propagation
phenomenon due to large signal groups a sensor subset isselected for this study An engineering sense selection basedon physical proximity and functional correlation is used todefine the sensor subset for this study Thirteen transmittersas listed in Table 1 are selected from various services innuclear steam supply systemof a real PWR typeNPP Figure 5shows the spatial distribution of the selected sensors
Starting from postrefueling full power startup the dataset covers approximately one year of selected sensors valuesSelected sensors are sampled every 10 seconds for consecutive12-hour time window Figure 6 shows data plot from fewselected sensors
6 Model Training
NPP sensorrsquos data are divided into training test and valida-tion set Each sensor data set is scaled in 01 to 09 ranges byusing lower and upper extremities corresponding to individ-ual sensor However the values 0 and 1 are explicitly reservedfor gross and saturation type sensor failures respectivelyTraining data consists of 4320 samples from full power steadystate reactor operation Meanwhile test and validation dataare used for sensor model optimization and performanceevaluation respectively The training setup for DAASM
8 Science and Technology of Nuclear Installations
Reactor vessel
PP1
PL1 PL2
Pressurizer
SGL2
SF1
FF1
Steam generator
Coolant pump
SGL1
FF2
PP2
SP1 SP2 SP3
SF2
Figure 5 Spatial distribution of selected sensor set
Pres
sure
(BA
RG)
1535154
1545
0 1000 1500 2000 2500 3000 3500 4000 4500500Training samples
596061
()
Leve
l
0 1000 1500 2000 2500 3000 3500 4000 4500500Pressurizer pressure
67675
68
Pres
sure
(BA
RG)
0 1000 1500 2000 2500 3000 3500 4000 4500500Steam generator level
500520540
(kg
s)Fl
ow
0 1000 1500 2000 2500 3000 3500 4000 4500500Steam pressure
400500600
Feedwater flow
(kg
s)Fl
ow
0 1000 1500 2000 2500 3000 3500 4000 4500500Steam flow
Figure 6 Plot of NPP sensors listed in Table 1
employs two learning stages an unsupervised learning phaseand supervised training phase DAE based greedy layerwisepretraining of each hidden layer as described in Section 4 isperformed usingminibatches from training data set Stochas-tic gradient descent based learning algorithm is employed
as suggested in practical training recommendations by [30]Finally standard backpropagation algorithm is employed forsupervised fine-tuning in fully stacked DAASM in Figure 4Supervised training is performed using clean sensor inputonly The model hyperparameters are set by random gridsearchmethod [31] A summary of the training hyperparame-ters corresponding to optimumDAASM is shown in Table 2
7 Invariance Test for Robustness
A layer by layer invariance study is conducted to test therobustness of fully trained DAASM against failed sensorstates Data corruption processes applied during pretrainingare essentially meant to learn hidden layer mappings whichare stable and invariant to faulty sensor conditions Thefollowing invariance test for successive hidden layers in finalDAASM stack can provide an insight into the effectivenessof data corruption processes exercised during denoisingbased pretraining phase Invariance for hidden layer map-pings ℎ
119897 is quantified through mean square error (MSE)
between Euclidean (1198712) normalized hidden layer activation⟨ℎ119894⟩119899⟨ℎ119894⟩1198992and ⟨ℎ
119894⟩119899⟨ℎ119894⟩1198992against clean and faulty
sensors respectively Invariance test samples are generatedby corrupting randomly selected sensors in input set withvarying level of offset failures [5ndash50] The MSE againsteach offset level is normalized across hidden layer dimension119863ℎand number of test samples 119879
119873as shown in (13) Finally
theseMSE values are normalized with maximalMSE value asin (14) Normalized MSE curves for each successive hiddenlayer are plotted in Figure 7 Consider
Science and Technology of Nuclear Installations 9
Table 2 Summary of DAASM hyperparameters
Hyperparameter type Tested hyperparameter values Successful hyperparameteragainst optimum model
Pretrained DAE units 3 3
Network architecture
119868119899minus 119864(119899 119901) minus 119861
119899minus 119863(119899 119901) minus 119900
119899
119868119899 input layer neurons
119861119899 bottleneck layer neurons119900119899 output layer neurons
119864(119899 119901) encoder cascade119863(119899 119901) decoder cascade
119899 number of layers119901 neurons per layer
13 minus 119864(2 20) minus 8 minus
119863(2 20) minus 13
Learning rate forunsupervised pretraining 01 5 times 10
minus2
1 times 10minus2
5 times 10minus3
1 times 10minus3
5 times 10minus4
[5 times 10minus2 1 times 10
minus2
]
Learning rate forsupervised training
Scheduled learning rate based on training errormonitoring
015 01 0005 0001 00001
01 0005 0001
Mean pretraining error foreach hidden layer
Corresponding to minima observed during crossvalidation 10
minus4
Weight decay 120582 10minus3
10minus4
10minus5
10minus3
Momentum 119898 [085 099] [095 098]
Input corruption level ]Corrupted input fraction 10 25 30 40
Gaussian corruption ( of sensorrsquos nominal value)005 010 020 035 050
Input fraction [25ndash35]Gaussian noise level
[010ndash025]Dropout fraction in DAE-3 010 020 01
MSE (119867119897Offset) =
(1119879119873)sum119879119873
119899=1(⟨ℎ119894⟩1198991003817100381710038171003817⟨ℎ119894⟩119899
10038171003817100381710038172minus ⟨ℎ119894⟩119899
10038171003817100381710038171003817⟨ℎ119894⟩119899
100381710038171003817100381710038172)2
119863ℎ
1 le 119897 le 119871 = No of encoder layers = 3 Offset = 5 10 20 sdot sdot sdot 50
(13)
MSENormalized (119867119897) =
MSE (119867119897Offset)
MSE (119867119897Max-Offset)
(14)
LayerwiseMSEplots in Figure 7 clearly show that invarianceto faulty sensor conditions increases towards higher layers inthe network hierarchy In these plots lower curves indicatehigher level of invariance To further investigate the effectof increasing invariance on reconstructed sensor values asensor model corresponding to the level ldquo119871rdquo of each hiddenlayer is assembled via encoder and decoder cascade Robust-ness of these partial models is quantified through (1 minus 119878
119894
Auto)Autosensitivity values 119878
119894
Auto (see Section 82) are calculatedagainst varying offset failure levels In Figure 8 layerwiseincrease in robustness confirms that increased invariancehelps in improving overall modelrsquos robustness
8 DAASM versus K-AANNPerformance Analysis
Here we will assess and compare the performance of DAASMwith popular five-layer AANNmodel originally proposed byKramer [8] The K-AANN model is trained with same data
set as used for DAASM and is regularized with Levenberg-Marquardt algorithm Furthermore to improve robustnesstraining with jitter heuristic is employed by introducing anoise of 10 magnitude on clean sensor input The five-layertopology 13-17-9-17-13 is found to be optimum for K-AANNmodel Both DAASM and K-AANN model are comparedthrough accuracy robustness spillover and fault detectabilitybased performance metrics in the following subsections Allperformance metrics are calculated against test data setconsisting of 4320 samples frompostrefueling full powerNPPoperations Performance metric values are reported
81 Accuracy Mean square error (MSE) of observed andmodel estimated sensor values against fault free test data setis used to quantify accuracy metric as follows
Accuracy =1
119873
119873
sum
119894=1
(119894minus 119878119894)2
(15)
10 Science and Technology of Nuclear Installations
Layerwise invariance
3rd hidden layer2nd hidden layer1st hidden layer
0010203040506070809
1
Nor
mal
ised
MSE
minus20 minus10minus40 minus30 0 10 20 30 40 50minus50
Offset failure level ()
Figure 7 Layerwise invariance in DAASM Lower curves depicthigher invariance
Robustness
With 3rd hidden layerWith 2nd hidden layerWith 1st hidden layer
0203040506070809
1
minus20 minus10minus40 minus30 0 10 20 30 40 50minus50
Offset failure level ()
Robu
stnes
s (1minusS a
)
Figure 8 Robustness measure (1 minus autosensitivity) at multiplehidden layers in DAASM Higher curves depict high robustness
The MSE values of all sensors are normalized to theirrespective span and are presented as percent span in Figure 9Being an error measure the lower MSE values by DAASMsignify its prediction accuracy
82 Robustness Robustness is quantified through autosensi-tivity as defined by [32 33] It is themeasure of modelrsquos abilityto predict correct sensor values under missing or corruptedsensor states The measure is averaged over an operatingregion defined by 119896 samples from test data set as follows
119878119894
Auto =1
119873
119873
sum
119896=1
1003816100381610038161003816100381610038161003816100381610038161003816
drift119896119894
minus 119896119894
119904drift119896119894
minus 119904119896119894
1003816100381610038161003816100381610038161003816100381610038161003816
(16)
where 119894 and 119896 are indexes corresponding to sensors and theirrespective test samples 119904
119896119894is the original sensor value without
fault 119896119894is the model estimated sensor value against 119904
119896119894 119904drift119896119894
is the driftedfaulted sensor value drift119896119894
is themodel estimatedsensor value against drifted value 119904
drift119896119894
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN 0382 0361 0343 0412 0186 0211 0166 0386 0411 0243 0223 0512 0621
DAASM 0281 0253 0246 0293 0121 0132 0122 0243 0315 0173 0156 0394 0465
Accuracy metric
0
01
02
03
04
05
06
07
MSE
( sp
an)
Figure 9 MSE depicting DAASM and K-AANN accuracy on eachsensor
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Autosensitivity metric
0005
01015
02025
03503
Auto
sens
itivi
ty
0273 0294 0321 0332 0205 0182 0198 0253 0233 0225 0231 0268 0301
0153 0167 0187 0163 0112 0079 0082 0131 0071 014 0117 0133 0187
Figure 10 Autosensitivity values of individual sensors in bothmodels
The autosensitivitymetric lies in [0 1] range For autosen-sitivity value of one the model predictions follow the faultwith zero residuals hence no fault can be detected Smallerautosensitivity values are preferred which essentially meansdecreased sensitivity towards small perturbations Largeautosensitivity values may lead to missed alarms due tounderestimation of the fault size caused by small residualvalues Compared to K-AANN model in case of DAASM asignificant decrease in autosensitivity values for all sensors isobserved The plot in Figure 10 shows that DAASM is morerobust to failed sensor inputs
To further investigate robustness against large offsetfailures both models are evaluated against offset failuresin [5ndash50] range For each sensor samples from testdata are corrupted with specific offset level and correspond-ing autosensitivities are averaged over whole sensor setAutosensitivity values less than 02 are considered as robustThe maximum autosensitivity value of 0187 is observed insteam flow sensor The plot in Figure 11 shows that averageautosensitivity for both models increases with increasinglevel of offset failure However the autosensitivity curve forDAASM autosensitivity is well below the corresponding K-AANN curve
83 Spillover Depending upon the size and type of failurea failed sensor input can cause discrepancy in estimatedoutput for other sensors The phenomenon is referred to in
Science and Technology of Nuclear Installations 11
0060086
014
021
028
041
016021
028
036
043
054
005 01 02 03 04 05Sensor perturbation magnitude ( of actual value)
Robustness performance
0
01
02
03
04
05
06
Avg
auto
sens
itivi
ty
DAASMK-AANN
Figure 11 Comparison of robustness against increasing offsetfailure
literature as ldquospillover effectrdquo and is quantified through ldquocross-sensitivityrdquo metric [32] It quantifies the influence of faultysensor 119894 on predictions of sensor 119895 as follows
119878119895119894
Cross =1
119873
119873
sum
119896=1
10038161003816100381610038161003816100381610038161003816100381610038161003816
drift119896119895
minus 119896119895
119904drift119896119894
minus 119904119896119894
10038161003816100381610038161003816100381610038161003816100381610038161003816
(17)
119878119895
Cross =1
119873 minus 1
119873
sum
119894=1
119878119895119894
Cross 119894 = 119895 (18)
where 119894 and 119895 indexes are used to refer to faulty andnonfaulty sensors respectively Meanwhile 119896 is the index forcorresponding test samples 119878119895119894Cross is the cross-sensitivity ofsensor 119895 with respect to drift in 119894th sensor 119904
119896119894is the value of
119894th sensor without any fault 119896119895is the model estimated value
of 119895th sensor against 119904119896119894 119904drift119896119894
is the driftedfaulted value of 119894thsensor drift
119896119895is themodel estimated value of 119895th sensor against
drifted value 119904drift119896119894
The highly distributed representation of the input in
neural network based sensor models has pronounced effecton the cross-sensitivity performance Cross-sensitivitymetricvalue lies in [0 1] range High value of cross-sensitivitymay set off false alarms in other sensors provided theresidual values overshoot the fault detectability threshold inother sensors So minimum cross-sensitivity value is desiredfor a robust model The plot in Figure 12 shows that thecross-sensitivity for DAASM is reduced by a large factor ascompared to K-AANNmodel
The spillover effect against particular level of offset failurein [5ndash50] range is averaged over all sensors as follows
AvgCross Sensitivity (119870) =
sum119873
119895=1119878119895
cross(119870)
119873
offset failure level 119870 = 5 10 20 sdot sdot sdot 50
(19)
0128 0124 0113 0116 0112 0108 0092 0079 0083 0098 011 0103 0105
00632 00527 00563 00614 0039 00311 00287 00186 00203 00344 00336 003 00286
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Cross-sensitivity metric
0002004006008
01012014
Cros
s-se
nsiti
vity
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
Figure 12 Cross-sensitivity values of individual sensors in bothmodels
00810105
0161
0253
0312
041
0018 0026 0038 00460071
012
005 01 02 03 04 05Sensor perturbation magnitude ( of observed sensor value)
Spillover offset sensor failure
Requires SPRT mean relaxation to avoid
false alarmsahead of this point
0
005
01
015
02
025
03
035
04
045
Avg
cros
s-se
nsiti
vity
DAASMK-AANN
Figure 13 Comparison of spillover effects against increasing offsetfailure
The cross-sensitivity values 119878119895
cross(119870) against 119870 offset fail-ure level are calculated using (18) Figure 13 shows theaverage cross-sensitivity plot for both models Small cross-sensitivities are observed in DAASM which effectivelyavoided false alarms in other channels without relaxing theSPRT faulted mean value up to an offset failure of 35ndash40 inany channel However for the case of offset noise larger than35 SPRTmean needs to be relaxed to avoid false alarms andisolate the faulty sensor However Robustness of K-AANNmodel deteriorates significantly due to spillover effect beyond15 offset failure
Similarly gross failure scenarios corresponding to twoextremities of sensor range can cause severe Spillover effectTo study robustness against gross type failure scenario asubset of input sensors is simultaneously failed with grosshigh or low value and average cross-sensitivity of remainingsensor set is calculated using relation (19) Plot in Figure 14shows that average cross-sensitivity of K-AANN modelincreases drastically beyond 10 gross failure HoweverDAASM resulted in a very nominal spillover even in caseof multiple sensor failure The DAASM effectively managedsimultaneous gross high or low failures in 25 of total sensorset as compared to 10 in case of K-AANN
12 Science and Technology of Nuclear Installations
0028 0032 0041 012021016 024
0457
061
077
01 02 03 04 05Sensors failed with gross highlow error ()
Spillover gross sensor failure
0010203040506070809
Avg
cros
s-se
nsiti
vity
DAASMK-AANN
Figure 14 Comparison of spillover effect against simultaneous grosshighlow failure in multiple sensors
84 Fault Detectability Fault detectability metric measuresthe smallest fault that can be detected by integrated sensorestimation and fault detection module as shown in Figure 1[32] The detectability metric is measured as percentage ofsensor span 119863 = 119872Span where value M correspondsto minimum detectable fault Minimum fault detectabilitylimit for each sensor is quantified through statistical basedsequential probability ratio test (SPRT) by Wald [34] SPRTtest is carried out to detect if the residual being generatedfrom normal distribution 119873(120583
1 1205902
) or 119873(1205830 1205902
) as definedfor faulty and fault free sensor operations respectively [35]Calibration failures are reflected in the mean parameter ofresidualrsquos distribution The SPRT procedure is applied todetect changes in the mean of residualrsquos distribution Theapplication of SPRT requires setting of following parametersvalue [36]
1205830 normal mode residual mean
1205902 normal mode residual variance
1205831 expected offset in residual mean in abnormal
mode120572 false alarm probability120573 missed alarm probability
Under normal mode the residuals from observed and modelestimated sensor values behave as awhiteGaussian noisewithmean 120583
0= 0 The residual variance 120590
2 is estimated for eachsensor under normal operating conditions and remainedfixed The false alarm 120572 and missed alarm 120573 probabilities areset to be 0001 and 001 respectively In order to determineminimum fault detectability limit a numerical procedureis opted which searches for minimum expected offset 120583
1
in the interval 1205831
[120590ndash3120590] provided the constraint onmissed and false alarm rate holds 120590 is the standard deviationcorresponding to residual variance of particular sensor Theplot in Figure 15 shows the detectability metric for eachsensor The plot in Figure 15 shows that DAASM can detect
0312 0315 033 035 004 005 004 008 0086 01 009 006 0058
0184 0165 019 02 0021 0023 002 0048 005 004 004 0032 0028
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Fault detectability metric
0005
01015
02025
03035
04
Det
ecta
bilit
y (
span
)
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
Figure 15 Comparison of fault detectability metrics
faults which are two times smaller in magnitude than thosedetectable by K-AANNmodel
Improvement in fault detectability metric for DAASMcan be attributed to observed improvement in model robust-ness as suggested by the following relation
119903119894
Δ119878drift119894
= (1 minus 119878119894
Auto) (20)
The term 119903119894Δ119878
drift119894
measures the ratio of observed residualto actual sensor drift in terms of autosensitivity For highlyrobust model this ratio reduces to one which means residualreflects the actual drift and results in high fault detectabilityContrarily ratio value close to zero means that the predictionis following the input and results in poor fault detectability
841 SPRT Based Fault Detectability Test Sequential prob-ability ratio [34 36] based fault hypothesis test is appliedto residual sequence 119877
119894 = 119903
1(1199051) 1199031(1199051)sdot sdot sdot 119903119899(119905119899)generated
by relation 119877119894(119905119894) = 119878
Obs(119905119894) minus 119878
Est(119905119894) at time 119905
119894 where
119878Obs
(119905119894) and 119878
Est(119905119894) are the actual and model predicted
sensor values respectively The SPRT procedure analyzeswhether the residual sequence is more likely to be generatedfrom a probability distribution that belongs to normal modehypothesis 119867
0or abnormal mode hypothesis 119867
1by using
likelihood ratio as follows
119871119899= exp[minus(12120590
2)[sum119899
119894=11205831(1205831minus2119903119894)]]
(21)
For fault free sensor values the normalmode hypothesis1198670is
approximated byGaussian distributionwithmean 1205830= 0 and
variance 1205902 Abnormal mode hypothesis 119867
1is approximated
with mean 1205831
gt 1205830using the same variance 120590
2 The SPRTindex for the positive mean test is finally obtained by takinglogarithm of the likelihood ratio in (21) as follows [35]
SPRT = ln (119871119899) = minus
1
21205902[
119899
sum
119894=1
1205831(1205831minus 2119903119894)]
=1205831
1205902
119899
sum
119894=1
(119903119894minus
1205831
2)
(22)
Pressurizer pressure sensor sampled at a frequency of 10 sec-onds is used as a test signal to validate the fault detectability
Science and Technology of Nuclear Installations 13
0 500 1000 1500 2000 2500 3000 3500 4000 45001535
1541545
SPRT drift detection (pressure sensor)(B
ARG
)Pr
essu
re
Drifted sensorDAASM estimate
Sample number
0 500 1000 1500 2000 2500 3000 3500 4000 45001535
1541545
(BA
RG)
Pres
sure
Drifted sensorK-AANN estimate
Sample number
0515
SPRT
hypo
thes
is
minus050 500 1000 1500 2000 2500 3000 3500 4000 4500
DAASM indexK-AANN index
Sample number
005
valu
eRe
sidua
l
0 500 1000 1500 2000 2500 3000 3500 4000 4500
Residual DAASMResidual K-AANN
Sample number
minus05
H1mdash1
H0mdash0
Figure 16 SPRTbased fault detection in pressurizer pressure sensor
performance Two drift faults at the rate of +001hour andminus001hour are introduced in the test signal for DAASMand K-AANN modelrsquos assessment respectively The first andsecond plots in Figure 16 show drifted and estimated pressuresignal from DAASM and K-AANN models respectivelyThird plot shows residual values generated by differencing thedrifted and estimated signals frombothmodelsThe final plotshows SPRT index values against residuals from K-AANNmodel and DAASM The hypotheses 119867
1and 119867
0correspond
to positive and negative fault acceptance respectively FromSPRT index plot successful early detection of the sensordrift at 2200th sample with lag of 611 hours since the driftinception shows that DAASM is more sensitive to smalldrifts On the other hand SPRT index on K-AANN basedsensor estimates registered the same drift at 3800th samplewith a lag of almost 1055 hours The result shows thatDAASM is more robust in terms of early fault detection withlow false and missed alarm rates
Finally both models are tested against five test data setsEach test set consists of 3630 samples corresponding todifferentmonths of full power reactor operation Bothmodelssuccessfully detected an offset failure of 012ndash03 BARG in allsteam pressure channels and a drift type failure up to 285 insteam generator level (Figure 22)The K-AANNmodel failedto register a very small drift up to 01 in steam flow (STMflow 1) channel A small drift up to 01 BARG is detected in
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Steam pressure estimation
Observed valueDAASM estimateK-AANN estimate
672673674675676677678679
68
Stea
m p
ress
ure (
BARG
)
Figure 17 Steam pressure estimates against offset failures up to 03(BARG)
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Feed flow estimation
400420440460480500520540560
Feed
flow
(kg
s)
Observed sensor valueK-AANN estimateDAASM estimate
Figure 18 Feed flow estimates
test set 5 of pressurizer pressure channel However in caseof drift type sensor failures fault detection lag for DAASMwas on average 05 times smaller in comparison with K-AANN model Plots in Figures 17ndash21 show the estimatedsensor values from both models on five test data sets of fewselected channels
9 Conclusion
This paper presented a neural network based denoisedautoassociative sensor model (DAASM) for empirical sen-sor modeling The proposed sensor model is trained togenerate a monitoring system for sensor fault detection innuclear power plantsMultilayer AANNbased sensormodelsmay result in suboptimal solutions due to poor regulariza-tion by traditional backpropagation based joint multilayerlearning procedures So a complementary deep learningapproach based on greedy layerwise unsupervised pretrain-ing is employed for effective regularization in the proposedmultilayer DAASM Autoencoder architecture is used fordenoising based unsupervised pretraining and regularizationof individual layers in the network hierarchy To addressrobustness against perturbations in input sensors data cor-ruption processes exercised during unsupervised pretraining
14 Science and Technology of Nuclear Installations
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5490495500505510515520525530535540
Stea
m fl
ow (k
gs)
Steam flow estimation
Observed sensor valueDAASM estimateK-AANN estimate
Figure 19 Steam flow estimates against drift failure up to 01
58859
592594596598
60602604606608
Pres
suriz
er le
vel (
)
Pressurizer level estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valueK-AANN estimateDAASM estimate
Figure 20 Pressurizer level estimates
phase were based on prior knowledge about different failurescenarios Results from invariance tests showed that theproposed data corruption schemeswere beneficial in learninglatent representations at hidden layers and were invariantto multiple levels of perturbation in input sensors Conse-quently these pretrained hidden layers worked as well regu-larized perturbation filters with increased invariance towardssensor faults It is also observed that sensitivity against sensorfaults decreased significantly towards higher layers in fullDAASMassembly In a practical context of sensormonitoringin nuclear power plants the proposed model proved itsrobustness against gross type simultaneous sensor failuresIt also showed significant improvement in all performancemetrics when compared with popular and widely used five-layered AANNmodel by KramerMoreover time lag in smalldriftrsquos detection is significantly reduced The overall resultssuggest that greedy layerwise pretraining technique in com-bination with domain specific corruption processes providesa viable framework for effective regularization and robustnessin such deep multilayered autoassociative sensor validationmodels
Appendix
See Table 3
153715375
153815385
153915395
15415405
154115415
Pres
sure
(BA
RG)
Pressurizer pressure estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valuesK-AANN estimateDAASM estimate
Figure 21 Pressurizer pressure estimates against drift failure up to01
74
76
78
80
82
84
Stea
m g
ener
ator
leve
l (
)
Steam generator level estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valueK-AANN estimateDAASM estimate
Figure 22 Steam generator level estimates against growing driftfailure up to 287
Abbreviations
AANN Autoassociative neural networkK-AANN Kramer proposed Autoassociative neural
networkDAASM Denoised autoassociative sensor modelNPP Nuclear power plantPWR Pressurized water reactor119878Auto Autosensitivity119878Cross Cross-sensitivityDAE Denoising autoencoder119878 Observed sensor value Model predicted sensor value Corrupted sensor valueSPN Salt-and-pepper noiseAGN Additive Gaussian noise
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Science and Technology of Nuclear Installations 15
Table3Re
sults
ofDAASM
andK-AANNperfo
rmance
metric
sPerfo
rmance
metric
Mod
eltype
Feed
flow1
Feed
flow2
STM
flow1
STM
flow2
STM
PSR1
STM
PSR2
STM
PSR3
PZRPS
R1
PZRPS
R2
PZRLV
L1
PZRLV
L2
SGLV
LNR1
SGLV
LNR2
AANN
Sp-D
AANN
Accuracy
(span)
0382
0361
0343
0412
0186
0211
0166
0386
0411
0243
0223
0512
0621
0281
0253
0246
0293
0121
0132
0122
0243
0315
0173
0156
0394
0465
Autosensitivity
0273
0294
0321
0332
0205
0182
0198
0253
0233
0225
0231
0268
0301
0153
0167
0187
0163
0112
0079
0082
0131
0071
014
0117
0133
0187
Cross-sensitivity
0128
0124
0113
0116
0112
0108
0092
0079
0083
0098
011
0103
0105
00632
00527
00563
00614
0039
00311
00287
00186
00203
00344
00336
003
00286
SPRT
detectability
0312
0315
033
035
004
005
004
008
0086
01
009
006
0058
0184
0165
019
02
0021
0023
002
004
8005
004
004
0032
0028
16 Science and Technology of Nuclear Installations
Acknowledgments
The authors acknowledge the support of Mr Jose GalindoRodrıguez affiliated to TECNATOM Inc Spain and MrChad Painter (Director of Nuclear Power Plant SimulatorDevelopment and Training Program at International AtomicEnergy Agency) for providing necessary tools and data toconduct this research
References
[1] N Mehranbod M Soroush M Piovoso and B A OgunnaikeldquoProbabilistic model for sensor fault detection and identifica-tionrdquo AIChE Journal vol 49 no 7 pp 1787ndash1802 2003
[2] JW Hines and E Davis ldquoLessons learned from the US nuclearpower plant on-line monitoring programsrdquo Progress in NuclearEnergy vol 46 no 3-4 pp 176ndash189 2005
[3] EPRI ldquoOn-line monitoring cost benefit guiderdquo Final Report1006777 EPRI Palo Alto Calif USA 2003
[4] S J Qin and W Li ldquoDetection identification and recon-struction of faulty sensors with maximized sensitivityrdquo AIChEJournal vol 45 no 9 pp 1963ndash1976 1999
[5] J Ma and J Jiang ldquoApplications of fault detection and diagnosismethods in nuclear power plants a reviewrdquo Progress in NuclearEnergy vol 53 no 3 pp 255ndash266 2011
[6] G YHeo ldquoConditionmonitoring using empiricalmodels tech-nical review and prospects for nuclear applicationsrdquo NuclearEngineering and Technology vol 40 no 1 pp 49ndash68 2008
[7] J B Coble R M Meyer P Ramuhalli et al ldquoA review ofsensor calibration monitoring for calibration interval extensionin nuclear power plantsrdquo Tech Rep PNNL-21687 PacificNorthwest National Laboratory Richland Wash USA 2012
[8] M A Kramer ldquoAutoassociative neural networksrdquo Computersand Chemical Engineering vol 16 no 4 pp 313ndash328 1992
[9] X Xu J W Hines and R E Uhrig ldquoOn-line sensor calibra-tion monitoring and fault detection for chemical processesrdquoin Proceedings of the Maintenance and Reliability Conference(MARCON rsquo98) pp 12ndash14 Knoxville Tenn USA May 1998
[10] M Hamidreza S Mehdi J-R Hooshang and N AliakbarldquoReconstruction based approach to sensor fault diagnosis usingauto-associative neural networksrdquo Journal of Central SouthUniversity vol 21 no 6 pp 2273ndash2281 2014
[11] UThissenW JMelssen andLMC Buydens ldquoNonlinear pro-cess monitoring using bottle-neck neural networksrdquo AnalyticaChimica Acta vol 446 no 1-2 pp 371ndash383 2001
[12] D J Wrest J W Hines and R E Uhrig ldquoInstrument surveil-lance and calibration verification through plant wide monitor-ing using autoassociative neural networksrdquo in Proceedings ofthe American Nuclear Society International Topical Meeting onNuclear Plant Instrumentation Control and Human MachineInterface Technologies University Park Pa USA May 1996
[13] J W Hines R E Uhrig and D J Wrest ldquoUse of autoassociativeneural networks for signal validationrdquo Journal of Intelligent andRobotic Systems Theory and Applications vol 21 no 2 pp 143ndash154 1998
[14] P F Fantoni M I Hoffmann R Shankar and E L Davis ldquoOn-line monitoring of instrument channel performance in nuclearpower plant using PEANOrdquo Progress in Nuclear Energy vol 43no 1ndash4 pp 83ndash89 2003
[15] MMarseguerra andA Zoia ldquoTheAutoAssociative Neural Net-work in signal analysis II Application to on-line monitoring of
a simulated BWR componentrdquo Annals of Nuclear Energy vol32 no 11 pp 1207ndash1223 2005
[16] M S Ikbal H Misra and B Yegnanarayana ldquoAnalysis ofautoassociative mapping neural networksrdquo in Proceedings of theInternational Joint Conference on Neural Networks (IJCNN rsquo99)vol 5 pp 3025ndash3029 IEEE Washington DC USA July 1999Proceedings (Cat no 99CH36339)
[17] Y Bengio ldquoLearning deep architectures for AIrdquo Foundationsand Trends in Machine Learning vol 2 no 1 pp 1ndash27 2009
[18] J W Hines A V Gribok I Attieh and R E Uhrig ldquoRegular-izationmethods for inferential sensing in nuclear power plantsrdquoin Fuzzy Systems and Soft Computing in Nuclear Engineering DRuan Ed vol 38 of Studies in Fuzziness and Soft Computingpp 285ndash314 Physica 2000
[19] A V Gribok J W Hines A Urmanov and R E UhrigldquoHeuristic systematic and informational regularization forprocessmonitoringrdquo International Journal of Intelligent Systemsvol 17 no 8 pp 723ndash749 2002
[20] X Glorot and Y Bengio ldquoUnderstanding the difficulty oftraining deep feedforward neural networksrdquo in Proceedings ofthe 13th International Conference on Artificial Intelligence andStatistics (AISTATS rsquo10) vol 9 pp 249ndash256 Sardinia Italy May2010
[21] S Tan andM L Mayrovouniotis ldquoReducing data dimensional-ity through optimizing neural network inputsrdquo AIChE Journalvol 41 no 6 pp 1471ndash1480 1995
[22] Y Bengio and P Lamblin ldquoGreedy layer-wise training of deepnetworksrdquo in Proceedings of the Advances in Neural InformationProcessing Systems 19 (NIPS rsquo07) pp 153ndash160 September 2007
[23] H Larochelle Y Bengio J Lourador and P Lamblin ldquoExplor-ing strategies for training deep neural networksrdquoThe Journal ofMachine Learning Research vol 10 pp 1ndash40 2009
[24] D Yu and L Deng ldquoDeep learning and its applications to signaland information processing [exploratory DSP]rdquo IEEE SignalProcessing Magazine vol 28 no 1 pp 145ndash154 2011
[25] D ErhanA Courville andPVincent ldquoWhydoes unsupervisedpre-training help deep learningrdquo Journal of Machine LearningResearch vol 11 pp 625ndash660 2010
[26] P Vincent H Larochelle Y Bengio and P-A ManzagolldquoExtracting and composing robust features with denoisingautoencodersrdquo in Proceedings of the 25th International Confer-ence on Machine Learning (ICML rsquo08) pp 1096ndash1103 July 2008
[27] P Vincent H Larochelle I Lajoie Y Bengio and P-AManzagol ldquoStacked denoising autoencoders learning usefulrepresentations in a deep network with a local denoisingcriterionrdquo Journal of Machine Learning Research vol 11 pp3371ndash3408 2010
[28] N Srivastava G Hinton A Krizhevsky I Sutskever and RSalakhutdinov ldquoDropout a simple way to prevent neural net-works from overfittingrdquo Journal of Machine Learning Researchvol 15 pp 1929ndash1958 2014
[29] M J Embrechts B J Hargis and J D Linton ldquoAn augmentedefficient backpropagation training strategy for deep autoasso-ciative neural networksrdquo in Proceedings of the 18th EuropeanSymposiumonArtificial NeuralNetworksmdashComputational Intel-ligence and Machine Learning (ESANN rsquo10) vol no pp 28ndash30April 2010
[30] Y Bengio ldquoPractical recommendations for gradient-basedtraining of deep architecturesrdquo inNeural Networks Tricks of theTrade vol 7700 of Lecture Notes in Computer Science pp 437ndash478 Springer Berlin Germany 2nd edition 2012
Science and Technology of Nuclear Installations 17
[31] J Bergstra and Y Bengio ldquoRandom search for hyper-parameteroptimizationrdquo Journal of Machine Learning Research vol 13 pp281ndash305 2012
[32] J W Hines and D R Garvey ldquoDevelopment and applicationof fault detectability performance metrics for instrument cali-bration verification and anomaly detectionrdquo Journal of PatternRecognition Research vol 1 no 1 pp 2ndash15 2006
[33] A Usynin and J W Hines ldquoOn-line monitoring robustnessmeasures and comparisonsrdquo in Proceedings of the Interna-tional Atomic Energy Agency Technical Meeting on ldquoIncreasingInstrument Calibration Interval through On-Line CalibrationTechnologyrdquo OECD Halden Reactor Project Halden NorwaySeptember 2004
[34] A Wald ldquoSequential tests of statistical hypothesesrdquo Annals ofMathematical Statistics vol 16 no 2 pp 117ndash186 1945
[35] F Di Maio P Baraldi E Zio and R Seraoui ldquoFault detection innuclear power plants components by a combination of statisticalmethodsrdquo IEEE Transactions on Reliability vol 62 no 4 pp833ndash845 2013
[36] S Cheng and M Pecht ldquoUsing cross-validation for modelparameter selection of sequential probability ratio testrdquo ExpertSystems with Applications vol 39 no 9 pp 8467ndash8473 2012
TribologyAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
FuelsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal ofPetroleum Engineering
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Power ElectronicsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
CombustionJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Renewable Energy
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
StructuresJournal of
International Journal of
RotatingMachinery
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporation httpwwwhindawicom
Journal ofEngineeringVolume 2014
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
International Journal ofPhotoenergy
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear InstallationsScience and Technology of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Solar EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Wind EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear EnergyInternational Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
High Energy PhysicsAdvances in
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
2 Science and Technology of Nuclear Installations
Sensornetwork
Signal reconstruction
model
ResidualFault
detection
Fault
Nofault
Residual analysis
SObserved
SEstimate
Figure 1 Integrated sensor estimation and fault detection architecture
statistics and artificial intelligence are an active area ofresearch [6 7]
In order to model complex nonlinearity in physicalprocess sensors autoassociative neural network based sensormodels had widely been used and reported for calibrationmonitoring in chemical processes [8ndash11] and nuclear powerplants [12ndash15] Data driven training procedures for suchneural network based sensor models discover the underlyingstatistical regularities among input sensors from history dataand try to model them by adjusting network parametersFive-layer AANN is one of the earliest autoassociative archi-tectures proposed for sensor and process modeling [8]
In contrast to shallow single layered architectures thesemultilayered neural architectures have flexibility for model-ing complex nonlinear functions [16 17] However harness-ing the complexity offered by these deep NN models with-out overfitting requires effective regularization techniquesSeveral heuristics based standard regularization methodsare suggested and exercised in literature [18 19] such astraining with jitter (noise) Levenberg-Marquardt trainingweight decay neuron pruning cross validation and Bayesianregularization Despite all these regularization heuristics thejoint learning of multiple hidden layers via backpropagationof error gradient inherently suffers from gradient vanishingproblem at the earlier layers [20] This gradient instabilityproblem restricts the very first hidden layer (closer to input)from fully exploiting the underlying structure in original datadistribution Result is the poor generalization and predictioninconsistency Problem gets evenmore complex and hard dueto inherent noise and colinearity in sensor data
Considering the complexity and training difficulty dueto gradient instability in five-layer AANN topology Tanand Mayrovouniotis proposed a shallow network topologyof three layers known as input trained neural network(ITN-network) [21] However the modeling flexibility getscompromised by shallow architecture of ITN-network
The regularization and robustness issues associated withthese traditional learning procedures motivate the need forcomplementary approaches Contrary to shallow architectureapproach by Tan andMayrovouniotis [21] here we are inter-ested in preserving the modeling flexibility offered by manylayered architectures without being compromised on gener-alization and robustness of the sensormodel Recent researchon greedy layerwise learning approaches [22 23] has beenfound successful for efficient learning in deep multilayered
neural architectures for image speech and natural languageprocessing [24] So for a multilayered DAASM we pro-posed to address poor regularization through deep learningframework Contrary to joint multilayer learning methodsfor traditional AANN models the deep learning frameworkemploys greedy layerwise pretraining approach Followingthe deep learning framework each layer in the proposedDAASM is regularized individually through unsupervisedpretraining under denoising based learning objective Thisdenoising based learning is commenced under autoencoderarchitectures as elaborated in Section 3 It essentially servesseveral purposes
(1) Helps deep models in capturing robust statisticalregularities among input sensors
(2) Initializes network parameters in basin of attractionwith good generalization properties [17 25]
(3) Implicitly addresses modelrsquos robustness by learninghidden layer mappings which are stable and invariantto perturbation caused by failed sensor states
Moreover robustness to failed sensor states is not anautomatic property of AANN based sensor models but isprimarily essential for fault detection Consequently tradi-tional AANN based sensor model requires explicit treatmentfor robustness against failed sensor states However forthe case of DAASM an explicit data corruption process isexercised during denoising based unsupervised pretrainingphase The proposed corruption process is derived fromdrift additive and gross type failure scenarios as elaboratedin Section 41 Robustness to faulty sensor conditions is animplicit process of denoising based unsupervised pretrain-ing phase Robustness of the proposed DAASM againstdifferent sensor failure scenarios is rigorously studied anddemonstrated through invariance measurement at multiplehidden layers in theDAASMnetwork (see Section 7)The fullDAASM architecture and layerwise pretraining is detailedin Section 4 We will compare the proposed DAASM basedsensor model with an extensively reported five-layer AANNbased sensor model by Kramer Both sensor models aretrained on sensor data sampled from full power steadyoperation of a pressurizedwater reactor Finally performanceassessment with respect to accuracy autosensitivity cross-sensitivity and fault detectability metrics is conducted underSection 8
Science and Technology of Nuclear Installations 3
2 Problem Formulation
In context of sensor fault detection application the purposeof a typical sensor reconstruction model is to estimatecorrect sensor value from its corrupted observation Theobjective is to model relationships among input sensorswhich are invariant and robust against sensor faults Soempirical learning for robust sensor relationships can beformulated as sensor denoising problem However contraryto the superimposed channelacquisition noise the termldquodenoisingrdquo specifically corresponds to the corruption causedby gross offset and drift type sensor failures Under suchdenoising based learning objective the empirical sensormodel can be forced to learn a function that captures therobust relationships among correlated sensors and is capableof restoring true sensor value from a corrupted version of it
Let 119878True and Obs be the normal and corrupted sensorstates related by some corruption process 120593(sdot) as follows
Obs = 120593 (119878True) (1)where 120593 119877
119899
rarr 119877119899 is a stochastic corruption caused by
an arbitrary type sensor failure The learning objective fordenoising task can be formulated as
119891 = arg min119891
119864119878True
10038171003817100381710038171003817119891 (Obs) minus 119878True
10038171003817100381710038171003817
2
2
(2)
Under minimization of above formulation the objective ofempirical learning is to search for 119891 that best approximates120593minus1 Furthermore we will formulate and learn such sensor
value estimation and restoration function under neural net-work based autoassociative model driven by deep learningframe work
21 Basic Deep Learning Framework Neural networkresearch suggests that the composition of several levels ofnonlinearity is key to the efficient modeling of complexfunctions However optimization of deep architecture withtraditional gradient based supervised learning methods hasresulted in suboptimal solutions with poor generalizationJoint learning of multiple hidden layers via backpropagationof error gradient inherently suffers from gradient vanishingproblem at the earlier layers and hence constrains the hiddenlayers from fully exploiting the underlying structure inoriginal data distribution In 2006 Hinton in his pioneeringwork proposed a systematic greedy layer by layer trainingof a deep network The idea is to divide the training ofsuccessive layers of a deep network in the form of smallsubnetworks and use unsupervised learning to minimizeinput reconstruction error This technique successfullyeliminates the shortcomings of the gradient based learningby averting the local minima Deep learning frameworkemploys a systematic three-step training approach as follows
(1) Pretraining one layer at a time in a greedy way(2) Using unsupervised learning at each layer in a way
that preserves information from the input and disen-tangles factors of variation
(3) Fine-tuning the whole network with respect to theultimate criterion of interest
x
x
g120579998400 (f120579(x))
120578SD S | S))
Figure 2 Suppose training data ( ) concentrate near a low-dimensional manifold Corrupted examples ( ) obtained by apply-ing corruption process 120578
119878
119863( | 119878) will lie farther from the manifold
The model learns 1198921205791015840 (119891120579()) with 119875(119883 | ) to ldquoproject them backrdquo
onto the manifold
Input layer
x
f120579h
x
Hidden layer
g120579998400
Outputlayer
x
L(x x)
Figure 3 Basic denoising autoencoder (DAE) scheme An emptycircle depicts a single neuron A filled circle depicts corrupted unitsin input vector
3 Building Block for DAASM
In relation to empirical modeling approach as formulatedin Section 2 denoising autoencoder (DAE) [26] is the mostpromising building block for pretraining and compositionof deep autoassociative sensor model DAE is a variant ofthe traditional autoencoder neural network where learningobjective is to reconstruct the original uncorrupted input119909 from partially corrupted or missing inputs Undertraining criterion of reconstruction errorminimization DAEis forced to conserve information details about the inputat its hidden layer mappings The regularization effect ofdenoising based learning objective pushes the DAE networktowards true manifold underlying the high dimension inputdata as depicted in Figure 2 Hence implicitly captures theunderlying data generating distribution by exploring robuststatistical regularities in input data A typical DAE architec-ture as depicted in Figure 3 comprises an input output anda hidden layer An empty circle depicts a neuron unit Theinput layer acts as a proxy layer to the original clean inputMeanwhile the red filed units in input layer are proxies toclean input units which are randomly selected for corruptionunder some artificial noise process 119871(119909 ) is an empiricalloss function to be optimized during training process
Let 119909119894be the original data vector with 119894 = 1 2 119873
elements while 119894represents the partially corrupted version
obtained through corruption process 120578119863 The encoder and
4 Science and Technology of Nuclear Installations
decoder functions corresponding to DAE in Figure 3 aredefined as
ℎ (119894) = 119891120579() = 120590 (119882
119894+ 119887)
(119894) = 1198921205791015840 (ℎ) = 120590 (119882
1015840
ℎ (119894) + 1198871015840
)
(3)
The encoder function 119891120579() transforms input data to ℎ(
119894)
mapping through a sigmoid type activation function 120590(119909) =
(1 + expminus119909)minus1 at hidden layer neurons (119894) is an approxi-
mate reconstruction of 119909 obtained through decoder function1198921205791015840(ℎ) through reverse mapping followed by sigmoid activa-
tion at output layer Meanwhile 120579 = 120579 1205791015840
= 119882 1198871198821015840
1198871015840
are the weight and bias parameters corresponding to theseencoder and decoder functions
In relation to sensor reconstruction model as formulatedin Section 2 the above-described DAE can be reinterpretedas follows
119909 sim 119878 = 119904119894119899
119894=1
sim = 120578119863(( | 119904))
sim = 120578minus1
119863()
(4)
119878 are the input sensor values under fault free steady stateoperation is a partially corrupted input which is generatedthrough an artificial corruption process 120578
119863on selected subset
in input sensor set 119904119894 are the estimated sensor values
by reconstruction function learnt on clean and corruptedinputs 119878 and Network parameters 120579 for DAE can belearned in an unsupervised setting through minimization ofthe reconstruction loss in
119871 (119909 120579) sim 119871 (119878 120579) = arg min120579
119873
sum
119894=1
10038171003817100381710038171003817119878 minus
10038171003817100381710038171003817
2
2
(5)
4 DAASM Architecture and Regularization
In order to capture complex nonlinear relationships amonginput sensors a multilayered architecture is proposed fordenoised autoassociative sensormodel (DAASM) Individuallayers in network hierarchy are pretrained successively frombottom to top For a well regularized sensor model thestructure and optimization objective in greedy layerwisepretraining play a crucial role Two heuristics are applied forrobust learning in DAASM as follows
(1) Each successive layer in multilayered DAASMassembly is pretrained in an unsupervised fashionunder denoising autoencoder (DAE) as elaborated inSection 3
(2) To address robustness data corruption processes fordenoising based pretraining task are incorporatedwith domain specific failure scenarios which arederived from different types of sensor faults Theseheuristics serve several purposes
(i) Forcing the DAE output to match the orig-inal uncorrupted input data acts as a strong
regularizer It helps avoid the trivial identitylearning especially under overcomplete hiddenlayer setting
(ii) Denoising procedure during pretraining leadsto latent representations that are robust to inputperturbations
(iii) Addition of corrupted data set increases train-ing set size and thus is useful in alleviatingoverfitting problem
Full DAASM is learnt in two stages (1) an unsupervisedpretraining phase and (2) a supervised fine-tuning phase Asshown in Figure 4 the pretraining phase follows a hierarchallearning process in which successive DAEs in the stackhierarchy are defined and trained in an unsupervised fashionon the preceding hidden layer activations Full sensor modelis constructed by stacking hidden layers from unsupervisedpretrained DAEs followed by a supervised fine-tuning phaseFor each DAE in the stack hierarchy the optimizationobjective for unsupervised pretraining will remain the sameas in relation (5) However weight decay regularization termis added to the loss function which constrains networkcomplexity by penalizing large weight values In relation (6)119882119882
1015840
are the network weight parameters corresponding toencoder and decoder function while 120582 is the weight decayhyperparameter
119871 (119878 120579) =1
119873
119873
sum
119894=1
10038171003817100381710038171003817119878 minus
10038171003817100381710038171003817
2
2
+120582
2(119882
2
+10038171003817100381710038171003817119882101584010038171003817100381710038171003817
2
) (6)
In a typical DAE architecture a number of input andoutput layer neurons are fixed corresponding to input datadimension 119889 however middle layer neuron counts 119889
1015840 canbe adjusted according to problem complexity Literature indeep learning suggests that under complete middle layer(1198891015840
lt 119889) for DAE architecture results in dense compressedrepresentation at the middle layer Such compressed repre-sentation has tendency to entangle information (change in asingle aspect of the input translates into significant changesin all components of the hidden representation) [27] Thisentangling tendency directly affects the cross-sensitivity ofsensor reconstruction model especially for the case of grosstype sensor failure Considering that here we choose foran overcomplete hidden layer setting (119889
1015840
gt 119889) Underovercomplete setting denoising based optimization objectiveacts as a strong regularizer and inherently prevents DAE fromlearning identity function
Anticlockwise flow in Figure 4 shows architecture andgreedy layer by layer unsupervised pretraining procedure forall hidden layers in DAASM stack For each hidden layer ℎ
119897
a DAE block is shown in which an encoder function 119891119897
120579(sdot)
and a decoder function 119892119897
120579(sdot) are learnt by minimizing the
loss function corresponding to fault free reconstruction of theinputs as in relation (6) For the case of first hidden layer ℎ
1
the corresponding DAE-1 is trained directly on sensor datausing 119871(119878 120579) loss function in (6) However hidden layersℎ2through ℎ
3are learnt on data from preceding hidden layer
activations using recursive relation in (7) So the loss function
Science and Technology of Nuclear Installations 5
DAE-2 for 2nd-layerpretraining
DAE-1 for 1st-layerpretraining
DAE-3 for 3rd-layerpretraining
Supervised fine-tuning
Randomlydropped out
f(1)120579
f(2)120579
f(3)120579
S
S
S S
h1
h1
h2
h3
h1
h2
h3
h3
h2
h2
h1
middot middot middot
g(2)120579998400
g(1)120579998400
g(3)120579998400
Si
middot middot middot
middot middot middotmiddot middot middot
middot middot middot
middot middot middot
h2f(2)120579
f(1)120579
h2
S
middot middot middot
middot middot middot
middot middot middot
middot middot middot
middot middot middot
h1f(1)120579
h1
S
h1(S)
middot middot middot
120578h3D h2 | h2(s)))
120578h2D h1 | h1(s)))
f(2)120579
h1W2)) g(2)120579998400(h2W998400
2)L(h1 h1 120579)
g(3)120579998400(h3W998400
3)L(h2 h2 120579)f(3)
120579h2W3)(
L(S S 120579)
g(1)120579998400(h1W998400
1)L(S S 120579)
f(1)120579
SW1))
120578SD S | S))
Figure 4 DAASM architecture and greedy learning procedure Greedy layerwise pretraining procedure is depicted by counterclockwise flowin the figure
corresponding to DAE-1 and DAE-2 can be represented as119871(ℎ119897 ℎ119897 120579) where ℎ is an approximate reconstruction of ℎ
ℎ119897
= 119891119897
120579(ℎ119897minus1
119882119897) = sigm (119882
119897
ℎ119897minus1
+ 119887119897
)
1 le 119897 le 119871 = 3
(7)
⟨119882⟩ are the network weights corresponding to encoder partin DAE
The noise process 120578119878
119863( | 119878) for DAE-1 corresponds to
a salt-and-pepper (SPN) type corruption process in whicha fraction of the input sensor set 119878 (chosen at random foreach example) is set to minimum ormaximum possible value(typically 0 or 1)The selected noise processmodels gross typefailure scenarios and drives the DAE-1 network to learninginvariance against such type of sensor failures The noisefunctions 120578
ℎ2
119863(ℎ1
| ℎ1()) employ a corruption process in
which ℎ1(119878) and ℎ
1() from pretrained DAE-1 will be used as
the clean and noisy input for DAE-2 pretraining Finally anadditive Gaussian type corruption process (AGN) | 119909 sim
119873(119909 1205902
) is used for DAE-3 noise function 120578ℎ3
119863(ℎ2| ℎ2()) We
will further mathematically formulate and discuss all thesecorruption processes in detail in Section 41
These pretrained layers will initialize the DAASM net-work parameters in basin of attractions which have goodgeneralization and robustness property In order to generate asensormodel that is fairly dependent on all inputs ldquoDropoutrdquo[28] heuristic is applied on ℎ
3hidden units during DAE-
3 pretraining Random dropouts make it hard for latentrepresentations at ℎ
3to get specialized on particular sensors
in the input set Finally pretrained DAEs are unfolded into adeep autoassociator network with 119871 number of encoder and119871 minus 1 decoder cascade as shown in unsupervised fine-tuning
6 Science and Technology of Nuclear Installations
phase in Figure 3 The final network comprises one inputlayer one output and 2119871 minus 1 hidden layers The input sensorvalues flow through encoder cascade 119891 = 119891
119897
120579119900119891119897minus1
120579119900 sdot sdot sdot 119891
1
120579
using recursive expression in (7) and a decoder cascade 119892 =
1198921
1205791015840119900119892119897+1
1205791015840 119900 sdot sdot sdot 119892
119871minus1
1205791015840 using the following equations
ℎ119871
= ℎ119871
ℎ119897
= 119892119897
1205791015840 (ℎ119897+1
1198821015840
119897) = sigm(119882
1015840
119897ℎ119897+1
+ 1198871015840
119897)
1 le 119897 le 119871 minus 1 = 2
= 1198920
(ℎ1
) = 1198821015840
0ℎ1
+ 1198871015840
0
(8)
where ⟨1198821015840
1198871015840
⟩ are network weights and biases of the decoderpart in DAE The entire network is fine-tuned using asemiheuristic based ldquoAugmented Efficient BackpropagationAlgorithmrdquo proposed by Embrechts et al [29] with followingminimization objective
119871 (119878 120579) =1
119873
119873
sum
119894=1
10038171003817100381710038171003817119878 minus
10038171003817100381710038171003817
2
2
+120582
2
2119871
sum
119896=1
10038171003817100381710038171198821198961003817100381710038171003817
2
2 (9)
A 119871minus2weight decay term is added to the above loss functionfor network regularization purpose during fine-tuning phaseTo circumvent the overfitting an early stopping procedurewhich uses validation error as proxy for the generalizationperformance is used during fine-tuning phase
41 Corruption Process 120578(sdot) for Invariance For the case of cal-ibration monitoring an ideal DAASM should learn encoderand decoder functions which are invariant to failed sensorstates So during DAE based pretraining phase engineeredtransformations from prior knowledge about the involved
failure types are imposed on clean input Different datacorruption processes 120578(sdot) are devised for learning of eachsuccessive hidden layer Denoising based learning objectivedrives the hidden layermappings to get invariant against suchengineered transformations on input data It is important tounderstand that denoising based learning approach does notcorrect the faulty signal explicitly rather it seeks to extractstatistical structure among input signals which is stable andinvariant under faults and hence implicitly estimates correctvalue for faulty signal Two failure types are identified anddefined as follows
(i) Gross sensor failure it includes catastrophic sensorfailures Salt-and-pepper type corruption process inwhich a fraction ] of the input sensor set 119878 (chosenat random for each example) is set to minimum ormaximum possible value (typically 0 or 1) is selectedfor modeling gross type failure scenarios
(ii) Miscalibration sensor failure it includes drift multi-plicative and outlier type sensor failures and is mod-eled through isotropic Gaussian noise (GS) | 119909 sim
119873(119909 1205902
) Instead of selecting an arbitrarily simplenoise distribution we estimated the distribution ofsensorrsquos natural noise and exaggerated it to generatenoisy training data
We propose to distribute the denoising based invariancelearning task across multiple hidden layers in the DAASMnetwork Both gross and miscalibration noise types areequally likely to occur in the input space Gaussian typecorruption process is not suitable for input data space 119878
because of its low denoising efficiency against gross typesensor failures Contrarily salt-and-pepper type corruptionprocess covers two extremes of sensors failure range andhence provides an upper bound on perturbation due tominor offset and miscalibration type sensor failures So salt-and-pepper type corruption process is devised for DAE-1pretraining as follows
120578119878
119863( | 119878) = SPN
119894= 119904119894 119894 notin 119895
119894= 119896 119894 isin 119895 119896 = 0 1 with
Pr (0) =1
2
Pr (1) =1
2
119894=V119899
⋃
119894=1
119895119894sdotsdotsdot119873=V119899 119895
119894= [1 119899] = rand ( ) ] = input fraction 119899 = input dimension
(10)
Gross type sensor failures usually have high impact on cross-sensitivity and can trigger false alarms in other sensorsSuch high cross-sensitivity effect may affect isolation ofmiscalibration type secondary failures in other sensors Inorder tominimize the effect a corruption procedure in whichℎ1(119878) and ℎ
1() from pretrained DAE-1 are proposed as the
clean and noisy input for DAE-2 pretrainingThis corruptionmethod is more natural since it causes next hidden layermappings to get invariant against cross-sensitivity effects
and network aberrations from previous layerThe corruptionprocess is supposed to improve invariance in ℎ
2layer map-
pings against cross-sensitivity effects from gross type sensorfailures
120578ℎ2
119863(ℎ1| ℎ1()) = ℎ
119894
1= ℎ119894
1()
where = 120578119878
119863( | 119878) = SPN
(11)
Science and Technology of Nuclear Installations 7
Table 1 List of NPP sensors
Transmitter ID Transmitter name Service Units Low range High rangeFF1 Feed flow 1 Feedwater flow KGS 0 600FF2 Feed flow 2 Feedwater flow KGS 0 600SF1 STM flow 1 Steam flow KGS 0 600SF2 STM flow 2 Steam flow KGS 0 600SP1 STM PSR 1 Steam pressure BARG 0 100SP2 STM PSR 2 Steam pressure BARG 0 100SP3 STM PSR 3 Steam pressure BARG 0 100PP1 PZR PSR 1 Pressurizer pressure BARG 116 170PP2 PZR PSR 2 Pressurizer pressure BARG 116 170PL1 PZR LVL 1 Pressurizer level 0 100PL2 PZR LVL 2 Pressurizer level 0 100SGL1 SG LVL NR 1 range Steam generator level narrow 0 100SGL2 SG LVL NR 2 range Steam generator level narrow 0 100
Here ℎ119894
1(119904) corresponds to hidden layer activations against
clean sensors at the input layer while ℎ119894
1() corresponds to
hidden layer activations against partially faulted sensor setFinally to add robustness against small offset and
miscalibration type sensor failures an isotropic Gaussian
type corruption process is devised for DAE-3 pretrain-ing The corruption procedure corrupts the ℎ
2hidden
layer mappings against clean sensors at the input layeras ℎ2(ℎ1(119878)) by employing an isotropic Gaussian noise as
follows
120578ℎ3
119863(ℎ2| ℎ2(119878)) = AGN
ℎ119894
2= ℎ119894
2(ℎ119894
1(119878)) 119894 notin 119895
ℎ119894
2| ℎ119894
2sim 119873(119904 120590
2
119868) 119894 isin 119895
119894=V119899
⋃
119894=1
119895119894sdotsdotsdot119873=V119899 119895
119894= [1 119899] = rand ( ) ] = input fraction 119899 = total inputs
(12)
Finally clean input is used for the supervised fine-tuningphase in Figure 4
5 Data Set Description
Intentionally for study purposes we limited the modelingscope of DAASM to full power steady operational state It isthe common state in which NPP operates from one refuelingto the next However in practice it is not possible for NPPsystems to be in perfect steady state Reactivity inducedpower perturbations natural process fluctuations sensor andcontroller noises and so forth are some of the evident causesforNPP parameter fluctuations and are responsible for steadystate dynamics Considering that the collected data set shouldbe fairly representative of all possible steady state dynamicsand noise the selected sensors are sampled during differenttime spans of one complete operating cycleThe training dataset consists of 6104 samples collected during the first twomonths of full power reactor operations after refueling cycleMeanwhile 3260 and 2616 samples are reserved for validationand test data sets respectively Five test data sets are used formodelrsquos performance evaluation Each test data set consistsof 4360 samples collected during eight-month period afterrefueling operation In order to account for fault propagation
phenomenon due to large signal groups a sensor subset isselected for this study An engineering sense selection basedon physical proximity and functional correlation is used todefine the sensor subset for this study Thirteen transmittersas listed in Table 1 are selected from various services innuclear steam supply systemof a real PWR typeNPP Figure 5shows the spatial distribution of the selected sensors
Starting from postrefueling full power startup the dataset covers approximately one year of selected sensors valuesSelected sensors are sampled every 10 seconds for consecutive12-hour time window Figure 6 shows data plot from fewselected sensors
6 Model Training
NPP sensorrsquos data are divided into training test and valida-tion set Each sensor data set is scaled in 01 to 09 ranges byusing lower and upper extremities corresponding to individ-ual sensor However the values 0 and 1 are explicitly reservedfor gross and saturation type sensor failures respectivelyTraining data consists of 4320 samples from full power steadystate reactor operation Meanwhile test and validation dataare used for sensor model optimization and performanceevaluation respectively The training setup for DAASM
8 Science and Technology of Nuclear Installations
Reactor vessel
PP1
PL1 PL2
Pressurizer
SGL2
SF1
FF1
Steam generator
Coolant pump
SGL1
FF2
PP2
SP1 SP2 SP3
SF2
Figure 5 Spatial distribution of selected sensor set
Pres
sure
(BA
RG)
1535154
1545
0 1000 1500 2000 2500 3000 3500 4000 4500500Training samples
596061
()
Leve
l
0 1000 1500 2000 2500 3000 3500 4000 4500500Pressurizer pressure
67675
68
Pres
sure
(BA
RG)
0 1000 1500 2000 2500 3000 3500 4000 4500500Steam generator level
500520540
(kg
s)Fl
ow
0 1000 1500 2000 2500 3000 3500 4000 4500500Steam pressure
400500600
Feedwater flow
(kg
s)Fl
ow
0 1000 1500 2000 2500 3000 3500 4000 4500500Steam flow
Figure 6 Plot of NPP sensors listed in Table 1
employs two learning stages an unsupervised learning phaseand supervised training phase DAE based greedy layerwisepretraining of each hidden layer as described in Section 4 isperformed usingminibatches from training data set Stochas-tic gradient descent based learning algorithm is employed
as suggested in practical training recommendations by [30]Finally standard backpropagation algorithm is employed forsupervised fine-tuning in fully stacked DAASM in Figure 4Supervised training is performed using clean sensor inputonly The model hyperparameters are set by random gridsearchmethod [31] A summary of the training hyperparame-ters corresponding to optimumDAASM is shown in Table 2
7 Invariance Test for Robustness
A layer by layer invariance study is conducted to test therobustness of fully trained DAASM against failed sensorstates Data corruption processes applied during pretrainingare essentially meant to learn hidden layer mappings whichare stable and invariant to faulty sensor conditions Thefollowing invariance test for successive hidden layers in finalDAASM stack can provide an insight into the effectivenessof data corruption processes exercised during denoisingbased pretraining phase Invariance for hidden layer map-pings ℎ
119897 is quantified through mean square error (MSE)
between Euclidean (1198712) normalized hidden layer activation⟨ℎ119894⟩119899⟨ℎ119894⟩1198992and ⟨ℎ
119894⟩119899⟨ℎ119894⟩1198992against clean and faulty
sensors respectively Invariance test samples are generatedby corrupting randomly selected sensors in input set withvarying level of offset failures [5ndash50] The MSE againsteach offset level is normalized across hidden layer dimension119863ℎand number of test samples 119879
119873as shown in (13) Finally
theseMSE values are normalized with maximalMSE value asin (14) Normalized MSE curves for each successive hiddenlayer are plotted in Figure 7 Consider
Science and Technology of Nuclear Installations 9
Table 2 Summary of DAASM hyperparameters
Hyperparameter type Tested hyperparameter values Successful hyperparameteragainst optimum model
Pretrained DAE units 3 3
Network architecture
119868119899minus 119864(119899 119901) minus 119861
119899minus 119863(119899 119901) minus 119900
119899
119868119899 input layer neurons
119861119899 bottleneck layer neurons119900119899 output layer neurons
119864(119899 119901) encoder cascade119863(119899 119901) decoder cascade
119899 number of layers119901 neurons per layer
13 minus 119864(2 20) minus 8 minus
119863(2 20) minus 13
Learning rate forunsupervised pretraining 01 5 times 10
minus2
1 times 10minus2
5 times 10minus3
1 times 10minus3
5 times 10minus4
[5 times 10minus2 1 times 10
minus2
]
Learning rate forsupervised training
Scheduled learning rate based on training errormonitoring
015 01 0005 0001 00001
01 0005 0001
Mean pretraining error foreach hidden layer
Corresponding to minima observed during crossvalidation 10
minus4
Weight decay 120582 10minus3
10minus4
10minus5
10minus3
Momentum 119898 [085 099] [095 098]
Input corruption level ]Corrupted input fraction 10 25 30 40
Gaussian corruption ( of sensorrsquos nominal value)005 010 020 035 050
Input fraction [25ndash35]Gaussian noise level
[010ndash025]Dropout fraction in DAE-3 010 020 01
MSE (119867119897Offset) =
(1119879119873)sum119879119873
119899=1(⟨ℎ119894⟩1198991003817100381710038171003817⟨ℎ119894⟩119899
10038171003817100381710038172minus ⟨ℎ119894⟩119899
10038171003817100381710038171003817⟨ℎ119894⟩119899
100381710038171003817100381710038172)2
119863ℎ
1 le 119897 le 119871 = No of encoder layers = 3 Offset = 5 10 20 sdot sdot sdot 50
(13)
MSENormalized (119867119897) =
MSE (119867119897Offset)
MSE (119867119897Max-Offset)
(14)
LayerwiseMSEplots in Figure 7 clearly show that invarianceto faulty sensor conditions increases towards higher layers inthe network hierarchy In these plots lower curves indicatehigher level of invariance To further investigate the effectof increasing invariance on reconstructed sensor values asensor model corresponding to the level ldquo119871rdquo of each hiddenlayer is assembled via encoder and decoder cascade Robust-ness of these partial models is quantified through (1 minus 119878
119894
Auto)Autosensitivity values 119878
119894
Auto (see Section 82) are calculatedagainst varying offset failure levels In Figure 8 layerwiseincrease in robustness confirms that increased invariancehelps in improving overall modelrsquos robustness
8 DAASM versus K-AANNPerformance Analysis
Here we will assess and compare the performance of DAASMwith popular five-layer AANNmodel originally proposed byKramer [8] The K-AANN model is trained with same data
set as used for DAASM and is regularized with Levenberg-Marquardt algorithm Furthermore to improve robustnesstraining with jitter heuristic is employed by introducing anoise of 10 magnitude on clean sensor input The five-layertopology 13-17-9-17-13 is found to be optimum for K-AANNmodel Both DAASM and K-AANN model are comparedthrough accuracy robustness spillover and fault detectabilitybased performance metrics in the following subsections Allperformance metrics are calculated against test data setconsisting of 4320 samples frompostrefueling full powerNPPoperations Performance metric values are reported
81 Accuracy Mean square error (MSE) of observed andmodel estimated sensor values against fault free test data setis used to quantify accuracy metric as follows
Accuracy =1
119873
119873
sum
119894=1
(119894minus 119878119894)2
(15)
10 Science and Technology of Nuclear Installations
Layerwise invariance
3rd hidden layer2nd hidden layer1st hidden layer
0010203040506070809
1
Nor
mal
ised
MSE
minus20 minus10minus40 minus30 0 10 20 30 40 50minus50
Offset failure level ()
Figure 7 Layerwise invariance in DAASM Lower curves depicthigher invariance
Robustness
With 3rd hidden layerWith 2nd hidden layerWith 1st hidden layer
0203040506070809
1
minus20 minus10minus40 minus30 0 10 20 30 40 50minus50
Offset failure level ()
Robu
stnes
s (1minusS a
)
Figure 8 Robustness measure (1 minus autosensitivity) at multiplehidden layers in DAASM Higher curves depict high robustness
The MSE values of all sensors are normalized to theirrespective span and are presented as percent span in Figure 9Being an error measure the lower MSE values by DAASMsignify its prediction accuracy
82 Robustness Robustness is quantified through autosensi-tivity as defined by [32 33] It is themeasure of modelrsquos abilityto predict correct sensor values under missing or corruptedsensor states The measure is averaged over an operatingregion defined by 119896 samples from test data set as follows
119878119894
Auto =1
119873
119873
sum
119896=1
1003816100381610038161003816100381610038161003816100381610038161003816
drift119896119894
minus 119896119894
119904drift119896119894
minus 119904119896119894
1003816100381610038161003816100381610038161003816100381610038161003816
(16)
where 119894 and 119896 are indexes corresponding to sensors and theirrespective test samples 119904
119896119894is the original sensor value without
fault 119896119894is the model estimated sensor value against 119904
119896119894 119904drift119896119894
is the driftedfaulted sensor value drift119896119894
is themodel estimatedsensor value against drifted value 119904
drift119896119894
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN 0382 0361 0343 0412 0186 0211 0166 0386 0411 0243 0223 0512 0621
DAASM 0281 0253 0246 0293 0121 0132 0122 0243 0315 0173 0156 0394 0465
Accuracy metric
0
01
02
03
04
05
06
07
MSE
( sp
an)
Figure 9 MSE depicting DAASM and K-AANN accuracy on eachsensor
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Autosensitivity metric
0005
01015
02025
03503
Auto
sens
itivi
ty
0273 0294 0321 0332 0205 0182 0198 0253 0233 0225 0231 0268 0301
0153 0167 0187 0163 0112 0079 0082 0131 0071 014 0117 0133 0187
Figure 10 Autosensitivity values of individual sensors in bothmodels
The autosensitivitymetric lies in [0 1] range For autosen-sitivity value of one the model predictions follow the faultwith zero residuals hence no fault can be detected Smallerautosensitivity values are preferred which essentially meansdecreased sensitivity towards small perturbations Largeautosensitivity values may lead to missed alarms due tounderestimation of the fault size caused by small residualvalues Compared to K-AANN model in case of DAASM asignificant decrease in autosensitivity values for all sensors isobserved The plot in Figure 10 shows that DAASM is morerobust to failed sensor inputs
To further investigate robustness against large offsetfailures both models are evaluated against offset failuresin [5ndash50] range For each sensor samples from testdata are corrupted with specific offset level and correspond-ing autosensitivities are averaged over whole sensor setAutosensitivity values less than 02 are considered as robustThe maximum autosensitivity value of 0187 is observed insteam flow sensor The plot in Figure 11 shows that averageautosensitivity for both models increases with increasinglevel of offset failure However the autosensitivity curve forDAASM autosensitivity is well below the corresponding K-AANN curve
83 Spillover Depending upon the size and type of failurea failed sensor input can cause discrepancy in estimatedoutput for other sensors The phenomenon is referred to in
Science and Technology of Nuclear Installations 11
0060086
014
021
028
041
016021
028
036
043
054
005 01 02 03 04 05Sensor perturbation magnitude ( of actual value)
Robustness performance
0
01
02
03
04
05
06
Avg
auto
sens
itivi
ty
DAASMK-AANN
Figure 11 Comparison of robustness against increasing offsetfailure
literature as ldquospillover effectrdquo and is quantified through ldquocross-sensitivityrdquo metric [32] It quantifies the influence of faultysensor 119894 on predictions of sensor 119895 as follows
119878119895119894
Cross =1
119873
119873
sum
119896=1
10038161003816100381610038161003816100381610038161003816100381610038161003816
drift119896119895
minus 119896119895
119904drift119896119894
minus 119904119896119894
10038161003816100381610038161003816100381610038161003816100381610038161003816
(17)
119878119895
Cross =1
119873 minus 1
119873
sum
119894=1
119878119895119894
Cross 119894 = 119895 (18)
where 119894 and 119895 indexes are used to refer to faulty andnonfaulty sensors respectively Meanwhile 119896 is the index forcorresponding test samples 119878119895119894Cross is the cross-sensitivity ofsensor 119895 with respect to drift in 119894th sensor 119904
119896119894is the value of
119894th sensor without any fault 119896119895is the model estimated value
of 119895th sensor against 119904119896119894 119904drift119896119894
is the driftedfaulted value of 119894thsensor drift
119896119895is themodel estimated value of 119895th sensor against
drifted value 119904drift119896119894
The highly distributed representation of the input in
neural network based sensor models has pronounced effecton the cross-sensitivity performance Cross-sensitivitymetricvalue lies in [0 1] range High value of cross-sensitivitymay set off false alarms in other sensors provided theresidual values overshoot the fault detectability threshold inother sensors So minimum cross-sensitivity value is desiredfor a robust model The plot in Figure 12 shows that thecross-sensitivity for DAASM is reduced by a large factor ascompared to K-AANNmodel
The spillover effect against particular level of offset failurein [5ndash50] range is averaged over all sensors as follows
AvgCross Sensitivity (119870) =
sum119873
119895=1119878119895
cross(119870)
119873
offset failure level 119870 = 5 10 20 sdot sdot sdot 50
(19)
0128 0124 0113 0116 0112 0108 0092 0079 0083 0098 011 0103 0105
00632 00527 00563 00614 0039 00311 00287 00186 00203 00344 00336 003 00286
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Cross-sensitivity metric
0002004006008
01012014
Cros
s-se
nsiti
vity
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
Figure 12 Cross-sensitivity values of individual sensors in bothmodels
00810105
0161
0253
0312
041
0018 0026 0038 00460071
012
005 01 02 03 04 05Sensor perturbation magnitude ( of observed sensor value)
Spillover offset sensor failure
Requires SPRT mean relaxation to avoid
false alarmsahead of this point
0
005
01
015
02
025
03
035
04
045
Avg
cros
s-se
nsiti
vity
DAASMK-AANN
Figure 13 Comparison of spillover effects against increasing offsetfailure
The cross-sensitivity values 119878119895
cross(119870) against 119870 offset fail-ure level are calculated using (18) Figure 13 shows theaverage cross-sensitivity plot for both models Small cross-sensitivities are observed in DAASM which effectivelyavoided false alarms in other channels without relaxing theSPRT faulted mean value up to an offset failure of 35ndash40 inany channel However for the case of offset noise larger than35 SPRTmean needs to be relaxed to avoid false alarms andisolate the faulty sensor However Robustness of K-AANNmodel deteriorates significantly due to spillover effect beyond15 offset failure
Similarly gross failure scenarios corresponding to twoextremities of sensor range can cause severe Spillover effectTo study robustness against gross type failure scenario asubset of input sensors is simultaneously failed with grosshigh or low value and average cross-sensitivity of remainingsensor set is calculated using relation (19) Plot in Figure 14shows that average cross-sensitivity of K-AANN modelincreases drastically beyond 10 gross failure HoweverDAASM resulted in a very nominal spillover even in caseof multiple sensor failure The DAASM effectively managedsimultaneous gross high or low failures in 25 of total sensorset as compared to 10 in case of K-AANN
12 Science and Technology of Nuclear Installations
0028 0032 0041 012021016 024
0457
061
077
01 02 03 04 05Sensors failed with gross highlow error ()
Spillover gross sensor failure
0010203040506070809
Avg
cros
s-se
nsiti
vity
DAASMK-AANN
Figure 14 Comparison of spillover effect against simultaneous grosshighlow failure in multiple sensors
84 Fault Detectability Fault detectability metric measuresthe smallest fault that can be detected by integrated sensorestimation and fault detection module as shown in Figure 1[32] The detectability metric is measured as percentage ofsensor span 119863 = 119872Span where value M correspondsto minimum detectable fault Minimum fault detectabilitylimit for each sensor is quantified through statistical basedsequential probability ratio test (SPRT) by Wald [34] SPRTtest is carried out to detect if the residual being generatedfrom normal distribution 119873(120583
1 1205902
) or 119873(1205830 1205902
) as definedfor faulty and fault free sensor operations respectively [35]Calibration failures are reflected in the mean parameter ofresidualrsquos distribution The SPRT procedure is applied todetect changes in the mean of residualrsquos distribution Theapplication of SPRT requires setting of following parametersvalue [36]
1205830 normal mode residual mean
1205902 normal mode residual variance
1205831 expected offset in residual mean in abnormal
mode120572 false alarm probability120573 missed alarm probability
Under normal mode the residuals from observed and modelestimated sensor values behave as awhiteGaussian noisewithmean 120583
0= 0 The residual variance 120590
2 is estimated for eachsensor under normal operating conditions and remainedfixed The false alarm 120572 and missed alarm 120573 probabilities areset to be 0001 and 001 respectively In order to determineminimum fault detectability limit a numerical procedureis opted which searches for minimum expected offset 120583
1
in the interval 1205831
[120590ndash3120590] provided the constraint onmissed and false alarm rate holds 120590 is the standard deviationcorresponding to residual variance of particular sensor Theplot in Figure 15 shows the detectability metric for eachsensor The plot in Figure 15 shows that DAASM can detect
0312 0315 033 035 004 005 004 008 0086 01 009 006 0058
0184 0165 019 02 0021 0023 002 0048 005 004 004 0032 0028
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Fault detectability metric
0005
01015
02025
03035
04
Det
ecta
bilit
y (
span
)
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
Figure 15 Comparison of fault detectability metrics
faults which are two times smaller in magnitude than thosedetectable by K-AANNmodel
Improvement in fault detectability metric for DAASMcan be attributed to observed improvement in model robust-ness as suggested by the following relation
119903119894
Δ119878drift119894
= (1 minus 119878119894
Auto) (20)
The term 119903119894Δ119878
drift119894
measures the ratio of observed residualto actual sensor drift in terms of autosensitivity For highlyrobust model this ratio reduces to one which means residualreflects the actual drift and results in high fault detectabilityContrarily ratio value close to zero means that the predictionis following the input and results in poor fault detectability
841 SPRT Based Fault Detectability Test Sequential prob-ability ratio [34 36] based fault hypothesis test is appliedto residual sequence 119877
119894 = 119903
1(1199051) 1199031(1199051)sdot sdot sdot 119903119899(119905119899)generated
by relation 119877119894(119905119894) = 119878
Obs(119905119894) minus 119878
Est(119905119894) at time 119905
119894 where
119878Obs
(119905119894) and 119878
Est(119905119894) are the actual and model predicted
sensor values respectively The SPRT procedure analyzeswhether the residual sequence is more likely to be generatedfrom a probability distribution that belongs to normal modehypothesis 119867
0or abnormal mode hypothesis 119867
1by using
likelihood ratio as follows
119871119899= exp[minus(12120590
2)[sum119899
119894=11205831(1205831minus2119903119894)]]
(21)
For fault free sensor values the normalmode hypothesis1198670is
approximated byGaussian distributionwithmean 1205830= 0 and
variance 1205902 Abnormal mode hypothesis 119867
1is approximated
with mean 1205831
gt 1205830using the same variance 120590
2 The SPRTindex for the positive mean test is finally obtained by takinglogarithm of the likelihood ratio in (21) as follows [35]
SPRT = ln (119871119899) = minus
1
21205902[
119899
sum
119894=1
1205831(1205831minus 2119903119894)]
=1205831
1205902
119899
sum
119894=1
(119903119894minus
1205831
2)
(22)
Pressurizer pressure sensor sampled at a frequency of 10 sec-onds is used as a test signal to validate the fault detectability
Science and Technology of Nuclear Installations 13
0 500 1000 1500 2000 2500 3000 3500 4000 45001535
1541545
SPRT drift detection (pressure sensor)(B
ARG
)Pr
essu
re
Drifted sensorDAASM estimate
Sample number
0 500 1000 1500 2000 2500 3000 3500 4000 45001535
1541545
(BA
RG)
Pres
sure
Drifted sensorK-AANN estimate
Sample number
0515
SPRT
hypo
thes
is
minus050 500 1000 1500 2000 2500 3000 3500 4000 4500
DAASM indexK-AANN index
Sample number
005
valu
eRe
sidua
l
0 500 1000 1500 2000 2500 3000 3500 4000 4500
Residual DAASMResidual K-AANN
Sample number
minus05
H1mdash1
H0mdash0
Figure 16 SPRTbased fault detection in pressurizer pressure sensor
performance Two drift faults at the rate of +001hour andminus001hour are introduced in the test signal for DAASMand K-AANN modelrsquos assessment respectively The first andsecond plots in Figure 16 show drifted and estimated pressuresignal from DAASM and K-AANN models respectivelyThird plot shows residual values generated by differencing thedrifted and estimated signals frombothmodelsThe final plotshows SPRT index values against residuals from K-AANNmodel and DAASM The hypotheses 119867
1and 119867
0correspond
to positive and negative fault acceptance respectively FromSPRT index plot successful early detection of the sensordrift at 2200th sample with lag of 611 hours since the driftinception shows that DAASM is more sensitive to smalldrifts On the other hand SPRT index on K-AANN basedsensor estimates registered the same drift at 3800th samplewith a lag of almost 1055 hours The result shows thatDAASM is more robust in terms of early fault detection withlow false and missed alarm rates
Finally both models are tested against five test data setsEach test set consists of 3630 samples corresponding todifferentmonths of full power reactor operation Bothmodelssuccessfully detected an offset failure of 012ndash03 BARG in allsteam pressure channels and a drift type failure up to 285 insteam generator level (Figure 22)The K-AANNmodel failedto register a very small drift up to 01 in steam flow (STMflow 1) channel A small drift up to 01 BARG is detected in
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Steam pressure estimation
Observed valueDAASM estimateK-AANN estimate
672673674675676677678679
68
Stea
m p
ress
ure (
BARG
)
Figure 17 Steam pressure estimates against offset failures up to 03(BARG)
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Feed flow estimation
400420440460480500520540560
Feed
flow
(kg
s)
Observed sensor valueK-AANN estimateDAASM estimate
Figure 18 Feed flow estimates
test set 5 of pressurizer pressure channel However in caseof drift type sensor failures fault detection lag for DAASMwas on average 05 times smaller in comparison with K-AANN model Plots in Figures 17ndash21 show the estimatedsensor values from both models on five test data sets of fewselected channels
9 Conclusion
This paper presented a neural network based denoisedautoassociative sensor model (DAASM) for empirical sen-sor modeling The proposed sensor model is trained togenerate a monitoring system for sensor fault detection innuclear power plantsMultilayer AANNbased sensormodelsmay result in suboptimal solutions due to poor regulariza-tion by traditional backpropagation based joint multilayerlearning procedures So a complementary deep learningapproach based on greedy layerwise unsupervised pretrain-ing is employed for effective regularization in the proposedmultilayer DAASM Autoencoder architecture is used fordenoising based unsupervised pretraining and regularizationof individual layers in the network hierarchy To addressrobustness against perturbations in input sensors data cor-ruption processes exercised during unsupervised pretraining
14 Science and Technology of Nuclear Installations
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5490495500505510515520525530535540
Stea
m fl
ow (k
gs)
Steam flow estimation
Observed sensor valueDAASM estimateK-AANN estimate
Figure 19 Steam flow estimates against drift failure up to 01
58859
592594596598
60602604606608
Pres
suriz
er le
vel (
)
Pressurizer level estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valueK-AANN estimateDAASM estimate
Figure 20 Pressurizer level estimates
phase were based on prior knowledge about different failurescenarios Results from invariance tests showed that theproposed data corruption schemeswere beneficial in learninglatent representations at hidden layers and were invariantto multiple levels of perturbation in input sensors Conse-quently these pretrained hidden layers worked as well regu-larized perturbation filters with increased invariance towardssensor faults It is also observed that sensitivity against sensorfaults decreased significantly towards higher layers in fullDAASMassembly In a practical context of sensormonitoringin nuclear power plants the proposed model proved itsrobustness against gross type simultaneous sensor failuresIt also showed significant improvement in all performancemetrics when compared with popular and widely used five-layered AANNmodel by KramerMoreover time lag in smalldriftrsquos detection is significantly reduced The overall resultssuggest that greedy layerwise pretraining technique in com-bination with domain specific corruption processes providesa viable framework for effective regularization and robustnessin such deep multilayered autoassociative sensor validationmodels
Appendix
See Table 3
153715375
153815385
153915395
15415405
154115415
Pres
sure
(BA
RG)
Pressurizer pressure estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valuesK-AANN estimateDAASM estimate
Figure 21 Pressurizer pressure estimates against drift failure up to01
74
76
78
80
82
84
Stea
m g
ener
ator
leve
l (
)
Steam generator level estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valueK-AANN estimateDAASM estimate
Figure 22 Steam generator level estimates against growing driftfailure up to 287
Abbreviations
AANN Autoassociative neural networkK-AANN Kramer proposed Autoassociative neural
networkDAASM Denoised autoassociative sensor modelNPP Nuclear power plantPWR Pressurized water reactor119878Auto Autosensitivity119878Cross Cross-sensitivityDAE Denoising autoencoder119878 Observed sensor value Model predicted sensor value Corrupted sensor valueSPN Salt-and-pepper noiseAGN Additive Gaussian noise
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Science and Technology of Nuclear Installations 15
Table3Re
sults
ofDAASM
andK-AANNperfo
rmance
metric
sPerfo
rmance
metric
Mod
eltype
Feed
flow1
Feed
flow2
STM
flow1
STM
flow2
STM
PSR1
STM
PSR2
STM
PSR3
PZRPS
R1
PZRPS
R2
PZRLV
L1
PZRLV
L2
SGLV
LNR1
SGLV
LNR2
AANN
Sp-D
AANN
Accuracy
(span)
0382
0361
0343
0412
0186
0211
0166
0386
0411
0243
0223
0512
0621
0281
0253
0246
0293
0121
0132
0122
0243
0315
0173
0156
0394
0465
Autosensitivity
0273
0294
0321
0332
0205
0182
0198
0253
0233
0225
0231
0268
0301
0153
0167
0187
0163
0112
0079
0082
0131
0071
014
0117
0133
0187
Cross-sensitivity
0128
0124
0113
0116
0112
0108
0092
0079
0083
0098
011
0103
0105
00632
00527
00563
00614
0039
00311
00287
00186
00203
00344
00336
003
00286
SPRT
detectability
0312
0315
033
035
004
005
004
008
0086
01
009
006
0058
0184
0165
019
02
0021
0023
002
004
8005
004
004
0032
0028
16 Science and Technology of Nuclear Installations
Acknowledgments
The authors acknowledge the support of Mr Jose GalindoRodrıguez affiliated to TECNATOM Inc Spain and MrChad Painter (Director of Nuclear Power Plant SimulatorDevelopment and Training Program at International AtomicEnergy Agency) for providing necessary tools and data toconduct this research
References
[1] N Mehranbod M Soroush M Piovoso and B A OgunnaikeldquoProbabilistic model for sensor fault detection and identifica-tionrdquo AIChE Journal vol 49 no 7 pp 1787ndash1802 2003
[2] JW Hines and E Davis ldquoLessons learned from the US nuclearpower plant on-line monitoring programsrdquo Progress in NuclearEnergy vol 46 no 3-4 pp 176ndash189 2005
[3] EPRI ldquoOn-line monitoring cost benefit guiderdquo Final Report1006777 EPRI Palo Alto Calif USA 2003
[4] S J Qin and W Li ldquoDetection identification and recon-struction of faulty sensors with maximized sensitivityrdquo AIChEJournal vol 45 no 9 pp 1963ndash1976 1999
[5] J Ma and J Jiang ldquoApplications of fault detection and diagnosismethods in nuclear power plants a reviewrdquo Progress in NuclearEnergy vol 53 no 3 pp 255ndash266 2011
[6] G YHeo ldquoConditionmonitoring using empiricalmodels tech-nical review and prospects for nuclear applicationsrdquo NuclearEngineering and Technology vol 40 no 1 pp 49ndash68 2008
[7] J B Coble R M Meyer P Ramuhalli et al ldquoA review ofsensor calibration monitoring for calibration interval extensionin nuclear power plantsrdquo Tech Rep PNNL-21687 PacificNorthwest National Laboratory Richland Wash USA 2012
[8] M A Kramer ldquoAutoassociative neural networksrdquo Computersand Chemical Engineering vol 16 no 4 pp 313ndash328 1992
[9] X Xu J W Hines and R E Uhrig ldquoOn-line sensor calibra-tion monitoring and fault detection for chemical processesrdquoin Proceedings of the Maintenance and Reliability Conference(MARCON rsquo98) pp 12ndash14 Knoxville Tenn USA May 1998
[10] M Hamidreza S Mehdi J-R Hooshang and N AliakbarldquoReconstruction based approach to sensor fault diagnosis usingauto-associative neural networksrdquo Journal of Central SouthUniversity vol 21 no 6 pp 2273ndash2281 2014
[11] UThissenW JMelssen andLMC Buydens ldquoNonlinear pro-cess monitoring using bottle-neck neural networksrdquo AnalyticaChimica Acta vol 446 no 1-2 pp 371ndash383 2001
[12] D J Wrest J W Hines and R E Uhrig ldquoInstrument surveil-lance and calibration verification through plant wide monitor-ing using autoassociative neural networksrdquo in Proceedings ofthe American Nuclear Society International Topical Meeting onNuclear Plant Instrumentation Control and Human MachineInterface Technologies University Park Pa USA May 1996
[13] J W Hines R E Uhrig and D J Wrest ldquoUse of autoassociativeneural networks for signal validationrdquo Journal of Intelligent andRobotic Systems Theory and Applications vol 21 no 2 pp 143ndash154 1998
[14] P F Fantoni M I Hoffmann R Shankar and E L Davis ldquoOn-line monitoring of instrument channel performance in nuclearpower plant using PEANOrdquo Progress in Nuclear Energy vol 43no 1ndash4 pp 83ndash89 2003
[15] MMarseguerra andA Zoia ldquoTheAutoAssociative Neural Net-work in signal analysis II Application to on-line monitoring of
a simulated BWR componentrdquo Annals of Nuclear Energy vol32 no 11 pp 1207ndash1223 2005
[16] M S Ikbal H Misra and B Yegnanarayana ldquoAnalysis ofautoassociative mapping neural networksrdquo in Proceedings of theInternational Joint Conference on Neural Networks (IJCNN rsquo99)vol 5 pp 3025ndash3029 IEEE Washington DC USA July 1999Proceedings (Cat no 99CH36339)
[17] Y Bengio ldquoLearning deep architectures for AIrdquo Foundationsand Trends in Machine Learning vol 2 no 1 pp 1ndash27 2009
[18] J W Hines A V Gribok I Attieh and R E Uhrig ldquoRegular-izationmethods for inferential sensing in nuclear power plantsrdquoin Fuzzy Systems and Soft Computing in Nuclear Engineering DRuan Ed vol 38 of Studies in Fuzziness and Soft Computingpp 285ndash314 Physica 2000
[19] A V Gribok J W Hines A Urmanov and R E UhrigldquoHeuristic systematic and informational regularization forprocessmonitoringrdquo International Journal of Intelligent Systemsvol 17 no 8 pp 723ndash749 2002
[20] X Glorot and Y Bengio ldquoUnderstanding the difficulty oftraining deep feedforward neural networksrdquo in Proceedings ofthe 13th International Conference on Artificial Intelligence andStatistics (AISTATS rsquo10) vol 9 pp 249ndash256 Sardinia Italy May2010
[21] S Tan andM L Mayrovouniotis ldquoReducing data dimensional-ity through optimizing neural network inputsrdquo AIChE Journalvol 41 no 6 pp 1471ndash1480 1995
[22] Y Bengio and P Lamblin ldquoGreedy layer-wise training of deepnetworksrdquo in Proceedings of the Advances in Neural InformationProcessing Systems 19 (NIPS rsquo07) pp 153ndash160 September 2007
[23] H Larochelle Y Bengio J Lourador and P Lamblin ldquoExplor-ing strategies for training deep neural networksrdquoThe Journal ofMachine Learning Research vol 10 pp 1ndash40 2009
[24] D Yu and L Deng ldquoDeep learning and its applications to signaland information processing [exploratory DSP]rdquo IEEE SignalProcessing Magazine vol 28 no 1 pp 145ndash154 2011
[25] D ErhanA Courville andPVincent ldquoWhydoes unsupervisedpre-training help deep learningrdquo Journal of Machine LearningResearch vol 11 pp 625ndash660 2010
[26] P Vincent H Larochelle Y Bengio and P-A ManzagolldquoExtracting and composing robust features with denoisingautoencodersrdquo in Proceedings of the 25th International Confer-ence on Machine Learning (ICML rsquo08) pp 1096ndash1103 July 2008
[27] P Vincent H Larochelle I Lajoie Y Bengio and P-AManzagol ldquoStacked denoising autoencoders learning usefulrepresentations in a deep network with a local denoisingcriterionrdquo Journal of Machine Learning Research vol 11 pp3371ndash3408 2010
[28] N Srivastava G Hinton A Krizhevsky I Sutskever and RSalakhutdinov ldquoDropout a simple way to prevent neural net-works from overfittingrdquo Journal of Machine Learning Researchvol 15 pp 1929ndash1958 2014
[29] M J Embrechts B J Hargis and J D Linton ldquoAn augmentedefficient backpropagation training strategy for deep autoasso-ciative neural networksrdquo in Proceedings of the 18th EuropeanSymposiumonArtificial NeuralNetworksmdashComputational Intel-ligence and Machine Learning (ESANN rsquo10) vol no pp 28ndash30April 2010
[30] Y Bengio ldquoPractical recommendations for gradient-basedtraining of deep architecturesrdquo inNeural Networks Tricks of theTrade vol 7700 of Lecture Notes in Computer Science pp 437ndash478 Springer Berlin Germany 2nd edition 2012
Science and Technology of Nuclear Installations 17
[31] J Bergstra and Y Bengio ldquoRandom search for hyper-parameteroptimizationrdquo Journal of Machine Learning Research vol 13 pp281ndash305 2012
[32] J W Hines and D R Garvey ldquoDevelopment and applicationof fault detectability performance metrics for instrument cali-bration verification and anomaly detectionrdquo Journal of PatternRecognition Research vol 1 no 1 pp 2ndash15 2006
[33] A Usynin and J W Hines ldquoOn-line monitoring robustnessmeasures and comparisonsrdquo in Proceedings of the Interna-tional Atomic Energy Agency Technical Meeting on ldquoIncreasingInstrument Calibration Interval through On-Line CalibrationTechnologyrdquo OECD Halden Reactor Project Halden NorwaySeptember 2004
[34] A Wald ldquoSequential tests of statistical hypothesesrdquo Annals ofMathematical Statistics vol 16 no 2 pp 117ndash186 1945
[35] F Di Maio P Baraldi E Zio and R Seraoui ldquoFault detection innuclear power plants components by a combination of statisticalmethodsrdquo IEEE Transactions on Reliability vol 62 no 4 pp833ndash845 2013
[36] S Cheng and M Pecht ldquoUsing cross-validation for modelparameter selection of sequential probability ratio testrdquo ExpertSystems with Applications vol 39 no 9 pp 8467ndash8473 2012
TribologyAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
FuelsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal ofPetroleum Engineering
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Power ElectronicsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
CombustionJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Renewable Energy
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
StructuresJournal of
International Journal of
RotatingMachinery
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporation httpwwwhindawicom
Journal ofEngineeringVolume 2014
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
International Journal ofPhotoenergy
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear InstallationsScience and Technology of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Solar EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Wind EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear EnergyInternational Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
High Energy PhysicsAdvances in
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Science and Technology of Nuclear Installations 3
2 Problem Formulation
In context of sensor fault detection application the purposeof a typical sensor reconstruction model is to estimatecorrect sensor value from its corrupted observation Theobjective is to model relationships among input sensorswhich are invariant and robust against sensor faults Soempirical learning for robust sensor relationships can beformulated as sensor denoising problem However contraryto the superimposed channelacquisition noise the termldquodenoisingrdquo specifically corresponds to the corruption causedby gross offset and drift type sensor failures Under suchdenoising based learning objective the empirical sensormodel can be forced to learn a function that captures therobust relationships among correlated sensors and is capableof restoring true sensor value from a corrupted version of it
Let 119878True and Obs be the normal and corrupted sensorstates related by some corruption process 120593(sdot) as follows
Obs = 120593 (119878True) (1)where 120593 119877
119899
rarr 119877119899 is a stochastic corruption caused by
an arbitrary type sensor failure The learning objective fordenoising task can be formulated as
119891 = arg min119891
119864119878True
10038171003817100381710038171003817119891 (Obs) minus 119878True
10038171003817100381710038171003817
2
2
(2)
Under minimization of above formulation the objective ofempirical learning is to search for 119891 that best approximates120593minus1 Furthermore we will formulate and learn such sensor
value estimation and restoration function under neural net-work based autoassociative model driven by deep learningframe work
21 Basic Deep Learning Framework Neural networkresearch suggests that the composition of several levels ofnonlinearity is key to the efficient modeling of complexfunctions However optimization of deep architecture withtraditional gradient based supervised learning methods hasresulted in suboptimal solutions with poor generalizationJoint learning of multiple hidden layers via backpropagationof error gradient inherently suffers from gradient vanishingproblem at the earlier layers and hence constrains the hiddenlayers from fully exploiting the underlying structure inoriginal data distribution In 2006 Hinton in his pioneeringwork proposed a systematic greedy layer by layer trainingof a deep network The idea is to divide the training ofsuccessive layers of a deep network in the form of smallsubnetworks and use unsupervised learning to minimizeinput reconstruction error This technique successfullyeliminates the shortcomings of the gradient based learningby averting the local minima Deep learning frameworkemploys a systematic three-step training approach as follows
(1) Pretraining one layer at a time in a greedy way(2) Using unsupervised learning at each layer in a way
that preserves information from the input and disen-tangles factors of variation
(3) Fine-tuning the whole network with respect to theultimate criterion of interest
x
x
g120579998400 (f120579(x))
120578SD S | S))
Figure 2 Suppose training data ( ) concentrate near a low-dimensional manifold Corrupted examples ( ) obtained by apply-ing corruption process 120578
119878
119863( | 119878) will lie farther from the manifold
The model learns 1198921205791015840 (119891120579()) with 119875(119883 | ) to ldquoproject them backrdquo
onto the manifold
Input layer
x
f120579h
x
Hidden layer
g120579998400
Outputlayer
x
L(x x)
Figure 3 Basic denoising autoencoder (DAE) scheme An emptycircle depicts a single neuron A filled circle depicts corrupted unitsin input vector
3 Building Block for DAASM
In relation to empirical modeling approach as formulatedin Section 2 denoising autoencoder (DAE) [26] is the mostpromising building block for pretraining and compositionof deep autoassociative sensor model DAE is a variant ofthe traditional autoencoder neural network where learningobjective is to reconstruct the original uncorrupted input119909 from partially corrupted or missing inputs Undertraining criterion of reconstruction errorminimization DAEis forced to conserve information details about the inputat its hidden layer mappings The regularization effect ofdenoising based learning objective pushes the DAE networktowards true manifold underlying the high dimension inputdata as depicted in Figure 2 Hence implicitly captures theunderlying data generating distribution by exploring robuststatistical regularities in input data A typical DAE architec-ture as depicted in Figure 3 comprises an input output anda hidden layer An empty circle depicts a neuron unit Theinput layer acts as a proxy layer to the original clean inputMeanwhile the red filed units in input layer are proxies toclean input units which are randomly selected for corruptionunder some artificial noise process 119871(119909 ) is an empiricalloss function to be optimized during training process
Let 119909119894be the original data vector with 119894 = 1 2 119873
elements while 119894represents the partially corrupted version
obtained through corruption process 120578119863 The encoder and
4 Science and Technology of Nuclear Installations
decoder functions corresponding to DAE in Figure 3 aredefined as
ℎ (119894) = 119891120579() = 120590 (119882
119894+ 119887)
(119894) = 1198921205791015840 (ℎ) = 120590 (119882
1015840
ℎ (119894) + 1198871015840
)
(3)
The encoder function 119891120579() transforms input data to ℎ(
119894)
mapping through a sigmoid type activation function 120590(119909) =
(1 + expminus119909)minus1 at hidden layer neurons (119894) is an approxi-
mate reconstruction of 119909 obtained through decoder function1198921205791015840(ℎ) through reverse mapping followed by sigmoid activa-
tion at output layer Meanwhile 120579 = 120579 1205791015840
= 119882 1198871198821015840
1198871015840
are the weight and bias parameters corresponding to theseencoder and decoder functions
In relation to sensor reconstruction model as formulatedin Section 2 the above-described DAE can be reinterpretedas follows
119909 sim 119878 = 119904119894119899
119894=1
sim = 120578119863(( | 119904))
sim = 120578minus1
119863()
(4)
119878 are the input sensor values under fault free steady stateoperation is a partially corrupted input which is generatedthrough an artificial corruption process 120578
119863on selected subset
in input sensor set 119904119894 are the estimated sensor values
by reconstruction function learnt on clean and corruptedinputs 119878 and Network parameters 120579 for DAE can belearned in an unsupervised setting through minimization ofthe reconstruction loss in
119871 (119909 120579) sim 119871 (119878 120579) = arg min120579
119873
sum
119894=1
10038171003817100381710038171003817119878 minus
10038171003817100381710038171003817
2
2
(5)
4 DAASM Architecture and Regularization
In order to capture complex nonlinear relationships amonginput sensors a multilayered architecture is proposed fordenoised autoassociative sensormodel (DAASM) Individuallayers in network hierarchy are pretrained successively frombottom to top For a well regularized sensor model thestructure and optimization objective in greedy layerwisepretraining play a crucial role Two heuristics are applied forrobust learning in DAASM as follows
(1) Each successive layer in multilayered DAASMassembly is pretrained in an unsupervised fashionunder denoising autoencoder (DAE) as elaborated inSection 3
(2) To address robustness data corruption processes fordenoising based pretraining task are incorporatedwith domain specific failure scenarios which arederived from different types of sensor faults Theseheuristics serve several purposes
(i) Forcing the DAE output to match the orig-inal uncorrupted input data acts as a strong
regularizer It helps avoid the trivial identitylearning especially under overcomplete hiddenlayer setting
(ii) Denoising procedure during pretraining leadsto latent representations that are robust to inputperturbations
(iii) Addition of corrupted data set increases train-ing set size and thus is useful in alleviatingoverfitting problem
Full DAASM is learnt in two stages (1) an unsupervisedpretraining phase and (2) a supervised fine-tuning phase Asshown in Figure 4 the pretraining phase follows a hierarchallearning process in which successive DAEs in the stackhierarchy are defined and trained in an unsupervised fashionon the preceding hidden layer activations Full sensor modelis constructed by stacking hidden layers from unsupervisedpretrained DAEs followed by a supervised fine-tuning phaseFor each DAE in the stack hierarchy the optimizationobjective for unsupervised pretraining will remain the sameas in relation (5) However weight decay regularization termis added to the loss function which constrains networkcomplexity by penalizing large weight values In relation (6)119882119882
1015840
are the network weight parameters corresponding toencoder and decoder function while 120582 is the weight decayhyperparameter
119871 (119878 120579) =1
119873
119873
sum
119894=1
10038171003817100381710038171003817119878 minus
10038171003817100381710038171003817
2
2
+120582
2(119882
2
+10038171003817100381710038171003817119882101584010038171003817100381710038171003817
2
) (6)
In a typical DAE architecture a number of input andoutput layer neurons are fixed corresponding to input datadimension 119889 however middle layer neuron counts 119889
1015840 canbe adjusted according to problem complexity Literature indeep learning suggests that under complete middle layer(1198891015840
lt 119889) for DAE architecture results in dense compressedrepresentation at the middle layer Such compressed repre-sentation has tendency to entangle information (change in asingle aspect of the input translates into significant changesin all components of the hidden representation) [27] Thisentangling tendency directly affects the cross-sensitivity ofsensor reconstruction model especially for the case of grosstype sensor failure Considering that here we choose foran overcomplete hidden layer setting (119889
1015840
gt 119889) Underovercomplete setting denoising based optimization objectiveacts as a strong regularizer and inherently prevents DAE fromlearning identity function
Anticlockwise flow in Figure 4 shows architecture andgreedy layer by layer unsupervised pretraining procedure forall hidden layers in DAASM stack For each hidden layer ℎ
119897
a DAE block is shown in which an encoder function 119891119897
120579(sdot)
and a decoder function 119892119897
120579(sdot) are learnt by minimizing the
loss function corresponding to fault free reconstruction of theinputs as in relation (6) For the case of first hidden layer ℎ
1
the corresponding DAE-1 is trained directly on sensor datausing 119871(119878 120579) loss function in (6) However hidden layersℎ2through ℎ
3are learnt on data from preceding hidden layer
activations using recursive relation in (7) So the loss function
Science and Technology of Nuclear Installations 5
DAE-2 for 2nd-layerpretraining
DAE-1 for 1st-layerpretraining
DAE-3 for 3rd-layerpretraining
Supervised fine-tuning
Randomlydropped out
f(1)120579
f(2)120579
f(3)120579
S
S
S S
h1
h1
h2
h3
h1
h2
h3
h3
h2
h2
h1
middot middot middot
g(2)120579998400
g(1)120579998400
g(3)120579998400
Si
middot middot middot
middot middot middotmiddot middot middot
middot middot middot
middot middot middot
h2f(2)120579
f(1)120579
h2
S
middot middot middot
middot middot middot
middot middot middot
middot middot middot
middot middot middot
h1f(1)120579
h1
S
h1(S)
middot middot middot
120578h3D h2 | h2(s)))
120578h2D h1 | h1(s)))
f(2)120579
h1W2)) g(2)120579998400(h2W998400
2)L(h1 h1 120579)
g(3)120579998400(h3W998400
3)L(h2 h2 120579)f(3)
120579h2W3)(
L(S S 120579)
g(1)120579998400(h1W998400
1)L(S S 120579)
f(1)120579
SW1))
120578SD S | S))
Figure 4 DAASM architecture and greedy learning procedure Greedy layerwise pretraining procedure is depicted by counterclockwise flowin the figure
corresponding to DAE-1 and DAE-2 can be represented as119871(ℎ119897 ℎ119897 120579) where ℎ is an approximate reconstruction of ℎ
ℎ119897
= 119891119897
120579(ℎ119897minus1
119882119897) = sigm (119882
119897
ℎ119897minus1
+ 119887119897
)
1 le 119897 le 119871 = 3
(7)
⟨119882⟩ are the network weights corresponding to encoder partin DAE
The noise process 120578119878
119863( | 119878) for DAE-1 corresponds to
a salt-and-pepper (SPN) type corruption process in whicha fraction of the input sensor set 119878 (chosen at random foreach example) is set to minimum ormaximum possible value(typically 0 or 1)The selected noise processmodels gross typefailure scenarios and drives the DAE-1 network to learninginvariance against such type of sensor failures The noisefunctions 120578
ℎ2
119863(ℎ1
| ℎ1()) employ a corruption process in
which ℎ1(119878) and ℎ
1() from pretrained DAE-1 will be used as
the clean and noisy input for DAE-2 pretraining Finally anadditive Gaussian type corruption process (AGN) | 119909 sim
119873(119909 1205902
) is used for DAE-3 noise function 120578ℎ3
119863(ℎ2| ℎ2()) We
will further mathematically formulate and discuss all thesecorruption processes in detail in Section 41
These pretrained layers will initialize the DAASM net-work parameters in basin of attractions which have goodgeneralization and robustness property In order to generate asensormodel that is fairly dependent on all inputs ldquoDropoutrdquo[28] heuristic is applied on ℎ
3hidden units during DAE-
3 pretraining Random dropouts make it hard for latentrepresentations at ℎ
3to get specialized on particular sensors
in the input set Finally pretrained DAEs are unfolded into adeep autoassociator network with 119871 number of encoder and119871 minus 1 decoder cascade as shown in unsupervised fine-tuning
6 Science and Technology of Nuclear Installations
phase in Figure 3 The final network comprises one inputlayer one output and 2119871 minus 1 hidden layers The input sensorvalues flow through encoder cascade 119891 = 119891
119897
120579119900119891119897minus1
120579119900 sdot sdot sdot 119891
1
120579
using recursive expression in (7) and a decoder cascade 119892 =
1198921
1205791015840119900119892119897+1
1205791015840 119900 sdot sdot sdot 119892
119871minus1
1205791015840 using the following equations
ℎ119871
= ℎ119871
ℎ119897
= 119892119897
1205791015840 (ℎ119897+1
1198821015840
119897) = sigm(119882
1015840
119897ℎ119897+1
+ 1198871015840
119897)
1 le 119897 le 119871 minus 1 = 2
= 1198920
(ℎ1
) = 1198821015840
0ℎ1
+ 1198871015840
0
(8)
where ⟨1198821015840
1198871015840
⟩ are network weights and biases of the decoderpart in DAE The entire network is fine-tuned using asemiheuristic based ldquoAugmented Efficient BackpropagationAlgorithmrdquo proposed by Embrechts et al [29] with followingminimization objective
119871 (119878 120579) =1
119873
119873
sum
119894=1
10038171003817100381710038171003817119878 minus
10038171003817100381710038171003817
2
2
+120582
2
2119871
sum
119896=1
10038171003817100381710038171198821198961003817100381710038171003817
2
2 (9)
A 119871minus2weight decay term is added to the above loss functionfor network regularization purpose during fine-tuning phaseTo circumvent the overfitting an early stopping procedurewhich uses validation error as proxy for the generalizationperformance is used during fine-tuning phase
41 Corruption Process 120578(sdot) for Invariance For the case of cal-ibration monitoring an ideal DAASM should learn encoderand decoder functions which are invariant to failed sensorstates So during DAE based pretraining phase engineeredtransformations from prior knowledge about the involved
failure types are imposed on clean input Different datacorruption processes 120578(sdot) are devised for learning of eachsuccessive hidden layer Denoising based learning objectivedrives the hidden layermappings to get invariant against suchengineered transformations on input data It is important tounderstand that denoising based learning approach does notcorrect the faulty signal explicitly rather it seeks to extractstatistical structure among input signals which is stable andinvariant under faults and hence implicitly estimates correctvalue for faulty signal Two failure types are identified anddefined as follows
(i) Gross sensor failure it includes catastrophic sensorfailures Salt-and-pepper type corruption process inwhich a fraction ] of the input sensor set 119878 (chosenat random for each example) is set to minimum ormaximum possible value (typically 0 or 1) is selectedfor modeling gross type failure scenarios
(ii) Miscalibration sensor failure it includes drift multi-plicative and outlier type sensor failures and is mod-eled through isotropic Gaussian noise (GS) | 119909 sim
119873(119909 1205902
) Instead of selecting an arbitrarily simplenoise distribution we estimated the distribution ofsensorrsquos natural noise and exaggerated it to generatenoisy training data
We propose to distribute the denoising based invariancelearning task across multiple hidden layers in the DAASMnetwork Both gross and miscalibration noise types areequally likely to occur in the input space Gaussian typecorruption process is not suitable for input data space 119878
because of its low denoising efficiency against gross typesensor failures Contrarily salt-and-pepper type corruptionprocess covers two extremes of sensors failure range andhence provides an upper bound on perturbation due tominor offset and miscalibration type sensor failures So salt-and-pepper type corruption process is devised for DAE-1pretraining as follows
120578119878
119863( | 119878) = SPN
119894= 119904119894 119894 notin 119895
119894= 119896 119894 isin 119895 119896 = 0 1 with
Pr (0) =1
2
Pr (1) =1
2
119894=V119899
⋃
119894=1
119895119894sdotsdotsdot119873=V119899 119895
119894= [1 119899] = rand ( ) ] = input fraction 119899 = input dimension
(10)
Gross type sensor failures usually have high impact on cross-sensitivity and can trigger false alarms in other sensorsSuch high cross-sensitivity effect may affect isolation ofmiscalibration type secondary failures in other sensors Inorder tominimize the effect a corruption procedure in whichℎ1(119878) and ℎ
1() from pretrained DAE-1 are proposed as the
clean and noisy input for DAE-2 pretrainingThis corruptionmethod is more natural since it causes next hidden layermappings to get invariant against cross-sensitivity effects
and network aberrations from previous layerThe corruptionprocess is supposed to improve invariance in ℎ
2layer map-
pings against cross-sensitivity effects from gross type sensorfailures
120578ℎ2
119863(ℎ1| ℎ1()) = ℎ
119894
1= ℎ119894
1()
where = 120578119878
119863( | 119878) = SPN
(11)
Science and Technology of Nuclear Installations 7
Table 1 List of NPP sensors
Transmitter ID Transmitter name Service Units Low range High rangeFF1 Feed flow 1 Feedwater flow KGS 0 600FF2 Feed flow 2 Feedwater flow KGS 0 600SF1 STM flow 1 Steam flow KGS 0 600SF2 STM flow 2 Steam flow KGS 0 600SP1 STM PSR 1 Steam pressure BARG 0 100SP2 STM PSR 2 Steam pressure BARG 0 100SP3 STM PSR 3 Steam pressure BARG 0 100PP1 PZR PSR 1 Pressurizer pressure BARG 116 170PP2 PZR PSR 2 Pressurizer pressure BARG 116 170PL1 PZR LVL 1 Pressurizer level 0 100PL2 PZR LVL 2 Pressurizer level 0 100SGL1 SG LVL NR 1 range Steam generator level narrow 0 100SGL2 SG LVL NR 2 range Steam generator level narrow 0 100
Here ℎ119894
1(119904) corresponds to hidden layer activations against
clean sensors at the input layer while ℎ119894
1() corresponds to
hidden layer activations against partially faulted sensor setFinally to add robustness against small offset and
miscalibration type sensor failures an isotropic Gaussian
type corruption process is devised for DAE-3 pretrain-ing The corruption procedure corrupts the ℎ
2hidden
layer mappings against clean sensors at the input layeras ℎ2(ℎ1(119878)) by employing an isotropic Gaussian noise as
follows
120578ℎ3
119863(ℎ2| ℎ2(119878)) = AGN
ℎ119894
2= ℎ119894
2(ℎ119894
1(119878)) 119894 notin 119895
ℎ119894
2| ℎ119894
2sim 119873(119904 120590
2
119868) 119894 isin 119895
119894=V119899
⋃
119894=1
119895119894sdotsdotsdot119873=V119899 119895
119894= [1 119899] = rand ( ) ] = input fraction 119899 = total inputs
(12)
Finally clean input is used for the supervised fine-tuningphase in Figure 4
5 Data Set Description
Intentionally for study purposes we limited the modelingscope of DAASM to full power steady operational state It isthe common state in which NPP operates from one refuelingto the next However in practice it is not possible for NPPsystems to be in perfect steady state Reactivity inducedpower perturbations natural process fluctuations sensor andcontroller noises and so forth are some of the evident causesforNPP parameter fluctuations and are responsible for steadystate dynamics Considering that the collected data set shouldbe fairly representative of all possible steady state dynamicsand noise the selected sensors are sampled during differenttime spans of one complete operating cycleThe training dataset consists of 6104 samples collected during the first twomonths of full power reactor operations after refueling cycleMeanwhile 3260 and 2616 samples are reserved for validationand test data sets respectively Five test data sets are used formodelrsquos performance evaluation Each test data set consistsof 4360 samples collected during eight-month period afterrefueling operation In order to account for fault propagation
phenomenon due to large signal groups a sensor subset isselected for this study An engineering sense selection basedon physical proximity and functional correlation is used todefine the sensor subset for this study Thirteen transmittersas listed in Table 1 are selected from various services innuclear steam supply systemof a real PWR typeNPP Figure 5shows the spatial distribution of the selected sensors
Starting from postrefueling full power startup the dataset covers approximately one year of selected sensors valuesSelected sensors are sampled every 10 seconds for consecutive12-hour time window Figure 6 shows data plot from fewselected sensors
6 Model Training
NPP sensorrsquos data are divided into training test and valida-tion set Each sensor data set is scaled in 01 to 09 ranges byusing lower and upper extremities corresponding to individ-ual sensor However the values 0 and 1 are explicitly reservedfor gross and saturation type sensor failures respectivelyTraining data consists of 4320 samples from full power steadystate reactor operation Meanwhile test and validation dataare used for sensor model optimization and performanceevaluation respectively The training setup for DAASM
8 Science and Technology of Nuclear Installations
Reactor vessel
PP1
PL1 PL2
Pressurizer
SGL2
SF1
FF1
Steam generator
Coolant pump
SGL1
FF2
PP2
SP1 SP2 SP3
SF2
Figure 5 Spatial distribution of selected sensor set
Pres
sure
(BA
RG)
1535154
1545
0 1000 1500 2000 2500 3000 3500 4000 4500500Training samples
596061
()
Leve
l
0 1000 1500 2000 2500 3000 3500 4000 4500500Pressurizer pressure
67675
68
Pres
sure
(BA
RG)
0 1000 1500 2000 2500 3000 3500 4000 4500500Steam generator level
500520540
(kg
s)Fl
ow
0 1000 1500 2000 2500 3000 3500 4000 4500500Steam pressure
400500600
Feedwater flow
(kg
s)Fl
ow
0 1000 1500 2000 2500 3000 3500 4000 4500500Steam flow
Figure 6 Plot of NPP sensors listed in Table 1
employs two learning stages an unsupervised learning phaseand supervised training phase DAE based greedy layerwisepretraining of each hidden layer as described in Section 4 isperformed usingminibatches from training data set Stochas-tic gradient descent based learning algorithm is employed
as suggested in practical training recommendations by [30]Finally standard backpropagation algorithm is employed forsupervised fine-tuning in fully stacked DAASM in Figure 4Supervised training is performed using clean sensor inputonly The model hyperparameters are set by random gridsearchmethod [31] A summary of the training hyperparame-ters corresponding to optimumDAASM is shown in Table 2
7 Invariance Test for Robustness
A layer by layer invariance study is conducted to test therobustness of fully trained DAASM against failed sensorstates Data corruption processes applied during pretrainingare essentially meant to learn hidden layer mappings whichare stable and invariant to faulty sensor conditions Thefollowing invariance test for successive hidden layers in finalDAASM stack can provide an insight into the effectivenessof data corruption processes exercised during denoisingbased pretraining phase Invariance for hidden layer map-pings ℎ
119897 is quantified through mean square error (MSE)
between Euclidean (1198712) normalized hidden layer activation⟨ℎ119894⟩119899⟨ℎ119894⟩1198992and ⟨ℎ
119894⟩119899⟨ℎ119894⟩1198992against clean and faulty
sensors respectively Invariance test samples are generatedby corrupting randomly selected sensors in input set withvarying level of offset failures [5ndash50] The MSE againsteach offset level is normalized across hidden layer dimension119863ℎand number of test samples 119879
119873as shown in (13) Finally
theseMSE values are normalized with maximalMSE value asin (14) Normalized MSE curves for each successive hiddenlayer are plotted in Figure 7 Consider
Science and Technology of Nuclear Installations 9
Table 2 Summary of DAASM hyperparameters
Hyperparameter type Tested hyperparameter values Successful hyperparameteragainst optimum model
Pretrained DAE units 3 3
Network architecture
119868119899minus 119864(119899 119901) minus 119861
119899minus 119863(119899 119901) minus 119900
119899
119868119899 input layer neurons
119861119899 bottleneck layer neurons119900119899 output layer neurons
119864(119899 119901) encoder cascade119863(119899 119901) decoder cascade
119899 number of layers119901 neurons per layer
13 minus 119864(2 20) minus 8 minus
119863(2 20) minus 13
Learning rate forunsupervised pretraining 01 5 times 10
minus2
1 times 10minus2
5 times 10minus3
1 times 10minus3
5 times 10minus4
[5 times 10minus2 1 times 10
minus2
]
Learning rate forsupervised training
Scheduled learning rate based on training errormonitoring
015 01 0005 0001 00001
01 0005 0001
Mean pretraining error foreach hidden layer
Corresponding to minima observed during crossvalidation 10
minus4
Weight decay 120582 10minus3
10minus4
10minus5
10minus3
Momentum 119898 [085 099] [095 098]
Input corruption level ]Corrupted input fraction 10 25 30 40
Gaussian corruption ( of sensorrsquos nominal value)005 010 020 035 050
Input fraction [25ndash35]Gaussian noise level
[010ndash025]Dropout fraction in DAE-3 010 020 01
MSE (119867119897Offset) =
(1119879119873)sum119879119873
119899=1(⟨ℎ119894⟩1198991003817100381710038171003817⟨ℎ119894⟩119899
10038171003817100381710038172minus ⟨ℎ119894⟩119899
10038171003817100381710038171003817⟨ℎ119894⟩119899
100381710038171003817100381710038172)2
119863ℎ
1 le 119897 le 119871 = No of encoder layers = 3 Offset = 5 10 20 sdot sdot sdot 50
(13)
MSENormalized (119867119897) =
MSE (119867119897Offset)
MSE (119867119897Max-Offset)
(14)
LayerwiseMSEplots in Figure 7 clearly show that invarianceto faulty sensor conditions increases towards higher layers inthe network hierarchy In these plots lower curves indicatehigher level of invariance To further investigate the effectof increasing invariance on reconstructed sensor values asensor model corresponding to the level ldquo119871rdquo of each hiddenlayer is assembled via encoder and decoder cascade Robust-ness of these partial models is quantified through (1 minus 119878
119894
Auto)Autosensitivity values 119878
119894
Auto (see Section 82) are calculatedagainst varying offset failure levels In Figure 8 layerwiseincrease in robustness confirms that increased invariancehelps in improving overall modelrsquos robustness
8 DAASM versus K-AANNPerformance Analysis
Here we will assess and compare the performance of DAASMwith popular five-layer AANNmodel originally proposed byKramer [8] The K-AANN model is trained with same data
set as used for DAASM and is regularized with Levenberg-Marquardt algorithm Furthermore to improve robustnesstraining with jitter heuristic is employed by introducing anoise of 10 magnitude on clean sensor input The five-layertopology 13-17-9-17-13 is found to be optimum for K-AANNmodel Both DAASM and K-AANN model are comparedthrough accuracy robustness spillover and fault detectabilitybased performance metrics in the following subsections Allperformance metrics are calculated against test data setconsisting of 4320 samples frompostrefueling full powerNPPoperations Performance metric values are reported
81 Accuracy Mean square error (MSE) of observed andmodel estimated sensor values against fault free test data setis used to quantify accuracy metric as follows
Accuracy =1
119873
119873
sum
119894=1
(119894minus 119878119894)2
(15)
10 Science and Technology of Nuclear Installations
Layerwise invariance
3rd hidden layer2nd hidden layer1st hidden layer
0010203040506070809
1
Nor
mal
ised
MSE
minus20 minus10minus40 minus30 0 10 20 30 40 50minus50
Offset failure level ()
Figure 7 Layerwise invariance in DAASM Lower curves depicthigher invariance
Robustness
With 3rd hidden layerWith 2nd hidden layerWith 1st hidden layer
0203040506070809
1
minus20 minus10minus40 minus30 0 10 20 30 40 50minus50
Offset failure level ()
Robu
stnes
s (1minusS a
)
Figure 8 Robustness measure (1 minus autosensitivity) at multiplehidden layers in DAASM Higher curves depict high robustness
The MSE values of all sensors are normalized to theirrespective span and are presented as percent span in Figure 9Being an error measure the lower MSE values by DAASMsignify its prediction accuracy
82 Robustness Robustness is quantified through autosensi-tivity as defined by [32 33] It is themeasure of modelrsquos abilityto predict correct sensor values under missing or corruptedsensor states The measure is averaged over an operatingregion defined by 119896 samples from test data set as follows
119878119894
Auto =1
119873
119873
sum
119896=1
1003816100381610038161003816100381610038161003816100381610038161003816
drift119896119894
minus 119896119894
119904drift119896119894
minus 119904119896119894
1003816100381610038161003816100381610038161003816100381610038161003816
(16)
where 119894 and 119896 are indexes corresponding to sensors and theirrespective test samples 119904
119896119894is the original sensor value without
fault 119896119894is the model estimated sensor value against 119904
119896119894 119904drift119896119894
is the driftedfaulted sensor value drift119896119894
is themodel estimatedsensor value against drifted value 119904
drift119896119894
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN 0382 0361 0343 0412 0186 0211 0166 0386 0411 0243 0223 0512 0621
DAASM 0281 0253 0246 0293 0121 0132 0122 0243 0315 0173 0156 0394 0465
Accuracy metric
0
01
02
03
04
05
06
07
MSE
( sp
an)
Figure 9 MSE depicting DAASM and K-AANN accuracy on eachsensor
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Autosensitivity metric
0005
01015
02025
03503
Auto
sens
itivi
ty
0273 0294 0321 0332 0205 0182 0198 0253 0233 0225 0231 0268 0301
0153 0167 0187 0163 0112 0079 0082 0131 0071 014 0117 0133 0187
Figure 10 Autosensitivity values of individual sensors in bothmodels
The autosensitivitymetric lies in [0 1] range For autosen-sitivity value of one the model predictions follow the faultwith zero residuals hence no fault can be detected Smallerautosensitivity values are preferred which essentially meansdecreased sensitivity towards small perturbations Largeautosensitivity values may lead to missed alarms due tounderestimation of the fault size caused by small residualvalues Compared to K-AANN model in case of DAASM asignificant decrease in autosensitivity values for all sensors isobserved The plot in Figure 10 shows that DAASM is morerobust to failed sensor inputs
To further investigate robustness against large offsetfailures both models are evaluated against offset failuresin [5ndash50] range For each sensor samples from testdata are corrupted with specific offset level and correspond-ing autosensitivities are averaged over whole sensor setAutosensitivity values less than 02 are considered as robustThe maximum autosensitivity value of 0187 is observed insteam flow sensor The plot in Figure 11 shows that averageautosensitivity for both models increases with increasinglevel of offset failure However the autosensitivity curve forDAASM autosensitivity is well below the corresponding K-AANN curve
83 Spillover Depending upon the size and type of failurea failed sensor input can cause discrepancy in estimatedoutput for other sensors The phenomenon is referred to in
Science and Technology of Nuclear Installations 11
0060086
014
021
028
041
016021
028
036
043
054
005 01 02 03 04 05Sensor perturbation magnitude ( of actual value)
Robustness performance
0
01
02
03
04
05
06
Avg
auto
sens
itivi
ty
DAASMK-AANN
Figure 11 Comparison of robustness against increasing offsetfailure
literature as ldquospillover effectrdquo and is quantified through ldquocross-sensitivityrdquo metric [32] It quantifies the influence of faultysensor 119894 on predictions of sensor 119895 as follows
119878119895119894
Cross =1
119873
119873
sum
119896=1
10038161003816100381610038161003816100381610038161003816100381610038161003816
drift119896119895
minus 119896119895
119904drift119896119894
minus 119904119896119894
10038161003816100381610038161003816100381610038161003816100381610038161003816
(17)
119878119895
Cross =1
119873 minus 1
119873
sum
119894=1
119878119895119894
Cross 119894 = 119895 (18)
where 119894 and 119895 indexes are used to refer to faulty andnonfaulty sensors respectively Meanwhile 119896 is the index forcorresponding test samples 119878119895119894Cross is the cross-sensitivity ofsensor 119895 with respect to drift in 119894th sensor 119904
119896119894is the value of
119894th sensor without any fault 119896119895is the model estimated value
of 119895th sensor against 119904119896119894 119904drift119896119894
is the driftedfaulted value of 119894thsensor drift
119896119895is themodel estimated value of 119895th sensor against
drifted value 119904drift119896119894
The highly distributed representation of the input in
neural network based sensor models has pronounced effecton the cross-sensitivity performance Cross-sensitivitymetricvalue lies in [0 1] range High value of cross-sensitivitymay set off false alarms in other sensors provided theresidual values overshoot the fault detectability threshold inother sensors So minimum cross-sensitivity value is desiredfor a robust model The plot in Figure 12 shows that thecross-sensitivity for DAASM is reduced by a large factor ascompared to K-AANNmodel
The spillover effect against particular level of offset failurein [5ndash50] range is averaged over all sensors as follows
AvgCross Sensitivity (119870) =
sum119873
119895=1119878119895
cross(119870)
119873
offset failure level 119870 = 5 10 20 sdot sdot sdot 50
(19)
0128 0124 0113 0116 0112 0108 0092 0079 0083 0098 011 0103 0105
00632 00527 00563 00614 0039 00311 00287 00186 00203 00344 00336 003 00286
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Cross-sensitivity metric
0002004006008
01012014
Cros
s-se
nsiti
vity
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
Figure 12 Cross-sensitivity values of individual sensors in bothmodels
00810105
0161
0253
0312
041
0018 0026 0038 00460071
012
005 01 02 03 04 05Sensor perturbation magnitude ( of observed sensor value)
Spillover offset sensor failure
Requires SPRT mean relaxation to avoid
false alarmsahead of this point
0
005
01
015
02
025
03
035
04
045
Avg
cros
s-se
nsiti
vity
DAASMK-AANN
Figure 13 Comparison of spillover effects against increasing offsetfailure
The cross-sensitivity values 119878119895
cross(119870) against 119870 offset fail-ure level are calculated using (18) Figure 13 shows theaverage cross-sensitivity plot for both models Small cross-sensitivities are observed in DAASM which effectivelyavoided false alarms in other channels without relaxing theSPRT faulted mean value up to an offset failure of 35ndash40 inany channel However for the case of offset noise larger than35 SPRTmean needs to be relaxed to avoid false alarms andisolate the faulty sensor However Robustness of K-AANNmodel deteriorates significantly due to spillover effect beyond15 offset failure
Similarly gross failure scenarios corresponding to twoextremities of sensor range can cause severe Spillover effectTo study robustness against gross type failure scenario asubset of input sensors is simultaneously failed with grosshigh or low value and average cross-sensitivity of remainingsensor set is calculated using relation (19) Plot in Figure 14shows that average cross-sensitivity of K-AANN modelincreases drastically beyond 10 gross failure HoweverDAASM resulted in a very nominal spillover even in caseof multiple sensor failure The DAASM effectively managedsimultaneous gross high or low failures in 25 of total sensorset as compared to 10 in case of K-AANN
12 Science and Technology of Nuclear Installations
0028 0032 0041 012021016 024
0457
061
077
01 02 03 04 05Sensors failed with gross highlow error ()
Spillover gross sensor failure
0010203040506070809
Avg
cros
s-se
nsiti
vity
DAASMK-AANN
Figure 14 Comparison of spillover effect against simultaneous grosshighlow failure in multiple sensors
84 Fault Detectability Fault detectability metric measuresthe smallest fault that can be detected by integrated sensorestimation and fault detection module as shown in Figure 1[32] The detectability metric is measured as percentage ofsensor span 119863 = 119872Span where value M correspondsto minimum detectable fault Minimum fault detectabilitylimit for each sensor is quantified through statistical basedsequential probability ratio test (SPRT) by Wald [34] SPRTtest is carried out to detect if the residual being generatedfrom normal distribution 119873(120583
1 1205902
) or 119873(1205830 1205902
) as definedfor faulty and fault free sensor operations respectively [35]Calibration failures are reflected in the mean parameter ofresidualrsquos distribution The SPRT procedure is applied todetect changes in the mean of residualrsquos distribution Theapplication of SPRT requires setting of following parametersvalue [36]
1205830 normal mode residual mean
1205902 normal mode residual variance
1205831 expected offset in residual mean in abnormal
mode120572 false alarm probability120573 missed alarm probability
Under normal mode the residuals from observed and modelestimated sensor values behave as awhiteGaussian noisewithmean 120583
0= 0 The residual variance 120590
2 is estimated for eachsensor under normal operating conditions and remainedfixed The false alarm 120572 and missed alarm 120573 probabilities areset to be 0001 and 001 respectively In order to determineminimum fault detectability limit a numerical procedureis opted which searches for minimum expected offset 120583
1
in the interval 1205831
[120590ndash3120590] provided the constraint onmissed and false alarm rate holds 120590 is the standard deviationcorresponding to residual variance of particular sensor Theplot in Figure 15 shows the detectability metric for eachsensor The plot in Figure 15 shows that DAASM can detect
0312 0315 033 035 004 005 004 008 0086 01 009 006 0058
0184 0165 019 02 0021 0023 002 0048 005 004 004 0032 0028
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Fault detectability metric
0005
01015
02025
03035
04
Det
ecta
bilit
y (
span
)
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
Figure 15 Comparison of fault detectability metrics
faults which are two times smaller in magnitude than thosedetectable by K-AANNmodel
Improvement in fault detectability metric for DAASMcan be attributed to observed improvement in model robust-ness as suggested by the following relation
119903119894
Δ119878drift119894
= (1 minus 119878119894
Auto) (20)
The term 119903119894Δ119878
drift119894
measures the ratio of observed residualto actual sensor drift in terms of autosensitivity For highlyrobust model this ratio reduces to one which means residualreflects the actual drift and results in high fault detectabilityContrarily ratio value close to zero means that the predictionis following the input and results in poor fault detectability
841 SPRT Based Fault Detectability Test Sequential prob-ability ratio [34 36] based fault hypothesis test is appliedto residual sequence 119877
119894 = 119903
1(1199051) 1199031(1199051)sdot sdot sdot 119903119899(119905119899)generated
by relation 119877119894(119905119894) = 119878
Obs(119905119894) minus 119878
Est(119905119894) at time 119905
119894 where
119878Obs
(119905119894) and 119878
Est(119905119894) are the actual and model predicted
sensor values respectively The SPRT procedure analyzeswhether the residual sequence is more likely to be generatedfrom a probability distribution that belongs to normal modehypothesis 119867
0or abnormal mode hypothesis 119867
1by using
likelihood ratio as follows
119871119899= exp[minus(12120590
2)[sum119899
119894=11205831(1205831minus2119903119894)]]
(21)
For fault free sensor values the normalmode hypothesis1198670is
approximated byGaussian distributionwithmean 1205830= 0 and
variance 1205902 Abnormal mode hypothesis 119867
1is approximated
with mean 1205831
gt 1205830using the same variance 120590
2 The SPRTindex for the positive mean test is finally obtained by takinglogarithm of the likelihood ratio in (21) as follows [35]
SPRT = ln (119871119899) = minus
1
21205902[
119899
sum
119894=1
1205831(1205831minus 2119903119894)]
=1205831
1205902
119899
sum
119894=1
(119903119894minus
1205831
2)
(22)
Pressurizer pressure sensor sampled at a frequency of 10 sec-onds is used as a test signal to validate the fault detectability
Science and Technology of Nuclear Installations 13
0 500 1000 1500 2000 2500 3000 3500 4000 45001535
1541545
SPRT drift detection (pressure sensor)(B
ARG
)Pr
essu
re
Drifted sensorDAASM estimate
Sample number
0 500 1000 1500 2000 2500 3000 3500 4000 45001535
1541545
(BA
RG)
Pres
sure
Drifted sensorK-AANN estimate
Sample number
0515
SPRT
hypo
thes
is
minus050 500 1000 1500 2000 2500 3000 3500 4000 4500
DAASM indexK-AANN index
Sample number
005
valu
eRe
sidua
l
0 500 1000 1500 2000 2500 3000 3500 4000 4500
Residual DAASMResidual K-AANN
Sample number
minus05
H1mdash1
H0mdash0
Figure 16 SPRTbased fault detection in pressurizer pressure sensor
performance Two drift faults at the rate of +001hour andminus001hour are introduced in the test signal for DAASMand K-AANN modelrsquos assessment respectively The first andsecond plots in Figure 16 show drifted and estimated pressuresignal from DAASM and K-AANN models respectivelyThird plot shows residual values generated by differencing thedrifted and estimated signals frombothmodelsThe final plotshows SPRT index values against residuals from K-AANNmodel and DAASM The hypotheses 119867
1and 119867
0correspond
to positive and negative fault acceptance respectively FromSPRT index plot successful early detection of the sensordrift at 2200th sample with lag of 611 hours since the driftinception shows that DAASM is more sensitive to smalldrifts On the other hand SPRT index on K-AANN basedsensor estimates registered the same drift at 3800th samplewith a lag of almost 1055 hours The result shows thatDAASM is more robust in terms of early fault detection withlow false and missed alarm rates
Finally both models are tested against five test data setsEach test set consists of 3630 samples corresponding todifferentmonths of full power reactor operation Bothmodelssuccessfully detected an offset failure of 012ndash03 BARG in allsteam pressure channels and a drift type failure up to 285 insteam generator level (Figure 22)The K-AANNmodel failedto register a very small drift up to 01 in steam flow (STMflow 1) channel A small drift up to 01 BARG is detected in
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Steam pressure estimation
Observed valueDAASM estimateK-AANN estimate
672673674675676677678679
68
Stea
m p
ress
ure (
BARG
)
Figure 17 Steam pressure estimates against offset failures up to 03(BARG)
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Feed flow estimation
400420440460480500520540560
Feed
flow
(kg
s)
Observed sensor valueK-AANN estimateDAASM estimate
Figure 18 Feed flow estimates
test set 5 of pressurizer pressure channel However in caseof drift type sensor failures fault detection lag for DAASMwas on average 05 times smaller in comparison with K-AANN model Plots in Figures 17ndash21 show the estimatedsensor values from both models on five test data sets of fewselected channels
9 Conclusion
This paper presented a neural network based denoisedautoassociative sensor model (DAASM) for empirical sen-sor modeling The proposed sensor model is trained togenerate a monitoring system for sensor fault detection innuclear power plantsMultilayer AANNbased sensormodelsmay result in suboptimal solutions due to poor regulariza-tion by traditional backpropagation based joint multilayerlearning procedures So a complementary deep learningapproach based on greedy layerwise unsupervised pretrain-ing is employed for effective regularization in the proposedmultilayer DAASM Autoencoder architecture is used fordenoising based unsupervised pretraining and regularizationof individual layers in the network hierarchy To addressrobustness against perturbations in input sensors data cor-ruption processes exercised during unsupervised pretraining
14 Science and Technology of Nuclear Installations
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5490495500505510515520525530535540
Stea
m fl
ow (k
gs)
Steam flow estimation
Observed sensor valueDAASM estimateK-AANN estimate
Figure 19 Steam flow estimates against drift failure up to 01
58859
592594596598
60602604606608
Pres
suriz
er le
vel (
)
Pressurizer level estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valueK-AANN estimateDAASM estimate
Figure 20 Pressurizer level estimates
phase were based on prior knowledge about different failurescenarios Results from invariance tests showed that theproposed data corruption schemeswere beneficial in learninglatent representations at hidden layers and were invariantto multiple levels of perturbation in input sensors Conse-quently these pretrained hidden layers worked as well regu-larized perturbation filters with increased invariance towardssensor faults It is also observed that sensitivity against sensorfaults decreased significantly towards higher layers in fullDAASMassembly In a practical context of sensormonitoringin nuclear power plants the proposed model proved itsrobustness against gross type simultaneous sensor failuresIt also showed significant improvement in all performancemetrics when compared with popular and widely used five-layered AANNmodel by KramerMoreover time lag in smalldriftrsquos detection is significantly reduced The overall resultssuggest that greedy layerwise pretraining technique in com-bination with domain specific corruption processes providesa viable framework for effective regularization and robustnessin such deep multilayered autoassociative sensor validationmodels
Appendix
See Table 3
153715375
153815385
153915395
15415405
154115415
Pres
sure
(BA
RG)
Pressurizer pressure estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valuesK-AANN estimateDAASM estimate
Figure 21 Pressurizer pressure estimates against drift failure up to01
74
76
78
80
82
84
Stea
m g
ener
ator
leve
l (
)
Steam generator level estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valueK-AANN estimateDAASM estimate
Figure 22 Steam generator level estimates against growing driftfailure up to 287
Abbreviations
AANN Autoassociative neural networkK-AANN Kramer proposed Autoassociative neural
networkDAASM Denoised autoassociative sensor modelNPP Nuclear power plantPWR Pressurized water reactor119878Auto Autosensitivity119878Cross Cross-sensitivityDAE Denoising autoencoder119878 Observed sensor value Model predicted sensor value Corrupted sensor valueSPN Salt-and-pepper noiseAGN Additive Gaussian noise
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Science and Technology of Nuclear Installations 15
Table3Re
sults
ofDAASM
andK-AANNperfo
rmance
metric
sPerfo
rmance
metric
Mod
eltype
Feed
flow1
Feed
flow2
STM
flow1
STM
flow2
STM
PSR1
STM
PSR2
STM
PSR3
PZRPS
R1
PZRPS
R2
PZRLV
L1
PZRLV
L2
SGLV
LNR1
SGLV
LNR2
AANN
Sp-D
AANN
Accuracy
(span)
0382
0361
0343
0412
0186
0211
0166
0386
0411
0243
0223
0512
0621
0281
0253
0246
0293
0121
0132
0122
0243
0315
0173
0156
0394
0465
Autosensitivity
0273
0294
0321
0332
0205
0182
0198
0253
0233
0225
0231
0268
0301
0153
0167
0187
0163
0112
0079
0082
0131
0071
014
0117
0133
0187
Cross-sensitivity
0128
0124
0113
0116
0112
0108
0092
0079
0083
0098
011
0103
0105
00632
00527
00563
00614
0039
00311
00287
00186
00203
00344
00336
003
00286
SPRT
detectability
0312
0315
033
035
004
005
004
008
0086
01
009
006
0058
0184
0165
019
02
0021
0023
002
004
8005
004
004
0032
0028
16 Science and Technology of Nuclear Installations
Acknowledgments
The authors acknowledge the support of Mr Jose GalindoRodrıguez affiliated to TECNATOM Inc Spain and MrChad Painter (Director of Nuclear Power Plant SimulatorDevelopment and Training Program at International AtomicEnergy Agency) for providing necessary tools and data toconduct this research
References
[1] N Mehranbod M Soroush M Piovoso and B A OgunnaikeldquoProbabilistic model for sensor fault detection and identifica-tionrdquo AIChE Journal vol 49 no 7 pp 1787ndash1802 2003
[2] JW Hines and E Davis ldquoLessons learned from the US nuclearpower plant on-line monitoring programsrdquo Progress in NuclearEnergy vol 46 no 3-4 pp 176ndash189 2005
[3] EPRI ldquoOn-line monitoring cost benefit guiderdquo Final Report1006777 EPRI Palo Alto Calif USA 2003
[4] S J Qin and W Li ldquoDetection identification and recon-struction of faulty sensors with maximized sensitivityrdquo AIChEJournal vol 45 no 9 pp 1963ndash1976 1999
[5] J Ma and J Jiang ldquoApplications of fault detection and diagnosismethods in nuclear power plants a reviewrdquo Progress in NuclearEnergy vol 53 no 3 pp 255ndash266 2011
[6] G YHeo ldquoConditionmonitoring using empiricalmodels tech-nical review and prospects for nuclear applicationsrdquo NuclearEngineering and Technology vol 40 no 1 pp 49ndash68 2008
[7] J B Coble R M Meyer P Ramuhalli et al ldquoA review ofsensor calibration monitoring for calibration interval extensionin nuclear power plantsrdquo Tech Rep PNNL-21687 PacificNorthwest National Laboratory Richland Wash USA 2012
[8] M A Kramer ldquoAutoassociative neural networksrdquo Computersand Chemical Engineering vol 16 no 4 pp 313ndash328 1992
[9] X Xu J W Hines and R E Uhrig ldquoOn-line sensor calibra-tion monitoring and fault detection for chemical processesrdquoin Proceedings of the Maintenance and Reliability Conference(MARCON rsquo98) pp 12ndash14 Knoxville Tenn USA May 1998
[10] M Hamidreza S Mehdi J-R Hooshang and N AliakbarldquoReconstruction based approach to sensor fault diagnosis usingauto-associative neural networksrdquo Journal of Central SouthUniversity vol 21 no 6 pp 2273ndash2281 2014
[11] UThissenW JMelssen andLMC Buydens ldquoNonlinear pro-cess monitoring using bottle-neck neural networksrdquo AnalyticaChimica Acta vol 446 no 1-2 pp 371ndash383 2001
[12] D J Wrest J W Hines and R E Uhrig ldquoInstrument surveil-lance and calibration verification through plant wide monitor-ing using autoassociative neural networksrdquo in Proceedings ofthe American Nuclear Society International Topical Meeting onNuclear Plant Instrumentation Control and Human MachineInterface Technologies University Park Pa USA May 1996
[13] J W Hines R E Uhrig and D J Wrest ldquoUse of autoassociativeneural networks for signal validationrdquo Journal of Intelligent andRobotic Systems Theory and Applications vol 21 no 2 pp 143ndash154 1998
[14] P F Fantoni M I Hoffmann R Shankar and E L Davis ldquoOn-line monitoring of instrument channel performance in nuclearpower plant using PEANOrdquo Progress in Nuclear Energy vol 43no 1ndash4 pp 83ndash89 2003
[15] MMarseguerra andA Zoia ldquoTheAutoAssociative Neural Net-work in signal analysis II Application to on-line monitoring of
a simulated BWR componentrdquo Annals of Nuclear Energy vol32 no 11 pp 1207ndash1223 2005
[16] M S Ikbal H Misra and B Yegnanarayana ldquoAnalysis ofautoassociative mapping neural networksrdquo in Proceedings of theInternational Joint Conference on Neural Networks (IJCNN rsquo99)vol 5 pp 3025ndash3029 IEEE Washington DC USA July 1999Proceedings (Cat no 99CH36339)
[17] Y Bengio ldquoLearning deep architectures for AIrdquo Foundationsand Trends in Machine Learning vol 2 no 1 pp 1ndash27 2009
[18] J W Hines A V Gribok I Attieh and R E Uhrig ldquoRegular-izationmethods for inferential sensing in nuclear power plantsrdquoin Fuzzy Systems and Soft Computing in Nuclear Engineering DRuan Ed vol 38 of Studies in Fuzziness and Soft Computingpp 285ndash314 Physica 2000
[19] A V Gribok J W Hines A Urmanov and R E UhrigldquoHeuristic systematic and informational regularization forprocessmonitoringrdquo International Journal of Intelligent Systemsvol 17 no 8 pp 723ndash749 2002
[20] X Glorot and Y Bengio ldquoUnderstanding the difficulty oftraining deep feedforward neural networksrdquo in Proceedings ofthe 13th International Conference on Artificial Intelligence andStatistics (AISTATS rsquo10) vol 9 pp 249ndash256 Sardinia Italy May2010
[21] S Tan andM L Mayrovouniotis ldquoReducing data dimensional-ity through optimizing neural network inputsrdquo AIChE Journalvol 41 no 6 pp 1471ndash1480 1995
[22] Y Bengio and P Lamblin ldquoGreedy layer-wise training of deepnetworksrdquo in Proceedings of the Advances in Neural InformationProcessing Systems 19 (NIPS rsquo07) pp 153ndash160 September 2007
[23] H Larochelle Y Bengio J Lourador and P Lamblin ldquoExplor-ing strategies for training deep neural networksrdquoThe Journal ofMachine Learning Research vol 10 pp 1ndash40 2009
[24] D Yu and L Deng ldquoDeep learning and its applications to signaland information processing [exploratory DSP]rdquo IEEE SignalProcessing Magazine vol 28 no 1 pp 145ndash154 2011
[25] D ErhanA Courville andPVincent ldquoWhydoes unsupervisedpre-training help deep learningrdquo Journal of Machine LearningResearch vol 11 pp 625ndash660 2010
[26] P Vincent H Larochelle Y Bengio and P-A ManzagolldquoExtracting and composing robust features with denoisingautoencodersrdquo in Proceedings of the 25th International Confer-ence on Machine Learning (ICML rsquo08) pp 1096ndash1103 July 2008
[27] P Vincent H Larochelle I Lajoie Y Bengio and P-AManzagol ldquoStacked denoising autoencoders learning usefulrepresentations in a deep network with a local denoisingcriterionrdquo Journal of Machine Learning Research vol 11 pp3371ndash3408 2010
[28] N Srivastava G Hinton A Krizhevsky I Sutskever and RSalakhutdinov ldquoDropout a simple way to prevent neural net-works from overfittingrdquo Journal of Machine Learning Researchvol 15 pp 1929ndash1958 2014
[29] M J Embrechts B J Hargis and J D Linton ldquoAn augmentedefficient backpropagation training strategy for deep autoasso-ciative neural networksrdquo in Proceedings of the 18th EuropeanSymposiumonArtificial NeuralNetworksmdashComputational Intel-ligence and Machine Learning (ESANN rsquo10) vol no pp 28ndash30April 2010
[30] Y Bengio ldquoPractical recommendations for gradient-basedtraining of deep architecturesrdquo inNeural Networks Tricks of theTrade vol 7700 of Lecture Notes in Computer Science pp 437ndash478 Springer Berlin Germany 2nd edition 2012
Science and Technology of Nuclear Installations 17
[31] J Bergstra and Y Bengio ldquoRandom search for hyper-parameteroptimizationrdquo Journal of Machine Learning Research vol 13 pp281ndash305 2012
[32] J W Hines and D R Garvey ldquoDevelopment and applicationof fault detectability performance metrics for instrument cali-bration verification and anomaly detectionrdquo Journal of PatternRecognition Research vol 1 no 1 pp 2ndash15 2006
[33] A Usynin and J W Hines ldquoOn-line monitoring robustnessmeasures and comparisonsrdquo in Proceedings of the Interna-tional Atomic Energy Agency Technical Meeting on ldquoIncreasingInstrument Calibration Interval through On-Line CalibrationTechnologyrdquo OECD Halden Reactor Project Halden NorwaySeptember 2004
[34] A Wald ldquoSequential tests of statistical hypothesesrdquo Annals ofMathematical Statistics vol 16 no 2 pp 117ndash186 1945
[35] F Di Maio P Baraldi E Zio and R Seraoui ldquoFault detection innuclear power plants components by a combination of statisticalmethodsrdquo IEEE Transactions on Reliability vol 62 no 4 pp833ndash845 2013
[36] S Cheng and M Pecht ldquoUsing cross-validation for modelparameter selection of sequential probability ratio testrdquo ExpertSystems with Applications vol 39 no 9 pp 8467ndash8473 2012
TribologyAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
FuelsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal ofPetroleum Engineering
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Power ElectronicsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
CombustionJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Renewable Energy
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
StructuresJournal of
International Journal of
RotatingMachinery
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporation httpwwwhindawicom
Journal ofEngineeringVolume 2014
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
International Journal ofPhotoenergy
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear InstallationsScience and Technology of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Solar EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Wind EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear EnergyInternational Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
High Energy PhysicsAdvances in
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
4 Science and Technology of Nuclear Installations
decoder functions corresponding to DAE in Figure 3 aredefined as
ℎ (119894) = 119891120579() = 120590 (119882
119894+ 119887)
(119894) = 1198921205791015840 (ℎ) = 120590 (119882
1015840
ℎ (119894) + 1198871015840
)
(3)
The encoder function 119891120579() transforms input data to ℎ(
119894)
mapping through a sigmoid type activation function 120590(119909) =
(1 + expminus119909)minus1 at hidden layer neurons (119894) is an approxi-
mate reconstruction of 119909 obtained through decoder function1198921205791015840(ℎ) through reverse mapping followed by sigmoid activa-
tion at output layer Meanwhile 120579 = 120579 1205791015840
= 119882 1198871198821015840
1198871015840
are the weight and bias parameters corresponding to theseencoder and decoder functions
In relation to sensor reconstruction model as formulatedin Section 2 the above-described DAE can be reinterpretedas follows
119909 sim 119878 = 119904119894119899
119894=1
sim = 120578119863(( | 119904))
sim = 120578minus1
119863()
(4)
119878 are the input sensor values under fault free steady stateoperation is a partially corrupted input which is generatedthrough an artificial corruption process 120578
119863on selected subset
in input sensor set 119904119894 are the estimated sensor values
by reconstruction function learnt on clean and corruptedinputs 119878 and Network parameters 120579 for DAE can belearned in an unsupervised setting through minimization ofthe reconstruction loss in
119871 (119909 120579) sim 119871 (119878 120579) = arg min120579
119873
sum
119894=1
10038171003817100381710038171003817119878 minus
10038171003817100381710038171003817
2
2
(5)
4 DAASM Architecture and Regularization
In order to capture complex nonlinear relationships amonginput sensors a multilayered architecture is proposed fordenoised autoassociative sensormodel (DAASM) Individuallayers in network hierarchy are pretrained successively frombottom to top For a well regularized sensor model thestructure and optimization objective in greedy layerwisepretraining play a crucial role Two heuristics are applied forrobust learning in DAASM as follows
(1) Each successive layer in multilayered DAASMassembly is pretrained in an unsupervised fashionunder denoising autoencoder (DAE) as elaborated inSection 3
(2) To address robustness data corruption processes fordenoising based pretraining task are incorporatedwith domain specific failure scenarios which arederived from different types of sensor faults Theseheuristics serve several purposes
(i) Forcing the DAE output to match the orig-inal uncorrupted input data acts as a strong
regularizer It helps avoid the trivial identitylearning especially under overcomplete hiddenlayer setting
(ii) Denoising procedure during pretraining leadsto latent representations that are robust to inputperturbations
(iii) Addition of corrupted data set increases train-ing set size and thus is useful in alleviatingoverfitting problem
Full DAASM is learnt in two stages (1) an unsupervisedpretraining phase and (2) a supervised fine-tuning phase Asshown in Figure 4 the pretraining phase follows a hierarchallearning process in which successive DAEs in the stackhierarchy are defined and trained in an unsupervised fashionon the preceding hidden layer activations Full sensor modelis constructed by stacking hidden layers from unsupervisedpretrained DAEs followed by a supervised fine-tuning phaseFor each DAE in the stack hierarchy the optimizationobjective for unsupervised pretraining will remain the sameas in relation (5) However weight decay regularization termis added to the loss function which constrains networkcomplexity by penalizing large weight values In relation (6)119882119882
1015840
are the network weight parameters corresponding toencoder and decoder function while 120582 is the weight decayhyperparameter
119871 (119878 120579) =1
119873
119873
sum
119894=1
10038171003817100381710038171003817119878 minus
10038171003817100381710038171003817
2
2
+120582
2(119882
2
+10038171003817100381710038171003817119882101584010038171003817100381710038171003817
2
) (6)
In a typical DAE architecture a number of input andoutput layer neurons are fixed corresponding to input datadimension 119889 however middle layer neuron counts 119889
1015840 canbe adjusted according to problem complexity Literature indeep learning suggests that under complete middle layer(1198891015840
lt 119889) for DAE architecture results in dense compressedrepresentation at the middle layer Such compressed repre-sentation has tendency to entangle information (change in asingle aspect of the input translates into significant changesin all components of the hidden representation) [27] Thisentangling tendency directly affects the cross-sensitivity ofsensor reconstruction model especially for the case of grosstype sensor failure Considering that here we choose foran overcomplete hidden layer setting (119889
1015840
gt 119889) Underovercomplete setting denoising based optimization objectiveacts as a strong regularizer and inherently prevents DAE fromlearning identity function
Anticlockwise flow in Figure 4 shows architecture andgreedy layer by layer unsupervised pretraining procedure forall hidden layers in DAASM stack For each hidden layer ℎ
119897
a DAE block is shown in which an encoder function 119891119897
120579(sdot)
and a decoder function 119892119897
120579(sdot) are learnt by minimizing the
loss function corresponding to fault free reconstruction of theinputs as in relation (6) For the case of first hidden layer ℎ
1
the corresponding DAE-1 is trained directly on sensor datausing 119871(119878 120579) loss function in (6) However hidden layersℎ2through ℎ
3are learnt on data from preceding hidden layer
activations using recursive relation in (7) So the loss function
Science and Technology of Nuclear Installations 5
DAE-2 for 2nd-layerpretraining
DAE-1 for 1st-layerpretraining
DAE-3 for 3rd-layerpretraining
Supervised fine-tuning
Randomlydropped out
f(1)120579
f(2)120579
f(3)120579
S
S
S S
h1
h1
h2
h3
h1
h2
h3
h3
h2
h2
h1
middot middot middot
g(2)120579998400
g(1)120579998400
g(3)120579998400
Si
middot middot middot
middot middot middotmiddot middot middot
middot middot middot
middot middot middot
h2f(2)120579
f(1)120579
h2
S
middot middot middot
middot middot middot
middot middot middot
middot middot middot
middot middot middot
h1f(1)120579
h1
S
h1(S)
middot middot middot
120578h3D h2 | h2(s)))
120578h2D h1 | h1(s)))
f(2)120579
h1W2)) g(2)120579998400(h2W998400
2)L(h1 h1 120579)
g(3)120579998400(h3W998400
3)L(h2 h2 120579)f(3)
120579h2W3)(
L(S S 120579)
g(1)120579998400(h1W998400
1)L(S S 120579)
f(1)120579
SW1))
120578SD S | S))
Figure 4 DAASM architecture and greedy learning procedure Greedy layerwise pretraining procedure is depicted by counterclockwise flowin the figure
corresponding to DAE-1 and DAE-2 can be represented as119871(ℎ119897 ℎ119897 120579) where ℎ is an approximate reconstruction of ℎ
ℎ119897
= 119891119897
120579(ℎ119897minus1
119882119897) = sigm (119882
119897
ℎ119897minus1
+ 119887119897
)
1 le 119897 le 119871 = 3
(7)
⟨119882⟩ are the network weights corresponding to encoder partin DAE
The noise process 120578119878
119863( | 119878) for DAE-1 corresponds to
a salt-and-pepper (SPN) type corruption process in whicha fraction of the input sensor set 119878 (chosen at random foreach example) is set to minimum ormaximum possible value(typically 0 or 1)The selected noise processmodels gross typefailure scenarios and drives the DAE-1 network to learninginvariance against such type of sensor failures The noisefunctions 120578
ℎ2
119863(ℎ1
| ℎ1()) employ a corruption process in
which ℎ1(119878) and ℎ
1() from pretrained DAE-1 will be used as
the clean and noisy input for DAE-2 pretraining Finally anadditive Gaussian type corruption process (AGN) | 119909 sim
119873(119909 1205902
) is used for DAE-3 noise function 120578ℎ3
119863(ℎ2| ℎ2()) We
will further mathematically formulate and discuss all thesecorruption processes in detail in Section 41
These pretrained layers will initialize the DAASM net-work parameters in basin of attractions which have goodgeneralization and robustness property In order to generate asensormodel that is fairly dependent on all inputs ldquoDropoutrdquo[28] heuristic is applied on ℎ
3hidden units during DAE-
3 pretraining Random dropouts make it hard for latentrepresentations at ℎ
3to get specialized on particular sensors
in the input set Finally pretrained DAEs are unfolded into adeep autoassociator network with 119871 number of encoder and119871 minus 1 decoder cascade as shown in unsupervised fine-tuning
6 Science and Technology of Nuclear Installations
phase in Figure 3 The final network comprises one inputlayer one output and 2119871 minus 1 hidden layers The input sensorvalues flow through encoder cascade 119891 = 119891
119897
120579119900119891119897minus1
120579119900 sdot sdot sdot 119891
1
120579
using recursive expression in (7) and a decoder cascade 119892 =
1198921
1205791015840119900119892119897+1
1205791015840 119900 sdot sdot sdot 119892
119871minus1
1205791015840 using the following equations
ℎ119871
= ℎ119871
ℎ119897
= 119892119897
1205791015840 (ℎ119897+1
1198821015840
119897) = sigm(119882
1015840
119897ℎ119897+1
+ 1198871015840
119897)
1 le 119897 le 119871 minus 1 = 2
= 1198920
(ℎ1
) = 1198821015840
0ℎ1
+ 1198871015840
0
(8)
where ⟨1198821015840
1198871015840
⟩ are network weights and biases of the decoderpart in DAE The entire network is fine-tuned using asemiheuristic based ldquoAugmented Efficient BackpropagationAlgorithmrdquo proposed by Embrechts et al [29] with followingminimization objective
119871 (119878 120579) =1
119873
119873
sum
119894=1
10038171003817100381710038171003817119878 minus
10038171003817100381710038171003817
2
2
+120582
2
2119871
sum
119896=1
10038171003817100381710038171198821198961003817100381710038171003817
2
2 (9)
A 119871minus2weight decay term is added to the above loss functionfor network regularization purpose during fine-tuning phaseTo circumvent the overfitting an early stopping procedurewhich uses validation error as proxy for the generalizationperformance is used during fine-tuning phase
41 Corruption Process 120578(sdot) for Invariance For the case of cal-ibration monitoring an ideal DAASM should learn encoderand decoder functions which are invariant to failed sensorstates So during DAE based pretraining phase engineeredtransformations from prior knowledge about the involved
failure types are imposed on clean input Different datacorruption processes 120578(sdot) are devised for learning of eachsuccessive hidden layer Denoising based learning objectivedrives the hidden layermappings to get invariant against suchengineered transformations on input data It is important tounderstand that denoising based learning approach does notcorrect the faulty signal explicitly rather it seeks to extractstatistical structure among input signals which is stable andinvariant under faults and hence implicitly estimates correctvalue for faulty signal Two failure types are identified anddefined as follows
(i) Gross sensor failure it includes catastrophic sensorfailures Salt-and-pepper type corruption process inwhich a fraction ] of the input sensor set 119878 (chosenat random for each example) is set to minimum ormaximum possible value (typically 0 or 1) is selectedfor modeling gross type failure scenarios
(ii) Miscalibration sensor failure it includes drift multi-plicative and outlier type sensor failures and is mod-eled through isotropic Gaussian noise (GS) | 119909 sim
119873(119909 1205902
) Instead of selecting an arbitrarily simplenoise distribution we estimated the distribution ofsensorrsquos natural noise and exaggerated it to generatenoisy training data
We propose to distribute the denoising based invariancelearning task across multiple hidden layers in the DAASMnetwork Both gross and miscalibration noise types areequally likely to occur in the input space Gaussian typecorruption process is not suitable for input data space 119878
because of its low denoising efficiency against gross typesensor failures Contrarily salt-and-pepper type corruptionprocess covers two extremes of sensors failure range andhence provides an upper bound on perturbation due tominor offset and miscalibration type sensor failures So salt-and-pepper type corruption process is devised for DAE-1pretraining as follows
120578119878
119863( | 119878) = SPN
119894= 119904119894 119894 notin 119895
119894= 119896 119894 isin 119895 119896 = 0 1 with
Pr (0) =1
2
Pr (1) =1
2
119894=V119899
⋃
119894=1
119895119894sdotsdotsdot119873=V119899 119895
119894= [1 119899] = rand ( ) ] = input fraction 119899 = input dimension
(10)
Gross type sensor failures usually have high impact on cross-sensitivity and can trigger false alarms in other sensorsSuch high cross-sensitivity effect may affect isolation ofmiscalibration type secondary failures in other sensors Inorder tominimize the effect a corruption procedure in whichℎ1(119878) and ℎ
1() from pretrained DAE-1 are proposed as the
clean and noisy input for DAE-2 pretrainingThis corruptionmethod is more natural since it causes next hidden layermappings to get invariant against cross-sensitivity effects
and network aberrations from previous layerThe corruptionprocess is supposed to improve invariance in ℎ
2layer map-
pings against cross-sensitivity effects from gross type sensorfailures
120578ℎ2
119863(ℎ1| ℎ1()) = ℎ
119894
1= ℎ119894
1()
where = 120578119878
119863( | 119878) = SPN
(11)
Science and Technology of Nuclear Installations 7
Table 1 List of NPP sensors
Transmitter ID Transmitter name Service Units Low range High rangeFF1 Feed flow 1 Feedwater flow KGS 0 600FF2 Feed flow 2 Feedwater flow KGS 0 600SF1 STM flow 1 Steam flow KGS 0 600SF2 STM flow 2 Steam flow KGS 0 600SP1 STM PSR 1 Steam pressure BARG 0 100SP2 STM PSR 2 Steam pressure BARG 0 100SP3 STM PSR 3 Steam pressure BARG 0 100PP1 PZR PSR 1 Pressurizer pressure BARG 116 170PP2 PZR PSR 2 Pressurizer pressure BARG 116 170PL1 PZR LVL 1 Pressurizer level 0 100PL2 PZR LVL 2 Pressurizer level 0 100SGL1 SG LVL NR 1 range Steam generator level narrow 0 100SGL2 SG LVL NR 2 range Steam generator level narrow 0 100
Here ℎ119894
1(119904) corresponds to hidden layer activations against
clean sensors at the input layer while ℎ119894
1() corresponds to
hidden layer activations against partially faulted sensor setFinally to add robustness against small offset and
miscalibration type sensor failures an isotropic Gaussian
type corruption process is devised for DAE-3 pretrain-ing The corruption procedure corrupts the ℎ
2hidden
layer mappings against clean sensors at the input layeras ℎ2(ℎ1(119878)) by employing an isotropic Gaussian noise as
follows
120578ℎ3
119863(ℎ2| ℎ2(119878)) = AGN
ℎ119894
2= ℎ119894
2(ℎ119894
1(119878)) 119894 notin 119895
ℎ119894
2| ℎ119894
2sim 119873(119904 120590
2
119868) 119894 isin 119895
119894=V119899
⋃
119894=1
119895119894sdotsdotsdot119873=V119899 119895
119894= [1 119899] = rand ( ) ] = input fraction 119899 = total inputs
(12)
Finally clean input is used for the supervised fine-tuningphase in Figure 4
5 Data Set Description
Intentionally for study purposes we limited the modelingscope of DAASM to full power steady operational state It isthe common state in which NPP operates from one refuelingto the next However in practice it is not possible for NPPsystems to be in perfect steady state Reactivity inducedpower perturbations natural process fluctuations sensor andcontroller noises and so forth are some of the evident causesforNPP parameter fluctuations and are responsible for steadystate dynamics Considering that the collected data set shouldbe fairly representative of all possible steady state dynamicsand noise the selected sensors are sampled during differenttime spans of one complete operating cycleThe training dataset consists of 6104 samples collected during the first twomonths of full power reactor operations after refueling cycleMeanwhile 3260 and 2616 samples are reserved for validationand test data sets respectively Five test data sets are used formodelrsquos performance evaluation Each test data set consistsof 4360 samples collected during eight-month period afterrefueling operation In order to account for fault propagation
phenomenon due to large signal groups a sensor subset isselected for this study An engineering sense selection basedon physical proximity and functional correlation is used todefine the sensor subset for this study Thirteen transmittersas listed in Table 1 are selected from various services innuclear steam supply systemof a real PWR typeNPP Figure 5shows the spatial distribution of the selected sensors
Starting from postrefueling full power startup the dataset covers approximately one year of selected sensors valuesSelected sensors are sampled every 10 seconds for consecutive12-hour time window Figure 6 shows data plot from fewselected sensors
6 Model Training
NPP sensorrsquos data are divided into training test and valida-tion set Each sensor data set is scaled in 01 to 09 ranges byusing lower and upper extremities corresponding to individ-ual sensor However the values 0 and 1 are explicitly reservedfor gross and saturation type sensor failures respectivelyTraining data consists of 4320 samples from full power steadystate reactor operation Meanwhile test and validation dataare used for sensor model optimization and performanceevaluation respectively The training setup for DAASM
8 Science and Technology of Nuclear Installations
Reactor vessel
PP1
PL1 PL2
Pressurizer
SGL2
SF1
FF1
Steam generator
Coolant pump
SGL1
FF2
PP2
SP1 SP2 SP3
SF2
Figure 5 Spatial distribution of selected sensor set
Pres
sure
(BA
RG)
1535154
1545
0 1000 1500 2000 2500 3000 3500 4000 4500500Training samples
596061
()
Leve
l
0 1000 1500 2000 2500 3000 3500 4000 4500500Pressurizer pressure
67675
68
Pres
sure
(BA
RG)
0 1000 1500 2000 2500 3000 3500 4000 4500500Steam generator level
500520540
(kg
s)Fl
ow
0 1000 1500 2000 2500 3000 3500 4000 4500500Steam pressure
400500600
Feedwater flow
(kg
s)Fl
ow
0 1000 1500 2000 2500 3000 3500 4000 4500500Steam flow
Figure 6 Plot of NPP sensors listed in Table 1
employs two learning stages an unsupervised learning phaseand supervised training phase DAE based greedy layerwisepretraining of each hidden layer as described in Section 4 isperformed usingminibatches from training data set Stochas-tic gradient descent based learning algorithm is employed
as suggested in practical training recommendations by [30]Finally standard backpropagation algorithm is employed forsupervised fine-tuning in fully stacked DAASM in Figure 4Supervised training is performed using clean sensor inputonly The model hyperparameters are set by random gridsearchmethod [31] A summary of the training hyperparame-ters corresponding to optimumDAASM is shown in Table 2
7 Invariance Test for Robustness
A layer by layer invariance study is conducted to test therobustness of fully trained DAASM against failed sensorstates Data corruption processes applied during pretrainingare essentially meant to learn hidden layer mappings whichare stable and invariant to faulty sensor conditions Thefollowing invariance test for successive hidden layers in finalDAASM stack can provide an insight into the effectivenessof data corruption processes exercised during denoisingbased pretraining phase Invariance for hidden layer map-pings ℎ
119897 is quantified through mean square error (MSE)
between Euclidean (1198712) normalized hidden layer activation⟨ℎ119894⟩119899⟨ℎ119894⟩1198992and ⟨ℎ
119894⟩119899⟨ℎ119894⟩1198992against clean and faulty
sensors respectively Invariance test samples are generatedby corrupting randomly selected sensors in input set withvarying level of offset failures [5ndash50] The MSE againsteach offset level is normalized across hidden layer dimension119863ℎand number of test samples 119879
119873as shown in (13) Finally
theseMSE values are normalized with maximalMSE value asin (14) Normalized MSE curves for each successive hiddenlayer are plotted in Figure 7 Consider
Science and Technology of Nuclear Installations 9
Table 2 Summary of DAASM hyperparameters
Hyperparameter type Tested hyperparameter values Successful hyperparameteragainst optimum model
Pretrained DAE units 3 3
Network architecture
119868119899minus 119864(119899 119901) minus 119861
119899minus 119863(119899 119901) minus 119900
119899
119868119899 input layer neurons
119861119899 bottleneck layer neurons119900119899 output layer neurons
119864(119899 119901) encoder cascade119863(119899 119901) decoder cascade
119899 number of layers119901 neurons per layer
13 minus 119864(2 20) minus 8 minus
119863(2 20) minus 13
Learning rate forunsupervised pretraining 01 5 times 10
minus2
1 times 10minus2
5 times 10minus3
1 times 10minus3
5 times 10minus4
[5 times 10minus2 1 times 10
minus2
]
Learning rate forsupervised training
Scheduled learning rate based on training errormonitoring
015 01 0005 0001 00001
01 0005 0001
Mean pretraining error foreach hidden layer
Corresponding to minima observed during crossvalidation 10
minus4
Weight decay 120582 10minus3
10minus4
10minus5
10minus3
Momentum 119898 [085 099] [095 098]
Input corruption level ]Corrupted input fraction 10 25 30 40
Gaussian corruption ( of sensorrsquos nominal value)005 010 020 035 050
Input fraction [25ndash35]Gaussian noise level
[010ndash025]Dropout fraction in DAE-3 010 020 01
MSE (119867119897Offset) =
(1119879119873)sum119879119873
119899=1(⟨ℎ119894⟩1198991003817100381710038171003817⟨ℎ119894⟩119899
10038171003817100381710038172minus ⟨ℎ119894⟩119899
10038171003817100381710038171003817⟨ℎ119894⟩119899
100381710038171003817100381710038172)2
119863ℎ
1 le 119897 le 119871 = No of encoder layers = 3 Offset = 5 10 20 sdot sdot sdot 50
(13)
MSENormalized (119867119897) =
MSE (119867119897Offset)
MSE (119867119897Max-Offset)
(14)
LayerwiseMSEplots in Figure 7 clearly show that invarianceto faulty sensor conditions increases towards higher layers inthe network hierarchy In these plots lower curves indicatehigher level of invariance To further investigate the effectof increasing invariance on reconstructed sensor values asensor model corresponding to the level ldquo119871rdquo of each hiddenlayer is assembled via encoder and decoder cascade Robust-ness of these partial models is quantified through (1 minus 119878
119894
Auto)Autosensitivity values 119878
119894
Auto (see Section 82) are calculatedagainst varying offset failure levels In Figure 8 layerwiseincrease in robustness confirms that increased invariancehelps in improving overall modelrsquos robustness
8 DAASM versus K-AANNPerformance Analysis
Here we will assess and compare the performance of DAASMwith popular five-layer AANNmodel originally proposed byKramer [8] The K-AANN model is trained with same data
set as used for DAASM and is regularized with Levenberg-Marquardt algorithm Furthermore to improve robustnesstraining with jitter heuristic is employed by introducing anoise of 10 magnitude on clean sensor input The five-layertopology 13-17-9-17-13 is found to be optimum for K-AANNmodel Both DAASM and K-AANN model are comparedthrough accuracy robustness spillover and fault detectabilitybased performance metrics in the following subsections Allperformance metrics are calculated against test data setconsisting of 4320 samples frompostrefueling full powerNPPoperations Performance metric values are reported
81 Accuracy Mean square error (MSE) of observed andmodel estimated sensor values against fault free test data setis used to quantify accuracy metric as follows
Accuracy =1
119873
119873
sum
119894=1
(119894minus 119878119894)2
(15)
10 Science and Technology of Nuclear Installations
Layerwise invariance
3rd hidden layer2nd hidden layer1st hidden layer
0010203040506070809
1
Nor
mal
ised
MSE
minus20 minus10minus40 minus30 0 10 20 30 40 50minus50
Offset failure level ()
Figure 7 Layerwise invariance in DAASM Lower curves depicthigher invariance
Robustness
With 3rd hidden layerWith 2nd hidden layerWith 1st hidden layer
0203040506070809
1
minus20 minus10minus40 minus30 0 10 20 30 40 50minus50
Offset failure level ()
Robu
stnes
s (1minusS a
)
Figure 8 Robustness measure (1 minus autosensitivity) at multiplehidden layers in DAASM Higher curves depict high robustness
The MSE values of all sensors are normalized to theirrespective span and are presented as percent span in Figure 9Being an error measure the lower MSE values by DAASMsignify its prediction accuracy
82 Robustness Robustness is quantified through autosensi-tivity as defined by [32 33] It is themeasure of modelrsquos abilityto predict correct sensor values under missing or corruptedsensor states The measure is averaged over an operatingregion defined by 119896 samples from test data set as follows
119878119894
Auto =1
119873
119873
sum
119896=1
1003816100381610038161003816100381610038161003816100381610038161003816
drift119896119894
minus 119896119894
119904drift119896119894
minus 119904119896119894
1003816100381610038161003816100381610038161003816100381610038161003816
(16)
where 119894 and 119896 are indexes corresponding to sensors and theirrespective test samples 119904
119896119894is the original sensor value without
fault 119896119894is the model estimated sensor value against 119904
119896119894 119904drift119896119894
is the driftedfaulted sensor value drift119896119894
is themodel estimatedsensor value against drifted value 119904
drift119896119894
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN 0382 0361 0343 0412 0186 0211 0166 0386 0411 0243 0223 0512 0621
DAASM 0281 0253 0246 0293 0121 0132 0122 0243 0315 0173 0156 0394 0465
Accuracy metric
0
01
02
03
04
05
06
07
MSE
( sp
an)
Figure 9 MSE depicting DAASM and K-AANN accuracy on eachsensor
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Autosensitivity metric
0005
01015
02025
03503
Auto
sens
itivi
ty
0273 0294 0321 0332 0205 0182 0198 0253 0233 0225 0231 0268 0301
0153 0167 0187 0163 0112 0079 0082 0131 0071 014 0117 0133 0187
Figure 10 Autosensitivity values of individual sensors in bothmodels
The autosensitivitymetric lies in [0 1] range For autosen-sitivity value of one the model predictions follow the faultwith zero residuals hence no fault can be detected Smallerautosensitivity values are preferred which essentially meansdecreased sensitivity towards small perturbations Largeautosensitivity values may lead to missed alarms due tounderestimation of the fault size caused by small residualvalues Compared to K-AANN model in case of DAASM asignificant decrease in autosensitivity values for all sensors isobserved The plot in Figure 10 shows that DAASM is morerobust to failed sensor inputs
To further investigate robustness against large offsetfailures both models are evaluated against offset failuresin [5ndash50] range For each sensor samples from testdata are corrupted with specific offset level and correspond-ing autosensitivities are averaged over whole sensor setAutosensitivity values less than 02 are considered as robustThe maximum autosensitivity value of 0187 is observed insteam flow sensor The plot in Figure 11 shows that averageautosensitivity for both models increases with increasinglevel of offset failure However the autosensitivity curve forDAASM autosensitivity is well below the corresponding K-AANN curve
83 Spillover Depending upon the size and type of failurea failed sensor input can cause discrepancy in estimatedoutput for other sensors The phenomenon is referred to in
Science and Technology of Nuclear Installations 11
0060086
014
021
028
041
016021
028
036
043
054
005 01 02 03 04 05Sensor perturbation magnitude ( of actual value)
Robustness performance
0
01
02
03
04
05
06
Avg
auto
sens
itivi
ty
DAASMK-AANN
Figure 11 Comparison of robustness against increasing offsetfailure
literature as ldquospillover effectrdquo and is quantified through ldquocross-sensitivityrdquo metric [32] It quantifies the influence of faultysensor 119894 on predictions of sensor 119895 as follows
119878119895119894
Cross =1
119873
119873
sum
119896=1
10038161003816100381610038161003816100381610038161003816100381610038161003816
drift119896119895
minus 119896119895
119904drift119896119894
minus 119904119896119894
10038161003816100381610038161003816100381610038161003816100381610038161003816
(17)
119878119895
Cross =1
119873 minus 1
119873
sum
119894=1
119878119895119894
Cross 119894 = 119895 (18)
where 119894 and 119895 indexes are used to refer to faulty andnonfaulty sensors respectively Meanwhile 119896 is the index forcorresponding test samples 119878119895119894Cross is the cross-sensitivity ofsensor 119895 with respect to drift in 119894th sensor 119904
119896119894is the value of
119894th sensor without any fault 119896119895is the model estimated value
of 119895th sensor against 119904119896119894 119904drift119896119894
is the driftedfaulted value of 119894thsensor drift
119896119895is themodel estimated value of 119895th sensor against
drifted value 119904drift119896119894
The highly distributed representation of the input in
neural network based sensor models has pronounced effecton the cross-sensitivity performance Cross-sensitivitymetricvalue lies in [0 1] range High value of cross-sensitivitymay set off false alarms in other sensors provided theresidual values overshoot the fault detectability threshold inother sensors So minimum cross-sensitivity value is desiredfor a robust model The plot in Figure 12 shows that thecross-sensitivity for DAASM is reduced by a large factor ascompared to K-AANNmodel
The spillover effect against particular level of offset failurein [5ndash50] range is averaged over all sensors as follows
AvgCross Sensitivity (119870) =
sum119873
119895=1119878119895
cross(119870)
119873
offset failure level 119870 = 5 10 20 sdot sdot sdot 50
(19)
0128 0124 0113 0116 0112 0108 0092 0079 0083 0098 011 0103 0105
00632 00527 00563 00614 0039 00311 00287 00186 00203 00344 00336 003 00286
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Cross-sensitivity metric
0002004006008
01012014
Cros
s-se
nsiti
vity
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
Figure 12 Cross-sensitivity values of individual sensors in bothmodels
00810105
0161
0253
0312
041
0018 0026 0038 00460071
012
005 01 02 03 04 05Sensor perturbation magnitude ( of observed sensor value)
Spillover offset sensor failure
Requires SPRT mean relaxation to avoid
false alarmsahead of this point
0
005
01
015
02
025
03
035
04
045
Avg
cros
s-se
nsiti
vity
DAASMK-AANN
Figure 13 Comparison of spillover effects against increasing offsetfailure
The cross-sensitivity values 119878119895
cross(119870) against 119870 offset fail-ure level are calculated using (18) Figure 13 shows theaverage cross-sensitivity plot for both models Small cross-sensitivities are observed in DAASM which effectivelyavoided false alarms in other channels without relaxing theSPRT faulted mean value up to an offset failure of 35ndash40 inany channel However for the case of offset noise larger than35 SPRTmean needs to be relaxed to avoid false alarms andisolate the faulty sensor However Robustness of K-AANNmodel deteriorates significantly due to spillover effect beyond15 offset failure
Similarly gross failure scenarios corresponding to twoextremities of sensor range can cause severe Spillover effectTo study robustness against gross type failure scenario asubset of input sensors is simultaneously failed with grosshigh or low value and average cross-sensitivity of remainingsensor set is calculated using relation (19) Plot in Figure 14shows that average cross-sensitivity of K-AANN modelincreases drastically beyond 10 gross failure HoweverDAASM resulted in a very nominal spillover even in caseof multiple sensor failure The DAASM effectively managedsimultaneous gross high or low failures in 25 of total sensorset as compared to 10 in case of K-AANN
12 Science and Technology of Nuclear Installations
0028 0032 0041 012021016 024
0457
061
077
01 02 03 04 05Sensors failed with gross highlow error ()
Spillover gross sensor failure
0010203040506070809
Avg
cros
s-se
nsiti
vity
DAASMK-AANN
Figure 14 Comparison of spillover effect against simultaneous grosshighlow failure in multiple sensors
84 Fault Detectability Fault detectability metric measuresthe smallest fault that can be detected by integrated sensorestimation and fault detection module as shown in Figure 1[32] The detectability metric is measured as percentage ofsensor span 119863 = 119872Span where value M correspondsto minimum detectable fault Minimum fault detectabilitylimit for each sensor is quantified through statistical basedsequential probability ratio test (SPRT) by Wald [34] SPRTtest is carried out to detect if the residual being generatedfrom normal distribution 119873(120583
1 1205902
) or 119873(1205830 1205902
) as definedfor faulty and fault free sensor operations respectively [35]Calibration failures are reflected in the mean parameter ofresidualrsquos distribution The SPRT procedure is applied todetect changes in the mean of residualrsquos distribution Theapplication of SPRT requires setting of following parametersvalue [36]
1205830 normal mode residual mean
1205902 normal mode residual variance
1205831 expected offset in residual mean in abnormal
mode120572 false alarm probability120573 missed alarm probability
Under normal mode the residuals from observed and modelestimated sensor values behave as awhiteGaussian noisewithmean 120583
0= 0 The residual variance 120590
2 is estimated for eachsensor under normal operating conditions and remainedfixed The false alarm 120572 and missed alarm 120573 probabilities areset to be 0001 and 001 respectively In order to determineminimum fault detectability limit a numerical procedureis opted which searches for minimum expected offset 120583
1
in the interval 1205831
[120590ndash3120590] provided the constraint onmissed and false alarm rate holds 120590 is the standard deviationcorresponding to residual variance of particular sensor Theplot in Figure 15 shows the detectability metric for eachsensor The plot in Figure 15 shows that DAASM can detect
0312 0315 033 035 004 005 004 008 0086 01 009 006 0058
0184 0165 019 02 0021 0023 002 0048 005 004 004 0032 0028
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Fault detectability metric
0005
01015
02025
03035
04
Det
ecta
bilit
y (
span
)
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
Figure 15 Comparison of fault detectability metrics
faults which are two times smaller in magnitude than thosedetectable by K-AANNmodel
Improvement in fault detectability metric for DAASMcan be attributed to observed improvement in model robust-ness as suggested by the following relation
119903119894
Δ119878drift119894
= (1 minus 119878119894
Auto) (20)
The term 119903119894Δ119878
drift119894
measures the ratio of observed residualto actual sensor drift in terms of autosensitivity For highlyrobust model this ratio reduces to one which means residualreflects the actual drift and results in high fault detectabilityContrarily ratio value close to zero means that the predictionis following the input and results in poor fault detectability
841 SPRT Based Fault Detectability Test Sequential prob-ability ratio [34 36] based fault hypothesis test is appliedto residual sequence 119877
119894 = 119903
1(1199051) 1199031(1199051)sdot sdot sdot 119903119899(119905119899)generated
by relation 119877119894(119905119894) = 119878
Obs(119905119894) minus 119878
Est(119905119894) at time 119905
119894 where
119878Obs
(119905119894) and 119878
Est(119905119894) are the actual and model predicted
sensor values respectively The SPRT procedure analyzeswhether the residual sequence is more likely to be generatedfrom a probability distribution that belongs to normal modehypothesis 119867
0or abnormal mode hypothesis 119867
1by using
likelihood ratio as follows
119871119899= exp[minus(12120590
2)[sum119899
119894=11205831(1205831minus2119903119894)]]
(21)
For fault free sensor values the normalmode hypothesis1198670is
approximated byGaussian distributionwithmean 1205830= 0 and
variance 1205902 Abnormal mode hypothesis 119867
1is approximated
with mean 1205831
gt 1205830using the same variance 120590
2 The SPRTindex for the positive mean test is finally obtained by takinglogarithm of the likelihood ratio in (21) as follows [35]
SPRT = ln (119871119899) = minus
1
21205902[
119899
sum
119894=1
1205831(1205831minus 2119903119894)]
=1205831
1205902
119899
sum
119894=1
(119903119894minus
1205831
2)
(22)
Pressurizer pressure sensor sampled at a frequency of 10 sec-onds is used as a test signal to validate the fault detectability
Science and Technology of Nuclear Installations 13
0 500 1000 1500 2000 2500 3000 3500 4000 45001535
1541545
SPRT drift detection (pressure sensor)(B
ARG
)Pr
essu
re
Drifted sensorDAASM estimate
Sample number
0 500 1000 1500 2000 2500 3000 3500 4000 45001535
1541545
(BA
RG)
Pres
sure
Drifted sensorK-AANN estimate
Sample number
0515
SPRT
hypo
thes
is
minus050 500 1000 1500 2000 2500 3000 3500 4000 4500
DAASM indexK-AANN index
Sample number
005
valu
eRe
sidua
l
0 500 1000 1500 2000 2500 3000 3500 4000 4500
Residual DAASMResidual K-AANN
Sample number
minus05
H1mdash1
H0mdash0
Figure 16 SPRTbased fault detection in pressurizer pressure sensor
performance Two drift faults at the rate of +001hour andminus001hour are introduced in the test signal for DAASMand K-AANN modelrsquos assessment respectively The first andsecond plots in Figure 16 show drifted and estimated pressuresignal from DAASM and K-AANN models respectivelyThird plot shows residual values generated by differencing thedrifted and estimated signals frombothmodelsThe final plotshows SPRT index values against residuals from K-AANNmodel and DAASM The hypotheses 119867
1and 119867
0correspond
to positive and negative fault acceptance respectively FromSPRT index plot successful early detection of the sensordrift at 2200th sample with lag of 611 hours since the driftinception shows that DAASM is more sensitive to smalldrifts On the other hand SPRT index on K-AANN basedsensor estimates registered the same drift at 3800th samplewith a lag of almost 1055 hours The result shows thatDAASM is more robust in terms of early fault detection withlow false and missed alarm rates
Finally both models are tested against five test data setsEach test set consists of 3630 samples corresponding todifferentmonths of full power reactor operation Bothmodelssuccessfully detected an offset failure of 012ndash03 BARG in allsteam pressure channels and a drift type failure up to 285 insteam generator level (Figure 22)The K-AANNmodel failedto register a very small drift up to 01 in steam flow (STMflow 1) channel A small drift up to 01 BARG is detected in
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Steam pressure estimation
Observed valueDAASM estimateK-AANN estimate
672673674675676677678679
68
Stea
m p
ress
ure (
BARG
)
Figure 17 Steam pressure estimates against offset failures up to 03(BARG)
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Feed flow estimation
400420440460480500520540560
Feed
flow
(kg
s)
Observed sensor valueK-AANN estimateDAASM estimate
Figure 18 Feed flow estimates
test set 5 of pressurizer pressure channel However in caseof drift type sensor failures fault detection lag for DAASMwas on average 05 times smaller in comparison with K-AANN model Plots in Figures 17ndash21 show the estimatedsensor values from both models on five test data sets of fewselected channels
9 Conclusion
This paper presented a neural network based denoisedautoassociative sensor model (DAASM) for empirical sen-sor modeling The proposed sensor model is trained togenerate a monitoring system for sensor fault detection innuclear power plantsMultilayer AANNbased sensormodelsmay result in suboptimal solutions due to poor regulariza-tion by traditional backpropagation based joint multilayerlearning procedures So a complementary deep learningapproach based on greedy layerwise unsupervised pretrain-ing is employed for effective regularization in the proposedmultilayer DAASM Autoencoder architecture is used fordenoising based unsupervised pretraining and regularizationof individual layers in the network hierarchy To addressrobustness against perturbations in input sensors data cor-ruption processes exercised during unsupervised pretraining
14 Science and Technology of Nuclear Installations
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5490495500505510515520525530535540
Stea
m fl
ow (k
gs)
Steam flow estimation
Observed sensor valueDAASM estimateK-AANN estimate
Figure 19 Steam flow estimates against drift failure up to 01
58859
592594596598
60602604606608
Pres
suriz
er le
vel (
)
Pressurizer level estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valueK-AANN estimateDAASM estimate
Figure 20 Pressurizer level estimates
phase were based on prior knowledge about different failurescenarios Results from invariance tests showed that theproposed data corruption schemeswere beneficial in learninglatent representations at hidden layers and were invariantto multiple levels of perturbation in input sensors Conse-quently these pretrained hidden layers worked as well regu-larized perturbation filters with increased invariance towardssensor faults It is also observed that sensitivity against sensorfaults decreased significantly towards higher layers in fullDAASMassembly In a practical context of sensormonitoringin nuclear power plants the proposed model proved itsrobustness against gross type simultaneous sensor failuresIt also showed significant improvement in all performancemetrics when compared with popular and widely used five-layered AANNmodel by KramerMoreover time lag in smalldriftrsquos detection is significantly reduced The overall resultssuggest that greedy layerwise pretraining technique in com-bination with domain specific corruption processes providesa viable framework for effective regularization and robustnessin such deep multilayered autoassociative sensor validationmodels
Appendix
See Table 3
153715375
153815385
153915395
15415405
154115415
Pres
sure
(BA
RG)
Pressurizer pressure estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valuesK-AANN estimateDAASM estimate
Figure 21 Pressurizer pressure estimates against drift failure up to01
74
76
78
80
82
84
Stea
m g
ener
ator
leve
l (
)
Steam generator level estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valueK-AANN estimateDAASM estimate
Figure 22 Steam generator level estimates against growing driftfailure up to 287
Abbreviations
AANN Autoassociative neural networkK-AANN Kramer proposed Autoassociative neural
networkDAASM Denoised autoassociative sensor modelNPP Nuclear power plantPWR Pressurized water reactor119878Auto Autosensitivity119878Cross Cross-sensitivityDAE Denoising autoencoder119878 Observed sensor value Model predicted sensor value Corrupted sensor valueSPN Salt-and-pepper noiseAGN Additive Gaussian noise
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Science and Technology of Nuclear Installations 15
Table3Re
sults
ofDAASM
andK-AANNperfo
rmance
metric
sPerfo
rmance
metric
Mod
eltype
Feed
flow1
Feed
flow2
STM
flow1
STM
flow2
STM
PSR1
STM
PSR2
STM
PSR3
PZRPS
R1
PZRPS
R2
PZRLV
L1
PZRLV
L2
SGLV
LNR1
SGLV
LNR2
AANN
Sp-D
AANN
Accuracy
(span)
0382
0361
0343
0412
0186
0211
0166
0386
0411
0243
0223
0512
0621
0281
0253
0246
0293
0121
0132
0122
0243
0315
0173
0156
0394
0465
Autosensitivity
0273
0294
0321
0332
0205
0182
0198
0253
0233
0225
0231
0268
0301
0153
0167
0187
0163
0112
0079
0082
0131
0071
014
0117
0133
0187
Cross-sensitivity
0128
0124
0113
0116
0112
0108
0092
0079
0083
0098
011
0103
0105
00632
00527
00563
00614
0039
00311
00287
00186
00203
00344
00336
003
00286
SPRT
detectability
0312
0315
033
035
004
005
004
008
0086
01
009
006
0058
0184
0165
019
02
0021
0023
002
004
8005
004
004
0032
0028
16 Science and Technology of Nuclear Installations
Acknowledgments
The authors acknowledge the support of Mr Jose GalindoRodrıguez affiliated to TECNATOM Inc Spain and MrChad Painter (Director of Nuclear Power Plant SimulatorDevelopment and Training Program at International AtomicEnergy Agency) for providing necessary tools and data toconduct this research
References
[1] N Mehranbod M Soroush M Piovoso and B A OgunnaikeldquoProbabilistic model for sensor fault detection and identifica-tionrdquo AIChE Journal vol 49 no 7 pp 1787ndash1802 2003
[2] JW Hines and E Davis ldquoLessons learned from the US nuclearpower plant on-line monitoring programsrdquo Progress in NuclearEnergy vol 46 no 3-4 pp 176ndash189 2005
[3] EPRI ldquoOn-line monitoring cost benefit guiderdquo Final Report1006777 EPRI Palo Alto Calif USA 2003
[4] S J Qin and W Li ldquoDetection identification and recon-struction of faulty sensors with maximized sensitivityrdquo AIChEJournal vol 45 no 9 pp 1963ndash1976 1999
[5] J Ma and J Jiang ldquoApplications of fault detection and diagnosismethods in nuclear power plants a reviewrdquo Progress in NuclearEnergy vol 53 no 3 pp 255ndash266 2011
[6] G YHeo ldquoConditionmonitoring using empiricalmodels tech-nical review and prospects for nuclear applicationsrdquo NuclearEngineering and Technology vol 40 no 1 pp 49ndash68 2008
[7] J B Coble R M Meyer P Ramuhalli et al ldquoA review ofsensor calibration monitoring for calibration interval extensionin nuclear power plantsrdquo Tech Rep PNNL-21687 PacificNorthwest National Laboratory Richland Wash USA 2012
[8] M A Kramer ldquoAutoassociative neural networksrdquo Computersand Chemical Engineering vol 16 no 4 pp 313ndash328 1992
[9] X Xu J W Hines and R E Uhrig ldquoOn-line sensor calibra-tion monitoring and fault detection for chemical processesrdquoin Proceedings of the Maintenance and Reliability Conference(MARCON rsquo98) pp 12ndash14 Knoxville Tenn USA May 1998
[10] M Hamidreza S Mehdi J-R Hooshang and N AliakbarldquoReconstruction based approach to sensor fault diagnosis usingauto-associative neural networksrdquo Journal of Central SouthUniversity vol 21 no 6 pp 2273ndash2281 2014
[11] UThissenW JMelssen andLMC Buydens ldquoNonlinear pro-cess monitoring using bottle-neck neural networksrdquo AnalyticaChimica Acta vol 446 no 1-2 pp 371ndash383 2001
[12] D J Wrest J W Hines and R E Uhrig ldquoInstrument surveil-lance and calibration verification through plant wide monitor-ing using autoassociative neural networksrdquo in Proceedings ofthe American Nuclear Society International Topical Meeting onNuclear Plant Instrumentation Control and Human MachineInterface Technologies University Park Pa USA May 1996
[13] J W Hines R E Uhrig and D J Wrest ldquoUse of autoassociativeneural networks for signal validationrdquo Journal of Intelligent andRobotic Systems Theory and Applications vol 21 no 2 pp 143ndash154 1998
[14] P F Fantoni M I Hoffmann R Shankar and E L Davis ldquoOn-line monitoring of instrument channel performance in nuclearpower plant using PEANOrdquo Progress in Nuclear Energy vol 43no 1ndash4 pp 83ndash89 2003
[15] MMarseguerra andA Zoia ldquoTheAutoAssociative Neural Net-work in signal analysis II Application to on-line monitoring of
a simulated BWR componentrdquo Annals of Nuclear Energy vol32 no 11 pp 1207ndash1223 2005
[16] M S Ikbal H Misra and B Yegnanarayana ldquoAnalysis ofautoassociative mapping neural networksrdquo in Proceedings of theInternational Joint Conference on Neural Networks (IJCNN rsquo99)vol 5 pp 3025ndash3029 IEEE Washington DC USA July 1999Proceedings (Cat no 99CH36339)
[17] Y Bengio ldquoLearning deep architectures for AIrdquo Foundationsand Trends in Machine Learning vol 2 no 1 pp 1ndash27 2009
[18] J W Hines A V Gribok I Attieh and R E Uhrig ldquoRegular-izationmethods for inferential sensing in nuclear power plantsrdquoin Fuzzy Systems and Soft Computing in Nuclear Engineering DRuan Ed vol 38 of Studies in Fuzziness and Soft Computingpp 285ndash314 Physica 2000
[19] A V Gribok J W Hines A Urmanov and R E UhrigldquoHeuristic systematic and informational regularization forprocessmonitoringrdquo International Journal of Intelligent Systemsvol 17 no 8 pp 723ndash749 2002
[20] X Glorot and Y Bengio ldquoUnderstanding the difficulty oftraining deep feedforward neural networksrdquo in Proceedings ofthe 13th International Conference on Artificial Intelligence andStatistics (AISTATS rsquo10) vol 9 pp 249ndash256 Sardinia Italy May2010
[21] S Tan andM L Mayrovouniotis ldquoReducing data dimensional-ity through optimizing neural network inputsrdquo AIChE Journalvol 41 no 6 pp 1471ndash1480 1995
[22] Y Bengio and P Lamblin ldquoGreedy layer-wise training of deepnetworksrdquo in Proceedings of the Advances in Neural InformationProcessing Systems 19 (NIPS rsquo07) pp 153ndash160 September 2007
[23] H Larochelle Y Bengio J Lourador and P Lamblin ldquoExplor-ing strategies for training deep neural networksrdquoThe Journal ofMachine Learning Research vol 10 pp 1ndash40 2009
[24] D Yu and L Deng ldquoDeep learning and its applications to signaland information processing [exploratory DSP]rdquo IEEE SignalProcessing Magazine vol 28 no 1 pp 145ndash154 2011
[25] D ErhanA Courville andPVincent ldquoWhydoes unsupervisedpre-training help deep learningrdquo Journal of Machine LearningResearch vol 11 pp 625ndash660 2010
[26] P Vincent H Larochelle Y Bengio and P-A ManzagolldquoExtracting and composing robust features with denoisingautoencodersrdquo in Proceedings of the 25th International Confer-ence on Machine Learning (ICML rsquo08) pp 1096ndash1103 July 2008
[27] P Vincent H Larochelle I Lajoie Y Bengio and P-AManzagol ldquoStacked denoising autoencoders learning usefulrepresentations in a deep network with a local denoisingcriterionrdquo Journal of Machine Learning Research vol 11 pp3371ndash3408 2010
[28] N Srivastava G Hinton A Krizhevsky I Sutskever and RSalakhutdinov ldquoDropout a simple way to prevent neural net-works from overfittingrdquo Journal of Machine Learning Researchvol 15 pp 1929ndash1958 2014
[29] M J Embrechts B J Hargis and J D Linton ldquoAn augmentedefficient backpropagation training strategy for deep autoasso-ciative neural networksrdquo in Proceedings of the 18th EuropeanSymposiumonArtificial NeuralNetworksmdashComputational Intel-ligence and Machine Learning (ESANN rsquo10) vol no pp 28ndash30April 2010
[30] Y Bengio ldquoPractical recommendations for gradient-basedtraining of deep architecturesrdquo inNeural Networks Tricks of theTrade vol 7700 of Lecture Notes in Computer Science pp 437ndash478 Springer Berlin Germany 2nd edition 2012
Science and Technology of Nuclear Installations 17
[31] J Bergstra and Y Bengio ldquoRandom search for hyper-parameteroptimizationrdquo Journal of Machine Learning Research vol 13 pp281ndash305 2012
[32] J W Hines and D R Garvey ldquoDevelopment and applicationof fault detectability performance metrics for instrument cali-bration verification and anomaly detectionrdquo Journal of PatternRecognition Research vol 1 no 1 pp 2ndash15 2006
[33] A Usynin and J W Hines ldquoOn-line monitoring robustnessmeasures and comparisonsrdquo in Proceedings of the Interna-tional Atomic Energy Agency Technical Meeting on ldquoIncreasingInstrument Calibration Interval through On-Line CalibrationTechnologyrdquo OECD Halden Reactor Project Halden NorwaySeptember 2004
[34] A Wald ldquoSequential tests of statistical hypothesesrdquo Annals ofMathematical Statistics vol 16 no 2 pp 117ndash186 1945
[35] F Di Maio P Baraldi E Zio and R Seraoui ldquoFault detection innuclear power plants components by a combination of statisticalmethodsrdquo IEEE Transactions on Reliability vol 62 no 4 pp833ndash845 2013
[36] S Cheng and M Pecht ldquoUsing cross-validation for modelparameter selection of sequential probability ratio testrdquo ExpertSystems with Applications vol 39 no 9 pp 8467ndash8473 2012
TribologyAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
FuelsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal ofPetroleum Engineering
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Power ElectronicsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
CombustionJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Renewable Energy
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
StructuresJournal of
International Journal of
RotatingMachinery
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporation httpwwwhindawicom
Journal ofEngineeringVolume 2014
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
International Journal ofPhotoenergy
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear InstallationsScience and Technology of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Solar EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Wind EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear EnergyInternational Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
High Energy PhysicsAdvances in
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Science and Technology of Nuclear Installations 5
DAE-2 for 2nd-layerpretraining
DAE-1 for 1st-layerpretraining
DAE-3 for 3rd-layerpretraining
Supervised fine-tuning
Randomlydropped out
f(1)120579
f(2)120579
f(3)120579
S
S
S S
h1
h1
h2
h3
h1
h2
h3
h3
h2
h2
h1
middot middot middot
g(2)120579998400
g(1)120579998400
g(3)120579998400
Si
middot middot middot
middot middot middotmiddot middot middot
middot middot middot
middot middot middot
h2f(2)120579
f(1)120579
h2
S
middot middot middot
middot middot middot
middot middot middot
middot middot middot
middot middot middot
h1f(1)120579
h1
S
h1(S)
middot middot middot
120578h3D h2 | h2(s)))
120578h2D h1 | h1(s)))
f(2)120579
h1W2)) g(2)120579998400(h2W998400
2)L(h1 h1 120579)
g(3)120579998400(h3W998400
3)L(h2 h2 120579)f(3)
120579h2W3)(
L(S S 120579)
g(1)120579998400(h1W998400
1)L(S S 120579)
f(1)120579
SW1))
120578SD S | S))
Figure 4 DAASM architecture and greedy learning procedure Greedy layerwise pretraining procedure is depicted by counterclockwise flowin the figure
corresponding to DAE-1 and DAE-2 can be represented as119871(ℎ119897 ℎ119897 120579) where ℎ is an approximate reconstruction of ℎ
ℎ119897
= 119891119897
120579(ℎ119897minus1
119882119897) = sigm (119882
119897
ℎ119897minus1
+ 119887119897
)
1 le 119897 le 119871 = 3
(7)
⟨119882⟩ are the network weights corresponding to encoder partin DAE
The noise process 120578119878
119863( | 119878) for DAE-1 corresponds to
a salt-and-pepper (SPN) type corruption process in whicha fraction of the input sensor set 119878 (chosen at random foreach example) is set to minimum ormaximum possible value(typically 0 or 1)The selected noise processmodels gross typefailure scenarios and drives the DAE-1 network to learninginvariance against such type of sensor failures The noisefunctions 120578
ℎ2
119863(ℎ1
| ℎ1()) employ a corruption process in
which ℎ1(119878) and ℎ
1() from pretrained DAE-1 will be used as
the clean and noisy input for DAE-2 pretraining Finally anadditive Gaussian type corruption process (AGN) | 119909 sim
119873(119909 1205902
) is used for DAE-3 noise function 120578ℎ3
119863(ℎ2| ℎ2()) We
will further mathematically formulate and discuss all thesecorruption processes in detail in Section 41
These pretrained layers will initialize the DAASM net-work parameters in basin of attractions which have goodgeneralization and robustness property In order to generate asensormodel that is fairly dependent on all inputs ldquoDropoutrdquo[28] heuristic is applied on ℎ
3hidden units during DAE-
3 pretraining Random dropouts make it hard for latentrepresentations at ℎ
3to get specialized on particular sensors
in the input set Finally pretrained DAEs are unfolded into adeep autoassociator network with 119871 number of encoder and119871 minus 1 decoder cascade as shown in unsupervised fine-tuning
6 Science and Technology of Nuclear Installations
phase in Figure 3 The final network comprises one inputlayer one output and 2119871 minus 1 hidden layers The input sensorvalues flow through encoder cascade 119891 = 119891
119897
120579119900119891119897minus1
120579119900 sdot sdot sdot 119891
1
120579
using recursive expression in (7) and a decoder cascade 119892 =
1198921
1205791015840119900119892119897+1
1205791015840 119900 sdot sdot sdot 119892
119871minus1
1205791015840 using the following equations
ℎ119871
= ℎ119871
ℎ119897
= 119892119897
1205791015840 (ℎ119897+1
1198821015840
119897) = sigm(119882
1015840
119897ℎ119897+1
+ 1198871015840
119897)
1 le 119897 le 119871 minus 1 = 2
= 1198920
(ℎ1
) = 1198821015840
0ℎ1
+ 1198871015840
0
(8)
where ⟨1198821015840
1198871015840
⟩ are network weights and biases of the decoderpart in DAE The entire network is fine-tuned using asemiheuristic based ldquoAugmented Efficient BackpropagationAlgorithmrdquo proposed by Embrechts et al [29] with followingminimization objective
119871 (119878 120579) =1
119873
119873
sum
119894=1
10038171003817100381710038171003817119878 minus
10038171003817100381710038171003817
2
2
+120582
2
2119871
sum
119896=1
10038171003817100381710038171198821198961003817100381710038171003817
2
2 (9)
A 119871minus2weight decay term is added to the above loss functionfor network regularization purpose during fine-tuning phaseTo circumvent the overfitting an early stopping procedurewhich uses validation error as proxy for the generalizationperformance is used during fine-tuning phase
41 Corruption Process 120578(sdot) for Invariance For the case of cal-ibration monitoring an ideal DAASM should learn encoderand decoder functions which are invariant to failed sensorstates So during DAE based pretraining phase engineeredtransformations from prior knowledge about the involved
failure types are imposed on clean input Different datacorruption processes 120578(sdot) are devised for learning of eachsuccessive hidden layer Denoising based learning objectivedrives the hidden layermappings to get invariant against suchengineered transformations on input data It is important tounderstand that denoising based learning approach does notcorrect the faulty signal explicitly rather it seeks to extractstatistical structure among input signals which is stable andinvariant under faults and hence implicitly estimates correctvalue for faulty signal Two failure types are identified anddefined as follows
(i) Gross sensor failure it includes catastrophic sensorfailures Salt-and-pepper type corruption process inwhich a fraction ] of the input sensor set 119878 (chosenat random for each example) is set to minimum ormaximum possible value (typically 0 or 1) is selectedfor modeling gross type failure scenarios
(ii) Miscalibration sensor failure it includes drift multi-plicative and outlier type sensor failures and is mod-eled through isotropic Gaussian noise (GS) | 119909 sim
119873(119909 1205902
) Instead of selecting an arbitrarily simplenoise distribution we estimated the distribution ofsensorrsquos natural noise and exaggerated it to generatenoisy training data
We propose to distribute the denoising based invariancelearning task across multiple hidden layers in the DAASMnetwork Both gross and miscalibration noise types areequally likely to occur in the input space Gaussian typecorruption process is not suitable for input data space 119878
because of its low denoising efficiency against gross typesensor failures Contrarily salt-and-pepper type corruptionprocess covers two extremes of sensors failure range andhence provides an upper bound on perturbation due tominor offset and miscalibration type sensor failures So salt-and-pepper type corruption process is devised for DAE-1pretraining as follows
120578119878
119863( | 119878) = SPN
119894= 119904119894 119894 notin 119895
119894= 119896 119894 isin 119895 119896 = 0 1 with
Pr (0) =1
2
Pr (1) =1
2
119894=V119899
⋃
119894=1
119895119894sdotsdotsdot119873=V119899 119895
119894= [1 119899] = rand ( ) ] = input fraction 119899 = input dimension
(10)
Gross type sensor failures usually have high impact on cross-sensitivity and can trigger false alarms in other sensorsSuch high cross-sensitivity effect may affect isolation ofmiscalibration type secondary failures in other sensors Inorder tominimize the effect a corruption procedure in whichℎ1(119878) and ℎ
1() from pretrained DAE-1 are proposed as the
clean and noisy input for DAE-2 pretrainingThis corruptionmethod is more natural since it causes next hidden layermappings to get invariant against cross-sensitivity effects
and network aberrations from previous layerThe corruptionprocess is supposed to improve invariance in ℎ
2layer map-
pings against cross-sensitivity effects from gross type sensorfailures
120578ℎ2
119863(ℎ1| ℎ1()) = ℎ
119894
1= ℎ119894
1()
where = 120578119878
119863( | 119878) = SPN
(11)
Science and Technology of Nuclear Installations 7
Table 1 List of NPP sensors
Transmitter ID Transmitter name Service Units Low range High rangeFF1 Feed flow 1 Feedwater flow KGS 0 600FF2 Feed flow 2 Feedwater flow KGS 0 600SF1 STM flow 1 Steam flow KGS 0 600SF2 STM flow 2 Steam flow KGS 0 600SP1 STM PSR 1 Steam pressure BARG 0 100SP2 STM PSR 2 Steam pressure BARG 0 100SP3 STM PSR 3 Steam pressure BARG 0 100PP1 PZR PSR 1 Pressurizer pressure BARG 116 170PP2 PZR PSR 2 Pressurizer pressure BARG 116 170PL1 PZR LVL 1 Pressurizer level 0 100PL2 PZR LVL 2 Pressurizer level 0 100SGL1 SG LVL NR 1 range Steam generator level narrow 0 100SGL2 SG LVL NR 2 range Steam generator level narrow 0 100
Here ℎ119894
1(119904) corresponds to hidden layer activations against
clean sensors at the input layer while ℎ119894
1() corresponds to
hidden layer activations against partially faulted sensor setFinally to add robustness against small offset and
miscalibration type sensor failures an isotropic Gaussian
type corruption process is devised for DAE-3 pretrain-ing The corruption procedure corrupts the ℎ
2hidden
layer mappings against clean sensors at the input layeras ℎ2(ℎ1(119878)) by employing an isotropic Gaussian noise as
follows
120578ℎ3
119863(ℎ2| ℎ2(119878)) = AGN
ℎ119894
2= ℎ119894
2(ℎ119894
1(119878)) 119894 notin 119895
ℎ119894
2| ℎ119894
2sim 119873(119904 120590
2
119868) 119894 isin 119895
119894=V119899
⋃
119894=1
119895119894sdotsdotsdot119873=V119899 119895
119894= [1 119899] = rand ( ) ] = input fraction 119899 = total inputs
(12)
Finally clean input is used for the supervised fine-tuningphase in Figure 4
5 Data Set Description
Intentionally for study purposes we limited the modelingscope of DAASM to full power steady operational state It isthe common state in which NPP operates from one refuelingto the next However in practice it is not possible for NPPsystems to be in perfect steady state Reactivity inducedpower perturbations natural process fluctuations sensor andcontroller noises and so forth are some of the evident causesforNPP parameter fluctuations and are responsible for steadystate dynamics Considering that the collected data set shouldbe fairly representative of all possible steady state dynamicsand noise the selected sensors are sampled during differenttime spans of one complete operating cycleThe training dataset consists of 6104 samples collected during the first twomonths of full power reactor operations after refueling cycleMeanwhile 3260 and 2616 samples are reserved for validationand test data sets respectively Five test data sets are used formodelrsquos performance evaluation Each test data set consistsof 4360 samples collected during eight-month period afterrefueling operation In order to account for fault propagation
phenomenon due to large signal groups a sensor subset isselected for this study An engineering sense selection basedon physical proximity and functional correlation is used todefine the sensor subset for this study Thirteen transmittersas listed in Table 1 are selected from various services innuclear steam supply systemof a real PWR typeNPP Figure 5shows the spatial distribution of the selected sensors
Starting from postrefueling full power startup the dataset covers approximately one year of selected sensors valuesSelected sensors are sampled every 10 seconds for consecutive12-hour time window Figure 6 shows data plot from fewselected sensors
6 Model Training
NPP sensorrsquos data are divided into training test and valida-tion set Each sensor data set is scaled in 01 to 09 ranges byusing lower and upper extremities corresponding to individ-ual sensor However the values 0 and 1 are explicitly reservedfor gross and saturation type sensor failures respectivelyTraining data consists of 4320 samples from full power steadystate reactor operation Meanwhile test and validation dataare used for sensor model optimization and performanceevaluation respectively The training setup for DAASM
8 Science and Technology of Nuclear Installations
Reactor vessel
PP1
PL1 PL2
Pressurizer
SGL2
SF1
FF1
Steam generator
Coolant pump
SGL1
FF2
PP2
SP1 SP2 SP3
SF2
Figure 5 Spatial distribution of selected sensor set
Pres
sure
(BA
RG)
1535154
1545
0 1000 1500 2000 2500 3000 3500 4000 4500500Training samples
596061
()
Leve
l
0 1000 1500 2000 2500 3000 3500 4000 4500500Pressurizer pressure
67675
68
Pres
sure
(BA
RG)
0 1000 1500 2000 2500 3000 3500 4000 4500500Steam generator level
500520540
(kg
s)Fl
ow
0 1000 1500 2000 2500 3000 3500 4000 4500500Steam pressure
400500600
Feedwater flow
(kg
s)Fl
ow
0 1000 1500 2000 2500 3000 3500 4000 4500500Steam flow
Figure 6 Plot of NPP sensors listed in Table 1
employs two learning stages an unsupervised learning phaseand supervised training phase DAE based greedy layerwisepretraining of each hidden layer as described in Section 4 isperformed usingminibatches from training data set Stochas-tic gradient descent based learning algorithm is employed
as suggested in practical training recommendations by [30]Finally standard backpropagation algorithm is employed forsupervised fine-tuning in fully stacked DAASM in Figure 4Supervised training is performed using clean sensor inputonly The model hyperparameters are set by random gridsearchmethod [31] A summary of the training hyperparame-ters corresponding to optimumDAASM is shown in Table 2
7 Invariance Test for Robustness
A layer by layer invariance study is conducted to test therobustness of fully trained DAASM against failed sensorstates Data corruption processes applied during pretrainingare essentially meant to learn hidden layer mappings whichare stable and invariant to faulty sensor conditions Thefollowing invariance test for successive hidden layers in finalDAASM stack can provide an insight into the effectivenessof data corruption processes exercised during denoisingbased pretraining phase Invariance for hidden layer map-pings ℎ
119897 is quantified through mean square error (MSE)
between Euclidean (1198712) normalized hidden layer activation⟨ℎ119894⟩119899⟨ℎ119894⟩1198992and ⟨ℎ
119894⟩119899⟨ℎ119894⟩1198992against clean and faulty
sensors respectively Invariance test samples are generatedby corrupting randomly selected sensors in input set withvarying level of offset failures [5ndash50] The MSE againsteach offset level is normalized across hidden layer dimension119863ℎand number of test samples 119879
119873as shown in (13) Finally
theseMSE values are normalized with maximalMSE value asin (14) Normalized MSE curves for each successive hiddenlayer are plotted in Figure 7 Consider
Science and Technology of Nuclear Installations 9
Table 2 Summary of DAASM hyperparameters
Hyperparameter type Tested hyperparameter values Successful hyperparameteragainst optimum model
Pretrained DAE units 3 3
Network architecture
119868119899minus 119864(119899 119901) minus 119861
119899minus 119863(119899 119901) minus 119900
119899
119868119899 input layer neurons
119861119899 bottleneck layer neurons119900119899 output layer neurons
119864(119899 119901) encoder cascade119863(119899 119901) decoder cascade
119899 number of layers119901 neurons per layer
13 minus 119864(2 20) minus 8 minus
119863(2 20) minus 13
Learning rate forunsupervised pretraining 01 5 times 10
minus2
1 times 10minus2
5 times 10minus3
1 times 10minus3
5 times 10minus4
[5 times 10minus2 1 times 10
minus2
]
Learning rate forsupervised training
Scheduled learning rate based on training errormonitoring
015 01 0005 0001 00001
01 0005 0001
Mean pretraining error foreach hidden layer
Corresponding to minima observed during crossvalidation 10
minus4
Weight decay 120582 10minus3
10minus4
10minus5
10minus3
Momentum 119898 [085 099] [095 098]
Input corruption level ]Corrupted input fraction 10 25 30 40
Gaussian corruption ( of sensorrsquos nominal value)005 010 020 035 050
Input fraction [25ndash35]Gaussian noise level
[010ndash025]Dropout fraction in DAE-3 010 020 01
MSE (119867119897Offset) =
(1119879119873)sum119879119873
119899=1(⟨ℎ119894⟩1198991003817100381710038171003817⟨ℎ119894⟩119899
10038171003817100381710038172minus ⟨ℎ119894⟩119899
10038171003817100381710038171003817⟨ℎ119894⟩119899
100381710038171003817100381710038172)2
119863ℎ
1 le 119897 le 119871 = No of encoder layers = 3 Offset = 5 10 20 sdot sdot sdot 50
(13)
MSENormalized (119867119897) =
MSE (119867119897Offset)
MSE (119867119897Max-Offset)
(14)
LayerwiseMSEplots in Figure 7 clearly show that invarianceto faulty sensor conditions increases towards higher layers inthe network hierarchy In these plots lower curves indicatehigher level of invariance To further investigate the effectof increasing invariance on reconstructed sensor values asensor model corresponding to the level ldquo119871rdquo of each hiddenlayer is assembled via encoder and decoder cascade Robust-ness of these partial models is quantified through (1 minus 119878
119894
Auto)Autosensitivity values 119878
119894
Auto (see Section 82) are calculatedagainst varying offset failure levels In Figure 8 layerwiseincrease in robustness confirms that increased invariancehelps in improving overall modelrsquos robustness
8 DAASM versus K-AANNPerformance Analysis
Here we will assess and compare the performance of DAASMwith popular five-layer AANNmodel originally proposed byKramer [8] The K-AANN model is trained with same data
set as used for DAASM and is regularized with Levenberg-Marquardt algorithm Furthermore to improve robustnesstraining with jitter heuristic is employed by introducing anoise of 10 magnitude on clean sensor input The five-layertopology 13-17-9-17-13 is found to be optimum for K-AANNmodel Both DAASM and K-AANN model are comparedthrough accuracy robustness spillover and fault detectabilitybased performance metrics in the following subsections Allperformance metrics are calculated against test data setconsisting of 4320 samples frompostrefueling full powerNPPoperations Performance metric values are reported
81 Accuracy Mean square error (MSE) of observed andmodel estimated sensor values against fault free test data setis used to quantify accuracy metric as follows
Accuracy =1
119873
119873
sum
119894=1
(119894minus 119878119894)2
(15)
10 Science and Technology of Nuclear Installations
Layerwise invariance
3rd hidden layer2nd hidden layer1st hidden layer
0010203040506070809
1
Nor
mal
ised
MSE
minus20 minus10minus40 minus30 0 10 20 30 40 50minus50
Offset failure level ()
Figure 7 Layerwise invariance in DAASM Lower curves depicthigher invariance
Robustness
With 3rd hidden layerWith 2nd hidden layerWith 1st hidden layer
0203040506070809
1
minus20 minus10minus40 minus30 0 10 20 30 40 50minus50
Offset failure level ()
Robu
stnes
s (1minusS a
)
Figure 8 Robustness measure (1 minus autosensitivity) at multiplehidden layers in DAASM Higher curves depict high robustness
The MSE values of all sensors are normalized to theirrespective span and are presented as percent span in Figure 9Being an error measure the lower MSE values by DAASMsignify its prediction accuracy
82 Robustness Robustness is quantified through autosensi-tivity as defined by [32 33] It is themeasure of modelrsquos abilityto predict correct sensor values under missing or corruptedsensor states The measure is averaged over an operatingregion defined by 119896 samples from test data set as follows
119878119894
Auto =1
119873
119873
sum
119896=1
1003816100381610038161003816100381610038161003816100381610038161003816
drift119896119894
minus 119896119894
119904drift119896119894
minus 119904119896119894
1003816100381610038161003816100381610038161003816100381610038161003816
(16)
where 119894 and 119896 are indexes corresponding to sensors and theirrespective test samples 119904
119896119894is the original sensor value without
fault 119896119894is the model estimated sensor value against 119904
119896119894 119904drift119896119894
is the driftedfaulted sensor value drift119896119894
is themodel estimatedsensor value against drifted value 119904
drift119896119894
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN 0382 0361 0343 0412 0186 0211 0166 0386 0411 0243 0223 0512 0621
DAASM 0281 0253 0246 0293 0121 0132 0122 0243 0315 0173 0156 0394 0465
Accuracy metric
0
01
02
03
04
05
06
07
MSE
( sp
an)
Figure 9 MSE depicting DAASM and K-AANN accuracy on eachsensor
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Autosensitivity metric
0005
01015
02025
03503
Auto
sens
itivi
ty
0273 0294 0321 0332 0205 0182 0198 0253 0233 0225 0231 0268 0301
0153 0167 0187 0163 0112 0079 0082 0131 0071 014 0117 0133 0187
Figure 10 Autosensitivity values of individual sensors in bothmodels
The autosensitivitymetric lies in [0 1] range For autosen-sitivity value of one the model predictions follow the faultwith zero residuals hence no fault can be detected Smallerautosensitivity values are preferred which essentially meansdecreased sensitivity towards small perturbations Largeautosensitivity values may lead to missed alarms due tounderestimation of the fault size caused by small residualvalues Compared to K-AANN model in case of DAASM asignificant decrease in autosensitivity values for all sensors isobserved The plot in Figure 10 shows that DAASM is morerobust to failed sensor inputs
To further investigate robustness against large offsetfailures both models are evaluated against offset failuresin [5ndash50] range For each sensor samples from testdata are corrupted with specific offset level and correspond-ing autosensitivities are averaged over whole sensor setAutosensitivity values less than 02 are considered as robustThe maximum autosensitivity value of 0187 is observed insteam flow sensor The plot in Figure 11 shows that averageautosensitivity for both models increases with increasinglevel of offset failure However the autosensitivity curve forDAASM autosensitivity is well below the corresponding K-AANN curve
83 Spillover Depending upon the size and type of failurea failed sensor input can cause discrepancy in estimatedoutput for other sensors The phenomenon is referred to in
Science and Technology of Nuclear Installations 11
0060086
014
021
028
041
016021
028
036
043
054
005 01 02 03 04 05Sensor perturbation magnitude ( of actual value)
Robustness performance
0
01
02
03
04
05
06
Avg
auto
sens
itivi
ty
DAASMK-AANN
Figure 11 Comparison of robustness against increasing offsetfailure
literature as ldquospillover effectrdquo and is quantified through ldquocross-sensitivityrdquo metric [32] It quantifies the influence of faultysensor 119894 on predictions of sensor 119895 as follows
119878119895119894
Cross =1
119873
119873
sum
119896=1
10038161003816100381610038161003816100381610038161003816100381610038161003816
drift119896119895
minus 119896119895
119904drift119896119894
minus 119904119896119894
10038161003816100381610038161003816100381610038161003816100381610038161003816
(17)
119878119895
Cross =1
119873 minus 1
119873
sum
119894=1
119878119895119894
Cross 119894 = 119895 (18)
where 119894 and 119895 indexes are used to refer to faulty andnonfaulty sensors respectively Meanwhile 119896 is the index forcorresponding test samples 119878119895119894Cross is the cross-sensitivity ofsensor 119895 with respect to drift in 119894th sensor 119904
119896119894is the value of
119894th sensor without any fault 119896119895is the model estimated value
of 119895th sensor against 119904119896119894 119904drift119896119894
is the driftedfaulted value of 119894thsensor drift
119896119895is themodel estimated value of 119895th sensor against
drifted value 119904drift119896119894
The highly distributed representation of the input in
neural network based sensor models has pronounced effecton the cross-sensitivity performance Cross-sensitivitymetricvalue lies in [0 1] range High value of cross-sensitivitymay set off false alarms in other sensors provided theresidual values overshoot the fault detectability threshold inother sensors So minimum cross-sensitivity value is desiredfor a robust model The plot in Figure 12 shows that thecross-sensitivity for DAASM is reduced by a large factor ascompared to K-AANNmodel
The spillover effect against particular level of offset failurein [5ndash50] range is averaged over all sensors as follows
AvgCross Sensitivity (119870) =
sum119873
119895=1119878119895
cross(119870)
119873
offset failure level 119870 = 5 10 20 sdot sdot sdot 50
(19)
0128 0124 0113 0116 0112 0108 0092 0079 0083 0098 011 0103 0105
00632 00527 00563 00614 0039 00311 00287 00186 00203 00344 00336 003 00286
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Cross-sensitivity metric
0002004006008
01012014
Cros
s-se
nsiti
vity
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
Figure 12 Cross-sensitivity values of individual sensors in bothmodels
00810105
0161
0253
0312
041
0018 0026 0038 00460071
012
005 01 02 03 04 05Sensor perturbation magnitude ( of observed sensor value)
Spillover offset sensor failure
Requires SPRT mean relaxation to avoid
false alarmsahead of this point
0
005
01
015
02
025
03
035
04
045
Avg
cros
s-se
nsiti
vity
DAASMK-AANN
Figure 13 Comparison of spillover effects against increasing offsetfailure
The cross-sensitivity values 119878119895
cross(119870) against 119870 offset fail-ure level are calculated using (18) Figure 13 shows theaverage cross-sensitivity plot for both models Small cross-sensitivities are observed in DAASM which effectivelyavoided false alarms in other channels without relaxing theSPRT faulted mean value up to an offset failure of 35ndash40 inany channel However for the case of offset noise larger than35 SPRTmean needs to be relaxed to avoid false alarms andisolate the faulty sensor However Robustness of K-AANNmodel deteriorates significantly due to spillover effect beyond15 offset failure
Similarly gross failure scenarios corresponding to twoextremities of sensor range can cause severe Spillover effectTo study robustness against gross type failure scenario asubset of input sensors is simultaneously failed with grosshigh or low value and average cross-sensitivity of remainingsensor set is calculated using relation (19) Plot in Figure 14shows that average cross-sensitivity of K-AANN modelincreases drastically beyond 10 gross failure HoweverDAASM resulted in a very nominal spillover even in caseof multiple sensor failure The DAASM effectively managedsimultaneous gross high or low failures in 25 of total sensorset as compared to 10 in case of K-AANN
12 Science and Technology of Nuclear Installations
0028 0032 0041 012021016 024
0457
061
077
01 02 03 04 05Sensors failed with gross highlow error ()
Spillover gross sensor failure
0010203040506070809
Avg
cros
s-se
nsiti
vity
DAASMK-AANN
Figure 14 Comparison of spillover effect against simultaneous grosshighlow failure in multiple sensors
84 Fault Detectability Fault detectability metric measuresthe smallest fault that can be detected by integrated sensorestimation and fault detection module as shown in Figure 1[32] The detectability metric is measured as percentage ofsensor span 119863 = 119872Span where value M correspondsto minimum detectable fault Minimum fault detectabilitylimit for each sensor is quantified through statistical basedsequential probability ratio test (SPRT) by Wald [34] SPRTtest is carried out to detect if the residual being generatedfrom normal distribution 119873(120583
1 1205902
) or 119873(1205830 1205902
) as definedfor faulty and fault free sensor operations respectively [35]Calibration failures are reflected in the mean parameter ofresidualrsquos distribution The SPRT procedure is applied todetect changes in the mean of residualrsquos distribution Theapplication of SPRT requires setting of following parametersvalue [36]
1205830 normal mode residual mean
1205902 normal mode residual variance
1205831 expected offset in residual mean in abnormal
mode120572 false alarm probability120573 missed alarm probability
Under normal mode the residuals from observed and modelestimated sensor values behave as awhiteGaussian noisewithmean 120583
0= 0 The residual variance 120590
2 is estimated for eachsensor under normal operating conditions and remainedfixed The false alarm 120572 and missed alarm 120573 probabilities areset to be 0001 and 001 respectively In order to determineminimum fault detectability limit a numerical procedureis opted which searches for minimum expected offset 120583
1
in the interval 1205831
[120590ndash3120590] provided the constraint onmissed and false alarm rate holds 120590 is the standard deviationcorresponding to residual variance of particular sensor Theplot in Figure 15 shows the detectability metric for eachsensor The plot in Figure 15 shows that DAASM can detect
0312 0315 033 035 004 005 004 008 0086 01 009 006 0058
0184 0165 019 02 0021 0023 002 0048 005 004 004 0032 0028
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Fault detectability metric
0005
01015
02025
03035
04
Det
ecta
bilit
y (
span
)
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
Figure 15 Comparison of fault detectability metrics
faults which are two times smaller in magnitude than thosedetectable by K-AANNmodel
Improvement in fault detectability metric for DAASMcan be attributed to observed improvement in model robust-ness as suggested by the following relation
119903119894
Δ119878drift119894
= (1 minus 119878119894
Auto) (20)
The term 119903119894Δ119878
drift119894
measures the ratio of observed residualto actual sensor drift in terms of autosensitivity For highlyrobust model this ratio reduces to one which means residualreflects the actual drift and results in high fault detectabilityContrarily ratio value close to zero means that the predictionis following the input and results in poor fault detectability
841 SPRT Based Fault Detectability Test Sequential prob-ability ratio [34 36] based fault hypothesis test is appliedto residual sequence 119877
119894 = 119903
1(1199051) 1199031(1199051)sdot sdot sdot 119903119899(119905119899)generated
by relation 119877119894(119905119894) = 119878
Obs(119905119894) minus 119878
Est(119905119894) at time 119905
119894 where
119878Obs
(119905119894) and 119878
Est(119905119894) are the actual and model predicted
sensor values respectively The SPRT procedure analyzeswhether the residual sequence is more likely to be generatedfrom a probability distribution that belongs to normal modehypothesis 119867
0or abnormal mode hypothesis 119867
1by using
likelihood ratio as follows
119871119899= exp[minus(12120590
2)[sum119899
119894=11205831(1205831minus2119903119894)]]
(21)
For fault free sensor values the normalmode hypothesis1198670is
approximated byGaussian distributionwithmean 1205830= 0 and
variance 1205902 Abnormal mode hypothesis 119867
1is approximated
with mean 1205831
gt 1205830using the same variance 120590
2 The SPRTindex for the positive mean test is finally obtained by takinglogarithm of the likelihood ratio in (21) as follows [35]
SPRT = ln (119871119899) = minus
1
21205902[
119899
sum
119894=1
1205831(1205831minus 2119903119894)]
=1205831
1205902
119899
sum
119894=1
(119903119894minus
1205831
2)
(22)
Pressurizer pressure sensor sampled at a frequency of 10 sec-onds is used as a test signal to validate the fault detectability
Science and Technology of Nuclear Installations 13
0 500 1000 1500 2000 2500 3000 3500 4000 45001535
1541545
SPRT drift detection (pressure sensor)(B
ARG
)Pr
essu
re
Drifted sensorDAASM estimate
Sample number
0 500 1000 1500 2000 2500 3000 3500 4000 45001535
1541545
(BA
RG)
Pres
sure
Drifted sensorK-AANN estimate
Sample number
0515
SPRT
hypo
thes
is
minus050 500 1000 1500 2000 2500 3000 3500 4000 4500
DAASM indexK-AANN index
Sample number
005
valu
eRe
sidua
l
0 500 1000 1500 2000 2500 3000 3500 4000 4500
Residual DAASMResidual K-AANN
Sample number
minus05
H1mdash1
H0mdash0
Figure 16 SPRTbased fault detection in pressurizer pressure sensor
performance Two drift faults at the rate of +001hour andminus001hour are introduced in the test signal for DAASMand K-AANN modelrsquos assessment respectively The first andsecond plots in Figure 16 show drifted and estimated pressuresignal from DAASM and K-AANN models respectivelyThird plot shows residual values generated by differencing thedrifted and estimated signals frombothmodelsThe final plotshows SPRT index values against residuals from K-AANNmodel and DAASM The hypotheses 119867
1and 119867
0correspond
to positive and negative fault acceptance respectively FromSPRT index plot successful early detection of the sensordrift at 2200th sample with lag of 611 hours since the driftinception shows that DAASM is more sensitive to smalldrifts On the other hand SPRT index on K-AANN basedsensor estimates registered the same drift at 3800th samplewith a lag of almost 1055 hours The result shows thatDAASM is more robust in terms of early fault detection withlow false and missed alarm rates
Finally both models are tested against five test data setsEach test set consists of 3630 samples corresponding todifferentmonths of full power reactor operation Bothmodelssuccessfully detected an offset failure of 012ndash03 BARG in allsteam pressure channels and a drift type failure up to 285 insteam generator level (Figure 22)The K-AANNmodel failedto register a very small drift up to 01 in steam flow (STMflow 1) channel A small drift up to 01 BARG is detected in
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Steam pressure estimation
Observed valueDAASM estimateK-AANN estimate
672673674675676677678679
68
Stea
m p
ress
ure (
BARG
)
Figure 17 Steam pressure estimates against offset failures up to 03(BARG)
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Feed flow estimation
400420440460480500520540560
Feed
flow
(kg
s)
Observed sensor valueK-AANN estimateDAASM estimate
Figure 18 Feed flow estimates
test set 5 of pressurizer pressure channel However in caseof drift type sensor failures fault detection lag for DAASMwas on average 05 times smaller in comparison with K-AANN model Plots in Figures 17ndash21 show the estimatedsensor values from both models on five test data sets of fewselected channels
9 Conclusion
This paper presented a neural network based denoisedautoassociative sensor model (DAASM) for empirical sen-sor modeling The proposed sensor model is trained togenerate a monitoring system for sensor fault detection innuclear power plantsMultilayer AANNbased sensormodelsmay result in suboptimal solutions due to poor regulariza-tion by traditional backpropagation based joint multilayerlearning procedures So a complementary deep learningapproach based on greedy layerwise unsupervised pretrain-ing is employed for effective regularization in the proposedmultilayer DAASM Autoencoder architecture is used fordenoising based unsupervised pretraining and regularizationof individual layers in the network hierarchy To addressrobustness against perturbations in input sensors data cor-ruption processes exercised during unsupervised pretraining
14 Science and Technology of Nuclear Installations
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5490495500505510515520525530535540
Stea
m fl
ow (k
gs)
Steam flow estimation
Observed sensor valueDAASM estimateK-AANN estimate
Figure 19 Steam flow estimates against drift failure up to 01
58859
592594596598
60602604606608
Pres
suriz
er le
vel (
)
Pressurizer level estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valueK-AANN estimateDAASM estimate
Figure 20 Pressurizer level estimates
phase were based on prior knowledge about different failurescenarios Results from invariance tests showed that theproposed data corruption schemeswere beneficial in learninglatent representations at hidden layers and were invariantto multiple levels of perturbation in input sensors Conse-quently these pretrained hidden layers worked as well regu-larized perturbation filters with increased invariance towardssensor faults It is also observed that sensitivity against sensorfaults decreased significantly towards higher layers in fullDAASMassembly In a practical context of sensormonitoringin nuclear power plants the proposed model proved itsrobustness against gross type simultaneous sensor failuresIt also showed significant improvement in all performancemetrics when compared with popular and widely used five-layered AANNmodel by KramerMoreover time lag in smalldriftrsquos detection is significantly reduced The overall resultssuggest that greedy layerwise pretraining technique in com-bination with domain specific corruption processes providesa viable framework for effective regularization and robustnessin such deep multilayered autoassociative sensor validationmodels
Appendix
See Table 3
153715375
153815385
153915395
15415405
154115415
Pres
sure
(BA
RG)
Pressurizer pressure estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valuesK-AANN estimateDAASM estimate
Figure 21 Pressurizer pressure estimates against drift failure up to01
74
76
78
80
82
84
Stea
m g
ener
ator
leve
l (
)
Steam generator level estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valueK-AANN estimateDAASM estimate
Figure 22 Steam generator level estimates against growing driftfailure up to 287
Abbreviations
AANN Autoassociative neural networkK-AANN Kramer proposed Autoassociative neural
networkDAASM Denoised autoassociative sensor modelNPP Nuclear power plantPWR Pressurized water reactor119878Auto Autosensitivity119878Cross Cross-sensitivityDAE Denoising autoencoder119878 Observed sensor value Model predicted sensor value Corrupted sensor valueSPN Salt-and-pepper noiseAGN Additive Gaussian noise
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Science and Technology of Nuclear Installations 15
Table3Re
sults
ofDAASM
andK-AANNperfo
rmance
metric
sPerfo
rmance
metric
Mod
eltype
Feed
flow1
Feed
flow2
STM
flow1
STM
flow2
STM
PSR1
STM
PSR2
STM
PSR3
PZRPS
R1
PZRPS
R2
PZRLV
L1
PZRLV
L2
SGLV
LNR1
SGLV
LNR2
AANN
Sp-D
AANN
Accuracy
(span)
0382
0361
0343
0412
0186
0211
0166
0386
0411
0243
0223
0512
0621
0281
0253
0246
0293
0121
0132
0122
0243
0315
0173
0156
0394
0465
Autosensitivity
0273
0294
0321
0332
0205
0182
0198
0253
0233
0225
0231
0268
0301
0153
0167
0187
0163
0112
0079
0082
0131
0071
014
0117
0133
0187
Cross-sensitivity
0128
0124
0113
0116
0112
0108
0092
0079
0083
0098
011
0103
0105
00632
00527
00563
00614
0039
00311
00287
00186
00203
00344
00336
003
00286
SPRT
detectability
0312
0315
033
035
004
005
004
008
0086
01
009
006
0058
0184
0165
019
02
0021
0023
002
004
8005
004
004
0032
0028
16 Science and Technology of Nuclear Installations
Acknowledgments
The authors acknowledge the support of Mr Jose GalindoRodrıguez affiliated to TECNATOM Inc Spain and MrChad Painter (Director of Nuclear Power Plant SimulatorDevelopment and Training Program at International AtomicEnergy Agency) for providing necessary tools and data toconduct this research
References
[1] N Mehranbod M Soroush M Piovoso and B A OgunnaikeldquoProbabilistic model for sensor fault detection and identifica-tionrdquo AIChE Journal vol 49 no 7 pp 1787ndash1802 2003
[2] JW Hines and E Davis ldquoLessons learned from the US nuclearpower plant on-line monitoring programsrdquo Progress in NuclearEnergy vol 46 no 3-4 pp 176ndash189 2005
[3] EPRI ldquoOn-line monitoring cost benefit guiderdquo Final Report1006777 EPRI Palo Alto Calif USA 2003
[4] S J Qin and W Li ldquoDetection identification and recon-struction of faulty sensors with maximized sensitivityrdquo AIChEJournal vol 45 no 9 pp 1963ndash1976 1999
[5] J Ma and J Jiang ldquoApplications of fault detection and diagnosismethods in nuclear power plants a reviewrdquo Progress in NuclearEnergy vol 53 no 3 pp 255ndash266 2011
[6] G YHeo ldquoConditionmonitoring using empiricalmodels tech-nical review and prospects for nuclear applicationsrdquo NuclearEngineering and Technology vol 40 no 1 pp 49ndash68 2008
[7] J B Coble R M Meyer P Ramuhalli et al ldquoA review ofsensor calibration monitoring for calibration interval extensionin nuclear power plantsrdquo Tech Rep PNNL-21687 PacificNorthwest National Laboratory Richland Wash USA 2012
[8] M A Kramer ldquoAutoassociative neural networksrdquo Computersand Chemical Engineering vol 16 no 4 pp 313ndash328 1992
[9] X Xu J W Hines and R E Uhrig ldquoOn-line sensor calibra-tion monitoring and fault detection for chemical processesrdquoin Proceedings of the Maintenance and Reliability Conference(MARCON rsquo98) pp 12ndash14 Knoxville Tenn USA May 1998
[10] M Hamidreza S Mehdi J-R Hooshang and N AliakbarldquoReconstruction based approach to sensor fault diagnosis usingauto-associative neural networksrdquo Journal of Central SouthUniversity vol 21 no 6 pp 2273ndash2281 2014
[11] UThissenW JMelssen andLMC Buydens ldquoNonlinear pro-cess monitoring using bottle-neck neural networksrdquo AnalyticaChimica Acta vol 446 no 1-2 pp 371ndash383 2001
[12] D J Wrest J W Hines and R E Uhrig ldquoInstrument surveil-lance and calibration verification through plant wide monitor-ing using autoassociative neural networksrdquo in Proceedings ofthe American Nuclear Society International Topical Meeting onNuclear Plant Instrumentation Control and Human MachineInterface Technologies University Park Pa USA May 1996
[13] J W Hines R E Uhrig and D J Wrest ldquoUse of autoassociativeneural networks for signal validationrdquo Journal of Intelligent andRobotic Systems Theory and Applications vol 21 no 2 pp 143ndash154 1998
[14] P F Fantoni M I Hoffmann R Shankar and E L Davis ldquoOn-line monitoring of instrument channel performance in nuclearpower plant using PEANOrdquo Progress in Nuclear Energy vol 43no 1ndash4 pp 83ndash89 2003
[15] MMarseguerra andA Zoia ldquoTheAutoAssociative Neural Net-work in signal analysis II Application to on-line monitoring of
a simulated BWR componentrdquo Annals of Nuclear Energy vol32 no 11 pp 1207ndash1223 2005
[16] M S Ikbal H Misra and B Yegnanarayana ldquoAnalysis ofautoassociative mapping neural networksrdquo in Proceedings of theInternational Joint Conference on Neural Networks (IJCNN rsquo99)vol 5 pp 3025ndash3029 IEEE Washington DC USA July 1999Proceedings (Cat no 99CH36339)
[17] Y Bengio ldquoLearning deep architectures for AIrdquo Foundationsand Trends in Machine Learning vol 2 no 1 pp 1ndash27 2009
[18] J W Hines A V Gribok I Attieh and R E Uhrig ldquoRegular-izationmethods for inferential sensing in nuclear power plantsrdquoin Fuzzy Systems and Soft Computing in Nuclear Engineering DRuan Ed vol 38 of Studies in Fuzziness and Soft Computingpp 285ndash314 Physica 2000
[19] A V Gribok J W Hines A Urmanov and R E UhrigldquoHeuristic systematic and informational regularization forprocessmonitoringrdquo International Journal of Intelligent Systemsvol 17 no 8 pp 723ndash749 2002
[20] X Glorot and Y Bengio ldquoUnderstanding the difficulty oftraining deep feedforward neural networksrdquo in Proceedings ofthe 13th International Conference on Artificial Intelligence andStatistics (AISTATS rsquo10) vol 9 pp 249ndash256 Sardinia Italy May2010
[21] S Tan andM L Mayrovouniotis ldquoReducing data dimensional-ity through optimizing neural network inputsrdquo AIChE Journalvol 41 no 6 pp 1471ndash1480 1995
[22] Y Bengio and P Lamblin ldquoGreedy layer-wise training of deepnetworksrdquo in Proceedings of the Advances in Neural InformationProcessing Systems 19 (NIPS rsquo07) pp 153ndash160 September 2007
[23] H Larochelle Y Bengio J Lourador and P Lamblin ldquoExplor-ing strategies for training deep neural networksrdquoThe Journal ofMachine Learning Research vol 10 pp 1ndash40 2009
[24] D Yu and L Deng ldquoDeep learning and its applications to signaland information processing [exploratory DSP]rdquo IEEE SignalProcessing Magazine vol 28 no 1 pp 145ndash154 2011
[25] D ErhanA Courville andPVincent ldquoWhydoes unsupervisedpre-training help deep learningrdquo Journal of Machine LearningResearch vol 11 pp 625ndash660 2010
[26] P Vincent H Larochelle Y Bengio and P-A ManzagolldquoExtracting and composing robust features with denoisingautoencodersrdquo in Proceedings of the 25th International Confer-ence on Machine Learning (ICML rsquo08) pp 1096ndash1103 July 2008
[27] P Vincent H Larochelle I Lajoie Y Bengio and P-AManzagol ldquoStacked denoising autoencoders learning usefulrepresentations in a deep network with a local denoisingcriterionrdquo Journal of Machine Learning Research vol 11 pp3371ndash3408 2010
[28] N Srivastava G Hinton A Krizhevsky I Sutskever and RSalakhutdinov ldquoDropout a simple way to prevent neural net-works from overfittingrdquo Journal of Machine Learning Researchvol 15 pp 1929ndash1958 2014
[29] M J Embrechts B J Hargis and J D Linton ldquoAn augmentedefficient backpropagation training strategy for deep autoasso-ciative neural networksrdquo in Proceedings of the 18th EuropeanSymposiumonArtificial NeuralNetworksmdashComputational Intel-ligence and Machine Learning (ESANN rsquo10) vol no pp 28ndash30April 2010
[30] Y Bengio ldquoPractical recommendations for gradient-basedtraining of deep architecturesrdquo inNeural Networks Tricks of theTrade vol 7700 of Lecture Notes in Computer Science pp 437ndash478 Springer Berlin Germany 2nd edition 2012
Science and Technology of Nuclear Installations 17
[31] J Bergstra and Y Bengio ldquoRandom search for hyper-parameteroptimizationrdquo Journal of Machine Learning Research vol 13 pp281ndash305 2012
[32] J W Hines and D R Garvey ldquoDevelopment and applicationof fault detectability performance metrics for instrument cali-bration verification and anomaly detectionrdquo Journal of PatternRecognition Research vol 1 no 1 pp 2ndash15 2006
[33] A Usynin and J W Hines ldquoOn-line monitoring robustnessmeasures and comparisonsrdquo in Proceedings of the Interna-tional Atomic Energy Agency Technical Meeting on ldquoIncreasingInstrument Calibration Interval through On-Line CalibrationTechnologyrdquo OECD Halden Reactor Project Halden NorwaySeptember 2004
[34] A Wald ldquoSequential tests of statistical hypothesesrdquo Annals ofMathematical Statistics vol 16 no 2 pp 117ndash186 1945
[35] F Di Maio P Baraldi E Zio and R Seraoui ldquoFault detection innuclear power plants components by a combination of statisticalmethodsrdquo IEEE Transactions on Reliability vol 62 no 4 pp833ndash845 2013
[36] S Cheng and M Pecht ldquoUsing cross-validation for modelparameter selection of sequential probability ratio testrdquo ExpertSystems with Applications vol 39 no 9 pp 8467ndash8473 2012
TribologyAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
FuelsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal ofPetroleum Engineering
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Power ElectronicsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
CombustionJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Renewable Energy
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
StructuresJournal of
International Journal of
RotatingMachinery
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporation httpwwwhindawicom
Journal ofEngineeringVolume 2014
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
International Journal ofPhotoenergy
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear InstallationsScience and Technology of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Solar EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Wind EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear EnergyInternational Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
High Energy PhysicsAdvances in
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
6 Science and Technology of Nuclear Installations
phase in Figure 3 The final network comprises one inputlayer one output and 2119871 minus 1 hidden layers The input sensorvalues flow through encoder cascade 119891 = 119891
119897
120579119900119891119897minus1
120579119900 sdot sdot sdot 119891
1
120579
using recursive expression in (7) and a decoder cascade 119892 =
1198921
1205791015840119900119892119897+1
1205791015840 119900 sdot sdot sdot 119892
119871minus1
1205791015840 using the following equations
ℎ119871
= ℎ119871
ℎ119897
= 119892119897
1205791015840 (ℎ119897+1
1198821015840
119897) = sigm(119882
1015840
119897ℎ119897+1
+ 1198871015840
119897)
1 le 119897 le 119871 minus 1 = 2
= 1198920
(ℎ1
) = 1198821015840
0ℎ1
+ 1198871015840
0
(8)
where ⟨1198821015840
1198871015840
⟩ are network weights and biases of the decoderpart in DAE The entire network is fine-tuned using asemiheuristic based ldquoAugmented Efficient BackpropagationAlgorithmrdquo proposed by Embrechts et al [29] with followingminimization objective
119871 (119878 120579) =1
119873
119873
sum
119894=1
10038171003817100381710038171003817119878 minus
10038171003817100381710038171003817
2
2
+120582
2
2119871
sum
119896=1
10038171003817100381710038171198821198961003817100381710038171003817
2
2 (9)
A 119871minus2weight decay term is added to the above loss functionfor network regularization purpose during fine-tuning phaseTo circumvent the overfitting an early stopping procedurewhich uses validation error as proxy for the generalizationperformance is used during fine-tuning phase
41 Corruption Process 120578(sdot) for Invariance For the case of cal-ibration monitoring an ideal DAASM should learn encoderand decoder functions which are invariant to failed sensorstates So during DAE based pretraining phase engineeredtransformations from prior knowledge about the involved
failure types are imposed on clean input Different datacorruption processes 120578(sdot) are devised for learning of eachsuccessive hidden layer Denoising based learning objectivedrives the hidden layermappings to get invariant against suchengineered transformations on input data It is important tounderstand that denoising based learning approach does notcorrect the faulty signal explicitly rather it seeks to extractstatistical structure among input signals which is stable andinvariant under faults and hence implicitly estimates correctvalue for faulty signal Two failure types are identified anddefined as follows
(i) Gross sensor failure it includes catastrophic sensorfailures Salt-and-pepper type corruption process inwhich a fraction ] of the input sensor set 119878 (chosenat random for each example) is set to minimum ormaximum possible value (typically 0 or 1) is selectedfor modeling gross type failure scenarios
(ii) Miscalibration sensor failure it includes drift multi-plicative and outlier type sensor failures and is mod-eled through isotropic Gaussian noise (GS) | 119909 sim
119873(119909 1205902
) Instead of selecting an arbitrarily simplenoise distribution we estimated the distribution ofsensorrsquos natural noise and exaggerated it to generatenoisy training data
We propose to distribute the denoising based invariancelearning task across multiple hidden layers in the DAASMnetwork Both gross and miscalibration noise types areequally likely to occur in the input space Gaussian typecorruption process is not suitable for input data space 119878
because of its low denoising efficiency against gross typesensor failures Contrarily salt-and-pepper type corruptionprocess covers two extremes of sensors failure range andhence provides an upper bound on perturbation due tominor offset and miscalibration type sensor failures So salt-and-pepper type corruption process is devised for DAE-1pretraining as follows
120578119878
119863( | 119878) = SPN
119894= 119904119894 119894 notin 119895
119894= 119896 119894 isin 119895 119896 = 0 1 with
Pr (0) =1
2
Pr (1) =1
2
119894=V119899
⋃
119894=1
119895119894sdotsdotsdot119873=V119899 119895
119894= [1 119899] = rand ( ) ] = input fraction 119899 = input dimension
(10)
Gross type sensor failures usually have high impact on cross-sensitivity and can trigger false alarms in other sensorsSuch high cross-sensitivity effect may affect isolation ofmiscalibration type secondary failures in other sensors Inorder tominimize the effect a corruption procedure in whichℎ1(119878) and ℎ
1() from pretrained DAE-1 are proposed as the
clean and noisy input for DAE-2 pretrainingThis corruptionmethod is more natural since it causes next hidden layermappings to get invariant against cross-sensitivity effects
and network aberrations from previous layerThe corruptionprocess is supposed to improve invariance in ℎ
2layer map-
pings against cross-sensitivity effects from gross type sensorfailures
120578ℎ2
119863(ℎ1| ℎ1()) = ℎ
119894
1= ℎ119894
1()
where = 120578119878
119863( | 119878) = SPN
(11)
Science and Technology of Nuclear Installations 7
Table 1 List of NPP sensors
Transmitter ID Transmitter name Service Units Low range High rangeFF1 Feed flow 1 Feedwater flow KGS 0 600FF2 Feed flow 2 Feedwater flow KGS 0 600SF1 STM flow 1 Steam flow KGS 0 600SF2 STM flow 2 Steam flow KGS 0 600SP1 STM PSR 1 Steam pressure BARG 0 100SP2 STM PSR 2 Steam pressure BARG 0 100SP3 STM PSR 3 Steam pressure BARG 0 100PP1 PZR PSR 1 Pressurizer pressure BARG 116 170PP2 PZR PSR 2 Pressurizer pressure BARG 116 170PL1 PZR LVL 1 Pressurizer level 0 100PL2 PZR LVL 2 Pressurizer level 0 100SGL1 SG LVL NR 1 range Steam generator level narrow 0 100SGL2 SG LVL NR 2 range Steam generator level narrow 0 100
Here ℎ119894
1(119904) corresponds to hidden layer activations against
clean sensors at the input layer while ℎ119894
1() corresponds to
hidden layer activations against partially faulted sensor setFinally to add robustness against small offset and
miscalibration type sensor failures an isotropic Gaussian
type corruption process is devised for DAE-3 pretrain-ing The corruption procedure corrupts the ℎ
2hidden
layer mappings against clean sensors at the input layeras ℎ2(ℎ1(119878)) by employing an isotropic Gaussian noise as
follows
120578ℎ3
119863(ℎ2| ℎ2(119878)) = AGN
ℎ119894
2= ℎ119894
2(ℎ119894
1(119878)) 119894 notin 119895
ℎ119894
2| ℎ119894
2sim 119873(119904 120590
2
119868) 119894 isin 119895
119894=V119899
⋃
119894=1
119895119894sdotsdotsdot119873=V119899 119895
119894= [1 119899] = rand ( ) ] = input fraction 119899 = total inputs
(12)
Finally clean input is used for the supervised fine-tuningphase in Figure 4
5 Data Set Description
Intentionally for study purposes we limited the modelingscope of DAASM to full power steady operational state It isthe common state in which NPP operates from one refuelingto the next However in practice it is not possible for NPPsystems to be in perfect steady state Reactivity inducedpower perturbations natural process fluctuations sensor andcontroller noises and so forth are some of the evident causesforNPP parameter fluctuations and are responsible for steadystate dynamics Considering that the collected data set shouldbe fairly representative of all possible steady state dynamicsand noise the selected sensors are sampled during differenttime spans of one complete operating cycleThe training dataset consists of 6104 samples collected during the first twomonths of full power reactor operations after refueling cycleMeanwhile 3260 and 2616 samples are reserved for validationand test data sets respectively Five test data sets are used formodelrsquos performance evaluation Each test data set consistsof 4360 samples collected during eight-month period afterrefueling operation In order to account for fault propagation
phenomenon due to large signal groups a sensor subset isselected for this study An engineering sense selection basedon physical proximity and functional correlation is used todefine the sensor subset for this study Thirteen transmittersas listed in Table 1 are selected from various services innuclear steam supply systemof a real PWR typeNPP Figure 5shows the spatial distribution of the selected sensors
Starting from postrefueling full power startup the dataset covers approximately one year of selected sensors valuesSelected sensors are sampled every 10 seconds for consecutive12-hour time window Figure 6 shows data plot from fewselected sensors
6 Model Training
NPP sensorrsquos data are divided into training test and valida-tion set Each sensor data set is scaled in 01 to 09 ranges byusing lower and upper extremities corresponding to individ-ual sensor However the values 0 and 1 are explicitly reservedfor gross and saturation type sensor failures respectivelyTraining data consists of 4320 samples from full power steadystate reactor operation Meanwhile test and validation dataare used for sensor model optimization and performanceevaluation respectively The training setup for DAASM
8 Science and Technology of Nuclear Installations
Reactor vessel
PP1
PL1 PL2
Pressurizer
SGL2
SF1
FF1
Steam generator
Coolant pump
SGL1
FF2
PP2
SP1 SP2 SP3
SF2
Figure 5 Spatial distribution of selected sensor set
Pres
sure
(BA
RG)
1535154
1545
0 1000 1500 2000 2500 3000 3500 4000 4500500Training samples
596061
()
Leve
l
0 1000 1500 2000 2500 3000 3500 4000 4500500Pressurizer pressure
67675
68
Pres
sure
(BA
RG)
0 1000 1500 2000 2500 3000 3500 4000 4500500Steam generator level
500520540
(kg
s)Fl
ow
0 1000 1500 2000 2500 3000 3500 4000 4500500Steam pressure
400500600
Feedwater flow
(kg
s)Fl
ow
0 1000 1500 2000 2500 3000 3500 4000 4500500Steam flow
Figure 6 Plot of NPP sensors listed in Table 1
employs two learning stages an unsupervised learning phaseand supervised training phase DAE based greedy layerwisepretraining of each hidden layer as described in Section 4 isperformed usingminibatches from training data set Stochas-tic gradient descent based learning algorithm is employed
as suggested in practical training recommendations by [30]Finally standard backpropagation algorithm is employed forsupervised fine-tuning in fully stacked DAASM in Figure 4Supervised training is performed using clean sensor inputonly The model hyperparameters are set by random gridsearchmethod [31] A summary of the training hyperparame-ters corresponding to optimumDAASM is shown in Table 2
7 Invariance Test for Robustness
A layer by layer invariance study is conducted to test therobustness of fully trained DAASM against failed sensorstates Data corruption processes applied during pretrainingare essentially meant to learn hidden layer mappings whichare stable and invariant to faulty sensor conditions Thefollowing invariance test for successive hidden layers in finalDAASM stack can provide an insight into the effectivenessof data corruption processes exercised during denoisingbased pretraining phase Invariance for hidden layer map-pings ℎ
119897 is quantified through mean square error (MSE)
between Euclidean (1198712) normalized hidden layer activation⟨ℎ119894⟩119899⟨ℎ119894⟩1198992and ⟨ℎ
119894⟩119899⟨ℎ119894⟩1198992against clean and faulty
sensors respectively Invariance test samples are generatedby corrupting randomly selected sensors in input set withvarying level of offset failures [5ndash50] The MSE againsteach offset level is normalized across hidden layer dimension119863ℎand number of test samples 119879
119873as shown in (13) Finally
theseMSE values are normalized with maximalMSE value asin (14) Normalized MSE curves for each successive hiddenlayer are plotted in Figure 7 Consider
Science and Technology of Nuclear Installations 9
Table 2 Summary of DAASM hyperparameters
Hyperparameter type Tested hyperparameter values Successful hyperparameteragainst optimum model
Pretrained DAE units 3 3
Network architecture
119868119899minus 119864(119899 119901) minus 119861
119899minus 119863(119899 119901) minus 119900
119899
119868119899 input layer neurons
119861119899 bottleneck layer neurons119900119899 output layer neurons
119864(119899 119901) encoder cascade119863(119899 119901) decoder cascade
119899 number of layers119901 neurons per layer
13 minus 119864(2 20) minus 8 minus
119863(2 20) minus 13
Learning rate forunsupervised pretraining 01 5 times 10
minus2
1 times 10minus2
5 times 10minus3
1 times 10minus3
5 times 10minus4
[5 times 10minus2 1 times 10
minus2
]
Learning rate forsupervised training
Scheduled learning rate based on training errormonitoring
015 01 0005 0001 00001
01 0005 0001
Mean pretraining error foreach hidden layer
Corresponding to minima observed during crossvalidation 10
minus4
Weight decay 120582 10minus3
10minus4
10minus5
10minus3
Momentum 119898 [085 099] [095 098]
Input corruption level ]Corrupted input fraction 10 25 30 40
Gaussian corruption ( of sensorrsquos nominal value)005 010 020 035 050
Input fraction [25ndash35]Gaussian noise level
[010ndash025]Dropout fraction in DAE-3 010 020 01
MSE (119867119897Offset) =
(1119879119873)sum119879119873
119899=1(⟨ℎ119894⟩1198991003817100381710038171003817⟨ℎ119894⟩119899
10038171003817100381710038172minus ⟨ℎ119894⟩119899
10038171003817100381710038171003817⟨ℎ119894⟩119899
100381710038171003817100381710038172)2
119863ℎ
1 le 119897 le 119871 = No of encoder layers = 3 Offset = 5 10 20 sdot sdot sdot 50
(13)
MSENormalized (119867119897) =
MSE (119867119897Offset)
MSE (119867119897Max-Offset)
(14)
LayerwiseMSEplots in Figure 7 clearly show that invarianceto faulty sensor conditions increases towards higher layers inthe network hierarchy In these plots lower curves indicatehigher level of invariance To further investigate the effectof increasing invariance on reconstructed sensor values asensor model corresponding to the level ldquo119871rdquo of each hiddenlayer is assembled via encoder and decoder cascade Robust-ness of these partial models is quantified through (1 minus 119878
119894
Auto)Autosensitivity values 119878
119894
Auto (see Section 82) are calculatedagainst varying offset failure levels In Figure 8 layerwiseincrease in robustness confirms that increased invariancehelps in improving overall modelrsquos robustness
8 DAASM versus K-AANNPerformance Analysis
Here we will assess and compare the performance of DAASMwith popular five-layer AANNmodel originally proposed byKramer [8] The K-AANN model is trained with same data
set as used for DAASM and is regularized with Levenberg-Marquardt algorithm Furthermore to improve robustnesstraining with jitter heuristic is employed by introducing anoise of 10 magnitude on clean sensor input The five-layertopology 13-17-9-17-13 is found to be optimum for K-AANNmodel Both DAASM and K-AANN model are comparedthrough accuracy robustness spillover and fault detectabilitybased performance metrics in the following subsections Allperformance metrics are calculated against test data setconsisting of 4320 samples frompostrefueling full powerNPPoperations Performance metric values are reported
81 Accuracy Mean square error (MSE) of observed andmodel estimated sensor values against fault free test data setis used to quantify accuracy metric as follows
Accuracy =1
119873
119873
sum
119894=1
(119894minus 119878119894)2
(15)
10 Science and Technology of Nuclear Installations
Layerwise invariance
3rd hidden layer2nd hidden layer1st hidden layer
0010203040506070809
1
Nor
mal
ised
MSE
minus20 minus10minus40 minus30 0 10 20 30 40 50minus50
Offset failure level ()
Figure 7 Layerwise invariance in DAASM Lower curves depicthigher invariance
Robustness
With 3rd hidden layerWith 2nd hidden layerWith 1st hidden layer
0203040506070809
1
minus20 minus10minus40 minus30 0 10 20 30 40 50minus50
Offset failure level ()
Robu
stnes
s (1minusS a
)
Figure 8 Robustness measure (1 minus autosensitivity) at multiplehidden layers in DAASM Higher curves depict high robustness
The MSE values of all sensors are normalized to theirrespective span and are presented as percent span in Figure 9Being an error measure the lower MSE values by DAASMsignify its prediction accuracy
82 Robustness Robustness is quantified through autosensi-tivity as defined by [32 33] It is themeasure of modelrsquos abilityto predict correct sensor values under missing or corruptedsensor states The measure is averaged over an operatingregion defined by 119896 samples from test data set as follows
119878119894
Auto =1
119873
119873
sum
119896=1
1003816100381610038161003816100381610038161003816100381610038161003816
drift119896119894
minus 119896119894
119904drift119896119894
minus 119904119896119894
1003816100381610038161003816100381610038161003816100381610038161003816
(16)
where 119894 and 119896 are indexes corresponding to sensors and theirrespective test samples 119904
119896119894is the original sensor value without
fault 119896119894is the model estimated sensor value against 119904
119896119894 119904drift119896119894
is the driftedfaulted sensor value drift119896119894
is themodel estimatedsensor value against drifted value 119904
drift119896119894
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN 0382 0361 0343 0412 0186 0211 0166 0386 0411 0243 0223 0512 0621
DAASM 0281 0253 0246 0293 0121 0132 0122 0243 0315 0173 0156 0394 0465
Accuracy metric
0
01
02
03
04
05
06
07
MSE
( sp
an)
Figure 9 MSE depicting DAASM and K-AANN accuracy on eachsensor
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Autosensitivity metric
0005
01015
02025
03503
Auto
sens
itivi
ty
0273 0294 0321 0332 0205 0182 0198 0253 0233 0225 0231 0268 0301
0153 0167 0187 0163 0112 0079 0082 0131 0071 014 0117 0133 0187
Figure 10 Autosensitivity values of individual sensors in bothmodels
The autosensitivitymetric lies in [0 1] range For autosen-sitivity value of one the model predictions follow the faultwith zero residuals hence no fault can be detected Smallerautosensitivity values are preferred which essentially meansdecreased sensitivity towards small perturbations Largeautosensitivity values may lead to missed alarms due tounderestimation of the fault size caused by small residualvalues Compared to K-AANN model in case of DAASM asignificant decrease in autosensitivity values for all sensors isobserved The plot in Figure 10 shows that DAASM is morerobust to failed sensor inputs
To further investigate robustness against large offsetfailures both models are evaluated against offset failuresin [5ndash50] range For each sensor samples from testdata are corrupted with specific offset level and correspond-ing autosensitivities are averaged over whole sensor setAutosensitivity values less than 02 are considered as robustThe maximum autosensitivity value of 0187 is observed insteam flow sensor The plot in Figure 11 shows that averageautosensitivity for both models increases with increasinglevel of offset failure However the autosensitivity curve forDAASM autosensitivity is well below the corresponding K-AANN curve
83 Spillover Depending upon the size and type of failurea failed sensor input can cause discrepancy in estimatedoutput for other sensors The phenomenon is referred to in
Science and Technology of Nuclear Installations 11
0060086
014
021
028
041
016021
028
036
043
054
005 01 02 03 04 05Sensor perturbation magnitude ( of actual value)
Robustness performance
0
01
02
03
04
05
06
Avg
auto
sens
itivi
ty
DAASMK-AANN
Figure 11 Comparison of robustness against increasing offsetfailure
literature as ldquospillover effectrdquo and is quantified through ldquocross-sensitivityrdquo metric [32] It quantifies the influence of faultysensor 119894 on predictions of sensor 119895 as follows
119878119895119894
Cross =1
119873
119873
sum
119896=1
10038161003816100381610038161003816100381610038161003816100381610038161003816
drift119896119895
minus 119896119895
119904drift119896119894
minus 119904119896119894
10038161003816100381610038161003816100381610038161003816100381610038161003816
(17)
119878119895
Cross =1
119873 minus 1
119873
sum
119894=1
119878119895119894
Cross 119894 = 119895 (18)
where 119894 and 119895 indexes are used to refer to faulty andnonfaulty sensors respectively Meanwhile 119896 is the index forcorresponding test samples 119878119895119894Cross is the cross-sensitivity ofsensor 119895 with respect to drift in 119894th sensor 119904
119896119894is the value of
119894th sensor without any fault 119896119895is the model estimated value
of 119895th sensor against 119904119896119894 119904drift119896119894
is the driftedfaulted value of 119894thsensor drift
119896119895is themodel estimated value of 119895th sensor against
drifted value 119904drift119896119894
The highly distributed representation of the input in
neural network based sensor models has pronounced effecton the cross-sensitivity performance Cross-sensitivitymetricvalue lies in [0 1] range High value of cross-sensitivitymay set off false alarms in other sensors provided theresidual values overshoot the fault detectability threshold inother sensors So minimum cross-sensitivity value is desiredfor a robust model The plot in Figure 12 shows that thecross-sensitivity for DAASM is reduced by a large factor ascompared to K-AANNmodel
The spillover effect against particular level of offset failurein [5ndash50] range is averaged over all sensors as follows
AvgCross Sensitivity (119870) =
sum119873
119895=1119878119895
cross(119870)
119873
offset failure level 119870 = 5 10 20 sdot sdot sdot 50
(19)
0128 0124 0113 0116 0112 0108 0092 0079 0083 0098 011 0103 0105
00632 00527 00563 00614 0039 00311 00287 00186 00203 00344 00336 003 00286
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Cross-sensitivity metric
0002004006008
01012014
Cros
s-se
nsiti
vity
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
Figure 12 Cross-sensitivity values of individual sensors in bothmodels
00810105
0161
0253
0312
041
0018 0026 0038 00460071
012
005 01 02 03 04 05Sensor perturbation magnitude ( of observed sensor value)
Spillover offset sensor failure
Requires SPRT mean relaxation to avoid
false alarmsahead of this point
0
005
01
015
02
025
03
035
04
045
Avg
cros
s-se
nsiti
vity
DAASMK-AANN
Figure 13 Comparison of spillover effects against increasing offsetfailure
The cross-sensitivity values 119878119895
cross(119870) against 119870 offset fail-ure level are calculated using (18) Figure 13 shows theaverage cross-sensitivity plot for both models Small cross-sensitivities are observed in DAASM which effectivelyavoided false alarms in other channels without relaxing theSPRT faulted mean value up to an offset failure of 35ndash40 inany channel However for the case of offset noise larger than35 SPRTmean needs to be relaxed to avoid false alarms andisolate the faulty sensor However Robustness of K-AANNmodel deteriorates significantly due to spillover effect beyond15 offset failure
Similarly gross failure scenarios corresponding to twoextremities of sensor range can cause severe Spillover effectTo study robustness against gross type failure scenario asubset of input sensors is simultaneously failed with grosshigh or low value and average cross-sensitivity of remainingsensor set is calculated using relation (19) Plot in Figure 14shows that average cross-sensitivity of K-AANN modelincreases drastically beyond 10 gross failure HoweverDAASM resulted in a very nominal spillover even in caseof multiple sensor failure The DAASM effectively managedsimultaneous gross high or low failures in 25 of total sensorset as compared to 10 in case of K-AANN
12 Science and Technology of Nuclear Installations
0028 0032 0041 012021016 024
0457
061
077
01 02 03 04 05Sensors failed with gross highlow error ()
Spillover gross sensor failure
0010203040506070809
Avg
cros
s-se
nsiti
vity
DAASMK-AANN
Figure 14 Comparison of spillover effect against simultaneous grosshighlow failure in multiple sensors
84 Fault Detectability Fault detectability metric measuresthe smallest fault that can be detected by integrated sensorestimation and fault detection module as shown in Figure 1[32] The detectability metric is measured as percentage ofsensor span 119863 = 119872Span where value M correspondsto minimum detectable fault Minimum fault detectabilitylimit for each sensor is quantified through statistical basedsequential probability ratio test (SPRT) by Wald [34] SPRTtest is carried out to detect if the residual being generatedfrom normal distribution 119873(120583
1 1205902
) or 119873(1205830 1205902
) as definedfor faulty and fault free sensor operations respectively [35]Calibration failures are reflected in the mean parameter ofresidualrsquos distribution The SPRT procedure is applied todetect changes in the mean of residualrsquos distribution Theapplication of SPRT requires setting of following parametersvalue [36]
1205830 normal mode residual mean
1205902 normal mode residual variance
1205831 expected offset in residual mean in abnormal
mode120572 false alarm probability120573 missed alarm probability
Under normal mode the residuals from observed and modelestimated sensor values behave as awhiteGaussian noisewithmean 120583
0= 0 The residual variance 120590
2 is estimated for eachsensor under normal operating conditions and remainedfixed The false alarm 120572 and missed alarm 120573 probabilities areset to be 0001 and 001 respectively In order to determineminimum fault detectability limit a numerical procedureis opted which searches for minimum expected offset 120583
1
in the interval 1205831
[120590ndash3120590] provided the constraint onmissed and false alarm rate holds 120590 is the standard deviationcorresponding to residual variance of particular sensor Theplot in Figure 15 shows the detectability metric for eachsensor The plot in Figure 15 shows that DAASM can detect
0312 0315 033 035 004 005 004 008 0086 01 009 006 0058
0184 0165 019 02 0021 0023 002 0048 005 004 004 0032 0028
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Fault detectability metric
0005
01015
02025
03035
04
Det
ecta
bilit
y (
span
)
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
Figure 15 Comparison of fault detectability metrics
faults which are two times smaller in magnitude than thosedetectable by K-AANNmodel
Improvement in fault detectability metric for DAASMcan be attributed to observed improvement in model robust-ness as suggested by the following relation
119903119894
Δ119878drift119894
= (1 minus 119878119894
Auto) (20)
The term 119903119894Δ119878
drift119894
measures the ratio of observed residualto actual sensor drift in terms of autosensitivity For highlyrobust model this ratio reduces to one which means residualreflects the actual drift and results in high fault detectabilityContrarily ratio value close to zero means that the predictionis following the input and results in poor fault detectability
841 SPRT Based Fault Detectability Test Sequential prob-ability ratio [34 36] based fault hypothesis test is appliedto residual sequence 119877
119894 = 119903
1(1199051) 1199031(1199051)sdot sdot sdot 119903119899(119905119899)generated
by relation 119877119894(119905119894) = 119878
Obs(119905119894) minus 119878
Est(119905119894) at time 119905
119894 where
119878Obs
(119905119894) and 119878
Est(119905119894) are the actual and model predicted
sensor values respectively The SPRT procedure analyzeswhether the residual sequence is more likely to be generatedfrom a probability distribution that belongs to normal modehypothesis 119867
0or abnormal mode hypothesis 119867
1by using
likelihood ratio as follows
119871119899= exp[minus(12120590
2)[sum119899
119894=11205831(1205831minus2119903119894)]]
(21)
For fault free sensor values the normalmode hypothesis1198670is
approximated byGaussian distributionwithmean 1205830= 0 and
variance 1205902 Abnormal mode hypothesis 119867
1is approximated
with mean 1205831
gt 1205830using the same variance 120590
2 The SPRTindex for the positive mean test is finally obtained by takinglogarithm of the likelihood ratio in (21) as follows [35]
SPRT = ln (119871119899) = minus
1
21205902[
119899
sum
119894=1
1205831(1205831minus 2119903119894)]
=1205831
1205902
119899
sum
119894=1
(119903119894minus
1205831
2)
(22)
Pressurizer pressure sensor sampled at a frequency of 10 sec-onds is used as a test signal to validate the fault detectability
Science and Technology of Nuclear Installations 13
0 500 1000 1500 2000 2500 3000 3500 4000 45001535
1541545
SPRT drift detection (pressure sensor)(B
ARG
)Pr
essu
re
Drifted sensorDAASM estimate
Sample number
0 500 1000 1500 2000 2500 3000 3500 4000 45001535
1541545
(BA
RG)
Pres
sure
Drifted sensorK-AANN estimate
Sample number
0515
SPRT
hypo
thes
is
minus050 500 1000 1500 2000 2500 3000 3500 4000 4500
DAASM indexK-AANN index
Sample number
005
valu
eRe
sidua
l
0 500 1000 1500 2000 2500 3000 3500 4000 4500
Residual DAASMResidual K-AANN
Sample number
minus05
H1mdash1
H0mdash0
Figure 16 SPRTbased fault detection in pressurizer pressure sensor
performance Two drift faults at the rate of +001hour andminus001hour are introduced in the test signal for DAASMand K-AANN modelrsquos assessment respectively The first andsecond plots in Figure 16 show drifted and estimated pressuresignal from DAASM and K-AANN models respectivelyThird plot shows residual values generated by differencing thedrifted and estimated signals frombothmodelsThe final plotshows SPRT index values against residuals from K-AANNmodel and DAASM The hypotheses 119867
1and 119867
0correspond
to positive and negative fault acceptance respectively FromSPRT index plot successful early detection of the sensordrift at 2200th sample with lag of 611 hours since the driftinception shows that DAASM is more sensitive to smalldrifts On the other hand SPRT index on K-AANN basedsensor estimates registered the same drift at 3800th samplewith a lag of almost 1055 hours The result shows thatDAASM is more robust in terms of early fault detection withlow false and missed alarm rates
Finally both models are tested against five test data setsEach test set consists of 3630 samples corresponding todifferentmonths of full power reactor operation Bothmodelssuccessfully detected an offset failure of 012ndash03 BARG in allsteam pressure channels and a drift type failure up to 285 insteam generator level (Figure 22)The K-AANNmodel failedto register a very small drift up to 01 in steam flow (STMflow 1) channel A small drift up to 01 BARG is detected in
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Steam pressure estimation
Observed valueDAASM estimateK-AANN estimate
672673674675676677678679
68
Stea
m p
ress
ure (
BARG
)
Figure 17 Steam pressure estimates against offset failures up to 03(BARG)
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Feed flow estimation
400420440460480500520540560
Feed
flow
(kg
s)
Observed sensor valueK-AANN estimateDAASM estimate
Figure 18 Feed flow estimates
test set 5 of pressurizer pressure channel However in caseof drift type sensor failures fault detection lag for DAASMwas on average 05 times smaller in comparison with K-AANN model Plots in Figures 17ndash21 show the estimatedsensor values from both models on five test data sets of fewselected channels
9 Conclusion
This paper presented a neural network based denoisedautoassociative sensor model (DAASM) for empirical sen-sor modeling The proposed sensor model is trained togenerate a monitoring system for sensor fault detection innuclear power plantsMultilayer AANNbased sensormodelsmay result in suboptimal solutions due to poor regulariza-tion by traditional backpropagation based joint multilayerlearning procedures So a complementary deep learningapproach based on greedy layerwise unsupervised pretrain-ing is employed for effective regularization in the proposedmultilayer DAASM Autoencoder architecture is used fordenoising based unsupervised pretraining and regularizationof individual layers in the network hierarchy To addressrobustness against perturbations in input sensors data cor-ruption processes exercised during unsupervised pretraining
14 Science and Technology of Nuclear Installations
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5490495500505510515520525530535540
Stea
m fl
ow (k
gs)
Steam flow estimation
Observed sensor valueDAASM estimateK-AANN estimate
Figure 19 Steam flow estimates against drift failure up to 01
58859
592594596598
60602604606608
Pres
suriz
er le
vel (
)
Pressurizer level estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valueK-AANN estimateDAASM estimate
Figure 20 Pressurizer level estimates
phase were based on prior knowledge about different failurescenarios Results from invariance tests showed that theproposed data corruption schemeswere beneficial in learninglatent representations at hidden layers and were invariantto multiple levels of perturbation in input sensors Conse-quently these pretrained hidden layers worked as well regu-larized perturbation filters with increased invariance towardssensor faults It is also observed that sensitivity against sensorfaults decreased significantly towards higher layers in fullDAASMassembly In a practical context of sensormonitoringin nuclear power plants the proposed model proved itsrobustness against gross type simultaneous sensor failuresIt also showed significant improvement in all performancemetrics when compared with popular and widely used five-layered AANNmodel by KramerMoreover time lag in smalldriftrsquos detection is significantly reduced The overall resultssuggest that greedy layerwise pretraining technique in com-bination with domain specific corruption processes providesa viable framework for effective regularization and robustnessin such deep multilayered autoassociative sensor validationmodels
Appendix
See Table 3
153715375
153815385
153915395
15415405
154115415
Pres
sure
(BA
RG)
Pressurizer pressure estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valuesK-AANN estimateDAASM estimate
Figure 21 Pressurizer pressure estimates against drift failure up to01
74
76
78
80
82
84
Stea
m g
ener
ator
leve
l (
)
Steam generator level estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valueK-AANN estimateDAASM estimate
Figure 22 Steam generator level estimates against growing driftfailure up to 287
Abbreviations
AANN Autoassociative neural networkK-AANN Kramer proposed Autoassociative neural
networkDAASM Denoised autoassociative sensor modelNPP Nuclear power plantPWR Pressurized water reactor119878Auto Autosensitivity119878Cross Cross-sensitivityDAE Denoising autoencoder119878 Observed sensor value Model predicted sensor value Corrupted sensor valueSPN Salt-and-pepper noiseAGN Additive Gaussian noise
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Science and Technology of Nuclear Installations 15
Table3Re
sults
ofDAASM
andK-AANNperfo
rmance
metric
sPerfo
rmance
metric
Mod
eltype
Feed
flow1
Feed
flow2
STM
flow1
STM
flow2
STM
PSR1
STM
PSR2
STM
PSR3
PZRPS
R1
PZRPS
R2
PZRLV
L1
PZRLV
L2
SGLV
LNR1
SGLV
LNR2
AANN
Sp-D
AANN
Accuracy
(span)
0382
0361
0343
0412
0186
0211
0166
0386
0411
0243
0223
0512
0621
0281
0253
0246
0293
0121
0132
0122
0243
0315
0173
0156
0394
0465
Autosensitivity
0273
0294
0321
0332
0205
0182
0198
0253
0233
0225
0231
0268
0301
0153
0167
0187
0163
0112
0079
0082
0131
0071
014
0117
0133
0187
Cross-sensitivity
0128
0124
0113
0116
0112
0108
0092
0079
0083
0098
011
0103
0105
00632
00527
00563
00614
0039
00311
00287
00186
00203
00344
00336
003
00286
SPRT
detectability
0312
0315
033
035
004
005
004
008
0086
01
009
006
0058
0184
0165
019
02
0021
0023
002
004
8005
004
004
0032
0028
16 Science and Technology of Nuclear Installations
Acknowledgments
The authors acknowledge the support of Mr Jose GalindoRodrıguez affiliated to TECNATOM Inc Spain and MrChad Painter (Director of Nuclear Power Plant SimulatorDevelopment and Training Program at International AtomicEnergy Agency) for providing necessary tools and data toconduct this research
References
[1] N Mehranbod M Soroush M Piovoso and B A OgunnaikeldquoProbabilistic model for sensor fault detection and identifica-tionrdquo AIChE Journal vol 49 no 7 pp 1787ndash1802 2003
[2] JW Hines and E Davis ldquoLessons learned from the US nuclearpower plant on-line monitoring programsrdquo Progress in NuclearEnergy vol 46 no 3-4 pp 176ndash189 2005
[3] EPRI ldquoOn-line monitoring cost benefit guiderdquo Final Report1006777 EPRI Palo Alto Calif USA 2003
[4] S J Qin and W Li ldquoDetection identification and recon-struction of faulty sensors with maximized sensitivityrdquo AIChEJournal vol 45 no 9 pp 1963ndash1976 1999
[5] J Ma and J Jiang ldquoApplications of fault detection and diagnosismethods in nuclear power plants a reviewrdquo Progress in NuclearEnergy vol 53 no 3 pp 255ndash266 2011
[6] G YHeo ldquoConditionmonitoring using empiricalmodels tech-nical review and prospects for nuclear applicationsrdquo NuclearEngineering and Technology vol 40 no 1 pp 49ndash68 2008
[7] J B Coble R M Meyer P Ramuhalli et al ldquoA review ofsensor calibration monitoring for calibration interval extensionin nuclear power plantsrdquo Tech Rep PNNL-21687 PacificNorthwest National Laboratory Richland Wash USA 2012
[8] M A Kramer ldquoAutoassociative neural networksrdquo Computersand Chemical Engineering vol 16 no 4 pp 313ndash328 1992
[9] X Xu J W Hines and R E Uhrig ldquoOn-line sensor calibra-tion monitoring and fault detection for chemical processesrdquoin Proceedings of the Maintenance and Reliability Conference(MARCON rsquo98) pp 12ndash14 Knoxville Tenn USA May 1998
[10] M Hamidreza S Mehdi J-R Hooshang and N AliakbarldquoReconstruction based approach to sensor fault diagnosis usingauto-associative neural networksrdquo Journal of Central SouthUniversity vol 21 no 6 pp 2273ndash2281 2014
[11] UThissenW JMelssen andLMC Buydens ldquoNonlinear pro-cess monitoring using bottle-neck neural networksrdquo AnalyticaChimica Acta vol 446 no 1-2 pp 371ndash383 2001
[12] D J Wrest J W Hines and R E Uhrig ldquoInstrument surveil-lance and calibration verification through plant wide monitor-ing using autoassociative neural networksrdquo in Proceedings ofthe American Nuclear Society International Topical Meeting onNuclear Plant Instrumentation Control and Human MachineInterface Technologies University Park Pa USA May 1996
[13] J W Hines R E Uhrig and D J Wrest ldquoUse of autoassociativeneural networks for signal validationrdquo Journal of Intelligent andRobotic Systems Theory and Applications vol 21 no 2 pp 143ndash154 1998
[14] P F Fantoni M I Hoffmann R Shankar and E L Davis ldquoOn-line monitoring of instrument channel performance in nuclearpower plant using PEANOrdquo Progress in Nuclear Energy vol 43no 1ndash4 pp 83ndash89 2003
[15] MMarseguerra andA Zoia ldquoTheAutoAssociative Neural Net-work in signal analysis II Application to on-line monitoring of
a simulated BWR componentrdquo Annals of Nuclear Energy vol32 no 11 pp 1207ndash1223 2005
[16] M S Ikbal H Misra and B Yegnanarayana ldquoAnalysis ofautoassociative mapping neural networksrdquo in Proceedings of theInternational Joint Conference on Neural Networks (IJCNN rsquo99)vol 5 pp 3025ndash3029 IEEE Washington DC USA July 1999Proceedings (Cat no 99CH36339)
[17] Y Bengio ldquoLearning deep architectures for AIrdquo Foundationsand Trends in Machine Learning vol 2 no 1 pp 1ndash27 2009
[18] J W Hines A V Gribok I Attieh and R E Uhrig ldquoRegular-izationmethods for inferential sensing in nuclear power plantsrdquoin Fuzzy Systems and Soft Computing in Nuclear Engineering DRuan Ed vol 38 of Studies in Fuzziness and Soft Computingpp 285ndash314 Physica 2000
[19] A V Gribok J W Hines A Urmanov and R E UhrigldquoHeuristic systematic and informational regularization forprocessmonitoringrdquo International Journal of Intelligent Systemsvol 17 no 8 pp 723ndash749 2002
[20] X Glorot and Y Bengio ldquoUnderstanding the difficulty oftraining deep feedforward neural networksrdquo in Proceedings ofthe 13th International Conference on Artificial Intelligence andStatistics (AISTATS rsquo10) vol 9 pp 249ndash256 Sardinia Italy May2010
[21] S Tan andM L Mayrovouniotis ldquoReducing data dimensional-ity through optimizing neural network inputsrdquo AIChE Journalvol 41 no 6 pp 1471ndash1480 1995
[22] Y Bengio and P Lamblin ldquoGreedy layer-wise training of deepnetworksrdquo in Proceedings of the Advances in Neural InformationProcessing Systems 19 (NIPS rsquo07) pp 153ndash160 September 2007
[23] H Larochelle Y Bengio J Lourador and P Lamblin ldquoExplor-ing strategies for training deep neural networksrdquoThe Journal ofMachine Learning Research vol 10 pp 1ndash40 2009
[24] D Yu and L Deng ldquoDeep learning and its applications to signaland information processing [exploratory DSP]rdquo IEEE SignalProcessing Magazine vol 28 no 1 pp 145ndash154 2011
[25] D ErhanA Courville andPVincent ldquoWhydoes unsupervisedpre-training help deep learningrdquo Journal of Machine LearningResearch vol 11 pp 625ndash660 2010
[26] P Vincent H Larochelle Y Bengio and P-A ManzagolldquoExtracting and composing robust features with denoisingautoencodersrdquo in Proceedings of the 25th International Confer-ence on Machine Learning (ICML rsquo08) pp 1096ndash1103 July 2008
[27] P Vincent H Larochelle I Lajoie Y Bengio and P-AManzagol ldquoStacked denoising autoencoders learning usefulrepresentations in a deep network with a local denoisingcriterionrdquo Journal of Machine Learning Research vol 11 pp3371ndash3408 2010
[28] N Srivastava G Hinton A Krizhevsky I Sutskever and RSalakhutdinov ldquoDropout a simple way to prevent neural net-works from overfittingrdquo Journal of Machine Learning Researchvol 15 pp 1929ndash1958 2014
[29] M J Embrechts B J Hargis and J D Linton ldquoAn augmentedefficient backpropagation training strategy for deep autoasso-ciative neural networksrdquo in Proceedings of the 18th EuropeanSymposiumonArtificial NeuralNetworksmdashComputational Intel-ligence and Machine Learning (ESANN rsquo10) vol no pp 28ndash30April 2010
[30] Y Bengio ldquoPractical recommendations for gradient-basedtraining of deep architecturesrdquo inNeural Networks Tricks of theTrade vol 7700 of Lecture Notes in Computer Science pp 437ndash478 Springer Berlin Germany 2nd edition 2012
Science and Technology of Nuclear Installations 17
[31] J Bergstra and Y Bengio ldquoRandom search for hyper-parameteroptimizationrdquo Journal of Machine Learning Research vol 13 pp281ndash305 2012
[32] J W Hines and D R Garvey ldquoDevelopment and applicationof fault detectability performance metrics for instrument cali-bration verification and anomaly detectionrdquo Journal of PatternRecognition Research vol 1 no 1 pp 2ndash15 2006
[33] A Usynin and J W Hines ldquoOn-line monitoring robustnessmeasures and comparisonsrdquo in Proceedings of the Interna-tional Atomic Energy Agency Technical Meeting on ldquoIncreasingInstrument Calibration Interval through On-Line CalibrationTechnologyrdquo OECD Halden Reactor Project Halden NorwaySeptember 2004
[34] A Wald ldquoSequential tests of statistical hypothesesrdquo Annals ofMathematical Statistics vol 16 no 2 pp 117ndash186 1945
[35] F Di Maio P Baraldi E Zio and R Seraoui ldquoFault detection innuclear power plants components by a combination of statisticalmethodsrdquo IEEE Transactions on Reliability vol 62 no 4 pp833ndash845 2013
[36] S Cheng and M Pecht ldquoUsing cross-validation for modelparameter selection of sequential probability ratio testrdquo ExpertSystems with Applications vol 39 no 9 pp 8467ndash8473 2012
TribologyAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
FuelsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal ofPetroleum Engineering
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Power ElectronicsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
CombustionJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Renewable Energy
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
StructuresJournal of
International Journal of
RotatingMachinery
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporation httpwwwhindawicom
Journal ofEngineeringVolume 2014
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
International Journal ofPhotoenergy
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear InstallationsScience and Technology of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Solar EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Wind EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear EnergyInternational Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
High Energy PhysicsAdvances in
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Science and Technology of Nuclear Installations 7
Table 1 List of NPP sensors
Transmitter ID Transmitter name Service Units Low range High rangeFF1 Feed flow 1 Feedwater flow KGS 0 600FF2 Feed flow 2 Feedwater flow KGS 0 600SF1 STM flow 1 Steam flow KGS 0 600SF2 STM flow 2 Steam flow KGS 0 600SP1 STM PSR 1 Steam pressure BARG 0 100SP2 STM PSR 2 Steam pressure BARG 0 100SP3 STM PSR 3 Steam pressure BARG 0 100PP1 PZR PSR 1 Pressurizer pressure BARG 116 170PP2 PZR PSR 2 Pressurizer pressure BARG 116 170PL1 PZR LVL 1 Pressurizer level 0 100PL2 PZR LVL 2 Pressurizer level 0 100SGL1 SG LVL NR 1 range Steam generator level narrow 0 100SGL2 SG LVL NR 2 range Steam generator level narrow 0 100
Here ℎ119894
1(119904) corresponds to hidden layer activations against
clean sensors at the input layer while ℎ119894
1() corresponds to
hidden layer activations against partially faulted sensor setFinally to add robustness against small offset and
miscalibration type sensor failures an isotropic Gaussian
type corruption process is devised for DAE-3 pretrain-ing The corruption procedure corrupts the ℎ
2hidden
layer mappings against clean sensors at the input layeras ℎ2(ℎ1(119878)) by employing an isotropic Gaussian noise as
follows
120578ℎ3
119863(ℎ2| ℎ2(119878)) = AGN
ℎ119894
2= ℎ119894
2(ℎ119894
1(119878)) 119894 notin 119895
ℎ119894
2| ℎ119894
2sim 119873(119904 120590
2
119868) 119894 isin 119895
119894=V119899
⋃
119894=1
119895119894sdotsdotsdot119873=V119899 119895
119894= [1 119899] = rand ( ) ] = input fraction 119899 = total inputs
(12)
Finally clean input is used for the supervised fine-tuningphase in Figure 4
5 Data Set Description
Intentionally for study purposes we limited the modelingscope of DAASM to full power steady operational state It isthe common state in which NPP operates from one refuelingto the next However in practice it is not possible for NPPsystems to be in perfect steady state Reactivity inducedpower perturbations natural process fluctuations sensor andcontroller noises and so forth are some of the evident causesforNPP parameter fluctuations and are responsible for steadystate dynamics Considering that the collected data set shouldbe fairly representative of all possible steady state dynamicsand noise the selected sensors are sampled during differenttime spans of one complete operating cycleThe training dataset consists of 6104 samples collected during the first twomonths of full power reactor operations after refueling cycleMeanwhile 3260 and 2616 samples are reserved for validationand test data sets respectively Five test data sets are used formodelrsquos performance evaluation Each test data set consistsof 4360 samples collected during eight-month period afterrefueling operation In order to account for fault propagation
phenomenon due to large signal groups a sensor subset isselected for this study An engineering sense selection basedon physical proximity and functional correlation is used todefine the sensor subset for this study Thirteen transmittersas listed in Table 1 are selected from various services innuclear steam supply systemof a real PWR typeNPP Figure 5shows the spatial distribution of the selected sensors
Starting from postrefueling full power startup the dataset covers approximately one year of selected sensors valuesSelected sensors are sampled every 10 seconds for consecutive12-hour time window Figure 6 shows data plot from fewselected sensors
6 Model Training
NPP sensorrsquos data are divided into training test and valida-tion set Each sensor data set is scaled in 01 to 09 ranges byusing lower and upper extremities corresponding to individ-ual sensor However the values 0 and 1 are explicitly reservedfor gross and saturation type sensor failures respectivelyTraining data consists of 4320 samples from full power steadystate reactor operation Meanwhile test and validation dataare used for sensor model optimization and performanceevaluation respectively The training setup for DAASM
8 Science and Technology of Nuclear Installations
Reactor vessel
PP1
PL1 PL2
Pressurizer
SGL2
SF1
FF1
Steam generator
Coolant pump
SGL1
FF2
PP2
SP1 SP2 SP3
SF2
Figure 5 Spatial distribution of selected sensor set
Pres
sure
(BA
RG)
1535154
1545
0 1000 1500 2000 2500 3000 3500 4000 4500500Training samples
596061
()
Leve
l
0 1000 1500 2000 2500 3000 3500 4000 4500500Pressurizer pressure
67675
68
Pres
sure
(BA
RG)
0 1000 1500 2000 2500 3000 3500 4000 4500500Steam generator level
500520540
(kg
s)Fl
ow
0 1000 1500 2000 2500 3000 3500 4000 4500500Steam pressure
400500600
Feedwater flow
(kg
s)Fl
ow
0 1000 1500 2000 2500 3000 3500 4000 4500500Steam flow
Figure 6 Plot of NPP sensors listed in Table 1
employs two learning stages an unsupervised learning phaseand supervised training phase DAE based greedy layerwisepretraining of each hidden layer as described in Section 4 isperformed usingminibatches from training data set Stochas-tic gradient descent based learning algorithm is employed
as suggested in practical training recommendations by [30]Finally standard backpropagation algorithm is employed forsupervised fine-tuning in fully stacked DAASM in Figure 4Supervised training is performed using clean sensor inputonly The model hyperparameters are set by random gridsearchmethod [31] A summary of the training hyperparame-ters corresponding to optimumDAASM is shown in Table 2
7 Invariance Test for Robustness
A layer by layer invariance study is conducted to test therobustness of fully trained DAASM against failed sensorstates Data corruption processes applied during pretrainingare essentially meant to learn hidden layer mappings whichare stable and invariant to faulty sensor conditions Thefollowing invariance test for successive hidden layers in finalDAASM stack can provide an insight into the effectivenessof data corruption processes exercised during denoisingbased pretraining phase Invariance for hidden layer map-pings ℎ
119897 is quantified through mean square error (MSE)
between Euclidean (1198712) normalized hidden layer activation⟨ℎ119894⟩119899⟨ℎ119894⟩1198992and ⟨ℎ
119894⟩119899⟨ℎ119894⟩1198992against clean and faulty
sensors respectively Invariance test samples are generatedby corrupting randomly selected sensors in input set withvarying level of offset failures [5ndash50] The MSE againsteach offset level is normalized across hidden layer dimension119863ℎand number of test samples 119879
119873as shown in (13) Finally
theseMSE values are normalized with maximalMSE value asin (14) Normalized MSE curves for each successive hiddenlayer are plotted in Figure 7 Consider
Science and Technology of Nuclear Installations 9
Table 2 Summary of DAASM hyperparameters
Hyperparameter type Tested hyperparameter values Successful hyperparameteragainst optimum model
Pretrained DAE units 3 3
Network architecture
119868119899minus 119864(119899 119901) minus 119861
119899minus 119863(119899 119901) minus 119900
119899
119868119899 input layer neurons
119861119899 bottleneck layer neurons119900119899 output layer neurons
119864(119899 119901) encoder cascade119863(119899 119901) decoder cascade
119899 number of layers119901 neurons per layer
13 minus 119864(2 20) minus 8 minus
119863(2 20) minus 13
Learning rate forunsupervised pretraining 01 5 times 10
minus2
1 times 10minus2
5 times 10minus3
1 times 10minus3
5 times 10minus4
[5 times 10minus2 1 times 10
minus2
]
Learning rate forsupervised training
Scheduled learning rate based on training errormonitoring
015 01 0005 0001 00001
01 0005 0001
Mean pretraining error foreach hidden layer
Corresponding to minima observed during crossvalidation 10
minus4
Weight decay 120582 10minus3
10minus4
10minus5
10minus3
Momentum 119898 [085 099] [095 098]
Input corruption level ]Corrupted input fraction 10 25 30 40
Gaussian corruption ( of sensorrsquos nominal value)005 010 020 035 050
Input fraction [25ndash35]Gaussian noise level
[010ndash025]Dropout fraction in DAE-3 010 020 01
MSE (119867119897Offset) =
(1119879119873)sum119879119873
119899=1(⟨ℎ119894⟩1198991003817100381710038171003817⟨ℎ119894⟩119899
10038171003817100381710038172minus ⟨ℎ119894⟩119899
10038171003817100381710038171003817⟨ℎ119894⟩119899
100381710038171003817100381710038172)2
119863ℎ
1 le 119897 le 119871 = No of encoder layers = 3 Offset = 5 10 20 sdot sdot sdot 50
(13)
MSENormalized (119867119897) =
MSE (119867119897Offset)
MSE (119867119897Max-Offset)
(14)
LayerwiseMSEplots in Figure 7 clearly show that invarianceto faulty sensor conditions increases towards higher layers inthe network hierarchy In these plots lower curves indicatehigher level of invariance To further investigate the effectof increasing invariance on reconstructed sensor values asensor model corresponding to the level ldquo119871rdquo of each hiddenlayer is assembled via encoder and decoder cascade Robust-ness of these partial models is quantified through (1 minus 119878
119894
Auto)Autosensitivity values 119878
119894
Auto (see Section 82) are calculatedagainst varying offset failure levels In Figure 8 layerwiseincrease in robustness confirms that increased invariancehelps in improving overall modelrsquos robustness
8 DAASM versus K-AANNPerformance Analysis
Here we will assess and compare the performance of DAASMwith popular five-layer AANNmodel originally proposed byKramer [8] The K-AANN model is trained with same data
set as used for DAASM and is regularized with Levenberg-Marquardt algorithm Furthermore to improve robustnesstraining with jitter heuristic is employed by introducing anoise of 10 magnitude on clean sensor input The five-layertopology 13-17-9-17-13 is found to be optimum for K-AANNmodel Both DAASM and K-AANN model are comparedthrough accuracy robustness spillover and fault detectabilitybased performance metrics in the following subsections Allperformance metrics are calculated against test data setconsisting of 4320 samples frompostrefueling full powerNPPoperations Performance metric values are reported
81 Accuracy Mean square error (MSE) of observed andmodel estimated sensor values against fault free test data setis used to quantify accuracy metric as follows
Accuracy =1
119873
119873
sum
119894=1
(119894minus 119878119894)2
(15)
10 Science and Technology of Nuclear Installations
Layerwise invariance
3rd hidden layer2nd hidden layer1st hidden layer
0010203040506070809
1
Nor
mal
ised
MSE
minus20 minus10minus40 minus30 0 10 20 30 40 50minus50
Offset failure level ()
Figure 7 Layerwise invariance in DAASM Lower curves depicthigher invariance
Robustness
With 3rd hidden layerWith 2nd hidden layerWith 1st hidden layer
0203040506070809
1
minus20 minus10minus40 minus30 0 10 20 30 40 50minus50
Offset failure level ()
Robu
stnes
s (1minusS a
)
Figure 8 Robustness measure (1 minus autosensitivity) at multiplehidden layers in DAASM Higher curves depict high robustness
The MSE values of all sensors are normalized to theirrespective span and are presented as percent span in Figure 9Being an error measure the lower MSE values by DAASMsignify its prediction accuracy
82 Robustness Robustness is quantified through autosensi-tivity as defined by [32 33] It is themeasure of modelrsquos abilityto predict correct sensor values under missing or corruptedsensor states The measure is averaged over an operatingregion defined by 119896 samples from test data set as follows
119878119894
Auto =1
119873
119873
sum
119896=1
1003816100381610038161003816100381610038161003816100381610038161003816
drift119896119894
minus 119896119894
119904drift119896119894
minus 119904119896119894
1003816100381610038161003816100381610038161003816100381610038161003816
(16)
where 119894 and 119896 are indexes corresponding to sensors and theirrespective test samples 119904
119896119894is the original sensor value without
fault 119896119894is the model estimated sensor value against 119904
119896119894 119904drift119896119894
is the driftedfaulted sensor value drift119896119894
is themodel estimatedsensor value against drifted value 119904
drift119896119894
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN 0382 0361 0343 0412 0186 0211 0166 0386 0411 0243 0223 0512 0621
DAASM 0281 0253 0246 0293 0121 0132 0122 0243 0315 0173 0156 0394 0465
Accuracy metric
0
01
02
03
04
05
06
07
MSE
( sp
an)
Figure 9 MSE depicting DAASM and K-AANN accuracy on eachsensor
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Autosensitivity metric
0005
01015
02025
03503
Auto
sens
itivi
ty
0273 0294 0321 0332 0205 0182 0198 0253 0233 0225 0231 0268 0301
0153 0167 0187 0163 0112 0079 0082 0131 0071 014 0117 0133 0187
Figure 10 Autosensitivity values of individual sensors in bothmodels
The autosensitivitymetric lies in [0 1] range For autosen-sitivity value of one the model predictions follow the faultwith zero residuals hence no fault can be detected Smallerautosensitivity values are preferred which essentially meansdecreased sensitivity towards small perturbations Largeautosensitivity values may lead to missed alarms due tounderestimation of the fault size caused by small residualvalues Compared to K-AANN model in case of DAASM asignificant decrease in autosensitivity values for all sensors isobserved The plot in Figure 10 shows that DAASM is morerobust to failed sensor inputs
To further investigate robustness against large offsetfailures both models are evaluated against offset failuresin [5ndash50] range For each sensor samples from testdata are corrupted with specific offset level and correspond-ing autosensitivities are averaged over whole sensor setAutosensitivity values less than 02 are considered as robustThe maximum autosensitivity value of 0187 is observed insteam flow sensor The plot in Figure 11 shows that averageautosensitivity for both models increases with increasinglevel of offset failure However the autosensitivity curve forDAASM autosensitivity is well below the corresponding K-AANN curve
83 Spillover Depending upon the size and type of failurea failed sensor input can cause discrepancy in estimatedoutput for other sensors The phenomenon is referred to in
Science and Technology of Nuclear Installations 11
0060086
014
021
028
041
016021
028
036
043
054
005 01 02 03 04 05Sensor perturbation magnitude ( of actual value)
Robustness performance
0
01
02
03
04
05
06
Avg
auto
sens
itivi
ty
DAASMK-AANN
Figure 11 Comparison of robustness against increasing offsetfailure
literature as ldquospillover effectrdquo and is quantified through ldquocross-sensitivityrdquo metric [32] It quantifies the influence of faultysensor 119894 on predictions of sensor 119895 as follows
119878119895119894
Cross =1
119873
119873
sum
119896=1
10038161003816100381610038161003816100381610038161003816100381610038161003816
drift119896119895
minus 119896119895
119904drift119896119894
minus 119904119896119894
10038161003816100381610038161003816100381610038161003816100381610038161003816
(17)
119878119895
Cross =1
119873 minus 1
119873
sum
119894=1
119878119895119894
Cross 119894 = 119895 (18)
where 119894 and 119895 indexes are used to refer to faulty andnonfaulty sensors respectively Meanwhile 119896 is the index forcorresponding test samples 119878119895119894Cross is the cross-sensitivity ofsensor 119895 with respect to drift in 119894th sensor 119904
119896119894is the value of
119894th sensor without any fault 119896119895is the model estimated value
of 119895th sensor against 119904119896119894 119904drift119896119894
is the driftedfaulted value of 119894thsensor drift
119896119895is themodel estimated value of 119895th sensor against
drifted value 119904drift119896119894
The highly distributed representation of the input in
neural network based sensor models has pronounced effecton the cross-sensitivity performance Cross-sensitivitymetricvalue lies in [0 1] range High value of cross-sensitivitymay set off false alarms in other sensors provided theresidual values overshoot the fault detectability threshold inother sensors So minimum cross-sensitivity value is desiredfor a robust model The plot in Figure 12 shows that thecross-sensitivity for DAASM is reduced by a large factor ascompared to K-AANNmodel
The spillover effect against particular level of offset failurein [5ndash50] range is averaged over all sensors as follows
AvgCross Sensitivity (119870) =
sum119873
119895=1119878119895
cross(119870)
119873
offset failure level 119870 = 5 10 20 sdot sdot sdot 50
(19)
0128 0124 0113 0116 0112 0108 0092 0079 0083 0098 011 0103 0105
00632 00527 00563 00614 0039 00311 00287 00186 00203 00344 00336 003 00286
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Cross-sensitivity metric
0002004006008
01012014
Cros
s-se
nsiti
vity
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
Figure 12 Cross-sensitivity values of individual sensors in bothmodels
00810105
0161
0253
0312
041
0018 0026 0038 00460071
012
005 01 02 03 04 05Sensor perturbation magnitude ( of observed sensor value)
Spillover offset sensor failure
Requires SPRT mean relaxation to avoid
false alarmsahead of this point
0
005
01
015
02
025
03
035
04
045
Avg
cros
s-se
nsiti
vity
DAASMK-AANN
Figure 13 Comparison of spillover effects against increasing offsetfailure
The cross-sensitivity values 119878119895
cross(119870) against 119870 offset fail-ure level are calculated using (18) Figure 13 shows theaverage cross-sensitivity plot for both models Small cross-sensitivities are observed in DAASM which effectivelyavoided false alarms in other channels without relaxing theSPRT faulted mean value up to an offset failure of 35ndash40 inany channel However for the case of offset noise larger than35 SPRTmean needs to be relaxed to avoid false alarms andisolate the faulty sensor However Robustness of K-AANNmodel deteriorates significantly due to spillover effect beyond15 offset failure
Similarly gross failure scenarios corresponding to twoextremities of sensor range can cause severe Spillover effectTo study robustness against gross type failure scenario asubset of input sensors is simultaneously failed with grosshigh or low value and average cross-sensitivity of remainingsensor set is calculated using relation (19) Plot in Figure 14shows that average cross-sensitivity of K-AANN modelincreases drastically beyond 10 gross failure HoweverDAASM resulted in a very nominal spillover even in caseof multiple sensor failure The DAASM effectively managedsimultaneous gross high or low failures in 25 of total sensorset as compared to 10 in case of K-AANN
12 Science and Technology of Nuclear Installations
0028 0032 0041 012021016 024
0457
061
077
01 02 03 04 05Sensors failed with gross highlow error ()
Spillover gross sensor failure
0010203040506070809
Avg
cros
s-se
nsiti
vity
DAASMK-AANN
Figure 14 Comparison of spillover effect against simultaneous grosshighlow failure in multiple sensors
84 Fault Detectability Fault detectability metric measuresthe smallest fault that can be detected by integrated sensorestimation and fault detection module as shown in Figure 1[32] The detectability metric is measured as percentage ofsensor span 119863 = 119872Span where value M correspondsto minimum detectable fault Minimum fault detectabilitylimit for each sensor is quantified through statistical basedsequential probability ratio test (SPRT) by Wald [34] SPRTtest is carried out to detect if the residual being generatedfrom normal distribution 119873(120583
1 1205902
) or 119873(1205830 1205902
) as definedfor faulty and fault free sensor operations respectively [35]Calibration failures are reflected in the mean parameter ofresidualrsquos distribution The SPRT procedure is applied todetect changes in the mean of residualrsquos distribution Theapplication of SPRT requires setting of following parametersvalue [36]
1205830 normal mode residual mean
1205902 normal mode residual variance
1205831 expected offset in residual mean in abnormal
mode120572 false alarm probability120573 missed alarm probability
Under normal mode the residuals from observed and modelestimated sensor values behave as awhiteGaussian noisewithmean 120583
0= 0 The residual variance 120590
2 is estimated for eachsensor under normal operating conditions and remainedfixed The false alarm 120572 and missed alarm 120573 probabilities areset to be 0001 and 001 respectively In order to determineminimum fault detectability limit a numerical procedureis opted which searches for minimum expected offset 120583
1
in the interval 1205831
[120590ndash3120590] provided the constraint onmissed and false alarm rate holds 120590 is the standard deviationcorresponding to residual variance of particular sensor Theplot in Figure 15 shows the detectability metric for eachsensor The plot in Figure 15 shows that DAASM can detect
0312 0315 033 035 004 005 004 008 0086 01 009 006 0058
0184 0165 019 02 0021 0023 002 0048 005 004 004 0032 0028
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Fault detectability metric
0005
01015
02025
03035
04
Det
ecta
bilit
y (
span
)
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
Figure 15 Comparison of fault detectability metrics
faults which are two times smaller in magnitude than thosedetectable by K-AANNmodel
Improvement in fault detectability metric for DAASMcan be attributed to observed improvement in model robust-ness as suggested by the following relation
119903119894
Δ119878drift119894
= (1 minus 119878119894
Auto) (20)
The term 119903119894Δ119878
drift119894
measures the ratio of observed residualto actual sensor drift in terms of autosensitivity For highlyrobust model this ratio reduces to one which means residualreflects the actual drift and results in high fault detectabilityContrarily ratio value close to zero means that the predictionis following the input and results in poor fault detectability
841 SPRT Based Fault Detectability Test Sequential prob-ability ratio [34 36] based fault hypothesis test is appliedto residual sequence 119877
119894 = 119903
1(1199051) 1199031(1199051)sdot sdot sdot 119903119899(119905119899)generated
by relation 119877119894(119905119894) = 119878
Obs(119905119894) minus 119878
Est(119905119894) at time 119905
119894 where
119878Obs
(119905119894) and 119878
Est(119905119894) are the actual and model predicted
sensor values respectively The SPRT procedure analyzeswhether the residual sequence is more likely to be generatedfrom a probability distribution that belongs to normal modehypothesis 119867
0or abnormal mode hypothesis 119867
1by using
likelihood ratio as follows
119871119899= exp[minus(12120590
2)[sum119899
119894=11205831(1205831minus2119903119894)]]
(21)
For fault free sensor values the normalmode hypothesis1198670is
approximated byGaussian distributionwithmean 1205830= 0 and
variance 1205902 Abnormal mode hypothesis 119867
1is approximated
with mean 1205831
gt 1205830using the same variance 120590
2 The SPRTindex for the positive mean test is finally obtained by takinglogarithm of the likelihood ratio in (21) as follows [35]
SPRT = ln (119871119899) = minus
1
21205902[
119899
sum
119894=1
1205831(1205831minus 2119903119894)]
=1205831
1205902
119899
sum
119894=1
(119903119894minus
1205831
2)
(22)
Pressurizer pressure sensor sampled at a frequency of 10 sec-onds is used as a test signal to validate the fault detectability
Science and Technology of Nuclear Installations 13
0 500 1000 1500 2000 2500 3000 3500 4000 45001535
1541545
SPRT drift detection (pressure sensor)(B
ARG
)Pr
essu
re
Drifted sensorDAASM estimate
Sample number
0 500 1000 1500 2000 2500 3000 3500 4000 45001535
1541545
(BA
RG)
Pres
sure
Drifted sensorK-AANN estimate
Sample number
0515
SPRT
hypo
thes
is
minus050 500 1000 1500 2000 2500 3000 3500 4000 4500
DAASM indexK-AANN index
Sample number
005
valu
eRe
sidua
l
0 500 1000 1500 2000 2500 3000 3500 4000 4500
Residual DAASMResidual K-AANN
Sample number
minus05
H1mdash1
H0mdash0
Figure 16 SPRTbased fault detection in pressurizer pressure sensor
performance Two drift faults at the rate of +001hour andminus001hour are introduced in the test signal for DAASMand K-AANN modelrsquos assessment respectively The first andsecond plots in Figure 16 show drifted and estimated pressuresignal from DAASM and K-AANN models respectivelyThird plot shows residual values generated by differencing thedrifted and estimated signals frombothmodelsThe final plotshows SPRT index values against residuals from K-AANNmodel and DAASM The hypotheses 119867
1and 119867
0correspond
to positive and negative fault acceptance respectively FromSPRT index plot successful early detection of the sensordrift at 2200th sample with lag of 611 hours since the driftinception shows that DAASM is more sensitive to smalldrifts On the other hand SPRT index on K-AANN basedsensor estimates registered the same drift at 3800th samplewith a lag of almost 1055 hours The result shows thatDAASM is more robust in terms of early fault detection withlow false and missed alarm rates
Finally both models are tested against five test data setsEach test set consists of 3630 samples corresponding todifferentmonths of full power reactor operation Bothmodelssuccessfully detected an offset failure of 012ndash03 BARG in allsteam pressure channels and a drift type failure up to 285 insteam generator level (Figure 22)The K-AANNmodel failedto register a very small drift up to 01 in steam flow (STMflow 1) channel A small drift up to 01 BARG is detected in
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Steam pressure estimation
Observed valueDAASM estimateK-AANN estimate
672673674675676677678679
68
Stea
m p
ress
ure (
BARG
)
Figure 17 Steam pressure estimates against offset failures up to 03(BARG)
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Feed flow estimation
400420440460480500520540560
Feed
flow
(kg
s)
Observed sensor valueK-AANN estimateDAASM estimate
Figure 18 Feed flow estimates
test set 5 of pressurizer pressure channel However in caseof drift type sensor failures fault detection lag for DAASMwas on average 05 times smaller in comparison with K-AANN model Plots in Figures 17ndash21 show the estimatedsensor values from both models on five test data sets of fewselected channels
9 Conclusion
This paper presented a neural network based denoisedautoassociative sensor model (DAASM) for empirical sen-sor modeling The proposed sensor model is trained togenerate a monitoring system for sensor fault detection innuclear power plantsMultilayer AANNbased sensormodelsmay result in suboptimal solutions due to poor regulariza-tion by traditional backpropagation based joint multilayerlearning procedures So a complementary deep learningapproach based on greedy layerwise unsupervised pretrain-ing is employed for effective regularization in the proposedmultilayer DAASM Autoencoder architecture is used fordenoising based unsupervised pretraining and regularizationof individual layers in the network hierarchy To addressrobustness against perturbations in input sensors data cor-ruption processes exercised during unsupervised pretraining
14 Science and Technology of Nuclear Installations
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5490495500505510515520525530535540
Stea
m fl
ow (k
gs)
Steam flow estimation
Observed sensor valueDAASM estimateK-AANN estimate
Figure 19 Steam flow estimates against drift failure up to 01
58859
592594596598
60602604606608
Pres
suriz
er le
vel (
)
Pressurizer level estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valueK-AANN estimateDAASM estimate
Figure 20 Pressurizer level estimates
phase were based on prior knowledge about different failurescenarios Results from invariance tests showed that theproposed data corruption schemeswere beneficial in learninglatent representations at hidden layers and were invariantto multiple levels of perturbation in input sensors Conse-quently these pretrained hidden layers worked as well regu-larized perturbation filters with increased invariance towardssensor faults It is also observed that sensitivity against sensorfaults decreased significantly towards higher layers in fullDAASMassembly In a practical context of sensormonitoringin nuclear power plants the proposed model proved itsrobustness against gross type simultaneous sensor failuresIt also showed significant improvement in all performancemetrics when compared with popular and widely used five-layered AANNmodel by KramerMoreover time lag in smalldriftrsquos detection is significantly reduced The overall resultssuggest that greedy layerwise pretraining technique in com-bination with domain specific corruption processes providesa viable framework for effective regularization and robustnessin such deep multilayered autoassociative sensor validationmodels
Appendix
See Table 3
153715375
153815385
153915395
15415405
154115415
Pres
sure
(BA
RG)
Pressurizer pressure estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valuesK-AANN estimateDAASM estimate
Figure 21 Pressurizer pressure estimates against drift failure up to01
74
76
78
80
82
84
Stea
m g
ener
ator
leve
l (
)
Steam generator level estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valueK-AANN estimateDAASM estimate
Figure 22 Steam generator level estimates against growing driftfailure up to 287
Abbreviations
AANN Autoassociative neural networkK-AANN Kramer proposed Autoassociative neural
networkDAASM Denoised autoassociative sensor modelNPP Nuclear power plantPWR Pressurized water reactor119878Auto Autosensitivity119878Cross Cross-sensitivityDAE Denoising autoencoder119878 Observed sensor value Model predicted sensor value Corrupted sensor valueSPN Salt-and-pepper noiseAGN Additive Gaussian noise
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Science and Technology of Nuclear Installations 15
Table3Re
sults
ofDAASM
andK-AANNperfo
rmance
metric
sPerfo
rmance
metric
Mod
eltype
Feed
flow1
Feed
flow2
STM
flow1
STM
flow2
STM
PSR1
STM
PSR2
STM
PSR3
PZRPS
R1
PZRPS
R2
PZRLV
L1
PZRLV
L2
SGLV
LNR1
SGLV
LNR2
AANN
Sp-D
AANN
Accuracy
(span)
0382
0361
0343
0412
0186
0211
0166
0386
0411
0243
0223
0512
0621
0281
0253
0246
0293
0121
0132
0122
0243
0315
0173
0156
0394
0465
Autosensitivity
0273
0294
0321
0332
0205
0182
0198
0253
0233
0225
0231
0268
0301
0153
0167
0187
0163
0112
0079
0082
0131
0071
014
0117
0133
0187
Cross-sensitivity
0128
0124
0113
0116
0112
0108
0092
0079
0083
0098
011
0103
0105
00632
00527
00563
00614
0039
00311
00287
00186
00203
00344
00336
003
00286
SPRT
detectability
0312
0315
033
035
004
005
004
008
0086
01
009
006
0058
0184
0165
019
02
0021
0023
002
004
8005
004
004
0032
0028
16 Science and Technology of Nuclear Installations
Acknowledgments
The authors acknowledge the support of Mr Jose GalindoRodrıguez affiliated to TECNATOM Inc Spain and MrChad Painter (Director of Nuclear Power Plant SimulatorDevelopment and Training Program at International AtomicEnergy Agency) for providing necessary tools and data toconduct this research
References
[1] N Mehranbod M Soroush M Piovoso and B A OgunnaikeldquoProbabilistic model for sensor fault detection and identifica-tionrdquo AIChE Journal vol 49 no 7 pp 1787ndash1802 2003
[2] JW Hines and E Davis ldquoLessons learned from the US nuclearpower plant on-line monitoring programsrdquo Progress in NuclearEnergy vol 46 no 3-4 pp 176ndash189 2005
[3] EPRI ldquoOn-line monitoring cost benefit guiderdquo Final Report1006777 EPRI Palo Alto Calif USA 2003
[4] S J Qin and W Li ldquoDetection identification and recon-struction of faulty sensors with maximized sensitivityrdquo AIChEJournal vol 45 no 9 pp 1963ndash1976 1999
[5] J Ma and J Jiang ldquoApplications of fault detection and diagnosismethods in nuclear power plants a reviewrdquo Progress in NuclearEnergy vol 53 no 3 pp 255ndash266 2011
[6] G YHeo ldquoConditionmonitoring using empiricalmodels tech-nical review and prospects for nuclear applicationsrdquo NuclearEngineering and Technology vol 40 no 1 pp 49ndash68 2008
[7] J B Coble R M Meyer P Ramuhalli et al ldquoA review ofsensor calibration monitoring for calibration interval extensionin nuclear power plantsrdquo Tech Rep PNNL-21687 PacificNorthwest National Laboratory Richland Wash USA 2012
[8] M A Kramer ldquoAutoassociative neural networksrdquo Computersand Chemical Engineering vol 16 no 4 pp 313ndash328 1992
[9] X Xu J W Hines and R E Uhrig ldquoOn-line sensor calibra-tion monitoring and fault detection for chemical processesrdquoin Proceedings of the Maintenance and Reliability Conference(MARCON rsquo98) pp 12ndash14 Knoxville Tenn USA May 1998
[10] M Hamidreza S Mehdi J-R Hooshang and N AliakbarldquoReconstruction based approach to sensor fault diagnosis usingauto-associative neural networksrdquo Journal of Central SouthUniversity vol 21 no 6 pp 2273ndash2281 2014
[11] UThissenW JMelssen andLMC Buydens ldquoNonlinear pro-cess monitoring using bottle-neck neural networksrdquo AnalyticaChimica Acta vol 446 no 1-2 pp 371ndash383 2001
[12] D J Wrest J W Hines and R E Uhrig ldquoInstrument surveil-lance and calibration verification through plant wide monitor-ing using autoassociative neural networksrdquo in Proceedings ofthe American Nuclear Society International Topical Meeting onNuclear Plant Instrumentation Control and Human MachineInterface Technologies University Park Pa USA May 1996
[13] J W Hines R E Uhrig and D J Wrest ldquoUse of autoassociativeneural networks for signal validationrdquo Journal of Intelligent andRobotic Systems Theory and Applications vol 21 no 2 pp 143ndash154 1998
[14] P F Fantoni M I Hoffmann R Shankar and E L Davis ldquoOn-line monitoring of instrument channel performance in nuclearpower plant using PEANOrdquo Progress in Nuclear Energy vol 43no 1ndash4 pp 83ndash89 2003
[15] MMarseguerra andA Zoia ldquoTheAutoAssociative Neural Net-work in signal analysis II Application to on-line monitoring of
a simulated BWR componentrdquo Annals of Nuclear Energy vol32 no 11 pp 1207ndash1223 2005
[16] M S Ikbal H Misra and B Yegnanarayana ldquoAnalysis ofautoassociative mapping neural networksrdquo in Proceedings of theInternational Joint Conference on Neural Networks (IJCNN rsquo99)vol 5 pp 3025ndash3029 IEEE Washington DC USA July 1999Proceedings (Cat no 99CH36339)
[17] Y Bengio ldquoLearning deep architectures for AIrdquo Foundationsand Trends in Machine Learning vol 2 no 1 pp 1ndash27 2009
[18] J W Hines A V Gribok I Attieh and R E Uhrig ldquoRegular-izationmethods for inferential sensing in nuclear power plantsrdquoin Fuzzy Systems and Soft Computing in Nuclear Engineering DRuan Ed vol 38 of Studies in Fuzziness and Soft Computingpp 285ndash314 Physica 2000
[19] A V Gribok J W Hines A Urmanov and R E UhrigldquoHeuristic systematic and informational regularization forprocessmonitoringrdquo International Journal of Intelligent Systemsvol 17 no 8 pp 723ndash749 2002
[20] X Glorot and Y Bengio ldquoUnderstanding the difficulty oftraining deep feedforward neural networksrdquo in Proceedings ofthe 13th International Conference on Artificial Intelligence andStatistics (AISTATS rsquo10) vol 9 pp 249ndash256 Sardinia Italy May2010
[21] S Tan andM L Mayrovouniotis ldquoReducing data dimensional-ity through optimizing neural network inputsrdquo AIChE Journalvol 41 no 6 pp 1471ndash1480 1995
[22] Y Bengio and P Lamblin ldquoGreedy layer-wise training of deepnetworksrdquo in Proceedings of the Advances in Neural InformationProcessing Systems 19 (NIPS rsquo07) pp 153ndash160 September 2007
[23] H Larochelle Y Bengio J Lourador and P Lamblin ldquoExplor-ing strategies for training deep neural networksrdquoThe Journal ofMachine Learning Research vol 10 pp 1ndash40 2009
[24] D Yu and L Deng ldquoDeep learning and its applications to signaland information processing [exploratory DSP]rdquo IEEE SignalProcessing Magazine vol 28 no 1 pp 145ndash154 2011
[25] D ErhanA Courville andPVincent ldquoWhydoes unsupervisedpre-training help deep learningrdquo Journal of Machine LearningResearch vol 11 pp 625ndash660 2010
[26] P Vincent H Larochelle Y Bengio and P-A ManzagolldquoExtracting and composing robust features with denoisingautoencodersrdquo in Proceedings of the 25th International Confer-ence on Machine Learning (ICML rsquo08) pp 1096ndash1103 July 2008
[27] P Vincent H Larochelle I Lajoie Y Bengio and P-AManzagol ldquoStacked denoising autoencoders learning usefulrepresentations in a deep network with a local denoisingcriterionrdquo Journal of Machine Learning Research vol 11 pp3371ndash3408 2010
[28] N Srivastava G Hinton A Krizhevsky I Sutskever and RSalakhutdinov ldquoDropout a simple way to prevent neural net-works from overfittingrdquo Journal of Machine Learning Researchvol 15 pp 1929ndash1958 2014
[29] M J Embrechts B J Hargis and J D Linton ldquoAn augmentedefficient backpropagation training strategy for deep autoasso-ciative neural networksrdquo in Proceedings of the 18th EuropeanSymposiumonArtificial NeuralNetworksmdashComputational Intel-ligence and Machine Learning (ESANN rsquo10) vol no pp 28ndash30April 2010
[30] Y Bengio ldquoPractical recommendations for gradient-basedtraining of deep architecturesrdquo inNeural Networks Tricks of theTrade vol 7700 of Lecture Notes in Computer Science pp 437ndash478 Springer Berlin Germany 2nd edition 2012
Science and Technology of Nuclear Installations 17
[31] J Bergstra and Y Bengio ldquoRandom search for hyper-parameteroptimizationrdquo Journal of Machine Learning Research vol 13 pp281ndash305 2012
[32] J W Hines and D R Garvey ldquoDevelopment and applicationof fault detectability performance metrics for instrument cali-bration verification and anomaly detectionrdquo Journal of PatternRecognition Research vol 1 no 1 pp 2ndash15 2006
[33] A Usynin and J W Hines ldquoOn-line monitoring robustnessmeasures and comparisonsrdquo in Proceedings of the Interna-tional Atomic Energy Agency Technical Meeting on ldquoIncreasingInstrument Calibration Interval through On-Line CalibrationTechnologyrdquo OECD Halden Reactor Project Halden NorwaySeptember 2004
[34] A Wald ldquoSequential tests of statistical hypothesesrdquo Annals ofMathematical Statistics vol 16 no 2 pp 117ndash186 1945
[35] F Di Maio P Baraldi E Zio and R Seraoui ldquoFault detection innuclear power plants components by a combination of statisticalmethodsrdquo IEEE Transactions on Reliability vol 62 no 4 pp833ndash845 2013
[36] S Cheng and M Pecht ldquoUsing cross-validation for modelparameter selection of sequential probability ratio testrdquo ExpertSystems with Applications vol 39 no 9 pp 8467ndash8473 2012
TribologyAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
FuelsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal ofPetroleum Engineering
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Power ElectronicsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
CombustionJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Renewable Energy
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
StructuresJournal of
International Journal of
RotatingMachinery
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporation httpwwwhindawicom
Journal ofEngineeringVolume 2014
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
International Journal ofPhotoenergy
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear InstallationsScience and Technology of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Solar EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Wind EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear EnergyInternational Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
High Energy PhysicsAdvances in
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
8 Science and Technology of Nuclear Installations
Reactor vessel
PP1
PL1 PL2
Pressurizer
SGL2
SF1
FF1
Steam generator
Coolant pump
SGL1
FF2
PP2
SP1 SP2 SP3
SF2
Figure 5 Spatial distribution of selected sensor set
Pres
sure
(BA
RG)
1535154
1545
0 1000 1500 2000 2500 3000 3500 4000 4500500Training samples
596061
()
Leve
l
0 1000 1500 2000 2500 3000 3500 4000 4500500Pressurizer pressure
67675
68
Pres
sure
(BA
RG)
0 1000 1500 2000 2500 3000 3500 4000 4500500Steam generator level
500520540
(kg
s)Fl
ow
0 1000 1500 2000 2500 3000 3500 4000 4500500Steam pressure
400500600
Feedwater flow
(kg
s)Fl
ow
0 1000 1500 2000 2500 3000 3500 4000 4500500Steam flow
Figure 6 Plot of NPP sensors listed in Table 1
employs two learning stages an unsupervised learning phaseand supervised training phase DAE based greedy layerwisepretraining of each hidden layer as described in Section 4 isperformed usingminibatches from training data set Stochas-tic gradient descent based learning algorithm is employed
as suggested in practical training recommendations by [30]Finally standard backpropagation algorithm is employed forsupervised fine-tuning in fully stacked DAASM in Figure 4Supervised training is performed using clean sensor inputonly The model hyperparameters are set by random gridsearchmethod [31] A summary of the training hyperparame-ters corresponding to optimumDAASM is shown in Table 2
7 Invariance Test for Robustness
A layer by layer invariance study is conducted to test therobustness of fully trained DAASM against failed sensorstates Data corruption processes applied during pretrainingare essentially meant to learn hidden layer mappings whichare stable and invariant to faulty sensor conditions Thefollowing invariance test for successive hidden layers in finalDAASM stack can provide an insight into the effectivenessof data corruption processes exercised during denoisingbased pretraining phase Invariance for hidden layer map-pings ℎ
119897 is quantified through mean square error (MSE)
between Euclidean (1198712) normalized hidden layer activation⟨ℎ119894⟩119899⟨ℎ119894⟩1198992and ⟨ℎ
119894⟩119899⟨ℎ119894⟩1198992against clean and faulty
sensors respectively Invariance test samples are generatedby corrupting randomly selected sensors in input set withvarying level of offset failures [5ndash50] The MSE againsteach offset level is normalized across hidden layer dimension119863ℎand number of test samples 119879
119873as shown in (13) Finally
theseMSE values are normalized with maximalMSE value asin (14) Normalized MSE curves for each successive hiddenlayer are plotted in Figure 7 Consider
Science and Technology of Nuclear Installations 9
Table 2 Summary of DAASM hyperparameters
Hyperparameter type Tested hyperparameter values Successful hyperparameteragainst optimum model
Pretrained DAE units 3 3
Network architecture
119868119899minus 119864(119899 119901) minus 119861
119899minus 119863(119899 119901) minus 119900
119899
119868119899 input layer neurons
119861119899 bottleneck layer neurons119900119899 output layer neurons
119864(119899 119901) encoder cascade119863(119899 119901) decoder cascade
119899 number of layers119901 neurons per layer
13 minus 119864(2 20) minus 8 minus
119863(2 20) minus 13
Learning rate forunsupervised pretraining 01 5 times 10
minus2
1 times 10minus2
5 times 10minus3
1 times 10minus3
5 times 10minus4
[5 times 10minus2 1 times 10
minus2
]
Learning rate forsupervised training
Scheduled learning rate based on training errormonitoring
015 01 0005 0001 00001
01 0005 0001
Mean pretraining error foreach hidden layer
Corresponding to minima observed during crossvalidation 10
minus4
Weight decay 120582 10minus3
10minus4
10minus5
10minus3
Momentum 119898 [085 099] [095 098]
Input corruption level ]Corrupted input fraction 10 25 30 40
Gaussian corruption ( of sensorrsquos nominal value)005 010 020 035 050
Input fraction [25ndash35]Gaussian noise level
[010ndash025]Dropout fraction in DAE-3 010 020 01
MSE (119867119897Offset) =
(1119879119873)sum119879119873
119899=1(⟨ℎ119894⟩1198991003817100381710038171003817⟨ℎ119894⟩119899
10038171003817100381710038172minus ⟨ℎ119894⟩119899
10038171003817100381710038171003817⟨ℎ119894⟩119899
100381710038171003817100381710038172)2
119863ℎ
1 le 119897 le 119871 = No of encoder layers = 3 Offset = 5 10 20 sdot sdot sdot 50
(13)
MSENormalized (119867119897) =
MSE (119867119897Offset)
MSE (119867119897Max-Offset)
(14)
LayerwiseMSEplots in Figure 7 clearly show that invarianceto faulty sensor conditions increases towards higher layers inthe network hierarchy In these plots lower curves indicatehigher level of invariance To further investigate the effectof increasing invariance on reconstructed sensor values asensor model corresponding to the level ldquo119871rdquo of each hiddenlayer is assembled via encoder and decoder cascade Robust-ness of these partial models is quantified through (1 minus 119878
119894
Auto)Autosensitivity values 119878
119894
Auto (see Section 82) are calculatedagainst varying offset failure levels In Figure 8 layerwiseincrease in robustness confirms that increased invariancehelps in improving overall modelrsquos robustness
8 DAASM versus K-AANNPerformance Analysis
Here we will assess and compare the performance of DAASMwith popular five-layer AANNmodel originally proposed byKramer [8] The K-AANN model is trained with same data
set as used for DAASM and is regularized with Levenberg-Marquardt algorithm Furthermore to improve robustnesstraining with jitter heuristic is employed by introducing anoise of 10 magnitude on clean sensor input The five-layertopology 13-17-9-17-13 is found to be optimum for K-AANNmodel Both DAASM and K-AANN model are comparedthrough accuracy robustness spillover and fault detectabilitybased performance metrics in the following subsections Allperformance metrics are calculated against test data setconsisting of 4320 samples frompostrefueling full powerNPPoperations Performance metric values are reported
81 Accuracy Mean square error (MSE) of observed andmodel estimated sensor values against fault free test data setis used to quantify accuracy metric as follows
Accuracy =1
119873
119873
sum
119894=1
(119894minus 119878119894)2
(15)
10 Science and Technology of Nuclear Installations
Layerwise invariance
3rd hidden layer2nd hidden layer1st hidden layer
0010203040506070809
1
Nor
mal
ised
MSE
minus20 minus10minus40 minus30 0 10 20 30 40 50minus50
Offset failure level ()
Figure 7 Layerwise invariance in DAASM Lower curves depicthigher invariance
Robustness
With 3rd hidden layerWith 2nd hidden layerWith 1st hidden layer
0203040506070809
1
minus20 minus10minus40 minus30 0 10 20 30 40 50minus50
Offset failure level ()
Robu
stnes
s (1minusS a
)
Figure 8 Robustness measure (1 minus autosensitivity) at multiplehidden layers in DAASM Higher curves depict high robustness
The MSE values of all sensors are normalized to theirrespective span and are presented as percent span in Figure 9Being an error measure the lower MSE values by DAASMsignify its prediction accuracy
82 Robustness Robustness is quantified through autosensi-tivity as defined by [32 33] It is themeasure of modelrsquos abilityto predict correct sensor values under missing or corruptedsensor states The measure is averaged over an operatingregion defined by 119896 samples from test data set as follows
119878119894
Auto =1
119873
119873
sum
119896=1
1003816100381610038161003816100381610038161003816100381610038161003816
drift119896119894
minus 119896119894
119904drift119896119894
minus 119904119896119894
1003816100381610038161003816100381610038161003816100381610038161003816
(16)
where 119894 and 119896 are indexes corresponding to sensors and theirrespective test samples 119904
119896119894is the original sensor value without
fault 119896119894is the model estimated sensor value against 119904
119896119894 119904drift119896119894
is the driftedfaulted sensor value drift119896119894
is themodel estimatedsensor value against drifted value 119904
drift119896119894
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN 0382 0361 0343 0412 0186 0211 0166 0386 0411 0243 0223 0512 0621
DAASM 0281 0253 0246 0293 0121 0132 0122 0243 0315 0173 0156 0394 0465
Accuracy metric
0
01
02
03
04
05
06
07
MSE
( sp
an)
Figure 9 MSE depicting DAASM and K-AANN accuracy on eachsensor
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Autosensitivity metric
0005
01015
02025
03503
Auto
sens
itivi
ty
0273 0294 0321 0332 0205 0182 0198 0253 0233 0225 0231 0268 0301
0153 0167 0187 0163 0112 0079 0082 0131 0071 014 0117 0133 0187
Figure 10 Autosensitivity values of individual sensors in bothmodels
The autosensitivitymetric lies in [0 1] range For autosen-sitivity value of one the model predictions follow the faultwith zero residuals hence no fault can be detected Smallerautosensitivity values are preferred which essentially meansdecreased sensitivity towards small perturbations Largeautosensitivity values may lead to missed alarms due tounderestimation of the fault size caused by small residualvalues Compared to K-AANN model in case of DAASM asignificant decrease in autosensitivity values for all sensors isobserved The plot in Figure 10 shows that DAASM is morerobust to failed sensor inputs
To further investigate robustness against large offsetfailures both models are evaluated against offset failuresin [5ndash50] range For each sensor samples from testdata are corrupted with specific offset level and correspond-ing autosensitivities are averaged over whole sensor setAutosensitivity values less than 02 are considered as robustThe maximum autosensitivity value of 0187 is observed insteam flow sensor The plot in Figure 11 shows that averageautosensitivity for both models increases with increasinglevel of offset failure However the autosensitivity curve forDAASM autosensitivity is well below the corresponding K-AANN curve
83 Spillover Depending upon the size and type of failurea failed sensor input can cause discrepancy in estimatedoutput for other sensors The phenomenon is referred to in
Science and Technology of Nuclear Installations 11
0060086
014
021
028
041
016021
028
036
043
054
005 01 02 03 04 05Sensor perturbation magnitude ( of actual value)
Robustness performance
0
01
02
03
04
05
06
Avg
auto
sens
itivi
ty
DAASMK-AANN
Figure 11 Comparison of robustness against increasing offsetfailure
literature as ldquospillover effectrdquo and is quantified through ldquocross-sensitivityrdquo metric [32] It quantifies the influence of faultysensor 119894 on predictions of sensor 119895 as follows
119878119895119894
Cross =1
119873
119873
sum
119896=1
10038161003816100381610038161003816100381610038161003816100381610038161003816
drift119896119895
minus 119896119895
119904drift119896119894
minus 119904119896119894
10038161003816100381610038161003816100381610038161003816100381610038161003816
(17)
119878119895
Cross =1
119873 minus 1
119873
sum
119894=1
119878119895119894
Cross 119894 = 119895 (18)
where 119894 and 119895 indexes are used to refer to faulty andnonfaulty sensors respectively Meanwhile 119896 is the index forcorresponding test samples 119878119895119894Cross is the cross-sensitivity ofsensor 119895 with respect to drift in 119894th sensor 119904
119896119894is the value of
119894th sensor without any fault 119896119895is the model estimated value
of 119895th sensor against 119904119896119894 119904drift119896119894
is the driftedfaulted value of 119894thsensor drift
119896119895is themodel estimated value of 119895th sensor against
drifted value 119904drift119896119894
The highly distributed representation of the input in
neural network based sensor models has pronounced effecton the cross-sensitivity performance Cross-sensitivitymetricvalue lies in [0 1] range High value of cross-sensitivitymay set off false alarms in other sensors provided theresidual values overshoot the fault detectability threshold inother sensors So minimum cross-sensitivity value is desiredfor a robust model The plot in Figure 12 shows that thecross-sensitivity for DAASM is reduced by a large factor ascompared to K-AANNmodel
The spillover effect against particular level of offset failurein [5ndash50] range is averaged over all sensors as follows
AvgCross Sensitivity (119870) =
sum119873
119895=1119878119895
cross(119870)
119873
offset failure level 119870 = 5 10 20 sdot sdot sdot 50
(19)
0128 0124 0113 0116 0112 0108 0092 0079 0083 0098 011 0103 0105
00632 00527 00563 00614 0039 00311 00287 00186 00203 00344 00336 003 00286
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Cross-sensitivity metric
0002004006008
01012014
Cros
s-se
nsiti
vity
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
Figure 12 Cross-sensitivity values of individual sensors in bothmodels
00810105
0161
0253
0312
041
0018 0026 0038 00460071
012
005 01 02 03 04 05Sensor perturbation magnitude ( of observed sensor value)
Spillover offset sensor failure
Requires SPRT mean relaxation to avoid
false alarmsahead of this point
0
005
01
015
02
025
03
035
04
045
Avg
cros
s-se
nsiti
vity
DAASMK-AANN
Figure 13 Comparison of spillover effects against increasing offsetfailure
The cross-sensitivity values 119878119895
cross(119870) against 119870 offset fail-ure level are calculated using (18) Figure 13 shows theaverage cross-sensitivity plot for both models Small cross-sensitivities are observed in DAASM which effectivelyavoided false alarms in other channels without relaxing theSPRT faulted mean value up to an offset failure of 35ndash40 inany channel However for the case of offset noise larger than35 SPRTmean needs to be relaxed to avoid false alarms andisolate the faulty sensor However Robustness of K-AANNmodel deteriorates significantly due to spillover effect beyond15 offset failure
Similarly gross failure scenarios corresponding to twoextremities of sensor range can cause severe Spillover effectTo study robustness against gross type failure scenario asubset of input sensors is simultaneously failed with grosshigh or low value and average cross-sensitivity of remainingsensor set is calculated using relation (19) Plot in Figure 14shows that average cross-sensitivity of K-AANN modelincreases drastically beyond 10 gross failure HoweverDAASM resulted in a very nominal spillover even in caseof multiple sensor failure The DAASM effectively managedsimultaneous gross high or low failures in 25 of total sensorset as compared to 10 in case of K-AANN
12 Science and Technology of Nuclear Installations
0028 0032 0041 012021016 024
0457
061
077
01 02 03 04 05Sensors failed with gross highlow error ()
Spillover gross sensor failure
0010203040506070809
Avg
cros
s-se
nsiti
vity
DAASMK-AANN
Figure 14 Comparison of spillover effect against simultaneous grosshighlow failure in multiple sensors
84 Fault Detectability Fault detectability metric measuresthe smallest fault that can be detected by integrated sensorestimation and fault detection module as shown in Figure 1[32] The detectability metric is measured as percentage ofsensor span 119863 = 119872Span where value M correspondsto minimum detectable fault Minimum fault detectabilitylimit for each sensor is quantified through statistical basedsequential probability ratio test (SPRT) by Wald [34] SPRTtest is carried out to detect if the residual being generatedfrom normal distribution 119873(120583
1 1205902
) or 119873(1205830 1205902
) as definedfor faulty and fault free sensor operations respectively [35]Calibration failures are reflected in the mean parameter ofresidualrsquos distribution The SPRT procedure is applied todetect changes in the mean of residualrsquos distribution Theapplication of SPRT requires setting of following parametersvalue [36]
1205830 normal mode residual mean
1205902 normal mode residual variance
1205831 expected offset in residual mean in abnormal
mode120572 false alarm probability120573 missed alarm probability
Under normal mode the residuals from observed and modelestimated sensor values behave as awhiteGaussian noisewithmean 120583
0= 0 The residual variance 120590
2 is estimated for eachsensor under normal operating conditions and remainedfixed The false alarm 120572 and missed alarm 120573 probabilities areset to be 0001 and 001 respectively In order to determineminimum fault detectability limit a numerical procedureis opted which searches for minimum expected offset 120583
1
in the interval 1205831
[120590ndash3120590] provided the constraint onmissed and false alarm rate holds 120590 is the standard deviationcorresponding to residual variance of particular sensor Theplot in Figure 15 shows the detectability metric for eachsensor The plot in Figure 15 shows that DAASM can detect
0312 0315 033 035 004 005 004 008 0086 01 009 006 0058
0184 0165 019 02 0021 0023 002 0048 005 004 004 0032 0028
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Fault detectability metric
0005
01015
02025
03035
04
Det
ecta
bilit
y (
span
)
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
Figure 15 Comparison of fault detectability metrics
faults which are two times smaller in magnitude than thosedetectable by K-AANNmodel
Improvement in fault detectability metric for DAASMcan be attributed to observed improvement in model robust-ness as suggested by the following relation
119903119894
Δ119878drift119894
= (1 minus 119878119894
Auto) (20)
The term 119903119894Δ119878
drift119894
measures the ratio of observed residualto actual sensor drift in terms of autosensitivity For highlyrobust model this ratio reduces to one which means residualreflects the actual drift and results in high fault detectabilityContrarily ratio value close to zero means that the predictionis following the input and results in poor fault detectability
841 SPRT Based Fault Detectability Test Sequential prob-ability ratio [34 36] based fault hypothesis test is appliedto residual sequence 119877
119894 = 119903
1(1199051) 1199031(1199051)sdot sdot sdot 119903119899(119905119899)generated
by relation 119877119894(119905119894) = 119878
Obs(119905119894) minus 119878
Est(119905119894) at time 119905
119894 where
119878Obs
(119905119894) and 119878
Est(119905119894) are the actual and model predicted
sensor values respectively The SPRT procedure analyzeswhether the residual sequence is more likely to be generatedfrom a probability distribution that belongs to normal modehypothesis 119867
0or abnormal mode hypothesis 119867
1by using
likelihood ratio as follows
119871119899= exp[minus(12120590
2)[sum119899
119894=11205831(1205831minus2119903119894)]]
(21)
For fault free sensor values the normalmode hypothesis1198670is
approximated byGaussian distributionwithmean 1205830= 0 and
variance 1205902 Abnormal mode hypothesis 119867
1is approximated
with mean 1205831
gt 1205830using the same variance 120590
2 The SPRTindex for the positive mean test is finally obtained by takinglogarithm of the likelihood ratio in (21) as follows [35]
SPRT = ln (119871119899) = minus
1
21205902[
119899
sum
119894=1
1205831(1205831minus 2119903119894)]
=1205831
1205902
119899
sum
119894=1
(119903119894minus
1205831
2)
(22)
Pressurizer pressure sensor sampled at a frequency of 10 sec-onds is used as a test signal to validate the fault detectability
Science and Technology of Nuclear Installations 13
0 500 1000 1500 2000 2500 3000 3500 4000 45001535
1541545
SPRT drift detection (pressure sensor)(B
ARG
)Pr
essu
re
Drifted sensorDAASM estimate
Sample number
0 500 1000 1500 2000 2500 3000 3500 4000 45001535
1541545
(BA
RG)
Pres
sure
Drifted sensorK-AANN estimate
Sample number
0515
SPRT
hypo
thes
is
minus050 500 1000 1500 2000 2500 3000 3500 4000 4500
DAASM indexK-AANN index
Sample number
005
valu
eRe
sidua
l
0 500 1000 1500 2000 2500 3000 3500 4000 4500
Residual DAASMResidual K-AANN
Sample number
minus05
H1mdash1
H0mdash0
Figure 16 SPRTbased fault detection in pressurizer pressure sensor
performance Two drift faults at the rate of +001hour andminus001hour are introduced in the test signal for DAASMand K-AANN modelrsquos assessment respectively The first andsecond plots in Figure 16 show drifted and estimated pressuresignal from DAASM and K-AANN models respectivelyThird plot shows residual values generated by differencing thedrifted and estimated signals frombothmodelsThe final plotshows SPRT index values against residuals from K-AANNmodel and DAASM The hypotheses 119867
1and 119867
0correspond
to positive and negative fault acceptance respectively FromSPRT index plot successful early detection of the sensordrift at 2200th sample with lag of 611 hours since the driftinception shows that DAASM is more sensitive to smalldrifts On the other hand SPRT index on K-AANN basedsensor estimates registered the same drift at 3800th samplewith a lag of almost 1055 hours The result shows thatDAASM is more robust in terms of early fault detection withlow false and missed alarm rates
Finally both models are tested against five test data setsEach test set consists of 3630 samples corresponding todifferentmonths of full power reactor operation Bothmodelssuccessfully detected an offset failure of 012ndash03 BARG in allsteam pressure channels and a drift type failure up to 285 insteam generator level (Figure 22)The K-AANNmodel failedto register a very small drift up to 01 in steam flow (STMflow 1) channel A small drift up to 01 BARG is detected in
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Steam pressure estimation
Observed valueDAASM estimateK-AANN estimate
672673674675676677678679
68
Stea
m p
ress
ure (
BARG
)
Figure 17 Steam pressure estimates against offset failures up to 03(BARG)
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Feed flow estimation
400420440460480500520540560
Feed
flow
(kg
s)
Observed sensor valueK-AANN estimateDAASM estimate
Figure 18 Feed flow estimates
test set 5 of pressurizer pressure channel However in caseof drift type sensor failures fault detection lag for DAASMwas on average 05 times smaller in comparison with K-AANN model Plots in Figures 17ndash21 show the estimatedsensor values from both models on five test data sets of fewselected channels
9 Conclusion
This paper presented a neural network based denoisedautoassociative sensor model (DAASM) for empirical sen-sor modeling The proposed sensor model is trained togenerate a monitoring system for sensor fault detection innuclear power plantsMultilayer AANNbased sensormodelsmay result in suboptimal solutions due to poor regulariza-tion by traditional backpropagation based joint multilayerlearning procedures So a complementary deep learningapproach based on greedy layerwise unsupervised pretrain-ing is employed for effective regularization in the proposedmultilayer DAASM Autoencoder architecture is used fordenoising based unsupervised pretraining and regularizationof individual layers in the network hierarchy To addressrobustness against perturbations in input sensors data cor-ruption processes exercised during unsupervised pretraining
14 Science and Technology of Nuclear Installations
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5490495500505510515520525530535540
Stea
m fl
ow (k
gs)
Steam flow estimation
Observed sensor valueDAASM estimateK-AANN estimate
Figure 19 Steam flow estimates against drift failure up to 01
58859
592594596598
60602604606608
Pres
suriz
er le
vel (
)
Pressurizer level estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valueK-AANN estimateDAASM estimate
Figure 20 Pressurizer level estimates
phase were based on prior knowledge about different failurescenarios Results from invariance tests showed that theproposed data corruption schemeswere beneficial in learninglatent representations at hidden layers and were invariantto multiple levels of perturbation in input sensors Conse-quently these pretrained hidden layers worked as well regu-larized perturbation filters with increased invariance towardssensor faults It is also observed that sensitivity against sensorfaults decreased significantly towards higher layers in fullDAASMassembly In a practical context of sensormonitoringin nuclear power plants the proposed model proved itsrobustness against gross type simultaneous sensor failuresIt also showed significant improvement in all performancemetrics when compared with popular and widely used five-layered AANNmodel by KramerMoreover time lag in smalldriftrsquos detection is significantly reduced The overall resultssuggest that greedy layerwise pretraining technique in com-bination with domain specific corruption processes providesa viable framework for effective regularization and robustnessin such deep multilayered autoassociative sensor validationmodels
Appendix
See Table 3
153715375
153815385
153915395
15415405
154115415
Pres
sure
(BA
RG)
Pressurizer pressure estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valuesK-AANN estimateDAASM estimate
Figure 21 Pressurizer pressure estimates against drift failure up to01
74
76
78
80
82
84
Stea
m g
ener
ator
leve
l (
)
Steam generator level estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valueK-AANN estimateDAASM estimate
Figure 22 Steam generator level estimates against growing driftfailure up to 287
Abbreviations
AANN Autoassociative neural networkK-AANN Kramer proposed Autoassociative neural
networkDAASM Denoised autoassociative sensor modelNPP Nuclear power plantPWR Pressurized water reactor119878Auto Autosensitivity119878Cross Cross-sensitivityDAE Denoising autoencoder119878 Observed sensor value Model predicted sensor value Corrupted sensor valueSPN Salt-and-pepper noiseAGN Additive Gaussian noise
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Science and Technology of Nuclear Installations 15
Table3Re
sults
ofDAASM
andK-AANNperfo
rmance
metric
sPerfo
rmance
metric
Mod
eltype
Feed
flow1
Feed
flow2
STM
flow1
STM
flow2
STM
PSR1
STM
PSR2
STM
PSR3
PZRPS
R1
PZRPS
R2
PZRLV
L1
PZRLV
L2
SGLV
LNR1
SGLV
LNR2
AANN
Sp-D
AANN
Accuracy
(span)
0382
0361
0343
0412
0186
0211
0166
0386
0411
0243
0223
0512
0621
0281
0253
0246
0293
0121
0132
0122
0243
0315
0173
0156
0394
0465
Autosensitivity
0273
0294
0321
0332
0205
0182
0198
0253
0233
0225
0231
0268
0301
0153
0167
0187
0163
0112
0079
0082
0131
0071
014
0117
0133
0187
Cross-sensitivity
0128
0124
0113
0116
0112
0108
0092
0079
0083
0098
011
0103
0105
00632
00527
00563
00614
0039
00311
00287
00186
00203
00344
00336
003
00286
SPRT
detectability
0312
0315
033
035
004
005
004
008
0086
01
009
006
0058
0184
0165
019
02
0021
0023
002
004
8005
004
004
0032
0028
16 Science and Technology of Nuclear Installations
Acknowledgments
The authors acknowledge the support of Mr Jose GalindoRodrıguez affiliated to TECNATOM Inc Spain and MrChad Painter (Director of Nuclear Power Plant SimulatorDevelopment and Training Program at International AtomicEnergy Agency) for providing necessary tools and data toconduct this research
References
[1] N Mehranbod M Soroush M Piovoso and B A OgunnaikeldquoProbabilistic model for sensor fault detection and identifica-tionrdquo AIChE Journal vol 49 no 7 pp 1787ndash1802 2003
[2] JW Hines and E Davis ldquoLessons learned from the US nuclearpower plant on-line monitoring programsrdquo Progress in NuclearEnergy vol 46 no 3-4 pp 176ndash189 2005
[3] EPRI ldquoOn-line monitoring cost benefit guiderdquo Final Report1006777 EPRI Palo Alto Calif USA 2003
[4] S J Qin and W Li ldquoDetection identification and recon-struction of faulty sensors with maximized sensitivityrdquo AIChEJournal vol 45 no 9 pp 1963ndash1976 1999
[5] J Ma and J Jiang ldquoApplications of fault detection and diagnosismethods in nuclear power plants a reviewrdquo Progress in NuclearEnergy vol 53 no 3 pp 255ndash266 2011
[6] G YHeo ldquoConditionmonitoring using empiricalmodels tech-nical review and prospects for nuclear applicationsrdquo NuclearEngineering and Technology vol 40 no 1 pp 49ndash68 2008
[7] J B Coble R M Meyer P Ramuhalli et al ldquoA review ofsensor calibration monitoring for calibration interval extensionin nuclear power plantsrdquo Tech Rep PNNL-21687 PacificNorthwest National Laboratory Richland Wash USA 2012
[8] M A Kramer ldquoAutoassociative neural networksrdquo Computersand Chemical Engineering vol 16 no 4 pp 313ndash328 1992
[9] X Xu J W Hines and R E Uhrig ldquoOn-line sensor calibra-tion monitoring and fault detection for chemical processesrdquoin Proceedings of the Maintenance and Reliability Conference(MARCON rsquo98) pp 12ndash14 Knoxville Tenn USA May 1998
[10] M Hamidreza S Mehdi J-R Hooshang and N AliakbarldquoReconstruction based approach to sensor fault diagnosis usingauto-associative neural networksrdquo Journal of Central SouthUniversity vol 21 no 6 pp 2273ndash2281 2014
[11] UThissenW JMelssen andLMC Buydens ldquoNonlinear pro-cess monitoring using bottle-neck neural networksrdquo AnalyticaChimica Acta vol 446 no 1-2 pp 371ndash383 2001
[12] D J Wrest J W Hines and R E Uhrig ldquoInstrument surveil-lance and calibration verification through plant wide monitor-ing using autoassociative neural networksrdquo in Proceedings ofthe American Nuclear Society International Topical Meeting onNuclear Plant Instrumentation Control and Human MachineInterface Technologies University Park Pa USA May 1996
[13] J W Hines R E Uhrig and D J Wrest ldquoUse of autoassociativeneural networks for signal validationrdquo Journal of Intelligent andRobotic Systems Theory and Applications vol 21 no 2 pp 143ndash154 1998
[14] P F Fantoni M I Hoffmann R Shankar and E L Davis ldquoOn-line monitoring of instrument channel performance in nuclearpower plant using PEANOrdquo Progress in Nuclear Energy vol 43no 1ndash4 pp 83ndash89 2003
[15] MMarseguerra andA Zoia ldquoTheAutoAssociative Neural Net-work in signal analysis II Application to on-line monitoring of
a simulated BWR componentrdquo Annals of Nuclear Energy vol32 no 11 pp 1207ndash1223 2005
[16] M S Ikbal H Misra and B Yegnanarayana ldquoAnalysis ofautoassociative mapping neural networksrdquo in Proceedings of theInternational Joint Conference on Neural Networks (IJCNN rsquo99)vol 5 pp 3025ndash3029 IEEE Washington DC USA July 1999Proceedings (Cat no 99CH36339)
[17] Y Bengio ldquoLearning deep architectures for AIrdquo Foundationsand Trends in Machine Learning vol 2 no 1 pp 1ndash27 2009
[18] J W Hines A V Gribok I Attieh and R E Uhrig ldquoRegular-izationmethods for inferential sensing in nuclear power plantsrdquoin Fuzzy Systems and Soft Computing in Nuclear Engineering DRuan Ed vol 38 of Studies in Fuzziness and Soft Computingpp 285ndash314 Physica 2000
[19] A V Gribok J W Hines A Urmanov and R E UhrigldquoHeuristic systematic and informational regularization forprocessmonitoringrdquo International Journal of Intelligent Systemsvol 17 no 8 pp 723ndash749 2002
[20] X Glorot and Y Bengio ldquoUnderstanding the difficulty oftraining deep feedforward neural networksrdquo in Proceedings ofthe 13th International Conference on Artificial Intelligence andStatistics (AISTATS rsquo10) vol 9 pp 249ndash256 Sardinia Italy May2010
[21] S Tan andM L Mayrovouniotis ldquoReducing data dimensional-ity through optimizing neural network inputsrdquo AIChE Journalvol 41 no 6 pp 1471ndash1480 1995
[22] Y Bengio and P Lamblin ldquoGreedy layer-wise training of deepnetworksrdquo in Proceedings of the Advances in Neural InformationProcessing Systems 19 (NIPS rsquo07) pp 153ndash160 September 2007
[23] H Larochelle Y Bengio J Lourador and P Lamblin ldquoExplor-ing strategies for training deep neural networksrdquoThe Journal ofMachine Learning Research vol 10 pp 1ndash40 2009
[24] D Yu and L Deng ldquoDeep learning and its applications to signaland information processing [exploratory DSP]rdquo IEEE SignalProcessing Magazine vol 28 no 1 pp 145ndash154 2011
[25] D ErhanA Courville andPVincent ldquoWhydoes unsupervisedpre-training help deep learningrdquo Journal of Machine LearningResearch vol 11 pp 625ndash660 2010
[26] P Vincent H Larochelle Y Bengio and P-A ManzagolldquoExtracting and composing robust features with denoisingautoencodersrdquo in Proceedings of the 25th International Confer-ence on Machine Learning (ICML rsquo08) pp 1096ndash1103 July 2008
[27] P Vincent H Larochelle I Lajoie Y Bengio and P-AManzagol ldquoStacked denoising autoencoders learning usefulrepresentations in a deep network with a local denoisingcriterionrdquo Journal of Machine Learning Research vol 11 pp3371ndash3408 2010
[28] N Srivastava G Hinton A Krizhevsky I Sutskever and RSalakhutdinov ldquoDropout a simple way to prevent neural net-works from overfittingrdquo Journal of Machine Learning Researchvol 15 pp 1929ndash1958 2014
[29] M J Embrechts B J Hargis and J D Linton ldquoAn augmentedefficient backpropagation training strategy for deep autoasso-ciative neural networksrdquo in Proceedings of the 18th EuropeanSymposiumonArtificial NeuralNetworksmdashComputational Intel-ligence and Machine Learning (ESANN rsquo10) vol no pp 28ndash30April 2010
[30] Y Bengio ldquoPractical recommendations for gradient-basedtraining of deep architecturesrdquo inNeural Networks Tricks of theTrade vol 7700 of Lecture Notes in Computer Science pp 437ndash478 Springer Berlin Germany 2nd edition 2012
Science and Technology of Nuclear Installations 17
[31] J Bergstra and Y Bengio ldquoRandom search for hyper-parameteroptimizationrdquo Journal of Machine Learning Research vol 13 pp281ndash305 2012
[32] J W Hines and D R Garvey ldquoDevelopment and applicationof fault detectability performance metrics for instrument cali-bration verification and anomaly detectionrdquo Journal of PatternRecognition Research vol 1 no 1 pp 2ndash15 2006
[33] A Usynin and J W Hines ldquoOn-line monitoring robustnessmeasures and comparisonsrdquo in Proceedings of the Interna-tional Atomic Energy Agency Technical Meeting on ldquoIncreasingInstrument Calibration Interval through On-Line CalibrationTechnologyrdquo OECD Halden Reactor Project Halden NorwaySeptember 2004
[34] A Wald ldquoSequential tests of statistical hypothesesrdquo Annals ofMathematical Statistics vol 16 no 2 pp 117ndash186 1945
[35] F Di Maio P Baraldi E Zio and R Seraoui ldquoFault detection innuclear power plants components by a combination of statisticalmethodsrdquo IEEE Transactions on Reliability vol 62 no 4 pp833ndash845 2013
[36] S Cheng and M Pecht ldquoUsing cross-validation for modelparameter selection of sequential probability ratio testrdquo ExpertSystems with Applications vol 39 no 9 pp 8467ndash8473 2012
TribologyAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
FuelsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal ofPetroleum Engineering
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Power ElectronicsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
CombustionJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Renewable Energy
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
StructuresJournal of
International Journal of
RotatingMachinery
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporation httpwwwhindawicom
Journal ofEngineeringVolume 2014
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
International Journal ofPhotoenergy
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear InstallationsScience and Technology of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Solar EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Wind EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear EnergyInternational Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
High Energy PhysicsAdvances in
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Science and Technology of Nuclear Installations 9
Table 2 Summary of DAASM hyperparameters
Hyperparameter type Tested hyperparameter values Successful hyperparameteragainst optimum model
Pretrained DAE units 3 3
Network architecture
119868119899minus 119864(119899 119901) minus 119861
119899minus 119863(119899 119901) minus 119900
119899
119868119899 input layer neurons
119861119899 bottleneck layer neurons119900119899 output layer neurons
119864(119899 119901) encoder cascade119863(119899 119901) decoder cascade
119899 number of layers119901 neurons per layer
13 minus 119864(2 20) minus 8 minus
119863(2 20) minus 13
Learning rate forunsupervised pretraining 01 5 times 10
minus2
1 times 10minus2
5 times 10minus3
1 times 10minus3
5 times 10minus4
[5 times 10minus2 1 times 10
minus2
]
Learning rate forsupervised training
Scheduled learning rate based on training errormonitoring
015 01 0005 0001 00001
01 0005 0001
Mean pretraining error foreach hidden layer
Corresponding to minima observed during crossvalidation 10
minus4
Weight decay 120582 10minus3
10minus4
10minus5
10minus3
Momentum 119898 [085 099] [095 098]
Input corruption level ]Corrupted input fraction 10 25 30 40
Gaussian corruption ( of sensorrsquos nominal value)005 010 020 035 050
Input fraction [25ndash35]Gaussian noise level
[010ndash025]Dropout fraction in DAE-3 010 020 01
MSE (119867119897Offset) =
(1119879119873)sum119879119873
119899=1(⟨ℎ119894⟩1198991003817100381710038171003817⟨ℎ119894⟩119899
10038171003817100381710038172minus ⟨ℎ119894⟩119899
10038171003817100381710038171003817⟨ℎ119894⟩119899
100381710038171003817100381710038172)2
119863ℎ
1 le 119897 le 119871 = No of encoder layers = 3 Offset = 5 10 20 sdot sdot sdot 50
(13)
MSENormalized (119867119897) =
MSE (119867119897Offset)
MSE (119867119897Max-Offset)
(14)
LayerwiseMSEplots in Figure 7 clearly show that invarianceto faulty sensor conditions increases towards higher layers inthe network hierarchy In these plots lower curves indicatehigher level of invariance To further investigate the effectof increasing invariance on reconstructed sensor values asensor model corresponding to the level ldquo119871rdquo of each hiddenlayer is assembled via encoder and decoder cascade Robust-ness of these partial models is quantified through (1 minus 119878
119894
Auto)Autosensitivity values 119878
119894
Auto (see Section 82) are calculatedagainst varying offset failure levels In Figure 8 layerwiseincrease in robustness confirms that increased invariancehelps in improving overall modelrsquos robustness
8 DAASM versus K-AANNPerformance Analysis
Here we will assess and compare the performance of DAASMwith popular five-layer AANNmodel originally proposed byKramer [8] The K-AANN model is trained with same data
set as used for DAASM and is regularized with Levenberg-Marquardt algorithm Furthermore to improve robustnesstraining with jitter heuristic is employed by introducing anoise of 10 magnitude on clean sensor input The five-layertopology 13-17-9-17-13 is found to be optimum for K-AANNmodel Both DAASM and K-AANN model are comparedthrough accuracy robustness spillover and fault detectabilitybased performance metrics in the following subsections Allperformance metrics are calculated against test data setconsisting of 4320 samples frompostrefueling full powerNPPoperations Performance metric values are reported
81 Accuracy Mean square error (MSE) of observed andmodel estimated sensor values against fault free test data setis used to quantify accuracy metric as follows
Accuracy =1
119873
119873
sum
119894=1
(119894minus 119878119894)2
(15)
10 Science and Technology of Nuclear Installations
Layerwise invariance
3rd hidden layer2nd hidden layer1st hidden layer
0010203040506070809
1
Nor
mal
ised
MSE
minus20 minus10minus40 minus30 0 10 20 30 40 50minus50
Offset failure level ()
Figure 7 Layerwise invariance in DAASM Lower curves depicthigher invariance
Robustness
With 3rd hidden layerWith 2nd hidden layerWith 1st hidden layer
0203040506070809
1
minus20 minus10minus40 minus30 0 10 20 30 40 50minus50
Offset failure level ()
Robu
stnes
s (1minusS a
)
Figure 8 Robustness measure (1 minus autosensitivity) at multiplehidden layers in DAASM Higher curves depict high robustness
The MSE values of all sensors are normalized to theirrespective span and are presented as percent span in Figure 9Being an error measure the lower MSE values by DAASMsignify its prediction accuracy
82 Robustness Robustness is quantified through autosensi-tivity as defined by [32 33] It is themeasure of modelrsquos abilityto predict correct sensor values under missing or corruptedsensor states The measure is averaged over an operatingregion defined by 119896 samples from test data set as follows
119878119894
Auto =1
119873
119873
sum
119896=1
1003816100381610038161003816100381610038161003816100381610038161003816
drift119896119894
minus 119896119894
119904drift119896119894
minus 119904119896119894
1003816100381610038161003816100381610038161003816100381610038161003816
(16)
where 119894 and 119896 are indexes corresponding to sensors and theirrespective test samples 119904
119896119894is the original sensor value without
fault 119896119894is the model estimated sensor value against 119904
119896119894 119904drift119896119894
is the driftedfaulted sensor value drift119896119894
is themodel estimatedsensor value against drifted value 119904
drift119896119894
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN 0382 0361 0343 0412 0186 0211 0166 0386 0411 0243 0223 0512 0621
DAASM 0281 0253 0246 0293 0121 0132 0122 0243 0315 0173 0156 0394 0465
Accuracy metric
0
01
02
03
04
05
06
07
MSE
( sp
an)
Figure 9 MSE depicting DAASM and K-AANN accuracy on eachsensor
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Autosensitivity metric
0005
01015
02025
03503
Auto
sens
itivi
ty
0273 0294 0321 0332 0205 0182 0198 0253 0233 0225 0231 0268 0301
0153 0167 0187 0163 0112 0079 0082 0131 0071 014 0117 0133 0187
Figure 10 Autosensitivity values of individual sensors in bothmodels
The autosensitivitymetric lies in [0 1] range For autosen-sitivity value of one the model predictions follow the faultwith zero residuals hence no fault can be detected Smallerautosensitivity values are preferred which essentially meansdecreased sensitivity towards small perturbations Largeautosensitivity values may lead to missed alarms due tounderestimation of the fault size caused by small residualvalues Compared to K-AANN model in case of DAASM asignificant decrease in autosensitivity values for all sensors isobserved The plot in Figure 10 shows that DAASM is morerobust to failed sensor inputs
To further investigate robustness against large offsetfailures both models are evaluated against offset failuresin [5ndash50] range For each sensor samples from testdata are corrupted with specific offset level and correspond-ing autosensitivities are averaged over whole sensor setAutosensitivity values less than 02 are considered as robustThe maximum autosensitivity value of 0187 is observed insteam flow sensor The plot in Figure 11 shows that averageautosensitivity for both models increases with increasinglevel of offset failure However the autosensitivity curve forDAASM autosensitivity is well below the corresponding K-AANN curve
83 Spillover Depending upon the size and type of failurea failed sensor input can cause discrepancy in estimatedoutput for other sensors The phenomenon is referred to in
Science and Technology of Nuclear Installations 11
0060086
014
021
028
041
016021
028
036
043
054
005 01 02 03 04 05Sensor perturbation magnitude ( of actual value)
Robustness performance
0
01
02
03
04
05
06
Avg
auto
sens
itivi
ty
DAASMK-AANN
Figure 11 Comparison of robustness against increasing offsetfailure
literature as ldquospillover effectrdquo and is quantified through ldquocross-sensitivityrdquo metric [32] It quantifies the influence of faultysensor 119894 on predictions of sensor 119895 as follows
119878119895119894
Cross =1
119873
119873
sum
119896=1
10038161003816100381610038161003816100381610038161003816100381610038161003816
drift119896119895
minus 119896119895
119904drift119896119894
minus 119904119896119894
10038161003816100381610038161003816100381610038161003816100381610038161003816
(17)
119878119895
Cross =1
119873 minus 1
119873
sum
119894=1
119878119895119894
Cross 119894 = 119895 (18)
where 119894 and 119895 indexes are used to refer to faulty andnonfaulty sensors respectively Meanwhile 119896 is the index forcorresponding test samples 119878119895119894Cross is the cross-sensitivity ofsensor 119895 with respect to drift in 119894th sensor 119904
119896119894is the value of
119894th sensor without any fault 119896119895is the model estimated value
of 119895th sensor against 119904119896119894 119904drift119896119894
is the driftedfaulted value of 119894thsensor drift
119896119895is themodel estimated value of 119895th sensor against
drifted value 119904drift119896119894
The highly distributed representation of the input in
neural network based sensor models has pronounced effecton the cross-sensitivity performance Cross-sensitivitymetricvalue lies in [0 1] range High value of cross-sensitivitymay set off false alarms in other sensors provided theresidual values overshoot the fault detectability threshold inother sensors So minimum cross-sensitivity value is desiredfor a robust model The plot in Figure 12 shows that thecross-sensitivity for DAASM is reduced by a large factor ascompared to K-AANNmodel
The spillover effect against particular level of offset failurein [5ndash50] range is averaged over all sensors as follows
AvgCross Sensitivity (119870) =
sum119873
119895=1119878119895
cross(119870)
119873
offset failure level 119870 = 5 10 20 sdot sdot sdot 50
(19)
0128 0124 0113 0116 0112 0108 0092 0079 0083 0098 011 0103 0105
00632 00527 00563 00614 0039 00311 00287 00186 00203 00344 00336 003 00286
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Cross-sensitivity metric
0002004006008
01012014
Cros
s-se
nsiti
vity
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
Figure 12 Cross-sensitivity values of individual sensors in bothmodels
00810105
0161
0253
0312
041
0018 0026 0038 00460071
012
005 01 02 03 04 05Sensor perturbation magnitude ( of observed sensor value)
Spillover offset sensor failure
Requires SPRT mean relaxation to avoid
false alarmsahead of this point
0
005
01
015
02
025
03
035
04
045
Avg
cros
s-se
nsiti
vity
DAASMK-AANN
Figure 13 Comparison of spillover effects against increasing offsetfailure
The cross-sensitivity values 119878119895
cross(119870) against 119870 offset fail-ure level are calculated using (18) Figure 13 shows theaverage cross-sensitivity plot for both models Small cross-sensitivities are observed in DAASM which effectivelyavoided false alarms in other channels without relaxing theSPRT faulted mean value up to an offset failure of 35ndash40 inany channel However for the case of offset noise larger than35 SPRTmean needs to be relaxed to avoid false alarms andisolate the faulty sensor However Robustness of K-AANNmodel deteriorates significantly due to spillover effect beyond15 offset failure
Similarly gross failure scenarios corresponding to twoextremities of sensor range can cause severe Spillover effectTo study robustness against gross type failure scenario asubset of input sensors is simultaneously failed with grosshigh or low value and average cross-sensitivity of remainingsensor set is calculated using relation (19) Plot in Figure 14shows that average cross-sensitivity of K-AANN modelincreases drastically beyond 10 gross failure HoweverDAASM resulted in a very nominal spillover even in caseof multiple sensor failure The DAASM effectively managedsimultaneous gross high or low failures in 25 of total sensorset as compared to 10 in case of K-AANN
12 Science and Technology of Nuclear Installations
0028 0032 0041 012021016 024
0457
061
077
01 02 03 04 05Sensors failed with gross highlow error ()
Spillover gross sensor failure
0010203040506070809
Avg
cros
s-se
nsiti
vity
DAASMK-AANN
Figure 14 Comparison of spillover effect against simultaneous grosshighlow failure in multiple sensors
84 Fault Detectability Fault detectability metric measuresthe smallest fault that can be detected by integrated sensorestimation and fault detection module as shown in Figure 1[32] The detectability metric is measured as percentage ofsensor span 119863 = 119872Span where value M correspondsto minimum detectable fault Minimum fault detectabilitylimit for each sensor is quantified through statistical basedsequential probability ratio test (SPRT) by Wald [34] SPRTtest is carried out to detect if the residual being generatedfrom normal distribution 119873(120583
1 1205902
) or 119873(1205830 1205902
) as definedfor faulty and fault free sensor operations respectively [35]Calibration failures are reflected in the mean parameter ofresidualrsquos distribution The SPRT procedure is applied todetect changes in the mean of residualrsquos distribution Theapplication of SPRT requires setting of following parametersvalue [36]
1205830 normal mode residual mean
1205902 normal mode residual variance
1205831 expected offset in residual mean in abnormal
mode120572 false alarm probability120573 missed alarm probability
Under normal mode the residuals from observed and modelestimated sensor values behave as awhiteGaussian noisewithmean 120583
0= 0 The residual variance 120590
2 is estimated for eachsensor under normal operating conditions and remainedfixed The false alarm 120572 and missed alarm 120573 probabilities areset to be 0001 and 001 respectively In order to determineminimum fault detectability limit a numerical procedureis opted which searches for minimum expected offset 120583
1
in the interval 1205831
[120590ndash3120590] provided the constraint onmissed and false alarm rate holds 120590 is the standard deviationcorresponding to residual variance of particular sensor Theplot in Figure 15 shows the detectability metric for eachsensor The plot in Figure 15 shows that DAASM can detect
0312 0315 033 035 004 005 004 008 0086 01 009 006 0058
0184 0165 019 02 0021 0023 002 0048 005 004 004 0032 0028
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Fault detectability metric
0005
01015
02025
03035
04
Det
ecta
bilit
y (
span
)
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
Figure 15 Comparison of fault detectability metrics
faults which are two times smaller in magnitude than thosedetectable by K-AANNmodel
Improvement in fault detectability metric for DAASMcan be attributed to observed improvement in model robust-ness as suggested by the following relation
119903119894
Δ119878drift119894
= (1 minus 119878119894
Auto) (20)
The term 119903119894Δ119878
drift119894
measures the ratio of observed residualto actual sensor drift in terms of autosensitivity For highlyrobust model this ratio reduces to one which means residualreflects the actual drift and results in high fault detectabilityContrarily ratio value close to zero means that the predictionis following the input and results in poor fault detectability
841 SPRT Based Fault Detectability Test Sequential prob-ability ratio [34 36] based fault hypothesis test is appliedto residual sequence 119877
119894 = 119903
1(1199051) 1199031(1199051)sdot sdot sdot 119903119899(119905119899)generated
by relation 119877119894(119905119894) = 119878
Obs(119905119894) minus 119878
Est(119905119894) at time 119905
119894 where
119878Obs
(119905119894) and 119878
Est(119905119894) are the actual and model predicted
sensor values respectively The SPRT procedure analyzeswhether the residual sequence is more likely to be generatedfrom a probability distribution that belongs to normal modehypothesis 119867
0or abnormal mode hypothesis 119867
1by using
likelihood ratio as follows
119871119899= exp[minus(12120590
2)[sum119899
119894=11205831(1205831minus2119903119894)]]
(21)
For fault free sensor values the normalmode hypothesis1198670is
approximated byGaussian distributionwithmean 1205830= 0 and
variance 1205902 Abnormal mode hypothesis 119867
1is approximated
with mean 1205831
gt 1205830using the same variance 120590
2 The SPRTindex for the positive mean test is finally obtained by takinglogarithm of the likelihood ratio in (21) as follows [35]
SPRT = ln (119871119899) = minus
1
21205902[
119899
sum
119894=1
1205831(1205831minus 2119903119894)]
=1205831
1205902
119899
sum
119894=1
(119903119894minus
1205831
2)
(22)
Pressurizer pressure sensor sampled at a frequency of 10 sec-onds is used as a test signal to validate the fault detectability
Science and Technology of Nuclear Installations 13
0 500 1000 1500 2000 2500 3000 3500 4000 45001535
1541545
SPRT drift detection (pressure sensor)(B
ARG
)Pr
essu
re
Drifted sensorDAASM estimate
Sample number
0 500 1000 1500 2000 2500 3000 3500 4000 45001535
1541545
(BA
RG)
Pres
sure
Drifted sensorK-AANN estimate
Sample number
0515
SPRT
hypo
thes
is
minus050 500 1000 1500 2000 2500 3000 3500 4000 4500
DAASM indexK-AANN index
Sample number
005
valu
eRe
sidua
l
0 500 1000 1500 2000 2500 3000 3500 4000 4500
Residual DAASMResidual K-AANN
Sample number
minus05
H1mdash1
H0mdash0
Figure 16 SPRTbased fault detection in pressurizer pressure sensor
performance Two drift faults at the rate of +001hour andminus001hour are introduced in the test signal for DAASMand K-AANN modelrsquos assessment respectively The first andsecond plots in Figure 16 show drifted and estimated pressuresignal from DAASM and K-AANN models respectivelyThird plot shows residual values generated by differencing thedrifted and estimated signals frombothmodelsThe final plotshows SPRT index values against residuals from K-AANNmodel and DAASM The hypotheses 119867
1and 119867
0correspond
to positive and negative fault acceptance respectively FromSPRT index plot successful early detection of the sensordrift at 2200th sample with lag of 611 hours since the driftinception shows that DAASM is more sensitive to smalldrifts On the other hand SPRT index on K-AANN basedsensor estimates registered the same drift at 3800th samplewith a lag of almost 1055 hours The result shows thatDAASM is more robust in terms of early fault detection withlow false and missed alarm rates
Finally both models are tested against five test data setsEach test set consists of 3630 samples corresponding todifferentmonths of full power reactor operation Bothmodelssuccessfully detected an offset failure of 012ndash03 BARG in allsteam pressure channels and a drift type failure up to 285 insteam generator level (Figure 22)The K-AANNmodel failedto register a very small drift up to 01 in steam flow (STMflow 1) channel A small drift up to 01 BARG is detected in
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Steam pressure estimation
Observed valueDAASM estimateK-AANN estimate
672673674675676677678679
68
Stea
m p
ress
ure (
BARG
)
Figure 17 Steam pressure estimates against offset failures up to 03(BARG)
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Feed flow estimation
400420440460480500520540560
Feed
flow
(kg
s)
Observed sensor valueK-AANN estimateDAASM estimate
Figure 18 Feed flow estimates
test set 5 of pressurizer pressure channel However in caseof drift type sensor failures fault detection lag for DAASMwas on average 05 times smaller in comparison with K-AANN model Plots in Figures 17ndash21 show the estimatedsensor values from both models on five test data sets of fewselected channels
9 Conclusion
This paper presented a neural network based denoisedautoassociative sensor model (DAASM) for empirical sen-sor modeling The proposed sensor model is trained togenerate a monitoring system for sensor fault detection innuclear power plantsMultilayer AANNbased sensormodelsmay result in suboptimal solutions due to poor regulariza-tion by traditional backpropagation based joint multilayerlearning procedures So a complementary deep learningapproach based on greedy layerwise unsupervised pretrain-ing is employed for effective regularization in the proposedmultilayer DAASM Autoencoder architecture is used fordenoising based unsupervised pretraining and regularizationof individual layers in the network hierarchy To addressrobustness against perturbations in input sensors data cor-ruption processes exercised during unsupervised pretraining
14 Science and Technology of Nuclear Installations
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5490495500505510515520525530535540
Stea
m fl
ow (k
gs)
Steam flow estimation
Observed sensor valueDAASM estimateK-AANN estimate
Figure 19 Steam flow estimates against drift failure up to 01
58859
592594596598
60602604606608
Pres
suriz
er le
vel (
)
Pressurizer level estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valueK-AANN estimateDAASM estimate
Figure 20 Pressurizer level estimates
phase were based on prior knowledge about different failurescenarios Results from invariance tests showed that theproposed data corruption schemeswere beneficial in learninglatent representations at hidden layers and were invariantto multiple levels of perturbation in input sensors Conse-quently these pretrained hidden layers worked as well regu-larized perturbation filters with increased invariance towardssensor faults It is also observed that sensitivity against sensorfaults decreased significantly towards higher layers in fullDAASMassembly In a practical context of sensormonitoringin nuclear power plants the proposed model proved itsrobustness against gross type simultaneous sensor failuresIt also showed significant improvement in all performancemetrics when compared with popular and widely used five-layered AANNmodel by KramerMoreover time lag in smalldriftrsquos detection is significantly reduced The overall resultssuggest that greedy layerwise pretraining technique in com-bination with domain specific corruption processes providesa viable framework for effective regularization and robustnessin such deep multilayered autoassociative sensor validationmodels
Appendix
See Table 3
153715375
153815385
153915395
15415405
154115415
Pres
sure
(BA
RG)
Pressurizer pressure estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valuesK-AANN estimateDAASM estimate
Figure 21 Pressurizer pressure estimates against drift failure up to01
74
76
78
80
82
84
Stea
m g
ener
ator
leve
l (
)
Steam generator level estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valueK-AANN estimateDAASM estimate
Figure 22 Steam generator level estimates against growing driftfailure up to 287
Abbreviations
AANN Autoassociative neural networkK-AANN Kramer proposed Autoassociative neural
networkDAASM Denoised autoassociative sensor modelNPP Nuclear power plantPWR Pressurized water reactor119878Auto Autosensitivity119878Cross Cross-sensitivityDAE Denoising autoencoder119878 Observed sensor value Model predicted sensor value Corrupted sensor valueSPN Salt-and-pepper noiseAGN Additive Gaussian noise
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Science and Technology of Nuclear Installations 15
Table3Re
sults
ofDAASM
andK-AANNperfo
rmance
metric
sPerfo
rmance
metric
Mod
eltype
Feed
flow1
Feed
flow2
STM
flow1
STM
flow2
STM
PSR1
STM
PSR2
STM
PSR3
PZRPS
R1
PZRPS
R2
PZRLV
L1
PZRLV
L2
SGLV
LNR1
SGLV
LNR2
AANN
Sp-D
AANN
Accuracy
(span)
0382
0361
0343
0412
0186
0211
0166
0386
0411
0243
0223
0512
0621
0281
0253
0246
0293
0121
0132
0122
0243
0315
0173
0156
0394
0465
Autosensitivity
0273
0294
0321
0332
0205
0182
0198
0253
0233
0225
0231
0268
0301
0153
0167
0187
0163
0112
0079
0082
0131
0071
014
0117
0133
0187
Cross-sensitivity
0128
0124
0113
0116
0112
0108
0092
0079
0083
0098
011
0103
0105
00632
00527
00563
00614
0039
00311
00287
00186
00203
00344
00336
003
00286
SPRT
detectability
0312
0315
033
035
004
005
004
008
0086
01
009
006
0058
0184
0165
019
02
0021
0023
002
004
8005
004
004
0032
0028
16 Science and Technology of Nuclear Installations
Acknowledgments
The authors acknowledge the support of Mr Jose GalindoRodrıguez affiliated to TECNATOM Inc Spain and MrChad Painter (Director of Nuclear Power Plant SimulatorDevelopment and Training Program at International AtomicEnergy Agency) for providing necessary tools and data toconduct this research
References
[1] N Mehranbod M Soroush M Piovoso and B A OgunnaikeldquoProbabilistic model for sensor fault detection and identifica-tionrdquo AIChE Journal vol 49 no 7 pp 1787ndash1802 2003
[2] JW Hines and E Davis ldquoLessons learned from the US nuclearpower plant on-line monitoring programsrdquo Progress in NuclearEnergy vol 46 no 3-4 pp 176ndash189 2005
[3] EPRI ldquoOn-line monitoring cost benefit guiderdquo Final Report1006777 EPRI Palo Alto Calif USA 2003
[4] S J Qin and W Li ldquoDetection identification and recon-struction of faulty sensors with maximized sensitivityrdquo AIChEJournal vol 45 no 9 pp 1963ndash1976 1999
[5] J Ma and J Jiang ldquoApplications of fault detection and diagnosismethods in nuclear power plants a reviewrdquo Progress in NuclearEnergy vol 53 no 3 pp 255ndash266 2011
[6] G YHeo ldquoConditionmonitoring using empiricalmodels tech-nical review and prospects for nuclear applicationsrdquo NuclearEngineering and Technology vol 40 no 1 pp 49ndash68 2008
[7] J B Coble R M Meyer P Ramuhalli et al ldquoA review ofsensor calibration monitoring for calibration interval extensionin nuclear power plantsrdquo Tech Rep PNNL-21687 PacificNorthwest National Laboratory Richland Wash USA 2012
[8] M A Kramer ldquoAutoassociative neural networksrdquo Computersand Chemical Engineering vol 16 no 4 pp 313ndash328 1992
[9] X Xu J W Hines and R E Uhrig ldquoOn-line sensor calibra-tion monitoring and fault detection for chemical processesrdquoin Proceedings of the Maintenance and Reliability Conference(MARCON rsquo98) pp 12ndash14 Knoxville Tenn USA May 1998
[10] M Hamidreza S Mehdi J-R Hooshang and N AliakbarldquoReconstruction based approach to sensor fault diagnosis usingauto-associative neural networksrdquo Journal of Central SouthUniversity vol 21 no 6 pp 2273ndash2281 2014
[11] UThissenW JMelssen andLMC Buydens ldquoNonlinear pro-cess monitoring using bottle-neck neural networksrdquo AnalyticaChimica Acta vol 446 no 1-2 pp 371ndash383 2001
[12] D J Wrest J W Hines and R E Uhrig ldquoInstrument surveil-lance and calibration verification through plant wide monitor-ing using autoassociative neural networksrdquo in Proceedings ofthe American Nuclear Society International Topical Meeting onNuclear Plant Instrumentation Control and Human MachineInterface Technologies University Park Pa USA May 1996
[13] J W Hines R E Uhrig and D J Wrest ldquoUse of autoassociativeneural networks for signal validationrdquo Journal of Intelligent andRobotic Systems Theory and Applications vol 21 no 2 pp 143ndash154 1998
[14] P F Fantoni M I Hoffmann R Shankar and E L Davis ldquoOn-line monitoring of instrument channel performance in nuclearpower plant using PEANOrdquo Progress in Nuclear Energy vol 43no 1ndash4 pp 83ndash89 2003
[15] MMarseguerra andA Zoia ldquoTheAutoAssociative Neural Net-work in signal analysis II Application to on-line monitoring of
a simulated BWR componentrdquo Annals of Nuclear Energy vol32 no 11 pp 1207ndash1223 2005
[16] M S Ikbal H Misra and B Yegnanarayana ldquoAnalysis ofautoassociative mapping neural networksrdquo in Proceedings of theInternational Joint Conference on Neural Networks (IJCNN rsquo99)vol 5 pp 3025ndash3029 IEEE Washington DC USA July 1999Proceedings (Cat no 99CH36339)
[17] Y Bengio ldquoLearning deep architectures for AIrdquo Foundationsand Trends in Machine Learning vol 2 no 1 pp 1ndash27 2009
[18] J W Hines A V Gribok I Attieh and R E Uhrig ldquoRegular-izationmethods for inferential sensing in nuclear power plantsrdquoin Fuzzy Systems and Soft Computing in Nuclear Engineering DRuan Ed vol 38 of Studies in Fuzziness and Soft Computingpp 285ndash314 Physica 2000
[19] A V Gribok J W Hines A Urmanov and R E UhrigldquoHeuristic systematic and informational regularization forprocessmonitoringrdquo International Journal of Intelligent Systemsvol 17 no 8 pp 723ndash749 2002
[20] X Glorot and Y Bengio ldquoUnderstanding the difficulty oftraining deep feedforward neural networksrdquo in Proceedings ofthe 13th International Conference on Artificial Intelligence andStatistics (AISTATS rsquo10) vol 9 pp 249ndash256 Sardinia Italy May2010
[21] S Tan andM L Mayrovouniotis ldquoReducing data dimensional-ity through optimizing neural network inputsrdquo AIChE Journalvol 41 no 6 pp 1471ndash1480 1995
[22] Y Bengio and P Lamblin ldquoGreedy layer-wise training of deepnetworksrdquo in Proceedings of the Advances in Neural InformationProcessing Systems 19 (NIPS rsquo07) pp 153ndash160 September 2007
[23] H Larochelle Y Bengio J Lourador and P Lamblin ldquoExplor-ing strategies for training deep neural networksrdquoThe Journal ofMachine Learning Research vol 10 pp 1ndash40 2009
[24] D Yu and L Deng ldquoDeep learning and its applications to signaland information processing [exploratory DSP]rdquo IEEE SignalProcessing Magazine vol 28 no 1 pp 145ndash154 2011
[25] D ErhanA Courville andPVincent ldquoWhydoes unsupervisedpre-training help deep learningrdquo Journal of Machine LearningResearch vol 11 pp 625ndash660 2010
[26] P Vincent H Larochelle Y Bengio and P-A ManzagolldquoExtracting and composing robust features with denoisingautoencodersrdquo in Proceedings of the 25th International Confer-ence on Machine Learning (ICML rsquo08) pp 1096ndash1103 July 2008
[27] P Vincent H Larochelle I Lajoie Y Bengio and P-AManzagol ldquoStacked denoising autoencoders learning usefulrepresentations in a deep network with a local denoisingcriterionrdquo Journal of Machine Learning Research vol 11 pp3371ndash3408 2010
[28] N Srivastava G Hinton A Krizhevsky I Sutskever and RSalakhutdinov ldquoDropout a simple way to prevent neural net-works from overfittingrdquo Journal of Machine Learning Researchvol 15 pp 1929ndash1958 2014
[29] M J Embrechts B J Hargis and J D Linton ldquoAn augmentedefficient backpropagation training strategy for deep autoasso-ciative neural networksrdquo in Proceedings of the 18th EuropeanSymposiumonArtificial NeuralNetworksmdashComputational Intel-ligence and Machine Learning (ESANN rsquo10) vol no pp 28ndash30April 2010
[30] Y Bengio ldquoPractical recommendations for gradient-basedtraining of deep architecturesrdquo inNeural Networks Tricks of theTrade vol 7700 of Lecture Notes in Computer Science pp 437ndash478 Springer Berlin Germany 2nd edition 2012
Science and Technology of Nuclear Installations 17
[31] J Bergstra and Y Bengio ldquoRandom search for hyper-parameteroptimizationrdquo Journal of Machine Learning Research vol 13 pp281ndash305 2012
[32] J W Hines and D R Garvey ldquoDevelopment and applicationof fault detectability performance metrics for instrument cali-bration verification and anomaly detectionrdquo Journal of PatternRecognition Research vol 1 no 1 pp 2ndash15 2006
[33] A Usynin and J W Hines ldquoOn-line monitoring robustnessmeasures and comparisonsrdquo in Proceedings of the Interna-tional Atomic Energy Agency Technical Meeting on ldquoIncreasingInstrument Calibration Interval through On-Line CalibrationTechnologyrdquo OECD Halden Reactor Project Halden NorwaySeptember 2004
[34] A Wald ldquoSequential tests of statistical hypothesesrdquo Annals ofMathematical Statistics vol 16 no 2 pp 117ndash186 1945
[35] F Di Maio P Baraldi E Zio and R Seraoui ldquoFault detection innuclear power plants components by a combination of statisticalmethodsrdquo IEEE Transactions on Reliability vol 62 no 4 pp833ndash845 2013
[36] S Cheng and M Pecht ldquoUsing cross-validation for modelparameter selection of sequential probability ratio testrdquo ExpertSystems with Applications vol 39 no 9 pp 8467ndash8473 2012
TribologyAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
FuelsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal ofPetroleum Engineering
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Power ElectronicsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
CombustionJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Renewable Energy
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
StructuresJournal of
International Journal of
RotatingMachinery
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporation httpwwwhindawicom
Journal ofEngineeringVolume 2014
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
International Journal ofPhotoenergy
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear InstallationsScience and Technology of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Solar EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Wind EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear EnergyInternational Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
High Energy PhysicsAdvances in
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
10 Science and Technology of Nuclear Installations
Layerwise invariance
3rd hidden layer2nd hidden layer1st hidden layer
0010203040506070809
1
Nor
mal
ised
MSE
minus20 minus10minus40 minus30 0 10 20 30 40 50minus50
Offset failure level ()
Figure 7 Layerwise invariance in DAASM Lower curves depicthigher invariance
Robustness
With 3rd hidden layerWith 2nd hidden layerWith 1st hidden layer
0203040506070809
1
minus20 minus10minus40 minus30 0 10 20 30 40 50minus50
Offset failure level ()
Robu
stnes
s (1minusS a
)
Figure 8 Robustness measure (1 minus autosensitivity) at multiplehidden layers in DAASM Higher curves depict high robustness
The MSE values of all sensors are normalized to theirrespective span and are presented as percent span in Figure 9Being an error measure the lower MSE values by DAASMsignify its prediction accuracy
82 Robustness Robustness is quantified through autosensi-tivity as defined by [32 33] It is themeasure of modelrsquos abilityto predict correct sensor values under missing or corruptedsensor states The measure is averaged over an operatingregion defined by 119896 samples from test data set as follows
119878119894
Auto =1
119873
119873
sum
119896=1
1003816100381610038161003816100381610038161003816100381610038161003816
drift119896119894
minus 119896119894
119904drift119896119894
minus 119904119896119894
1003816100381610038161003816100381610038161003816100381610038161003816
(16)
where 119894 and 119896 are indexes corresponding to sensors and theirrespective test samples 119904
119896119894is the original sensor value without
fault 119896119894is the model estimated sensor value against 119904
119896119894 119904drift119896119894
is the driftedfaulted sensor value drift119896119894
is themodel estimatedsensor value against drifted value 119904
drift119896119894
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN 0382 0361 0343 0412 0186 0211 0166 0386 0411 0243 0223 0512 0621
DAASM 0281 0253 0246 0293 0121 0132 0122 0243 0315 0173 0156 0394 0465
Accuracy metric
0
01
02
03
04
05
06
07
MSE
( sp
an)
Figure 9 MSE depicting DAASM and K-AANN accuracy on eachsensor
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Autosensitivity metric
0005
01015
02025
03503
Auto
sens
itivi
ty
0273 0294 0321 0332 0205 0182 0198 0253 0233 0225 0231 0268 0301
0153 0167 0187 0163 0112 0079 0082 0131 0071 014 0117 0133 0187
Figure 10 Autosensitivity values of individual sensors in bothmodels
The autosensitivitymetric lies in [0 1] range For autosen-sitivity value of one the model predictions follow the faultwith zero residuals hence no fault can be detected Smallerautosensitivity values are preferred which essentially meansdecreased sensitivity towards small perturbations Largeautosensitivity values may lead to missed alarms due tounderestimation of the fault size caused by small residualvalues Compared to K-AANN model in case of DAASM asignificant decrease in autosensitivity values for all sensors isobserved The plot in Figure 10 shows that DAASM is morerobust to failed sensor inputs
To further investigate robustness against large offsetfailures both models are evaluated against offset failuresin [5ndash50] range For each sensor samples from testdata are corrupted with specific offset level and correspond-ing autosensitivities are averaged over whole sensor setAutosensitivity values less than 02 are considered as robustThe maximum autosensitivity value of 0187 is observed insteam flow sensor The plot in Figure 11 shows that averageautosensitivity for both models increases with increasinglevel of offset failure However the autosensitivity curve forDAASM autosensitivity is well below the corresponding K-AANN curve
83 Spillover Depending upon the size and type of failurea failed sensor input can cause discrepancy in estimatedoutput for other sensors The phenomenon is referred to in
Science and Technology of Nuclear Installations 11
0060086
014
021
028
041
016021
028
036
043
054
005 01 02 03 04 05Sensor perturbation magnitude ( of actual value)
Robustness performance
0
01
02
03
04
05
06
Avg
auto
sens
itivi
ty
DAASMK-AANN
Figure 11 Comparison of robustness against increasing offsetfailure
literature as ldquospillover effectrdquo and is quantified through ldquocross-sensitivityrdquo metric [32] It quantifies the influence of faultysensor 119894 on predictions of sensor 119895 as follows
119878119895119894
Cross =1
119873
119873
sum
119896=1
10038161003816100381610038161003816100381610038161003816100381610038161003816
drift119896119895
minus 119896119895
119904drift119896119894
minus 119904119896119894
10038161003816100381610038161003816100381610038161003816100381610038161003816
(17)
119878119895
Cross =1
119873 minus 1
119873
sum
119894=1
119878119895119894
Cross 119894 = 119895 (18)
where 119894 and 119895 indexes are used to refer to faulty andnonfaulty sensors respectively Meanwhile 119896 is the index forcorresponding test samples 119878119895119894Cross is the cross-sensitivity ofsensor 119895 with respect to drift in 119894th sensor 119904
119896119894is the value of
119894th sensor without any fault 119896119895is the model estimated value
of 119895th sensor against 119904119896119894 119904drift119896119894
is the driftedfaulted value of 119894thsensor drift
119896119895is themodel estimated value of 119895th sensor against
drifted value 119904drift119896119894
The highly distributed representation of the input in
neural network based sensor models has pronounced effecton the cross-sensitivity performance Cross-sensitivitymetricvalue lies in [0 1] range High value of cross-sensitivitymay set off false alarms in other sensors provided theresidual values overshoot the fault detectability threshold inother sensors So minimum cross-sensitivity value is desiredfor a robust model The plot in Figure 12 shows that thecross-sensitivity for DAASM is reduced by a large factor ascompared to K-AANNmodel
The spillover effect against particular level of offset failurein [5ndash50] range is averaged over all sensors as follows
AvgCross Sensitivity (119870) =
sum119873
119895=1119878119895
cross(119870)
119873
offset failure level 119870 = 5 10 20 sdot sdot sdot 50
(19)
0128 0124 0113 0116 0112 0108 0092 0079 0083 0098 011 0103 0105
00632 00527 00563 00614 0039 00311 00287 00186 00203 00344 00336 003 00286
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Cross-sensitivity metric
0002004006008
01012014
Cros
s-se
nsiti
vity
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
Figure 12 Cross-sensitivity values of individual sensors in bothmodels
00810105
0161
0253
0312
041
0018 0026 0038 00460071
012
005 01 02 03 04 05Sensor perturbation magnitude ( of observed sensor value)
Spillover offset sensor failure
Requires SPRT mean relaxation to avoid
false alarmsahead of this point
0
005
01
015
02
025
03
035
04
045
Avg
cros
s-se
nsiti
vity
DAASMK-AANN
Figure 13 Comparison of spillover effects against increasing offsetfailure
The cross-sensitivity values 119878119895
cross(119870) against 119870 offset fail-ure level are calculated using (18) Figure 13 shows theaverage cross-sensitivity plot for both models Small cross-sensitivities are observed in DAASM which effectivelyavoided false alarms in other channels without relaxing theSPRT faulted mean value up to an offset failure of 35ndash40 inany channel However for the case of offset noise larger than35 SPRTmean needs to be relaxed to avoid false alarms andisolate the faulty sensor However Robustness of K-AANNmodel deteriorates significantly due to spillover effect beyond15 offset failure
Similarly gross failure scenarios corresponding to twoextremities of sensor range can cause severe Spillover effectTo study robustness against gross type failure scenario asubset of input sensors is simultaneously failed with grosshigh or low value and average cross-sensitivity of remainingsensor set is calculated using relation (19) Plot in Figure 14shows that average cross-sensitivity of K-AANN modelincreases drastically beyond 10 gross failure HoweverDAASM resulted in a very nominal spillover even in caseof multiple sensor failure The DAASM effectively managedsimultaneous gross high or low failures in 25 of total sensorset as compared to 10 in case of K-AANN
12 Science and Technology of Nuclear Installations
0028 0032 0041 012021016 024
0457
061
077
01 02 03 04 05Sensors failed with gross highlow error ()
Spillover gross sensor failure
0010203040506070809
Avg
cros
s-se
nsiti
vity
DAASMK-AANN
Figure 14 Comparison of spillover effect against simultaneous grosshighlow failure in multiple sensors
84 Fault Detectability Fault detectability metric measuresthe smallest fault that can be detected by integrated sensorestimation and fault detection module as shown in Figure 1[32] The detectability metric is measured as percentage ofsensor span 119863 = 119872Span where value M correspondsto minimum detectable fault Minimum fault detectabilitylimit for each sensor is quantified through statistical basedsequential probability ratio test (SPRT) by Wald [34] SPRTtest is carried out to detect if the residual being generatedfrom normal distribution 119873(120583
1 1205902
) or 119873(1205830 1205902
) as definedfor faulty and fault free sensor operations respectively [35]Calibration failures are reflected in the mean parameter ofresidualrsquos distribution The SPRT procedure is applied todetect changes in the mean of residualrsquos distribution Theapplication of SPRT requires setting of following parametersvalue [36]
1205830 normal mode residual mean
1205902 normal mode residual variance
1205831 expected offset in residual mean in abnormal
mode120572 false alarm probability120573 missed alarm probability
Under normal mode the residuals from observed and modelestimated sensor values behave as awhiteGaussian noisewithmean 120583
0= 0 The residual variance 120590
2 is estimated for eachsensor under normal operating conditions and remainedfixed The false alarm 120572 and missed alarm 120573 probabilities areset to be 0001 and 001 respectively In order to determineminimum fault detectability limit a numerical procedureis opted which searches for minimum expected offset 120583
1
in the interval 1205831
[120590ndash3120590] provided the constraint onmissed and false alarm rate holds 120590 is the standard deviationcorresponding to residual variance of particular sensor Theplot in Figure 15 shows the detectability metric for eachsensor The plot in Figure 15 shows that DAASM can detect
0312 0315 033 035 004 005 004 008 0086 01 009 006 0058
0184 0165 019 02 0021 0023 002 0048 005 004 004 0032 0028
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Fault detectability metric
0005
01015
02025
03035
04
Det
ecta
bilit
y (
span
)
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
Figure 15 Comparison of fault detectability metrics
faults which are two times smaller in magnitude than thosedetectable by K-AANNmodel
Improvement in fault detectability metric for DAASMcan be attributed to observed improvement in model robust-ness as suggested by the following relation
119903119894
Δ119878drift119894
= (1 minus 119878119894
Auto) (20)
The term 119903119894Δ119878
drift119894
measures the ratio of observed residualto actual sensor drift in terms of autosensitivity For highlyrobust model this ratio reduces to one which means residualreflects the actual drift and results in high fault detectabilityContrarily ratio value close to zero means that the predictionis following the input and results in poor fault detectability
841 SPRT Based Fault Detectability Test Sequential prob-ability ratio [34 36] based fault hypothesis test is appliedto residual sequence 119877
119894 = 119903
1(1199051) 1199031(1199051)sdot sdot sdot 119903119899(119905119899)generated
by relation 119877119894(119905119894) = 119878
Obs(119905119894) minus 119878
Est(119905119894) at time 119905
119894 where
119878Obs
(119905119894) and 119878
Est(119905119894) are the actual and model predicted
sensor values respectively The SPRT procedure analyzeswhether the residual sequence is more likely to be generatedfrom a probability distribution that belongs to normal modehypothesis 119867
0or abnormal mode hypothesis 119867
1by using
likelihood ratio as follows
119871119899= exp[minus(12120590
2)[sum119899
119894=11205831(1205831minus2119903119894)]]
(21)
For fault free sensor values the normalmode hypothesis1198670is
approximated byGaussian distributionwithmean 1205830= 0 and
variance 1205902 Abnormal mode hypothesis 119867
1is approximated
with mean 1205831
gt 1205830using the same variance 120590
2 The SPRTindex for the positive mean test is finally obtained by takinglogarithm of the likelihood ratio in (21) as follows [35]
SPRT = ln (119871119899) = minus
1
21205902[
119899
sum
119894=1
1205831(1205831minus 2119903119894)]
=1205831
1205902
119899
sum
119894=1
(119903119894minus
1205831
2)
(22)
Pressurizer pressure sensor sampled at a frequency of 10 sec-onds is used as a test signal to validate the fault detectability
Science and Technology of Nuclear Installations 13
0 500 1000 1500 2000 2500 3000 3500 4000 45001535
1541545
SPRT drift detection (pressure sensor)(B
ARG
)Pr
essu
re
Drifted sensorDAASM estimate
Sample number
0 500 1000 1500 2000 2500 3000 3500 4000 45001535
1541545
(BA
RG)
Pres
sure
Drifted sensorK-AANN estimate
Sample number
0515
SPRT
hypo
thes
is
minus050 500 1000 1500 2000 2500 3000 3500 4000 4500
DAASM indexK-AANN index
Sample number
005
valu
eRe
sidua
l
0 500 1000 1500 2000 2500 3000 3500 4000 4500
Residual DAASMResidual K-AANN
Sample number
minus05
H1mdash1
H0mdash0
Figure 16 SPRTbased fault detection in pressurizer pressure sensor
performance Two drift faults at the rate of +001hour andminus001hour are introduced in the test signal for DAASMand K-AANN modelrsquos assessment respectively The first andsecond plots in Figure 16 show drifted and estimated pressuresignal from DAASM and K-AANN models respectivelyThird plot shows residual values generated by differencing thedrifted and estimated signals frombothmodelsThe final plotshows SPRT index values against residuals from K-AANNmodel and DAASM The hypotheses 119867
1and 119867
0correspond
to positive and negative fault acceptance respectively FromSPRT index plot successful early detection of the sensordrift at 2200th sample with lag of 611 hours since the driftinception shows that DAASM is more sensitive to smalldrifts On the other hand SPRT index on K-AANN basedsensor estimates registered the same drift at 3800th samplewith a lag of almost 1055 hours The result shows thatDAASM is more robust in terms of early fault detection withlow false and missed alarm rates
Finally both models are tested against five test data setsEach test set consists of 3630 samples corresponding todifferentmonths of full power reactor operation Bothmodelssuccessfully detected an offset failure of 012ndash03 BARG in allsteam pressure channels and a drift type failure up to 285 insteam generator level (Figure 22)The K-AANNmodel failedto register a very small drift up to 01 in steam flow (STMflow 1) channel A small drift up to 01 BARG is detected in
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Steam pressure estimation
Observed valueDAASM estimateK-AANN estimate
672673674675676677678679
68
Stea
m p
ress
ure (
BARG
)
Figure 17 Steam pressure estimates against offset failures up to 03(BARG)
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Feed flow estimation
400420440460480500520540560
Feed
flow
(kg
s)
Observed sensor valueK-AANN estimateDAASM estimate
Figure 18 Feed flow estimates
test set 5 of pressurizer pressure channel However in caseof drift type sensor failures fault detection lag for DAASMwas on average 05 times smaller in comparison with K-AANN model Plots in Figures 17ndash21 show the estimatedsensor values from both models on five test data sets of fewselected channels
9 Conclusion
This paper presented a neural network based denoisedautoassociative sensor model (DAASM) for empirical sen-sor modeling The proposed sensor model is trained togenerate a monitoring system for sensor fault detection innuclear power plantsMultilayer AANNbased sensormodelsmay result in suboptimal solutions due to poor regulariza-tion by traditional backpropagation based joint multilayerlearning procedures So a complementary deep learningapproach based on greedy layerwise unsupervised pretrain-ing is employed for effective regularization in the proposedmultilayer DAASM Autoencoder architecture is used fordenoising based unsupervised pretraining and regularizationof individual layers in the network hierarchy To addressrobustness against perturbations in input sensors data cor-ruption processes exercised during unsupervised pretraining
14 Science and Technology of Nuclear Installations
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5490495500505510515520525530535540
Stea
m fl
ow (k
gs)
Steam flow estimation
Observed sensor valueDAASM estimateK-AANN estimate
Figure 19 Steam flow estimates against drift failure up to 01
58859
592594596598
60602604606608
Pres
suriz
er le
vel (
)
Pressurizer level estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valueK-AANN estimateDAASM estimate
Figure 20 Pressurizer level estimates
phase were based on prior knowledge about different failurescenarios Results from invariance tests showed that theproposed data corruption schemeswere beneficial in learninglatent representations at hidden layers and were invariantto multiple levels of perturbation in input sensors Conse-quently these pretrained hidden layers worked as well regu-larized perturbation filters with increased invariance towardssensor faults It is also observed that sensitivity against sensorfaults decreased significantly towards higher layers in fullDAASMassembly In a practical context of sensormonitoringin nuclear power plants the proposed model proved itsrobustness against gross type simultaneous sensor failuresIt also showed significant improvement in all performancemetrics when compared with popular and widely used five-layered AANNmodel by KramerMoreover time lag in smalldriftrsquos detection is significantly reduced The overall resultssuggest that greedy layerwise pretraining technique in com-bination with domain specific corruption processes providesa viable framework for effective regularization and robustnessin such deep multilayered autoassociative sensor validationmodels
Appendix
See Table 3
153715375
153815385
153915395
15415405
154115415
Pres
sure
(BA
RG)
Pressurizer pressure estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valuesK-AANN estimateDAASM estimate
Figure 21 Pressurizer pressure estimates against drift failure up to01
74
76
78
80
82
84
Stea
m g
ener
ator
leve
l (
)
Steam generator level estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valueK-AANN estimateDAASM estimate
Figure 22 Steam generator level estimates against growing driftfailure up to 287
Abbreviations
AANN Autoassociative neural networkK-AANN Kramer proposed Autoassociative neural
networkDAASM Denoised autoassociative sensor modelNPP Nuclear power plantPWR Pressurized water reactor119878Auto Autosensitivity119878Cross Cross-sensitivityDAE Denoising autoencoder119878 Observed sensor value Model predicted sensor value Corrupted sensor valueSPN Salt-and-pepper noiseAGN Additive Gaussian noise
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Science and Technology of Nuclear Installations 15
Table3Re
sults
ofDAASM
andK-AANNperfo
rmance
metric
sPerfo
rmance
metric
Mod
eltype
Feed
flow1
Feed
flow2
STM
flow1
STM
flow2
STM
PSR1
STM
PSR2
STM
PSR3
PZRPS
R1
PZRPS
R2
PZRLV
L1
PZRLV
L2
SGLV
LNR1
SGLV
LNR2
AANN
Sp-D
AANN
Accuracy
(span)
0382
0361
0343
0412
0186
0211
0166
0386
0411
0243
0223
0512
0621
0281
0253
0246
0293
0121
0132
0122
0243
0315
0173
0156
0394
0465
Autosensitivity
0273
0294
0321
0332
0205
0182
0198
0253
0233
0225
0231
0268
0301
0153
0167
0187
0163
0112
0079
0082
0131
0071
014
0117
0133
0187
Cross-sensitivity
0128
0124
0113
0116
0112
0108
0092
0079
0083
0098
011
0103
0105
00632
00527
00563
00614
0039
00311
00287
00186
00203
00344
00336
003
00286
SPRT
detectability
0312
0315
033
035
004
005
004
008
0086
01
009
006
0058
0184
0165
019
02
0021
0023
002
004
8005
004
004
0032
0028
16 Science and Technology of Nuclear Installations
Acknowledgments
The authors acknowledge the support of Mr Jose GalindoRodrıguez affiliated to TECNATOM Inc Spain and MrChad Painter (Director of Nuclear Power Plant SimulatorDevelopment and Training Program at International AtomicEnergy Agency) for providing necessary tools and data toconduct this research
References
[1] N Mehranbod M Soroush M Piovoso and B A OgunnaikeldquoProbabilistic model for sensor fault detection and identifica-tionrdquo AIChE Journal vol 49 no 7 pp 1787ndash1802 2003
[2] JW Hines and E Davis ldquoLessons learned from the US nuclearpower plant on-line monitoring programsrdquo Progress in NuclearEnergy vol 46 no 3-4 pp 176ndash189 2005
[3] EPRI ldquoOn-line monitoring cost benefit guiderdquo Final Report1006777 EPRI Palo Alto Calif USA 2003
[4] S J Qin and W Li ldquoDetection identification and recon-struction of faulty sensors with maximized sensitivityrdquo AIChEJournal vol 45 no 9 pp 1963ndash1976 1999
[5] J Ma and J Jiang ldquoApplications of fault detection and diagnosismethods in nuclear power plants a reviewrdquo Progress in NuclearEnergy vol 53 no 3 pp 255ndash266 2011
[6] G YHeo ldquoConditionmonitoring using empiricalmodels tech-nical review and prospects for nuclear applicationsrdquo NuclearEngineering and Technology vol 40 no 1 pp 49ndash68 2008
[7] J B Coble R M Meyer P Ramuhalli et al ldquoA review ofsensor calibration monitoring for calibration interval extensionin nuclear power plantsrdquo Tech Rep PNNL-21687 PacificNorthwest National Laboratory Richland Wash USA 2012
[8] M A Kramer ldquoAutoassociative neural networksrdquo Computersand Chemical Engineering vol 16 no 4 pp 313ndash328 1992
[9] X Xu J W Hines and R E Uhrig ldquoOn-line sensor calibra-tion monitoring and fault detection for chemical processesrdquoin Proceedings of the Maintenance and Reliability Conference(MARCON rsquo98) pp 12ndash14 Knoxville Tenn USA May 1998
[10] M Hamidreza S Mehdi J-R Hooshang and N AliakbarldquoReconstruction based approach to sensor fault diagnosis usingauto-associative neural networksrdquo Journal of Central SouthUniversity vol 21 no 6 pp 2273ndash2281 2014
[11] UThissenW JMelssen andLMC Buydens ldquoNonlinear pro-cess monitoring using bottle-neck neural networksrdquo AnalyticaChimica Acta vol 446 no 1-2 pp 371ndash383 2001
[12] D J Wrest J W Hines and R E Uhrig ldquoInstrument surveil-lance and calibration verification through plant wide monitor-ing using autoassociative neural networksrdquo in Proceedings ofthe American Nuclear Society International Topical Meeting onNuclear Plant Instrumentation Control and Human MachineInterface Technologies University Park Pa USA May 1996
[13] J W Hines R E Uhrig and D J Wrest ldquoUse of autoassociativeneural networks for signal validationrdquo Journal of Intelligent andRobotic Systems Theory and Applications vol 21 no 2 pp 143ndash154 1998
[14] P F Fantoni M I Hoffmann R Shankar and E L Davis ldquoOn-line monitoring of instrument channel performance in nuclearpower plant using PEANOrdquo Progress in Nuclear Energy vol 43no 1ndash4 pp 83ndash89 2003
[15] MMarseguerra andA Zoia ldquoTheAutoAssociative Neural Net-work in signal analysis II Application to on-line monitoring of
a simulated BWR componentrdquo Annals of Nuclear Energy vol32 no 11 pp 1207ndash1223 2005
[16] M S Ikbal H Misra and B Yegnanarayana ldquoAnalysis ofautoassociative mapping neural networksrdquo in Proceedings of theInternational Joint Conference on Neural Networks (IJCNN rsquo99)vol 5 pp 3025ndash3029 IEEE Washington DC USA July 1999Proceedings (Cat no 99CH36339)
[17] Y Bengio ldquoLearning deep architectures for AIrdquo Foundationsand Trends in Machine Learning vol 2 no 1 pp 1ndash27 2009
[18] J W Hines A V Gribok I Attieh and R E Uhrig ldquoRegular-izationmethods for inferential sensing in nuclear power plantsrdquoin Fuzzy Systems and Soft Computing in Nuclear Engineering DRuan Ed vol 38 of Studies in Fuzziness and Soft Computingpp 285ndash314 Physica 2000
[19] A V Gribok J W Hines A Urmanov and R E UhrigldquoHeuristic systematic and informational regularization forprocessmonitoringrdquo International Journal of Intelligent Systemsvol 17 no 8 pp 723ndash749 2002
[20] X Glorot and Y Bengio ldquoUnderstanding the difficulty oftraining deep feedforward neural networksrdquo in Proceedings ofthe 13th International Conference on Artificial Intelligence andStatistics (AISTATS rsquo10) vol 9 pp 249ndash256 Sardinia Italy May2010
[21] S Tan andM L Mayrovouniotis ldquoReducing data dimensional-ity through optimizing neural network inputsrdquo AIChE Journalvol 41 no 6 pp 1471ndash1480 1995
[22] Y Bengio and P Lamblin ldquoGreedy layer-wise training of deepnetworksrdquo in Proceedings of the Advances in Neural InformationProcessing Systems 19 (NIPS rsquo07) pp 153ndash160 September 2007
[23] H Larochelle Y Bengio J Lourador and P Lamblin ldquoExplor-ing strategies for training deep neural networksrdquoThe Journal ofMachine Learning Research vol 10 pp 1ndash40 2009
[24] D Yu and L Deng ldquoDeep learning and its applications to signaland information processing [exploratory DSP]rdquo IEEE SignalProcessing Magazine vol 28 no 1 pp 145ndash154 2011
[25] D ErhanA Courville andPVincent ldquoWhydoes unsupervisedpre-training help deep learningrdquo Journal of Machine LearningResearch vol 11 pp 625ndash660 2010
[26] P Vincent H Larochelle Y Bengio and P-A ManzagolldquoExtracting and composing robust features with denoisingautoencodersrdquo in Proceedings of the 25th International Confer-ence on Machine Learning (ICML rsquo08) pp 1096ndash1103 July 2008
[27] P Vincent H Larochelle I Lajoie Y Bengio and P-AManzagol ldquoStacked denoising autoencoders learning usefulrepresentations in a deep network with a local denoisingcriterionrdquo Journal of Machine Learning Research vol 11 pp3371ndash3408 2010
[28] N Srivastava G Hinton A Krizhevsky I Sutskever and RSalakhutdinov ldquoDropout a simple way to prevent neural net-works from overfittingrdquo Journal of Machine Learning Researchvol 15 pp 1929ndash1958 2014
[29] M J Embrechts B J Hargis and J D Linton ldquoAn augmentedefficient backpropagation training strategy for deep autoasso-ciative neural networksrdquo in Proceedings of the 18th EuropeanSymposiumonArtificial NeuralNetworksmdashComputational Intel-ligence and Machine Learning (ESANN rsquo10) vol no pp 28ndash30April 2010
[30] Y Bengio ldquoPractical recommendations for gradient-basedtraining of deep architecturesrdquo inNeural Networks Tricks of theTrade vol 7700 of Lecture Notes in Computer Science pp 437ndash478 Springer Berlin Germany 2nd edition 2012
Science and Technology of Nuclear Installations 17
[31] J Bergstra and Y Bengio ldquoRandom search for hyper-parameteroptimizationrdquo Journal of Machine Learning Research vol 13 pp281ndash305 2012
[32] J W Hines and D R Garvey ldquoDevelopment and applicationof fault detectability performance metrics for instrument cali-bration verification and anomaly detectionrdquo Journal of PatternRecognition Research vol 1 no 1 pp 2ndash15 2006
[33] A Usynin and J W Hines ldquoOn-line monitoring robustnessmeasures and comparisonsrdquo in Proceedings of the Interna-tional Atomic Energy Agency Technical Meeting on ldquoIncreasingInstrument Calibration Interval through On-Line CalibrationTechnologyrdquo OECD Halden Reactor Project Halden NorwaySeptember 2004
[34] A Wald ldquoSequential tests of statistical hypothesesrdquo Annals ofMathematical Statistics vol 16 no 2 pp 117ndash186 1945
[35] F Di Maio P Baraldi E Zio and R Seraoui ldquoFault detection innuclear power plants components by a combination of statisticalmethodsrdquo IEEE Transactions on Reliability vol 62 no 4 pp833ndash845 2013
[36] S Cheng and M Pecht ldquoUsing cross-validation for modelparameter selection of sequential probability ratio testrdquo ExpertSystems with Applications vol 39 no 9 pp 8467ndash8473 2012
TribologyAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
FuelsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal ofPetroleum Engineering
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Power ElectronicsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
CombustionJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Renewable Energy
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
StructuresJournal of
International Journal of
RotatingMachinery
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporation httpwwwhindawicom
Journal ofEngineeringVolume 2014
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
International Journal ofPhotoenergy
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear InstallationsScience and Technology of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Solar EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Wind EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear EnergyInternational Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
High Energy PhysicsAdvances in
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Science and Technology of Nuclear Installations 11
0060086
014
021
028
041
016021
028
036
043
054
005 01 02 03 04 05Sensor perturbation magnitude ( of actual value)
Robustness performance
0
01
02
03
04
05
06
Avg
auto
sens
itivi
ty
DAASMK-AANN
Figure 11 Comparison of robustness against increasing offsetfailure
literature as ldquospillover effectrdquo and is quantified through ldquocross-sensitivityrdquo metric [32] It quantifies the influence of faultysensor 119894 on predictions of sensor 119895 as follows
119878119895119894
Cross =1
119873
119873
sum
119896=1
10038161003816100381610038161003816100381610038161003816100381610038161003816
drift119896119895
minus 119896119895
119904drift119896119894
minus 119904119896119894
10038161003816100381610038161003816100381610038161003816100381610038161003816
(17)
119878119895
Cross =1
119873 minus 1
119873
sum
119894=1
119878119895119894
Cross 119894 = 119895 (18)
where 119894 and 119895 indexes are used to refer to faulty andnonfaulty sensors respectively Meanwhile 119896 is the index forcorresponding test samples 119878119895119894Cross is the cross-sensitivity ofsensor 119895 with respect to drift in 119894th sensor 119904
119896119894is the value of
119894th sensor without any fault 119896119895is the model estimated value
of 119895th sensor against 119904119896119894 119904drift119896119894
is the driftedfaulted value of 119894thsensor drift
119896119895is themodel estimated value of 119895th sensor against
drifted value 119904drift119896119894
The highly distributed representation of the input in
neural network based sensor models has pronounced effecton the cross-sensitivity performance Cross-sensitivitymetricvalue lies in [0 1] range High value of cross-sensitivitymay set off false alarms in other sensors provided theresidual values overshoot the fault detectability threshold inother sensors So minimum cross-sensitivity value is desiredfor a robust model The plot in Figure 12 shows that thecross-sensitivity for DAASM is reduced by a large factor ascompared to K-AANNmodel
The spillover effect against particular level of offset failurein [5ndash50] range is averaged over all sensors as follows
AvgCross Sensitivity (119870) =
sum119873
119895=1119878119895
cross(119870)
119873
offset failure level 119870 = 5 10 20 sdot sdot sdot 50
(19)
0128 0124 0113 0116 0112 0108 0092 0079 0083 0098 011 0103 0105
00632 00527 00563 00614 0039 00311 00287 00186 00203 00344 00336 003 00286
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Cross-sensitivity metric
0002004006008
01012014
Cros
s-se
nsiti
vity
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
Figure 12 Cross-sensitivity values of individual sensors in bothmodels
00810105
0161
0253
0312
041
0018 0026 0038 00460071
012
005 01 02 03 04 05Sensor perturbation magnitude ( of observed sensor value)
Spillover offset sensor failure
Requires SPRT mean relaxation to avoid
false alarmsahead of this point
0
005
01
015
02
025
03
035
04
045
Avg
cros
s-se
nsiti
vity
DAASMK-AANN
Figure 13 Comparison of spillover effects against increasing offsetfailure
The cross-sensitivity values 119878119895
cross(119870) against 119870 offset fail-ure level are calculated using (18) Figure 13 shows theaverage cross-sensitivity plot for both models Small cross-sensitivities are observed in DAASM which effectivelyavoided false alarms in other channels without relaxing theSPRT faulted mean value up to an offset failure of 35ndash40 inany channel However for the case of offset noise larger than35 SPRTmean needs to be relaxed to avoid false alarms andisolate the faulty sensor However Robustness of K-AANNmodel deteriorates significantly due to spillover effect beyond15 offset failure
Similarly gross failure scenarios corresponding to twoextremities of sensor range can cause severe Spillover effectTo study robustness against gross type failure scenario asubset of input sensors is simultaneously failed with grosshigh or low value and average cross-sensitivity of remainingsensor set is calculated using relation (19) Plot in Figure 14shows that average cross-sensitivity of K-AANN modelincreases drastically beyond 10 gross failure HoweverDAASM resulted in a very nominal spillover even in caseof multiple sensor failure The DAASM effectively managedsimultaneous gross high or low failures in 25 of total sensorset as compared to 10 in case of K-AANN
12 Science and Technology of Nuclear Installations
0028 0032 0041 012021016 024
0457
061
077
01 02 03 04 05Sensors failed with gross highlow error ()
Spillover gross sensor failure
0010203040506070809
Avg
cros
s-se
nsiti
vity
DAASMK-AANN
Figure 14 Comparison of spillover effect against simultaneous grosshighlow failure in multiple sensors
84 Fault Detectability Fault detectability metric measuresthe smallest fault that can be detected by integrated sensorestimation and fault detection module as shown in Figure 1[32] The detectability metric is measured as percentage ofsensor span 119863 = 119872Span where value M correspondsto minimum detectable fault Minimum fault detectabilitylimit for each sensor is quantified through statistical basedsequential probability ratio test (SPRT) by Wald [34] SPRTtest is carried out to detect if the residual being generatedfrom normal distribution 119873(120583
1 1205902
) or 119873(1205830 1205902
) as definedfor faulty and fault free sensor operations respectively [35]Calibration failures are reflected in the mean parameter ofresidualrsquos distribution The SPRT procedure is applied todetect changes in the mean of residualrsquos distribution Theapplication of SPRT requires setting of following parametersvalue [36]
1205830 normal mode residual mean
1205902 normal mode residual variance
1205831 expected offset in residual mean in abnormal
mode120572 false alarm probability120573 missed alarm probability
Under normal mode the residuals from observed and modelestimated sensor values behave as awhiteGaussian noisewithmean 120583
0= 0 The residual variance 120590
2 is estimated for eachsensor under normal operating conditions and remainedfixed The false alarm 120572 and missed alarm 120573 probabilities areset to be 0001 and 001 respectively In order to determineminimum fault detectability limit a numerical procedureis opted which searches for minimum expected offset 120583
1
in the interval 1205831
[120590ndash3120590] provided the constraint onmissed and false alarm rate holds 120590 is the standard deviationcorresponding to residual variance of particular sensor Theplot in Figure 15 shows the detectability metric for eachsensor The plot in Figure 15 shows that DAASM can detect
0312 0315 033 035 004 005 004 008 0086 01 009 006 0058
0184 0165 019 02 0021 0023 002 0048 005 004 004 0032 0028
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Fault detectability metric
0005
01015
02025
03035
04
Det
ecta
bilit
y (
span
)
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
Figure 15 Comparison of fault detectability metrics
faults which are two times smaller in magnitude than thosedetectable by K-AANNmodel
Improvement in fault detectability metric for DAASMcan be attributed to observed improvement in model robust-ness as suggested by the following relation
119903119894
Δ119878drift119894
= (1 minus 119878119894
Auto) (20)
The term 119903119894Δ119878
drift119894
measures the ratio of observed residualto actual sensor drift in terms of autosensitivity For highlyrobust model this ratio reduces to one which means residualreflects the actual drift and results in high fault detectabilityContrarily ratio value close to zero means that the predictionis following the input and results in poor fault detectability
841 SPRT Based Fault Detectability Test Sequential prob-ability ratio [34 36] based fault hypothesis test is appliedto residual sequence 119877
119894 = 119903
1(1199051) 1199031(1199051)sdot sdot sdot 119903119899(119905119899)generated
by relation 119877119894(119905119894) = 119878
Obs(119905119894) minus 119878
Est(119905119894) at time 119905
119894 where
119878Obs
(119905119894) and 119878
Est(119905119894) are the actual and model predicted
sensor values respectively The SPRT procedure analyzeswhether the residual sequence is more likely to be generatedfrom a probability distribution that belongs to normal modehypothesis 119867
0or abnormal mode hypothesis 119867
1by using
likelihood ratio as follows
119871119899= exp[minus(12120590
2)[sum119899
119894=11205831(1205831minus2119903119894)]]
(21)
For fault free sensor values the normalmode hypothesis1198670is
approximated byGaussian distributionwithmean 1205830= 0 and
variance 1205902 Abnormal mode hypothesis 119867
1is approximated
with mean 1205831
gt 1205830using the same variance 120590
2 The SPRTindex for the positive mean test is finally obtained by takinglogarithm of the likelihood ratio in (21) as follows [35]
SPRT = ln (119871119899) = minus
1
21205902[
119899
sum
119894=1
1205831(1205831minus 2119903119894)]
=1205831
1205902
119899
sum
119894=1
(119903119894minus
1205831
2)
(22)
Pressurizer pressure sensor sampled at a frequency of 10 sec-onds is used as a test signal to validate the fault detectability
Science and Technology of Nuclear Installations 13
0 500 1000 1500 2000 2500 3000 3500 4000 45001535
1541545
SPRT drift detection (pressure sensor)(B
ARG
)Pr
essu
re
Drifted sensorDAASM estimate
Sample number
0 500 1000 1500 2000 2500 3000 3500 4000 45001535
1541545
(BA
RG)
Pres
sure
Drifted sensorK-AANN estimate
Sample number
0515
SPRT
hypo
thes
is
minus050 500 1000 1500 2000 2500 3000 3500 4000 4500
DAASM indexK-AANN index
Sample number
005
valu
eRe
sidua
l
0 500 1000 1500 2000 2500 3000 3500 4000 4500
Residual DAASMResidual K-AANN
Sample number
minus05
H1mdash1
H0mdash0
Figure 16 SPRTbased fault detection in pressurizer pressure sensor
performance Two drift faults at the rate of +001hour andminus001hour are introduced in the test signal for DAASMand K-AANN modelrsquos assessment respectively The first andsecond plots in Figure 16 show drifted and estimated pressuresignal from DAASM and K-AANN models respectivelyThird plot shows residual values generated by differencing thedrifted and estimated signals frombothmodelsThe final plotshows SPRT index values against residuals from K-AANNmodel and DAASM The hypotheses 119867
1and 119867
0correspond
to positive and negative fault acceptance respectively FromSPRT index plot successful early detection of the sensordrift at 2200th sample with lag of 611 hours since the driftinception shows that DAASM is more sensitive to smalldrifts On the other hand SPRT index on K-AANN basedsensor estimates registered the same drift at 3800th samplewith a lag of almost 1055 hours The result shows thatDAASM is more robust in terms of early fault detection withlow false and missed alarm rates
Finally both models are tested against five test data setsEach test set consists of 3630 samples corresponding todifferentmonths of full power reactor operation Bothmodelssuccessfully detected an offset failure of 012ndash03 BARG in allsteam pressure channels and a drift type failure up to 285 insteam generator level (Figure 22)The K-AANNmodel failedto register a very small drift up to 01 in steam flow (STMflow 1) channel A small drift up to 01 BARG is detected in
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Steam pressure estimation
Observed valueDAASM estimateK-AANN estimate
672673674675676677678679
68
Stea
m p
ress
ure (
BARG
)
Figure 17 Steam pressure estimates against offset failures up to 03(BARG)
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Feed flow estimation
400420440460480500520540560
Feed
flow
(kg
s)
Observed sensor valueK-AANN estimateDAASM estimate
Figure 18 Feed flow estimates
test set 5 of pressurizer pressure channel However in caseof drift type sensor failures fault detection lag for DAASMwas on average 05 times smaller in comparison with K-AANN model Plots in Figures 17ndash21 show the estimatedsensor values from both models on five test data sets of fewselected channels
9 Conclusion
This paper presented a neural network based denoisedautoassociative sensor model (DAASM) for empirical sen-sor modeling The proposed sensor model is trained togenerate a monitoring system for sensor fault detection innuclear power plantsMultilayer AANNbased sensormodelsmay result in suboptimal solutions due to poor regulariza-tion by traditional backpropagation based joint multilayerlearning procedures So a complementary deep learningapproach based on greedy layerwise unsupervised pretrain-ing is employed for effective regularization in the proposedmultilayer DAASM Autoencoder architecture is used fordenoising based unsupervised pretraining and regularizationof individual layers in the network hierarchy To addressrobustness against perturbations in input sensors data cor-ruption processes exercised during unsupervised pretraining
14 Science and Technology of Nuclear Installations
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5490495500505510515520525530535540
Stea
m fl
ow (k
gs)
Steam flow estimation
Observed sensor valueDAASM estimateK-AANN estimate
Figure 19 Steam flow estimates against drift failure up to 01
58859
592594596598
60602604606608
Pres
suriz
er le
vel (
)
Pressurizer level estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valueK-AANN estimateDAASM estimate
Figure 20 Pressurizer level estimates
phase were based on prior knowledge about different failurescenarios Results from invariance tests showed that theproposed data corruption schemeswere beneficial in learninglatent representations at hidden layers and were invariantto multiple levels of perturbation in input sensors Conse-quently these pretrained hidden layers worked as well regu-larized perturbation filters with increased invariance towardssensor faults It is also observed that sensitivity against sensorfaults decreased significantly towards higher layers in fullDAASMassembly In a practical context of sensormonitoringin nuclear power plants the proposed model proved itsrobustness against gross type simultaneous sensor failuresIt also showed significant improvement in all performancemetrics when compared with popular and widely used five-layered AANNmodel by KramerMoreover time lag in smalldriftrsquos detection is significantly reduced The overall resultssuggest that greedy layerwise pretraining technique in com-bination with domain specific corruption processes providesa viable framework for effective regularization and robustnessin such deep multilayered autoassociative sensor validationmodels
Appendix
See Table 3
153715375
153815385
153915395
15415405
154115415
Pres
sure
(BA
RG)
Pressurizer pressure estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valuesK-AANN estimateDAASM estimate
Figure 21 Pressurizer pressure estimates against drift failure up to01
74
76
78
80
82
84
Stea
m g
ener
ator
leve
l (
)
Steam generator level estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valueK-AANN estimateDAASM estimate
Figure 22 Steam generator level estimates against growing driftfailure up to 287
Abbreviations
AANN Autoassociative neural networkK-AANN Kramer proposed Autoassociative neural
networkDAASM Denoised autoassociative sensor modelNPP Nuclear power plantPWR Pressurized water reactor119878Auto Autosensitivity119878Cross Cross-sensitivityDAE Denoising autoencoder119878 Observed sensor value Model predicted sensor value Corrupted sensor valueSPN Salt-and-pepper noiseAGN Additive Gaussian noise
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Science and Technology of Nuclear Installations 15
Table3Re
sults
ofDAASM
andK-AANNperfo
rmance
metric
sPerfo
rmance
metric
Mod
eltype
Feed
flow1
Feed
flow2
STM
flow1
STM
flow2
STM
PSR1
STM
PSR2
STM
PSR3
PZRPS
R1
PZRPS
R2
PZRLV
L1
PZRLV
L2
SGLV
LNR1
SGLV
LNR2
AANN
Sp-D
AANN
Accuracy
(span)
0382
0361
0343
0412
0186
0211
0166
0386
0411
0243
0223
0512
0621
0281
0253
0246
0293
0121
0132
0122
0243
0315
0173
0156
0394
0465
Autosensitivity
0273
0294
0321
0332
0205
0182
0198
0253
0233
0225
0231
0268
0301
0153
0167
0187
0163
0112
0079
0082
0131
0071
014
0117
0133
0187
Cross-sensitivity
0128
0124
0113
0116
0112
0108
0092
0079
0083
0098
011
0103
0105
00632
00527
00563
00614
0039
00311
00287
00186
00203
00344
00336
003
00286
SPRT
detectability
0312
0315
033
035
004
005
004
008
0086
01
009
006
0058
0184
0165
019
02
0021
0023
002
004
8005
004
004
0032
0028
16 Science and Technology of Nuclear Installations
Acknowledgments
The authors acknowledge the support of Mr Jose GalindoRodrıguez affiliated to TECNATOM Inc Spain and MrChad Painter (Director of Nuclear Power Plant SimulatorDevelopment and Training Program at International AtomicEnergy Agency) for providing necessary tools and data toconduct this research
References
[1] N Mehranbod M Soroush M Piovoso and B A OgunnaikeldquoProbabilistic model for sensor fault detection and identifica-tionrdquo AIChE Journal vol 49 no 7 pp 1787ndash1802 2003
[2] JW Hines and E Davis ldquoLessons learned from the US nuclearpower plant on-line monitoring programsrdquo Progress in NuclearEnergy vol 46 no 3-4 pp 176ndash189 2005
[3] EPRI ldquoOn-line monitoring cost benefit guiderdquo Final Report1006777 EPRI Palo Alto Calif USA 2003
[4] S J Qin and W Li ldquoDetection identification and recon-struction of faulty sensors with maximized sensitivityrdquo AIChEJournal vol 45 no 9 pp 1963ndash1976 1999
[5] J Ma and J Jiang ldquoApplications of fault detection and diagnosismethods in nuclear power plants a reviewrdquo Progress in NuclearEnergy vol 53 no 3 pp 255ndash266 2011
[6] G YHeo ldquoConditionmonitoring using empiricalmodels tech-nical review and prospects for nuclear applicationsrdquo NuclearEngineering and Technology vol 40 no 1 pp 49ndash68 2008
[7] J B Coble R M Meyer P Ramuhalli et al ldquoA review ofsensor calibration monitoring for calibration interval extensionin nuclear power plantsrdquo Tech Rep PNNL-21687 PacificNorthwest National Laboratory Richland Wash USA 2012
[8] M A Kramer ldquoAutoassociative neural networksrdquo Computersand Chemical Engineering vol 16 no 4 pp 313ndash328 1992
[9] X Xu J W Hines and R E Uhrig ldquoOn-line sensor calibra-tion monitoring and fault detection for chemical processesrdquoin Proceedings of the Maintenance and Reliability Conference(MARCON rsquo98) pp 12ndash14 Knoxville Tenn USA May 1998
[10] M Hamidreza S Mehdi J-R Hooshang and N AliakbarldquoReconstruction based approach to sensor fault diagnosis usingauto-associative neural networksrdquo Journal of Central SouthUniversity vol 21 no 6 pp 2273ndash2281 2014
[11] UThissenW JMelssen andLMC Buydens ldquoNonlinear pro-cess monitoring using bottle-neck neural networksrdquo AnalyticaChimica Acta vol 446 no 1-2 pp 371ndash383 2001
[12] D J Wrest J W Hines and R E Uhrig ldquoInstrument surveil-lance and calibration verification through plant wide monitor-ing using autoassociative neural networksrdquo in Proceedings ofthe American Nuclear Society International Topical Meeting onNuclear Plant Instrumentation Control and Human MachineInterface Technologies University Park Pa USA May 1996
[13] J W Hines R E Uhrig and D J Wrest ldquoUse of autoassociativeneural networks for signal validationrdquo Journal of Intelligent andRobotic Systems Theory and Applications vol 21 no 2 pp 143ndash154 1998
[14] P F Fantoni M I Hoffmann R Shankar and E L Davis ldquoOn-line monitoring of instrument channel performance in nuclearpower plant using PEANOrdquo Progress in Nuclear Energy vol 43no 1ndash4 pp 83ndash89 2003
[15] MMarseguerra andA Zoia ldquoTheAutoAssociative Neural Net-work in signal analysis II Application to on-line monitoring of
a simulated BWR componentrdquo Annals of Nuclear Energy vol32 no 11 pp 1207ndash1223 2005
[16] M S Ikbal H Misra and B Yegnanarayana ldquoAnalysis ofautoassociative mapping neural networksrdquo in Proceedings of theInternational Joint Conference on Neural Networks (IJCNN rsquo99)vol 5 pp 3025ndash3029 IEEE Washington DC USA July 1999Proceedings (Cat no 99CH36339)
[17] Y Bengio ldquoLearning deep architectures for AIrdquo Foundationsand Trends in Machine Learning vol 2 no 1 pp 1ndash27 2009
[18] J W Hines A V Gribok I Attieh and R E Uhrig ldquoRegular-izationmethods for inferential sensing in nuclear power plantsrdquoin Fuzzy Systems and Soft Computing in Nuclear Engineering DRuan Ed vol 38 of Studies in Fuzziness and Soft Computingpp 285ndash314 Physica 2000
[19] A V Gribok J W Hines A Urmanov and R E UhrigldquoHeuristic systematic and informational regularization forprocessmonitoringrdquo International Journal of Intelligent Systemsvol 17 no 8 pp 723ndash749 2002
[20] X Glorot and Y Bengio ldquoUnderstanding the difficulty oftraining deep feedforward neural networksrdquo in Proceedings ofthe 13th International Conference on Artificial Intelligence andStatistics (AISTATS rsquo10) vol 9 pp 249ndash256 Sardinia Italy May2010
[21] S Tan andM L Mayrovouniotis ldquoReducing data dimensional-ity through optimizing neural network inputsrdquo AIChE Journalvol 41 no 6 pp 1471ndash1480 1995
[22] Y Bengio and P Lamblin ldquoGreedy layer-wise training of deepnetworksrdquo in Proceedings of the Advances in Neural InformationProcessing Systems 19 (NIPS rsquo07) pp 153ndash160 September 2007
[23] H Larochelle Y Bengio J Lourador and P Lamblin ldquoExplor-ing strategies for training deep neural networksrdquoThe Journal ofMachine Learning Research vol 10 pp 1ndash40 2009
[24] D Yu and L Deng ldquoDeep learning and its applications to signaland information processing [exploratory DSP]rdquo IEEE SignalProcessing Magazine vol 28 no 1 pp 145ndash154 2011
[25] D ErhanA Courville andPVincent ldquoWhydoes unsupervisedpre-training help deep learningrdquo Journal of Machine LearningResearch vol 11 pp 625ndash660 2010
[26] P Vincent H Larochelle Y Bengio and P-A ManzagolldquoExtracting and composing robust features with denoisingautoencodersrdquo in Proceedings of the 25th International Confer-ence on Machine Learning (ICML rsquo08) pp 1096ndash1103 July 2008
[27] P Vincent H Larochelle I Lajoie Y Bengio and P-AManzagol ldquoStacked denoising autoencoders learning usefulrepresentations in a deep network with a local denoisingcriterionrdquo Journal of Machine Learning Research vol 11 pp3371ndash3408 2010
[28] N Srivastava G Hinton A Krizhevsky I Sutskever and RSalakhutdinov ldquoDropout a simple way to prevent neural net-works from overfittingrdquo Journal of Machine Learning Researchvol 15 pp 1929ndash1958 2014
[29] M J Embrechts B J Hargis and J D Linton ldquoAn augmentedefficient backpropagation training strategy for deep autoasso-ciative neural networksrdquo in Proceedings of the 18th EuropeanSymposiumonArtificial NeuralNetworksmdashComputational Intel-ligence and Machine Learning (ESANN rsquo10) vol no pp 28ndash30April 2010
[30] Y Bengio ldquoPractical recommendations for gradient-basedtraining of deep architecturesrdquo inNeural Networks Tricks of theTrade vol 7700 of Lecture Notes in Computer Science pp 437ndash478 Springer Berlin Germany 2nd edition 2012
Science and Technology of Nuclear Installations 17
[31] J Bergstra and Y Bengio ldquoRandom search for hyper-parameteroptimizationrdquo Journal of Machine Learning Research vol 13 pp281ndash305 2012
[32] J W Hines and D R Garvey ldquoDevelopment and applicationof fault detectability performance metrics for instrument cali-bration verification and anomaly detectionrdquo Journal of PatternRecognition Research vol 1 no 1 pp 2ndash15 2006
[33] A Usynin and J W Hines ldquoOn-line monitoring robustnessmeasures and comparisonsrdquo in Proceedings of the Interna-tional Atomic Energy Agency Technical Meeting on ldquoIncreasingInstrument Calibration Interval through On-Line CalibrationTechnologyrdquo OECD Halden Reactor Project Halden NorwaySeptember 2004
[34] A Wald ldquoSequential tests of statistical hypothesesrdquo Annals ofMathematical Statistics vol 16 no 2 pp 117ndash186 1945
[35] F Di Maio P Baraldi E Zio and R Seraoui ldquoFault detection innuclear power plants components by a combination of statisticalmethodsrdquo IEEE Transactions on Reliability vol 62 no 4 pp833ndash845 2013
[36] S Cheng and M Pecht ldquoUsing cross-validation for modelparameter selection of sequential probability ratio testrdquo ExpertSystems with Applications vol 39 no 9 pp 8467ndash8473 2012
TribologyAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
FuelsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal ofPetroleum Engineering
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Power ElectronicsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
CombustionJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Renewable Energy
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
StructuresJournal of
International Journal of
RotatingMachinery
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporation httpwwwhindawicom
Journal ofEngineeringVolume 2014
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
International Journal ofPhotoenergy
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear InstallationsScience and Technology of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Solar EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Wind EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear EnergyInternational Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
High Energy PhysicsAdvances in
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
12 Science and Technology of Nuclear Installations
0028 0032 0041 012021016 024
0457
061
077
01 02 03 04 05Sensors failed with gross highlow error ()
Spillover gross sensor failure
0010203040506070809
Avg
cros
s-se
nsiti
vity
DAASMK-AANN
Figure 14 Comparison of spillover effect against simultaneous grosshighlow failure in multiple sensors
84 Fault Detectability Fault detectability metric measuresthe smallest fault that can be detected by integrated sensorestimation and fault detection module as shown in Figure 1[32] The detectability metric is measured as percentage ofsensor span 119863 = 119872Span where value M correspondsto minimum detectable fault Minimum fault detectabilitylimit for each sensor is quantified through statistical basedsequential probability ratio test (SPRT) by Wald [34] SPRTtest is carried out to detect if the residual being generatedfrom normal distribution 119873(120583
1 1205902
) or 119873(1205830 1205902
) as definedfor faulty and fault free sensor operations respectively [35]Calibration failures are reflected in the mean parameter ofresidualrsquos distribution The SPRT procedure is applied todetect changes in the mean of residualrsquos distribution Theapplication of SPRT requires setting of following parametersvalue [36]
1205830 normal mode residual mean
1205902 normal mode residual variance
1205831 expected offset in residual mean in abnormal
mode120572 false alarm probability120573 missed alarm probability
Under normal mode the residuals from observed and modelestimated sensor values behave as awhiteGaussian noisewithmean 120583
0= 0 The residual variance 120590
2 is estimated for eachsensor under normal operating conditions and remainedfixed The false alarm 120572 and missed alarm 120573 probabilities areset to be 0001 and 001 respectively In order to determineminimum fault detectability limit a numerical procedureis opted which searches for minimum expected offset 120583
1
in the interval 1205831
[120590ndash3120590] provided the constraint onmissed and false alarm rate holds 120590 is the standard deviationcorresponding to residual variance of particular sensor Theplot in Figure 15 shows the detectability metric for eachsensor The plot in Figure 15 shows that DAASM can detect
0312 0315 033 035 004 005 004 008 0086 01 009 006 0058
0184 0165 019 02 0021 0023 002 0048 005 004 004 0032 0028
STMPSR 1
STMPSR 2
STMPSR 3
PZRPSR 1
PZRPSR 2
PZRLVL 1
PZRLVL 2 LVL
SG
NR 2LVLSG
NR 1K-AANN
DAASM
Fault detectability metric
0005
01015
02025
03035
04
Det
ecta
bilit
y (
span
)
Feedflow
1
Feedflow
2
STMflow
1
STMflow
2
Figure 15 Comparison of fault detectability metrics
faults which are two times smaller in magnitude than thosedetectable by K-AANNmodel
Improvement in fault detectability metric for DAASMcan be attributed to observed improvement in model robust-ness as suggested by the following relation
119903119894
Δ119878drift119894
= (1 minus 119878119894
Auto) (20)
The term 119903119894Δ119878
drift119894
measures the ratio of observed residualto actual sensor drift in terms of autosensitivity For highlyrobust model this ratio reduces to one which means residualreflects the actual drift and results in high fault detectabilityContrarily ratio value close to zero means that the predictionis following the input and results in poor fault detectability
841 SPRT Based Fault Detectability Test Sequential prob-ability ratio [34 36] based fault hypothesis test is appliedto residual sequence 119877
119894 = 119903
1(1199051) 1199031(1199051)sdot sdot sdot 119903119899(119905119899)generated
by relation 119877119894(119905119894) = 119878
Obs(119905119894) minus 119878
Est(119905119894) at time 119905
119894 where
119878Obs
(119905119894) and 119878
Est(119905119894) are the actual and model predicted
sensor values respectively The SPRT procedure analyzeswhether the residual sequence is more likely to be generatedfrom a probability distribution that belongs to normal modehypothesis 119867
0or abnormal mode hypothesis 119867
1by using
likelihood ratio as follows
119871119899= exp[minus(12120590
2)[sum119899
119894=11205831(1205831minus2119903119894)]]
(21)
For fault free sensor values the normalmode hypothesis1198670is
approximated byGaussian distributionwithmean 1205830= 0 and
variance 1205902 Abnormal mode hypothesis 119867
1is approximated
with mean 1205831
gt 1205830using the same variance 120590
2 The SPRTindex for the positive mean test is finally obtained by takinglogarithm of the likelihood ratio in (21) as follows [35]
SPRT = ln (119871119899) = minus
1
21205902[
119899
sum
119894=1
1205831(1205831minus 2119903119894)]
=1205831
1205902
119899
sum
119894=1
(119903119894minus
1205831
2)
(22)
Pressurizer pressure sensor sampled at a frequency of 10 sec-onds is used as a test signal to validate the fault detectability
Science and Technology of Nuclear Installations 13
0 500 1000 1500 2000 2500 3000 3500 4000 45001535
1541545
SPRT drift detection (pressure sensor)(B
ARG
)Pr
essu
re
Drifted sensorDAASM estimate
Sample number
0 500 1000 1500 2000 2500 3000 3500 4000 45001535
1541545
(BA
RG)
Pres
sure
Drifted sensorK-AANN estimate
Sample number
0515
SPRT
hypo
thes
is
minus050 500 1000 1500 2000 2500 3000 3500 4000 4500
DAASM indexK-AANN index
Sample number
005
valu
eRe
sidua
l
0 500 1000 1500 2000 2500 3000 3500 4000 4500
Residual DAASMResidual K-AANN
Sample number
minus05
H1mdash1
H0mdash0
Figure 16 SPRTbased fault detection in pressurizer pressure sensor
performance Two drift faults at the rate of +001hour andminus001hour are introduced in the test signal for DAASMand K-AANN modelrsquos assessment respectively The first andsecond plots in Figure 16 show drifted and estimated pressuresignal from DAASM and K-AANN models respectivelyThird plot shows residual values generated by differencing thedrifted and estimated signals frombothmodelsThe final plotshows SPRT index values against residuals from K-AANNmodel and DAASM The hypotheses 119867
1and 119867
0correspond
to positive and negative fault acceptance respectively FromSPRT index plot successful early detection of the sensordrift at 2200th sample with lag of 611 hours since the driftinception shows that DAASM is more sensitive to smalldrifts On the other hand SPRT index on K-AANN basedsensor estimates registered the same drift at 3800th samplewith a lag of almost 1055 hours The result shows thatDAASM is more robust in terms of early fault detection withlow false and missed alarm rates
Finally both models are tested against five test data setsEach test set consists of 3630 samples corresponding todifferentmonths of full power reactor operation Bothmodelssuccessfully detected an offset failure of 012ndash03 BARG in allsteam pressure channels and a drift type failure up to 285 insteam generator level (Figure 22)The K-AANNmodel failedto register a very small drift up to 01 in steam flow (STMflow 1) channel A small drift up to 01 BARG is detected in
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Steam pressure estimation
Observed valueDAASM estimateK-AANN estimate
672673674675676677678679
68
Stea
m p
ress
ure (
BARG
)
Figure 17 Steam pressure estimates against offset failures up to 03(BARG)
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Feed flow estimation
400420440460480500520540560
Feed
flow
(kg
s)
Observed sensor valueK-AANN estimateDAASM estimate
Figure 18 Feed flow estimates
test set 5 of pressurizer pressure channel However in caseof drift type sensor failures fault detection lag for DAASMwas on average 05 times smaller in comparison with K-AANN model Plots in Figures 17ndash21 show the estimatedsensor values from both models on five test data sets of fewselected channels
9 Conclusion
This paper presented a neural network based denoisedautoassociative sensor model (DAASM) for empirical sen-sor modeling The proposed sensor model is trained togenerate a monitoring system for sensor fault detection innuclear power plantsMultilayer AANNbased sensormodelsmay result in suboptimal solutions due to poor regulariza-tion by traditional backpropagation based joint multilayerlearning procedures So a complementary deep learningapproach based on greedy layerwise unsupervised pretrain-ing is employed for effective regularization in the proposedmultilayer DAASM Autoencoder architecture is used fordenoising based unsupervised pretraining and regularizationof individual layers in the network hierarchy To addressrobustness against perturbations in input sensors data cor-ruption processes exercised during unsupervised pretraining
14 Science and Technology of Nuclear Installations
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5490495500505510515520525530535540
Stea
m fl
ow (k
gs)
Steam flow estimation
Observed sensor valueDAASM estimateK-AANN estimate
Figure 19 Steam flow estimates against drift failure up to 01
58859
592594596598
60602604606608
Pres
suriz
er le
vel (
)
Pressurizer level estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valueK-AANN estimateDAASM estimate
Figure 20 Pressurizer level estimates
phase were based on prior knowledge about different failurescenarios Results from invariance tests showed that theproposed data corruption schemeswere beneficial in learninglatent representations at hidden layers and were invariantto multiple levels of perturbation in input sensors Conse-quently these pretrained hidden layers worked as well regu-larized perturbation filters with increased invariance towardssensor faults It is also observed that sensitivity against sensorfaults decreased significantly towards higher layers in fullDAASMassembly In a practical context of sensormonitoringin nuclear power plants the proposed model proved itsrobustness against gross type simultaneous sensor failuresIt also showed significant improvement in all performancemetrics when compared with popular and widely used five-layered AANNmodel by KramerMoreover time lag in smalldriftrsquos detection is significantly reduced The overall resultssuggest that greedy layerwise pretraining technique in com-bination with domain specific corruption processes providesa viable framework for effective regularization and robustnessin such deep multilayered autoassociative sensor validationmodels
Appendix
See Table 3
153715375
153815385
153915395
15415405
154115415
Pres
sure
(BA
RG)
Pressurizer pressure estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valuesK-AANN estimateDAASM estimate
Figure 21 Pressurizer pressure estimates against drift failure up to01
74
76
78
80
82
84
Stea
m g
ener
ator
leve
l (
)
Steam generator level estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valueK-AANN estimateDAASM estimate
Figure 22 Steam generator level estimates against growing driftfailure up to 287
Abbreviations
AANN Autoassociative neural networkK-AANN Kramer proposed Autoassociative neural
networkDAASM Denoised autoassociative sensor modelNPP Nuclear power plantPWR Pressurized water reactor119878Auto Autosensitivity119878Cross Cross-sensitivityDAE Denoising autoencoder119878 Observed sensor value Model predicted sensor value Corrupted sensor valueSPN Salt-and-pepper noiseAGN Additive Gaussian noise
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Science and Technology of Nuclear Installations 15
Table3Re
sults
ofDAASM
andK-AANNperfo
rmance
metric
sPerfo
rmance
metric
Mod
eltype
Feed
flow1
Feed
flow2
STM
flow1
STM
flow2
STM
PSR1
STM
PSR2
STM
PSR3
PZRPS
R1
PZRPS
R2
PZRLV
L1
PZRLV
L2
SGLV
LNR1
SGLV
LNR2
AANN
Sp-D
AANN
Accuracy
(span)
0382
0361
0343
0412
0186
0211
0166
0386
0411
0243
0223
0512
0621
0281
0253
0246
0293
0121
0132
0122
0243
0315
0173
0156
0394
0465
Autosensitivity
0273
0294
0321
0332
0205
0182
0198
0253
0233
0225
0231
0268
0301
0153
0167
0187
0163
0112
0079
0082
0131
0071
014
0117
0133
0187
Cross-sensitivity
0128
0124
0113
0116
0112
0108
0092
0079
0083
0098
011
0103
0105
00632
00527
00563
00614
0039
00311
00287
00186
00203
00344
00336
003
00286
SPRT
detectability
0312
0315
033
035
004
005
004
008
0086
01
009
006
0058
0184
0165
019
02
0021
0023
002
004
8005
004
004
0032
0028
16 Science and Technology of Nuclear Installations
Acknowledgments
The authors acknowledge the support of Mr Jose GalindoRodrıguez affiliated to TECNATOM Inc Spain and MrChad Painter (Director of Nuclear Power Plant SimulatorDevelopment and Training Program at International AtomicEnergy Agency) for providing necessary tools and data toconduct this research
References
[1] N Mehranbod M Soroush M Piovoso and B A OgunnaikeldquoProbabilistic model for sensor fault detection and identifica-tionrdquo AIChE Journal vol 49 no 7 pp 1787ndash1802 2003
[2] JW Hines and E Davis ldquoLessons learned from the US nuclearpower plant on-line monitoring programsrdquo Progress in NuclearEnergy vol 46 no 3-4 pp 176ndash189 2005
[3] EPRI ldquoOn-line monitoring cost benefit guiderdquo Final Report1006777 EPRI Palo Alto Calif USA 2003
[4] S J Qin and W Li ldquoDetection identification and recon-struction of faulty sensors with maximized sensitivityrdquo AIChEJournal vol 45 no 9 pp 1963ndash1976 1999
[5] J Ma and J Jiang ldquoApplications of fault detection and diagnosismethods in nuclear power plants a reviewrdquo Progress in NuclearEnergy vol 53 no 3 pp 255ndash266 2011
[6] G YHeo ldquoConditionmonitoring using empiricalmodels tech-nical review and prospects for nuclear applicationsrdquo NuclearEngineering and Technology vol 40 no 1 pp 49ndash68 2008
[7] J B Coble R M Meyer P Ramuhalli et al ldquoA review ofsensor calibration monitoring for calibration interval extensionin nuclear power plantsrdquo Tech Rep PNNL-21687 PacificNorthwest National Laboratory Richland Wash USA 2012
[8] M A Kramer ldquoAutoassociative neural networksrdquo Computersand Chemical Engineering vol 16 no 4 pp 313ndash328 1992
[9] X Xu J W Hines and R E Uhrig ldquoOn-line sensor calibra-tion monitoring and fault detection for chemical processesrdquoin Proceedings of the Maintenance and Reliability Conference(MARCON rsquo98) pp 12ndash14 Knoxville Tenn USA May 1998
[10] M Hamidreza S Mehdi J-R Hooshang and N AliakbarldquoReconstruction based approach to sensor fault diagnosis usingauto-associative neural networksrdquo Journal of Central SouthUniversity vol 21 no 6 pp 2273ndash2281 2014
[11] UThissenW JMelssen andLMC Buydens ldquoNonlinear pro-cess monitoring using bottle-neck neural networksrdquo AnalyticaChimica Acta vol 446 no 1-2 pp 371ndash383 2001
[12] D J Wrest J W Hines and R E Uhrig ldquoInstrument surveil-lance and calibration verification through plant wide monitor-ing using autoassociative neural networksrdquo in Proceedings ofthe American Nuclear Society International Topical Meeting onNuclear Plant Instrumentation Control and Human MachineInterface Technologies University Park Pa USA May 1996
[13] J W Hines R E Uhrig and D J Wrest ldquoUse of autoassociativeneural networks for signal validationrdquo Journal of Intelligent andRobotic Systems Theory and Applications vol 21 no 2 pp 143ndash154 1998
[14] P F Fantoni M I Hoffmann R Shankar and E L Davis ldquoOn-line monitoring of instrument channel performance in nuclearpower plant using PEANOrdquo Progress in Nuclear Energy vol 43no 1ndash4 pp 83ndash89 2003
[15] MMarseguerra andA Zoia ldquoTheAutoAssociative Neural Net-work in signal analysis II Application to on-line monitoring of
a simulated BWR componentrdquo Annals of Nuclear Energy vol32 no 11 pp 1207ndash1223 2005
[16] M S Ikbal H Misra and B Yegnanarayana ldquoAnalysis ofautoassociative mapping neural networksrdquo in Proceedings of theInternational Joint Conference on Neural Networks (IJCNN rsquo99)vol 5 pp 3025ndash3029 IEEE Washington DC USA July 1999Proceedings (Cat no 99CH36339)
[17] Y Bengio ldquoLearning deep architectures for AIrdquo Foundationsand Trends in Machine Learning vol 2 no 1 pp 1ndash27 2009
[18] J W Hines A V Gribok I Attieh and R E Uhrig ldquoRegular-izationmethods for inferential sensing in nuclear power plantsrdquoin Fuzzy Systems and Soft Computing in Nuclear Engineering DRuan Ed vol 38 of Studies in Fuzziness and Soft Computingpp 285ndash314 Physica 2000
[19] A V Gribok J W Hines A Urmanov and R E UhrigldquoHeuristic systematic and informational regularization forprocessmonitoringrdquo International Journal of Intelligent Systemsvol 17 no 8 pp 723ndash749 2002
[20] X Glorot and Y Bengio ldquoUnderstanding the difficulty oftraining deep feedforward neural networksrdquo in Proceedings ofthe 13th International Conference on Artificial Intelligence andStatistics (AISTATS rsquo10) vol 9 pp 249ndash256 Sardinia Italy May2010
[21] S Tan andM L Mayrovouniotis ldquoReducing data dimensional-ity through optimizing neural network inputsrdquo AIChE Journalvol 41 no 6 pp 1471ndash1480 1995
[22] Y Bengio and P Lamblin ldquoGreedy layer-wise training of deepnetworksrdquo in Proceedings of the Advances in Neural InformationProcessing Systems 19 (NIPS rsquo07) pp 153ndash160 September 2007
[23] H Larochelle Y Bengio J Lourador and P Lamblin ldquoExplor-ing strategies for training deep neural networksrdquoThe Journal ofMachine Learning Research vol 10 pp 1ndash40 2009
[24] D Yu and L Deng ldquoDeep learning and its applications to signaland information processing [exploratory DSP]rdquo IEEE SignalProcessing Magazine vol 28 no 1 pp 145ndash154 2011
[25] D ErhanA Courville andPVincent ldquoWhydoes unsupervisedpre-training help deep learningrdquo Journal of Machine LearningResearch vol 11 pp 625ndash660 2010
[26] P Vincent H Larochelle Y Bengio and P-A ManzagolldquoExtracting and composing robust features with denoisingautoencodersrdquo in Proceedings of the 25th International Confer-ence on Machine Learning (ICML rsquo08) pp 1096ndash1103 July 2008
[27] P Vincent H Larochelle I Lajoie Y Bengio and P-AManzagol ldquoStacked denoising autoencoders learning usefulrepresentations in a deep network with a local denoisingcriterionrdquo Journal of Machine Learning Research vol 11 pp3371ndash3408 2010
[28] N Srivastava G Hinton A Krizhevsky I Sutskever and RSalakhutdinov ldquoDropout a simple way to prevent neural net-works from overfittingrdquo Journal of Machine Learning Researchvol 15 pp 1929ndash1958 2014
[29] M J Embrechts B J Hargis and J D Linton ldquoAn augmentedefficient backpropagation training strategy for deep autoasso-ciative neural networksrdquo in Proceedings of the 18th EuropeanSymposiumonArtificial NeuralNetworksmdashComputational Intel-ligence and Machine Learning (ESANN rsquo10) vol no pp 28ndash30April 2010
[30] Y Bengio ldquoPractical recommendations for gradient-basedtraining of deep architecturesrdquo inNeural Networks Tricks of theTrade vol 7700 of Lecture Notes in Computer Science pp 437ndash478 Springer Berlin Germany 2nd edition 2012
Science and Technology of Nuclear Installations 17
[31] J Bergstra and Y Bengio ldquoRandom search for hyper-parameteroptimizationrdquo Journal of Machine Learning Research vol 13 pp281ndash305 2012
[32] J W Hines and D R Garvey ldquoDevelopment and applicationof fault detectability performance metrics for instrument cali-bration verification and anomaly detectionrdquo Journal of PatternRecognition Research vol 1 no 1 pp 2ndash15 2006
[33] A Usynin and J W Hines ldquoOn-line monitoring robustnessmeasures and comparisonsrdquo in Proceedings of the Interna-tional Atomic Energy Agency Technical Meeting on ldquoIncreasingInstrument Calibration Interval through On-Line CalibrationTechnologyrdquo OECD Halden Reactor Project Halden NorwaySeptember 2004
[34] A Wald ldquoSequential tests of statistical hypothesesrdquo Annals ofMathematical Statistics vol 16 no 2 pp 117ndash186 1945
[35] F Di Maio P Baraldi E Zio and R Seraoui ldquoFault detection innuclear power plants components by a combination of statisticalmethodsrdquo IEEE Transactions on Reliability vol 62 no 4 pp833ndash845 2013
[36] S Cheng and M Pecht ldquoUsing cross-validation for modelparameter selection of sequential probability ratio testrdquo ExpertSystems with Applications vol 39 no 9 pp 8467ndash8473 2012
TribologyAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
FuelsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal ofPetroleum Engineering
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Power ElectronicsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
CombustionJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Renewable Energy
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
StructuresJournal of
International Journal of
RotatingMachinery
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporation httpwwwhindawicom
Journal ofEngineeringVolume 2014
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
International Journal ofPhotoenergy
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear InstallationsScience and Technology of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Solar EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Wind EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear EnergyInternational Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
High Energy PhysicsAdvances in
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Science and Technology of Nuclear Installations 13
0 500 1000 1500 2000 2500 3000 3500 4000 45001535
1541545
SPRT drift detection (pressure sensor)(B
ARG
)Pr
essu
re
Drifted sensorDAASM estimate
Sample number
0 500 1000 1500 2000 2500 3000 3500 4000 45001535
1541545
(BA
RG)
Pres
sure
Drifted sensorK-AANN estimate
Sample number
0515
SPRT
hypo
thes
is
minus050 500 1000 1500 2000 2500 3000 3500 4000 4500
DAASM indexK-AANN index
Sample number
005
valu
eRe
sidua
l
0 500 1000 1500 2000 2500 3000 3500 4000 4500
Residual DAASMResidual K-AANN
Sample number
minus05
H1mdash1
H0mdash0
Figure 16 SPRTbased fault detection in pressurizer pressure sensor
performance Two drift faults at the rate of +001hour andminus001hour are introduced in the test signal for DAASMand K-AANN modelrsquos assessment respectively The first andsecond plots in Figure 16 show drifted and estimated pressuresignal from DAASM and K-AANN models respectivelyThird plot shows residual values generated by differencing thedrifted and estimated signals frombothmodelsThe final plotshows SPRT index values against residuals from K-AANNmodel and DAASM The hypotheses 119867
1and 119867
0correspond
to positive and negative fault acceptance respectively FromSPRT index plot successful early detection of the sensordrift at 2200th sample with lag of 611 hours since the driftinception shows that DAASM is more sensitive to smalldrifts On the other hand SPRT index on K-AANN basedsensor estimates registered the same drift at 3800th samplewith a lag of almost 1055 hours The result shows thatDAASM is more robust in terms of early fault detection withlow false and missed alarm rates
Finally both models are tested against five test data setsEach test set consists of 3630 samples corresponding todifferentmonths of full power reactor operation Bothmodelssuccessfully detected an offset failure of 012ndash03 BARG in allsteam pressure channels and a drift type failure up to 285 insteam generator level (Figure 22)The K-AANNmodel failedto register a very small drift up to 01 in steam flow (STMflow 1) channel A small drift up to 01 BARG is detected in
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Steam pressure estimation
Observed valueDAASM estimateK-AANN estimate
672673674675676677678679
68
Stea
m p
ress
ure (
BARG
)
Figure 17 Steam pressure estimates against offset failures up to 03(BARG)
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Feed flow estimation
400420440460480500520540560
Feed
flow
(kg
s)
Observed sensor valueK-AANN estimateDAASM estimate
Figure 18 Feed flow estimates
test set 5 of pressurizer pressure channel However in caseof drift type sensor failures fault detection lag for DAASMwas on average 05 times smaller in comparison with K-AANN model Plots in Figures 17ndash21 show the estimatedsensor values from both models on five test data sets of fewselected channels
9 Conclusion
This paper presented a neural network based denoisedautoassociative sensor model (DAASM) for empirical sen-sor modeling The proposed sensor model is trained togenerate a monitoring system for sensor fault detection innuclear power plantsMultilayer AANNbased sensormodelsmay result in suboptimal solutions due to poor regulariza-tion by traditional backpropagation based joint multilayerlearning procedures So a complementary deep learningapproach based on greedy layerwise unsupervised pretrain-ing is employed for effective regularization in the proposedmultilayer DAASM Autoencoder architecture is used fordenoising based unsupervised pretraining and regularizationof individual layers in the network hierarchy To addressrobustness against perturbations in input sensors data cor-ruption processes exercised during unsupervised pretraining
14 Science and Technology of Nuclear Installations
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5490495500505510515520525530535540
Stea
m fl
ow (k
gs)
Steam flow estimation
Observed sensor valueDAASM estimateK-AANN estimate
Figure 19 Steam flow estimates against drift failure up to 01
58859
592594596598
60602604606608
Pres
suriz
er le
vel (
)
Pressurizer level estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valueK-AANN estimateDAASM estimate
Figure 20 Pressurizer level estimates
phase were based on prior knowledge about different failurescenarios Results from invariance tests showed that theproposed data corruption schemeswere beneficial in learninglatent representations at hidden layers and were invariantto multiple levels of perturbation in input sensors Conse-quently these pretrained hidden layers worked as well regu-larized perturbation filters with increased invariance towardssensor faults It is also observed that sensitivity against sensorfaults decreased significantly towards higher layers in fullDAASMassembly In a practical context of sensormonitoringin nuclear power plants the proposed model proved itsrobustness against gross type simultaneous sensor failuresIt also showed significant improvement in all performancemetrics when compared with popular and widely used five-layered AANNmodel by KramerMoreover time lag in smalldriftrsquos detection is significantly reduced The overall resultssuggest that greedy layerwise pretraining technique in com-bination with domain specific corruption processes providesa viable framework for effective regularization and robustnessin such deep multilayered autoassociative sensor validationmodels
Appendix
See Table 3
153715375
153815385
153915395
15415405
154115415
Pres
sure
(BA
RG)
Pressurizer pressure estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valuesK-AANN estimateDAASM estimate
Figure 21 Pressurizer pressure estimates against drift failure up to01
74
76
78
80
82
84
Stea
m g
ener
ator
leve
l (
)
Steam generator level estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valueK-AANN estimateDAASM estimate
Figure 22 Steam generator level estimates against growing driftfailure up to 287
Abbreviations
AANN Autoassociative neural networkK-AANN Kramer proposed Autoassociative neural
networkDAASM Denoised autoassociative sensor modelNPP Nuclear power plantPWR Pressurized water reactor119878Auto Autosensitivity119878Cross Cross-sensitivityDAE Denoising autoencoder119878 Observed sensor value Model predicted sensor value Corrupted sensor valueSPN Salt-and-pepper noiseAGN Additive Gaussian noise
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Science and Technology of Nuclear Installations 15
Table3Re
sults
ofDAASM
andK-AANNperfo
rmance
metric
sPerfo
rmance
metric
Mod
eltype
Feed
flow1
Feed
flow2
STM
flow1
STM
flow2
STM
PSR1
STM
PSR2
STM
PSR3
PZRPS
R1
PZRPS
R2
PZRLV
L1
PZRLV
L2
SGLV
LNR1
SGLV
LNR2
AANN
Sp-D
AANN
Accuracy
(span)
0382
0361
0343
0412
0186
0211
0166
0386
0411
0243
0223
0512
0621
0281
0253
0246
0293
0121
0132
0122
0243
0315
0173
0156
0394
0465
Autosensitivity
0273
0294
0321
0332
0205
0182
0198
0253
0233
0225
0231
0268
0301
0153
0167
0187
0163
0112
0079
0082
0131
0071
014
0117
0133
0187
Cross-sensitivity
0128
0124
0113
0116
0112
0108
0092
0079
0083
0098
011
0103
0105
00632
00527
00563
00614
0039
00311
00287
00186
00203
00344
00336
003
00286
SPRT
detectability
0312
0315
033
035
004
005
004
008
0086
01
009
006
0058
0184
0165
019
02
0021
0023
002
004
8005
004
004
0032
0028
16 Science and Technology of Nuclear Installations
Acknowledgments
The authors acknowledge the support of Mr Jose GalindoRodrıguez affiliated to TECNATOM Inc Spain and MrChad Painter (Director of Nuclear Power Plant SimulatorDevelopment and Training Program at International AtomicEnergy Agency) for providing necessary tools and data toconduct this research
References
[1] N Mehranbod M Soroush M Piovoso and B A OgunnaikeldquoProbabilistic model for sensor fault detection and identifica-tionrdquo AIChE Journal vol 49 no 7 pp 1787ndash1802 2003
[2] JW Hines and E Davis ldquoLessons learned from the US nuclearpower plant on-line monitoring programsrdquo Progress in NuclearEnergy vol 46 no 3-4 pp 176ndash189 2005
[3] EPRI ldquoOn-line monitoring cost benefit guiderdquo Final Report1006777 EPRI Palo Alto Calif USA 2003
[4] S J Qin and W Li ldquoDetection identification and recon-struction of faulty sensors with maximized sensitivityrdquo AIChEJournal vol 45 no 9 pp 1963ndash1976 1999
[5] J Ma and J Jiang ldquoApplications of fault detection and diagnosismethods in nuclear power plants a reviewrdquo Progress in NuclearEnergy vol 53 no 3 pp 255ndash266 2011
[6] G YHeo ldquoConditionmonitoring using empiricalmodels tech-nical review and prospects for nuclear applicationsrdquo NuclearEngineering and Technology vol 40 no 1 pp 49ndash68 2008
[7] J B Coble R M Meyer P Ramuhalli et al ldquoA review ofsensor calibration monitoring for calibration interval extensionin nuclear power plantsrdquo Tech Rep PNNL-21687 PacificNorthwest National Laboratory Richland Wash USA 2012
[8] M A Kramer ldquoAutoassociative neural networksrdquo Computersand Chemical Engineering vol 16 no 4 pp 313ndash328 1992
[9] X Xu J W Hines and R E Uhrig ldquoOn-line sensor calibra-tion monitoring and fault detection for chemical processesrdquoin Proceedings of the Maintenance and Reliability Conference(MARCON rsquo98) pp 12ndash14 Knoxville Tenn USA May 1998
[10] M Hamidreza S Mehdi J-R Hooshang and N AliakbarldquoReconstruction based approach to sensor fault diagnosis usingauto-associative neural networksrdquo Journal of Central SouthUniversity vol 21 no 6 pp 2273ndash2281 2014
[11] UThissenW JMelssen andLMC Buydens ldquoNonlinear pro-cess monitoring using bottle-neck neural networksrdquo AnalyticaChimica Acta vol 446 no 1-2 pp 371ndash383 2001
[12] D J Wrest J W Hines and R E Uhrig ldquoInstrument surveil-lance and calibration verification through plant wide monitor-ing using autoassociative neural networksrdquo in Proceedings ofthe American Nuclear Society International Topical Meeting onNuclear Plant Instrumentation Control and Human MachineInterface Technologies University Park Pa USA May 1996
[13] J W Hines R E Uhrig and D J Wrest ldquoUse of autoassociativeneural networks for signal validationrdquo Journal of Intelligent andRobotic Systems Theory and Applications vol 21 no 2 pp 143ndash154 1998
[14] P F Fantoni M I Hoffmann R Shankar and E L Davis ldquoOn-line monitoring of instrument channel performance in nuclearpower plant using PEANOrdquo Progress in Nuclear Energy vol 43no 1ndash4 pp 83ndash89 2003
[15] MMarseguerra andA Zoia ldquoTheAutoAssociative Neural Net-work in signal analysis II Application to on-line monitoring of
a simulated BWR componentrdquo Annals of Nuclear Energy vol32 no 11 pp 1207ndash1223 2005
[16] M S Ikbal H Misra and B Yegnanarayana ldquoAnalysis ofautoassociative mapping neural networksrdquo in Proceedings of theInternational Joint Conference on Neural Networks (IJCNN rsquo99)vol 5 pp 3025ndash3029 IEEE Washington DC USA July 1999Proceedings (Cat no 99CH36339)
[17] Y Bengio ldquoLearning deep architectures for AIrdquo Foundationsand Trends in Machine Learning vol 2 no 1 pp 1ndash27 2009
[18] J W Hines A V Gribok I Attieh and R E Uhrig ldquoRegular-izationmethods for inferential sensing in nuclear power plantsrdquoin Fuzzy Systems and Soft Computing in Nuclear Engineering DRuan Ed vol 38 of Studies in Fuzziness and Soft Computingpp 285ndash314 Physica 2000
[19] A V Gribok J W Hines A Urmanov and R E UhrigldquoHeuristic systematic and informational regularization forprocessmonitoringrdquo International Journal of Intelligent Systemsvol 17 no 8 pp 723ndash749 2002
[20] X Glorot and Y Bengio ldquoUnderstanding the difficulty oftraining deep feedforward neural networksrdquo in Proceedings ofthe 13th International Conference on Artificial Intelligence andStatistics (AISTATS rsquo10) vol 9 pp 249ndash256 Sardinia Italy May2010
[21] S Tan andM L Mayrovouniotis ldquoReducing data dimensional-ity through optimizing neural network inputsrdquo AIChE Journalvol 41 no 6 pp 1471ndash1480 1995
[22] Y Bengio and P Lamblin ldquoGreedy layer-wise training of deepnetworksrdquo in Proceedings of the Advances in Neural InformationProcessing Systems 19 (NIPS rsquo07) pp 153ndash160 September 2007
[23] H Larochelle Y Bengio J Lourador and P Lamblin ldquoExplor-ing strategies for training deep neural networksrdquoThe Journal ofMachine Learning Research vol 10 pp 1ndash40 2009
[24] D Yu and L Deng ldquoDeep learning and its applications to signaland information processing [exploratory DSP]rdquo IEEE SignalProcessing Magazine vol 28 no 1 pp 145ndash154 2011
[25] D ErhanA Courville andPVincent ldquoWhydoes unsupervisedpre-training help deep learningrdquo Journal of Machine LearningResearch vol 11 pp 625ndash660 2010
[26] P Vincent H Larochelle Y Bengio and P-A ManzagolldquoExtracting and composing robust features with denoisingautoencodersrdquo in Proceedings of the 25th International Confer-ence on Machine Learning (ICML rsquo08) pp 1096ndash1103 July 2008
[27] P Vincent H Larochelle I Lajoie Y Bengio and P-AManzagol ldquoStacked denoising autoencoders learning usefulrepresentations in a deep network with a local denoisingcriterionrdquo Journal of Machine Learning Research vol 11 pp3371ndash3408 2010
[28] N Srivastava G Hinton A Krizhevsky I Sutskever and RSalakhutdinov ldquoDropout a simple way to prevent neural net-works from overfittingrdquo Journal of Machine Learning Researchvol 15 pp 1929ndash1958 2014
[29] M J Embrechts B J Hargis and J D Linton ldquoAn augmentedefficient backpropagation training strategy for deep autoasso-ciative neural networksrdquo in Proceedings of the 18th EuropeanSymposiumonArtificial NeuralNetworksmdashComputational Intel-ligence and Machine Learning (ESANN rsquo10) vol no pp 28ndash30April 2010
[30] Y Bengio ldquoPractical recommendations for gradient-basedtraining of deep architecturesrdquo inNeural Networks Tricks of theTrade vol 7700 of Lecture Notes in Computer Science pp 437ndash478 Springer Berlin Germany 2nd edition 2012
Science and Technology of Nuclear Installations 17
[31] J Bergstra and Y Bengio ldquoRandom search for hyper-parameteroptimizationrdquo Journal of Machine Learning Research vol 13 pp281ndash305 2012
[32] J W Hines and D R Garvey ldquoDevelopment and applicationof fault detectability performance metrics for instrument cali-bration verification and anomaly detectionrdquo Journal of PatternRecognition Research vol 1 no 1 pp 2ndash15 2006
[33] A Usynin and J W Hines ldquoOn-line monitoring robustnessmeasures and comparisonsrdquo in Proceedings of the Interna-tional Atomic Energy Agency Technical Meeting on ldquoIncreasingInstrument Calibration Interval through On-Line CalibrationTechnologyrdquo OECD Halden Reactor Project Halden NorwaySeptember 2004
[34] A Wald ldquoSequential tests of statistical hypothesesrdquo Annals ofMathematical Statistics vol 16 no 2 pp 117ndash186 1945
[35] F Di Maio P Baraldi E Zio and R Seraoui ldquoFault detection innuclear power plants components by a combination of statisticalmethodsrdquo IEEE Transactions on Reliability vol 62 no 4 pp833ndash845 2013
[36] S Cheng and M Pecht ldquoUsing cross-validation for modelparameter selection of sequential probability ratio testrdquo ExpertSystems with Applications vol 39 no 9 pp 8467ndash8473 2012
TribologyAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
FuelsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal ofPetroleum Engineering
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Power ElectronicsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
CombustionJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Renewable Energy
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
StructuresJournal of
International Journal of
RotatingMachinery
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporation httpwwwhindawicom
Journal ofEngineeringVolume 2014
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
International Journal ofPhotoenergy
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear InstallationsScience and Technology of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Solar EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Wind EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear EnergyInternational Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
High Energy PhysicsAdvances in
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
14 Science and Technology of Nuclear Installations
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5490495500505510515520525530535540
Stea
m fl
ow (k
gs)
Steam flow estimation
Observed sensor valueDAASM estimateK-AANN estimate
Figure 19 Steam flow estimates against drift failure up to 01
58859
592594596598
60602604606608
Pres
suriz
er le
vel (
)
Pressurizer level estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valueK-AANN estimateDAASM estimate
Figure 20 Pressurizer level estimates
phase were based on prior knowledge about different failurescenarios Results from invariance tests showed that theproposed data corruption schemeswere beneficial in learninglatent representations at hidden layers and were invariantto multiple levels of perturbation in input sensors Conse-quently these pretrained hidden layers worked as well regu-larized perturbation filters with increased invariance towardssensor faults It is also observed that sensitivity against sensorfaults decreased significantly towards higher layers in fullDAASMassembly In a practical context of sensormonitoringin nuclear power plants the proposed model proved itsrobustness against gross type simultaneous sensor failuresIt also showed significant improvement in all performancemetrics when compared with popular and widely used five-layered AANNmodel by KramerMoreover time lag in smalldriftrsquos detection is significantly reduced The overall resultssuggest that greedy layerwise pretraining technique in com-bination with domain specific corruption processes providesa viable framework for effective regularization and robustnessin such deep multilayered autoassociative sensor validationmodels
Appendix
See Table 3
153715375
153815385
153915395
15415405
154115415
Pres
sure
(BA
RG)
Pressurizer pressure estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valuesK-AANN estimateDAASM estimate
Figure 21 Pressurizer pressure estimates against drift failure up to01
74
76
78
80
82
84
Stea
m g
ener
ator
leve
l (
)
Steam generator level estimation
Test set 1 Test set 2 Test set 3 Test set 4 Test set 5
Observed sensor valueK-AANN estimateDAASM estimate
Figure 22 Steam generator level estimates against growing driftfailure up to 287
Abbreviations
AANN Autoassociative neural networkK-AANN Kramer proposed Autoassociative neural
networkDAASM Denoised autoassociative sensor modelNPP Nuclear power plantPWR Pressurized water reactor119878Auto Autosensitivity119878Cross Cross-sensitivityDAE Denoising autoencoder119878 Observed sensor value Model predicted sensor value Corrupted sensor valueSPN Salt-and-pepper noiseAGN Additive Gaussian noise
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Science and Technology of Nuclear Installations 15
Table3Re
sults
ofDAASM
andK-AANNperfo
rmance
metric
sPerfo
rmance
metric
Mod
eltype
Feed
flow1
Feed
flow2
STM
flow1
STM
flow2
STM
PSR1
STM
PSR2
STM
PSR3
PZRPS
R1
PZRPS
R2
PZRLV
L1
PZRLV
L2
SGLV
LNR1
SGLV
LNR2
AANN
Sp-D
AANN
Accuracy
(span)
0382
0361
0343
0412
0186
0211
0166
0386
0411
0243
0223
0512
0621
0281
0253
0246
0293
0121
0132
0122
0243
0315
0173
0156
0394
0465
Autosensitivity
0273
0294
0321
0332
0205
0182
0198
0253
0233
0225
0231
0268
0301
0153
0167
0187
0163
0112
0079
0082
0131
0071
014
0117
0133
0187
Cross-sensitivity
0128
0124
0113
0116
0112
0108
0092
0079
0083
0098
011
0103
0105
00632
00527
00563
00614
0039
00311
00287
00186
00203
00344
00336
003
00286
SPRT
detectability
0312
0315
033
035
004
005
004
008
0086
01
009
006
0058
0184
0165
019
02
0021
0023
002
004
8005
004
004
0032
0028
16 Science and Technology of Nuclear Installations
Acknowledgments
The authors acknowledge the support of Mr Jose GalindoRodrıguez affiliated to TECNATOM Inc Spain and MrChad Painter (Director of Nuclear Power Plant SimulatorDevelopment and Training Program at International AtomicEnergy Agency) for providing necessary tools and data toconduct this research
References
[1] N Mehranbod M Soroush M Piovoso and B A OgunnaikeldquoProbabilistic model for sensor fault detection and identifica-tionrdquo AIChE Journal vol 49 no 7 pp 1787ndash1802 2003
[2] JW Hines and E Davis ldquoLessons learned from the US nuclearpower plant on-line monitoring programsrdquo Progress in NuclearEnergy vol 46 no 3-4 pp 176ndash189 2005
[3] EPRI ldquoOn-line monitoring cost benefit guiderdquo Final Report1006777 EPRI Palo Alto Calif USA 2003
[4] S J Qin and W Li ldquoDetection identification and recon-struction of faulty sensors with maximized sensitivityrdquo AIChEJournal vol 45 no 9 pp 1963ndash1976 1999
[5] J Ma and J Jiang ldquoApplications of fault detection and diagnosismethods in nuclear power plants a reviewrdquo Progress in NuclearEnergy vol 53 no 3 pp 255ndash266 2011
[6] G YHeo ldquoConditionmonitoring using empiricalmodels tech-nical review and prospects for nuclear applicationsrdquo NuclearEngineering and Technology vol 40 no 1 pp 49ndash68 2008
[7] J B Coble R M Meyer P Ramuhalli et al ldquoA review ofsensor calibration monitoring for calibration interval extensionin nuclear power plantsrdquo Tech Rep PNNL-21687 PacificNorthwest National Laboratory Richland Wash USA 2012
[8] M A Kramer ldquoAutoassociative neural networksrdquo Computersand Chemical Engineering vol 16 no 4 pp 313ndash328 1992
[9] X Xu J W Hines and R E Uhrig ldquoOn-line sensor calibra-tion monitoring and fault detection for chemical processesrdquoin Proceedings of the Maintenance and Reliability Conference(MARCON rsquo98) pp 12ndash14 Knoxville Tenn USA May 1998
[10] M Hamidreza S Mehdi J-R Hooshang and N AliakbarldquoReconstruction based approach to sensor fault diagnosis usingauto-associative neural networksrdquo Journal of Central SouthUniversity vol 21 no 6 pp 2273ndash2281 2014
[11] UThissenW JMelssen andLMC Buydens ldquoNonlinear pro-cess monitoring using bottle-neck neural networksrdquo AnalyticaChimica Acta vol 446 no 1-2 pp 371ndash383 2001
[12] D J Wrest J W Hines and R E Uhrig ldquoInstrument surveil-lance and calibration verification through plant wide monitor-ing using autoassociative neural networksrdquo in Proceedings ofthe American Nuclear Society International Topical Meeting onNuclear Plant Instrumentation Control and Human MachineInterface Technologies University Park Pa USA May 1996
[13] J W Hines R E Uhrig and D J Wrest ldquoUse of autoassociativeneural networks for signal validationrdquo Journal of Intelligent andRobotic Systems Theory and Applications vol 21 no 2 pp 143ndash154 1998
[14] P F Fantoni M I Hoffmann R Shankar and E L Davis ldquoOn-line monitoring of instrument channel performance in nuclearpower plant using PEANOrdquo Progress in Nuclear Energy vol 43no 1ndash4 pp 83ndash89 2003
[15] MMarseguerra andA Zoia ldquoTheAutoAssociative Neural Net-work in signal analysis II Application to on-line monitoring of
a simulated BWR componentrdquo Annals of Nuclear Energy vol32 no 11 pp 1207ndash1223 2005
[16] M S Ikbal H Misra and B Yegnanarayana ldquoAnalysis ofautoassociative mapping neural networksrdquo in Proceedings of theInternational Joint Conference on Neural Networks (IJCNN rsquo99)vol 5 pp 3025ndash3029 IEEE Washington DC USA July 1999Proceedings (Cat no 99CH36339)
[17] Y Bengio ldquoLearning deep architectures for AIrdquo Foundationsand Trends in Machine Learning vol 2 no 1 pp 1ndash27 2009
[18] J W Hines A V Gribok I Attieh and R E Uhrig ldquoRegular-izationmethods for inferential sensing in nuclear power plantsrdquoin Fuzzy Systems and Soft Computing in Nuclear Engineering DRuan Ed vol 38 of Studies in Fuzziness and Soft Computingpp 285ndash314 Physica 2000
[19] A V Gribok J W Hines A Urmanov and R E UhrigldquoHeuristic systematic and informational regularization forprocessmonitoringrdquo International Journal of Intelligent Systemsvol 17 no 8 pp 723ndash749 2002
[20] X Glorot and Y Bengio ldquoUnderstanding the difficulty oftraining deep feedforward neural networksrdquo in Proceedings ofthe 13th International Conference on Artificial Intelligence andStatistics (AISTATS rsquo10) vol 9 pp 249ndash256 Sardinia Italy May2010
[21] S Tan andM L Mayrovouniotis ldquoReducing data dimensional-ity through optimizing neural network inputsrdquo AIChE Journalvol 41 no 6 pp 1471ndash1480 1995
[22] Y Bengio and P Lamblin ldquoGreedy layer-wise training of deepnetworksrdquo in Proceedings of the Advances in Neural InformationProcessing Systems 19 (NIPS rsquo07) pp 153ndash160 September 2007
[23] H Larochelle Y Bengio J Lourador and P Lamblin ldquoExplor-ing strategies for training deep neural networksrdquoThe Journal ofMachine Learning Research vol 10 pp 1ndash40 2009
[24] D Yu and L Deng ldquoDeep learning and its applications to signaland information processing [exploratory DSP]rdquo IEEE SignalProcessing Magazine vol 28 no 1 pp 145ndash154 2011
[25] D ErhanA Courville andPVincent ldquoWhydoes unsupervisedpre-training help deep learningrdquo Journal of Machine LearningResearch vol 11 pp 625ndash660 2010
[26] P Vincent H Larochelle Y Bengio and P-A ManzagolldquoExtracting and composing robust features with denoisingautoencodersrdquo in Proceedings of the 25th International Confer-ence on Machine Learning (ICML rsquo08) pp 1096ndash1103 July 2008
[27] P Vincent H Larochelle I Lajoie Y Bengio and P-AManzagol ldquoStacked denoising autoencoders learning usefulrepresentations in a deep network with a local denoisingcriterionrdquo Journal of Machine Learning Research vol 11 pp3371ndash3408 2010
[28] N Srivastava G Hinton A Krizhevsky I Sutskever and RSalakhutdinov ldquoDropout a simple way to prevent neural net-works from overfittingrdquo Journal of Machine Learning Researchvol 15 pp 1929ndash1958 2014
[29] M J Embrechts B J Hargis and J D Linton ldquoAn augmentedefficient backpropagation training strategy for deep autoasso-ciative neural networksrdquo in Proceedings of the 18th EuropeanSymposiumonArtificial NeuralNetworksmdashComputational Intel-ligence and Machine Learning (ESANN rsquo10) vol no pp 28ndash30April 2010
[30] Y Bengio ldquoPractical recommendations for gradient-basedtraining of deep architecturesrdquo inNeural Networks Tricks of theTrade vol 7700 of Lecture Notes in Computer Science pp 437ndash478 Springer Berlin Germany 2nd edition 2012
Science and Technology of Nuclear Installations 17
[31] J Bergstra and Y Bengio ldquoRandom search for hyper-parameteroptimizationrdquo Journal of Machine Learning Research vol 13 pp281ndash305 2012
[32] J W Hines and D R Garvey ldquoDevelopment and applicationof fault detectability performance metrics for instrument cali-bration verification and anomaly detectionrdquo Journal of PatternRecognition Research vol 1 no 1 pp 2ndash15 2006
[33] A Usynin and J W Hines ldquoOn-line monitoring robustnessmeasures and comparisonsrdquo in Proceedings of the Interna-tional Atomic Energy Agency Technical Meeting on ldquoIncreasingInstrument Calibration Interval through On-Line CalibrationTechnologyrdquo OECD Halden Reactor Project Halden NorwaySeptember 2004
[34] A Wald ldquoSequential tests of statistical hypothesesrdquo Annals ofMathematical Statistics vol 16 no 2 pp 117ndash186 1945
[35] F Di Maio P Baraldi E Zio and R Seraoui ldquoFault detection innuclear power plants components by a combination of statisticalmethodsrdquo IEEE Transactions on Reliability vol 62 no 4 pp833ndash845 2013
[36] S Cheng and M Pecht ldquoUsing cross-validation for modelparameter selection of sequential probability ratio testrdquo ExpertSystems with Applications vol 39 no 9 pp 8467ndash8473 2012
TribologyAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
FuelsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal ofPetroleum Engineering
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Power ElectronicsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
CombustionJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Renewable Energy
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
StructuresJournal of
International Journal of
RotatingMachinery
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporation httpwwwhindawicom
Journal ofEngineeringVolume 2014
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
International Journal ofPhotoenergy
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear InstallationsScience and Technology of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Solar EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Wind EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear EnergyInternational Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
High Energy PhysicsAdvances in
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Science and Technology of Nuclear Installations 15
Table3Re
sults
ofDAASM
andK-AANNperfo
rmance
metric
sPerfo
rmance
metric
Mod
eltype
Feed
flow1
Feed
flow2
STM
flow1
STM
flow2
STM
PSR1
STM
PSR2
STM
PSR3
PZRPS
R1
PZRPS
R2
PZRLV
L1
PZRLV
L2
SGLV
LNR1
SGLV
LNR2
AANN
Sp-D
AANN
Accuracy
(span)
0382
0361
0343
0412
0186
0211
0166
0386
0411
0243
0223
0512
0621
0281
0253
0246
0293
0121
0132
0122
0243
0315
0173
0156
0394
0465
Autosensitivity
0273
0294
0321
0332
0205
0182
0198
0253
0233
0225
0231
0268
0301
0153
0167
0187
0163
0112
0079
0082
0131
0071
014
0117
0133
0187
Cross-sensitivity
0128
0124
0113
0116
0112
0108
0092
0079
0083
0098
011
0103
0105
00632
00527
00563
00614
0039
00311
00287
00186
00203
00344
00336
003
00286
SPRT
detectability
0312
0315
033
035
004
005
004
008
0086
01
009
006
0058
0184
0165
019
02
0021
0023
002
004
8005
004
004
0032
0028
16 Science and Technology of Nuclear Installations
Acknowledgments
The authors acknowledge the support of Mr Jose GalindoRodrıguez affiliated to TECNATOM Inc Spain and MrChad Painter (Director of Nuclear Power Plant SimulatorDevelopment and Training Program at International AtomicEnergy Agency) for providing necessary tools and data toconduct this research
References
[1] N Mehranbod M Soroush M Piovoso and B A OgunnaikeldquoProbabilistic model for sensor fault detection and identifica-tionrdquo AIChE Journal vol 49 no 7 pp 1787ndash1802 2003
[2] JW Hines and E Davis ldquoLessons learned from the US nuclearpower plant on-line monitoring programsrdquo Progress in NuclearEnergy vol 46 no 3-4 pp 176ndash189 2005
[3] EPRI ldquoOn-line monitoring cost benefit guiderdquo Final Report1006777 EPRI Palo Alto Calif USA 2003
[4] S J Qin and W Li ldquoDetection identification and recon-struction of faulty sensors with maximized sensitivityrdquo AIChEJournal vol 45 no 9 pp 1963ndash1976 1999
[5] J Ma and J Jiang ldquoApplications of fault detection and diagnosismethods in nuclear power plants a reviewrdquo Progress in NuclearEnergy vol 53 no 3 pp 255ndash266 2011
[6] G YHeo ldquoConditionmonitoring using empiricalmodels tech-nical review and prospects for nuclear applicationsrdquo NuclearEngineering and Technology vol 40 no 1 pp 49ndash68 2008
[7] J B Coble R M Meyer P Ramuhalli et al ldquoA review ofsensor calibration monitoring for calibration interval extensionin nuclear power plantsrdquo Tech Rep PNNL-21687 PacificNorthwest National Laboratory Richland Wash USA 2012
[8] M A Kramer ldquoAutoassociative neural networksrdquo Computersand Chemical Engineering vol 16 no 4 pp 313ndash328 1992
[9] X Xu J W Hines and R E Uhrig ldquoOn-line sensor calibra-tion monitoring and fault detection for chemical processesrdquoin Proceedings of the Maintenance and Reliability Conference(MARCON rsquo98) pp 12ndash14 Knoxville Tenn USA May 1998
[10] M Hamidreza S Mehdi J-R Hooshang and N AliakbarldquoReconstruction based approach to sensor fault diagnosis usingauto-associative neural networksrdquo Journal of Central SouthUniversity vol 21 no 6 pp 2273ndash2281 2014
[11] UThissenW JMelssen andLMC Buydens ldquoNonlinear pro-cess monitoring using bottle-neck neural networksrdquo AnalyticaChimica Acta vol 446 no 1-2 pp 371ndash383 2001
[12] D J Wrest J W Hines and R E Uhrig ldquoInstrument surveil-lance and calibration verification through plant wide monitor-ing using autoassociative neural networksrdquo in Proceedings ofthe American Nuclear Society International Topical Meeting onNuclear Plant Instrumentation Control and Human MachineInterface Technologies University Park Pa USA May 1996
[13] J W Hines R E Uhrig and D J Wrest ldquoUse of autoassociativeneural networks for signal validationrdquo Journal of Intelligent andRobotic Systems Theory and Applications vol 21 no 2 pp 143ndash154 1998
[14] P F Fantoni M I Hoffmann R Shankar and E L Davis ldquoOn-line monitoring of instrument channel performance in nuclearpower plant using PEANOrdquo Progress in Nuclear Energy vol 43no 1ndash4 pp 83ndash89 2003
[15] MMarseguerra andA Zoia ldquoTheAutoAssociative Neural Net-work in signal analysis II Application to on-line monitoring of
a simulated BWR componentrdquo Annals of Nuclear Energy vol32 no 11 pp 1207ndash1223 2005
[16] M S Ikbal H Misra and B Yegnanarayana ldquoAnalysis ofautoassociative mapping neural networksrdquo in Proceedings of theInternational Joint Conference on Neural Networks (IJCNN rsquo99)vol 5 pp 3025ndash3029 IEEE Washington DC USA July 1999Proceedings (Cat no 99CH36339)
[17] Y Bengio ldquoLearning deep architectures for AIrdquo Foundationsand Trends in Machine Learning vol 2 no 1 pp 1ndash27 2009
[18] J W Hines A V Gribok I Attieh and R E Uhrig ldquoRegular-izationmethods for inferential sensing in nuclear power plantsrdquoin Fuzzy Systems and Soft Computing in Nuclear Engineering DRuan Ed vol 38 of Studies in Fuzziness and Soft Computingpp 285ndash314 Physica 2000
[19] A V Gribok J W Hines A Urmanov and R E UhrigldquoHeuristic systematic and informational regularization forprocessmonitoringrdquo International Journal of Intelligent Systemsvol 17 no 8 pp 723ndash749 2002
[20] X Glorot and Y Bengio ldquoUnderstanding the difficulty oftraining deep feedforward neural networksrdquo in Proceedings ofthe 13th International Conference on Artificial Intelligence andStatistics (AISTATS rsquo10) vol 9 pp 249ndash256 Sardinia Italy May2010
[21] S Tan andM L Mayrovouniotis ldquoReducing data dimensional-ity through optimizing neural network inputsrdquo AIChE Journalvol 41 no 6 pp 1471ndash1480 1995
[22] Y Bengio and P Lamblin ldquoGreedy layer-wise training of deepnetworksrdquo in Proceedings of the Advances in Neural InformationProcessing Systems 19 (NIPS rsquo07) pp 153ndash160 September 2007
[23] H Larochelle Y Bengio J Lourador and P Lamblin ldquoExplor-ing strategies for training deep neural networksrdquoThe Journal ofMachine Learning Research vol 10 pp 1ndash40 2009
[24] D Yu and L Deng ldquoDeep learning and its applications to signaland information processing [exploratory DSP]rdquo IEEE SignalProcessing Magazine vol 28 no 1 pp 145ndash154 2011
[25] D ErhanA Courville andPVincent ldquoWhydoes unsupervisedpre-training help deep learningrdquo Journal of Machine LearningResearch vol 11 pp 625ndash660 2010
[26] P Vincent H Larochelle Y Bengio and P-A ManzagolldquoExtracting and composing robust features with denoisingautoencodersrdquo in Proceedings of the 25th International Confer-ence on Machine Learning (ICML rsquo08) pp 1096ndash1103 July 2008
[27] P Vincent H Larochelle I Lajoie Y Bengio and P-AManzagol ldquoStacked denoising autoencoders learning usefulrepresentations in a deep network with a local denoisingcriterionrdquo Journal of Machine Learning Research vol 11 pp3371ndash3408 2010
[28] N Srivastava G Hinton A Krizhevsky I Sutskever and RSalakhutdinov ldquoDropout a simple way to prevent neural net-works from overfittingrdquo Journal of Machine Learning Researchvol 15 pp 1929ndash1958 2014
[29] M J Embrechts B J Hargis and J D Linton ldquoAn augmentedefficient backpropagation training strategy for deep autoasso-ciative neural networksrdquo in Proceedings of the 18th EuropeanSymposiumonArtificial NeuralNetworksmdashComputational Intel-ligence and Machine Learning (ESANN rsquo10) vol no pp 28ndash30April 2010
[30] Y Bengio ldquoPractical recommendations for gradient-basedtraining of deep architecturesrdquo inNeural Networks Tricks of theTrade vol 7700 of Lecture Notes in Computer Science pp 437ndash478 Springer Berlin Germany 2nd edition 2012
Science and Technology of Nuclear Installations 17
[31] J Bergstra and Y Bengio ldquoRandom search for hyper-parameteroptimizationrdquo Journal of Machine Learning Research vol 13 pp281ndash305 2012
[32] J W Hines and D R Garvey ldquoDevelopment and applicationof fault detectability performance metrics for instrument cali-bration verification and anomaly detectionrdquo Journal of PatternRecognition Research vol 1 no 1 pp 2ndash15 2006
[33] A Usynin and J W Hines ldquoOn-line monitoring robustnessmeasures and comparisonsrdquo in Proceedings of the Interna-tional Atomic Energy Agency Technical Meeting on ldquoIncreasingInstrument Calibration Interval through On-Line CalibrationTechnologyrdquo OECD Halden Reactor Project Halden NorwaySeptember 2004
[34] A Wald ldquoSequential tests of statistical hypothesesrdquo Annals ofMathematical Statistics vol 16 no 2 pp 117ndash186 1945
[35] F Di Maio P Baraldi E Zio and R Seraoui ldquoFault detection innuclear power plants components by a combination of statisticalmethodsrdquo IEEE Transactions on Reliability vol 62 no 4 pp833ndash845 2013
[36] S Cheng and M Pecht ldquoUsing cross-validation for modelparameter selection of sequential probability ratio testrdquo ExpertSystems with Applications vol 39 no 9 pp 8467ndash8473 2012
TribologyAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
FuelsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal ofPetroleum Engineering
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Power ElectronicsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
CombustionJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Renewable Energy
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
StructuresJournal of
International Journal of
RotatingMachinery
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporation httpwwwhindawicom
Journal ofEngineeringVolume 2014
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
International Journal ofPhotoenergy
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear InstallationsScience and Technology of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Solar EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Wind EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear EnergyInternational Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
High Energy PhysicsAdvances in
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
16 Science and Technology of Nuclear Installations
Acknowledgments
The authors acknowledge the support of Mr Jose GalindoRodrıguez affiliated to TECNATOM Inc Spain and MrChad Painter (Director of Nuclear Power Plant SimulatorDevelopment and Training Program at International AtomicEnergy Agency) for providing necessary tools and data toconduct this research
References
[1] N Mehranbod M Soroush M Piovoso and B A OgunnaikeldquoProbabilistic model for sensor fault detection and identifica-tionrdquo AIChE Journal vol 49 no 7 pp 1787ndash1802 2003
[2] JW Hines and E Davis ldquoLessons learned from the US nuclearpower plant on-line monitoring programsrdquo Progress in NuclearEnergy vol 46 no 3-4 pp 176ndash189 2005
[3] EPRI ldquoOn-line monitoring cost benefit guiderdquo Final Report1006777 EPRI Palo Alto Calif USA 2003
[4] S J Qin and W Li ldquoDetection identification and recon-struction of faulty sensors with maximized sensitivityrdquo AIChEJournal vol 45 no 9 pp 1963ndash1976 1999
[5] J Ma and J Jiang ldquoApplications of fault detection and diagnosismethods in nuclear power plants a reviewrdquo Progress in NuclearEnergy vol 53 no 3 pp 255ndash266 2011
[6] G YHeo ldquoConditionmonitoring using empiricalmodels tech-nical review and prospects for nuclear applicationsrdquo NuclearEngineering and Technology vol 40 no 1 pp 49ndash68 2008
[7] J B Coble R M Meyer P Ramuhalli et al ldquoA review ofsensor calibration monitoring for calibration interval extensionin nuclear power plantsrdquo Tech Rep PNNL-21687 PacificNorthwest National Laboratory Richland Wash USA 2012
[8] M A Kramer ldquoAutoassociative neural networksrdquo Computersand Chemical Engineering vol 16 no 4 pp 313ndash328 1992
[9] X Xu J W Hines and R E Uhrig ldquoOn-line sensor calibra-tion monitoring and fault detection for chemical processesrdquoin Proceedings of the Maintenance and Reliability Conference(MARCON rsquo98) pp 12ndash14 Knoxville Tenn USA May 1998
[10] M Hamidreza S Mehdi J-R Hooshang and N AliakbarldquoReconstruction based approach to sensor fault diagnosis usingauto-associative neural networksrdquo Journal of Central SouthUniversity vol 21 no 6 pp 2273ndash2281 2014
[11] UThissenW JMelssen andLMC Buydens ldquoNonlinear pro-cess monitoring using bottle-neck neural networksrdquo AnalyticaChimica Acta vol 446 no 1-2 pp 371ndash383 2001
[12] D J Wrest J W Hines and R E Uhrig ldquoInstrument surveil-lance and calibration verification through plant wide monitor-ing using autoassociative neural networksrdquo in Proceedings ofthe American Nuclear Society International Topical Meeting onNuclear Plant Instrumentation Control and Human MachineInterface Technologies University Park Pa USA May 1996
[13] J W Hines R E Uhrig and D J Wrest ldquoUse of autoassociativeneural networks for signal validationrdquo Journal of Intelligent andRobotic Systems Theory and Applications vol 21 no 2 pp 143ndash154 1998
[14] P F Fantoni M I Hoffmann R Shankar and E L Davis ldquoOn-line monitoring of instrument channel performance in nuclearpower plant using PEANOrdquo Progress in Nuclear Energy vol 43no 1ndash4 pp 83ndash89 2003
[15] MMarseguerra andA Zoia ldquoTheAutoAssociative Neural Net-work in signal analysis II Application to on-line monitoring of
a simulated BWR componentrdquo Annals of Nuclear Energy vol32 no 11 pp 1207ndash1223 2005
[16] M S Ikbal H Misra and B Yegnanarayana ldquoAnalysis ofautoassociative mapping neural networksrdquo in Proceedings of theInternational Joint Conference on Neural Networks (IJCNN rsquo99)vol 5 pp 3025ndash3029 IEEE Washington DC USA July 1999Proceedings (Cat no 99CH36339)
[17] Y Bengio ldquoLearning deep architectures for AIrdquo Foundationsand Trends in Machine Learning vol 2 no 1 pp 1ndash27 2009
[18] J W Hines A V Gribok I Attieh and R E Uhrig ldquoRegular-izationmethods for inferential sensing in nuclear power plantsrdquoin Fuzzy Systems and Soft Computing in Nuclear Engineering DRuan Ed vol 38 of Studies in Fuzziness and Soft Computingpp 285ndash314 Physica 2000
[19] A V Gribok J W Hines A Urmanov and R E UhrigldquoHeuristic systematic and informational regularization forprocessmonitoringrdquo International Journal of Intelligent Systemsvol 17 no 8 pp 723ndash749 2002
[20] X Glorot and Y Bengio ldquoUnderstanding the difficulty oftraining deep feedforward neural networksrdquo in Proceedings ofthe 13th International Conference on Artificial Intelligence andStatistics (AISTATS rsquo10) vol 9 pp 249ndash256 Sardinia Italy May2010
[21] S Tan andM L Mayrovouniotis ldquoReducing data dimensional-ity through optimizing neural network inputsrdquo AIChE Journalvol 41 no 6 pp 1471ndash1480 1995
[22] Y Bengio and P Lamblin ldquoGreedy layer-wise training of deepnetworksrdquo in Proceedings of the Advances in Neural InformationProcessing Systems 19 (NIPS rsquo07) pp 153ndash160 September 2007
[23] H Larochelle Y Bengio J Lourador and P Lamblin ldquoExplor-ing strategies for training deep neural networksrdquoThe Journal ofMachine Learning Research vol 10 pp 1ndash40 2009
[24] D Yu and L Deng ldquoDeep learning and its applications to signaland information processing [exploratory DSP]rdquo IEEE SignalProcessing Magazine vol 28 no 1 pp 145ndash154 2011
[25] D ErhanA Courville andPVincent ldquoWhydoes unsupervisedpre-training help deep learningrdquo Journal of Machine LearningResearch vol 11 pp 625ndash660 2010
[26] P Vincent H Larochelle Y Bengio and P-A ManzagolldquoExtracting and composing robust features with denoisingautoencodersrdquo in Proceedings of the 25th International Confer-ence on Machine Learning (ICML rsquo08) pp 1096ndash1103 July 2008
[27] P Vincent H Larochelle I Lajoie Y Bengio and P-AManzagol ldquoStacked denoising autoencoders learning usefulrepresentations in a deep network with a local denoisingcriterionrdquo Journal of Machine Learning Research vol 11 pp3371ndash3408 2010
[28] N Srivastava G Hinton A Krizhevsky I Sutskever and RSalakhutdinov ldquoDropout a simple way to prevent neural net-works from overfittingrdquo Journal of Machine Learning Researchvol 15 pp 1929ndash1958 2014
[29] M J Embrechts B J Hargis and J D Linton ldquoAn augmentedefficient backpropagation training strategy for deep autoasso-ciative neural networksrdquo in Proceedings of the 18th EuropeanSymposiumonArtificial NeuralNetworksmdashComputational Intel-ligence and Machine Learning (ESANN rsquo10) vol no pp 28ndash30April 2010
[30] Y Bengio ldquoPractical recommendations for gradient-basedtraining of deep architecturesrdquo inNeural Networks Tricks of theTrade vol 7700 of Lecture Notes in Computer Science pp 437ndash478 Springer Berlin Germany 2nd edition 2012
Science and Technology of Nuclear Installations 17
[31] J Bergstra and Y Bengio ldquoRandom search for hyper-parameteroptimizationrdquo Journal of Machine Learning Research vol 13 pp281ndash305 2012
[32] J W Hines and D R Garvey ldquoDevelopment and applicationof fault detectability performance metrics for instrument cali-bration verification and anomaly detectionrdquo Journal of PatternRecognition Research vol 1 no 1 pp 2ndash15 2006
[33] A Usynin and J W Hines ldquoOn-line monitoring robustnessmeasures and comparisonsrdquo in Proceedings of the Interna-tional Atomic Energy Agency Technical Meeting on ldquoIncreasingInstrument Calibration Interval through On-Line CalibrationTechnologyrdquo OECD Halden Reactor Project Halden NorwaySeptember 2004
[34] A Wald ldquoSequential tests of statistical hypothesesrdquo Annals ofMathematical Statistics vol 16 no 2 pp 117ndash186 1945
[35] F Di Maio P Baraldi E Zio and R Seraoui ldquoFault detection innuclear power plants components by a combination of statisticalmethodsrdquo IEEE Transactions on Reliability vol 62 no 4 pp833ndash845 2013
[36] S Cheng and M Pecht ldquoUsing cross-validation for modelparameter selection of sequential probability ratio testrdquo ExpertSystems with Applications vol 39 no 9 pp 8467ndash8473 2012
TribologyAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
FuelsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal ofPetroleum Engineering
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Power ElectronicsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
CombustionJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Renewable Energy
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
StructuresJournal of
International Journal of
RotatingMachinery
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporation httpwwwhindawicom
Journal ofEngineeringVolume 2014
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
International Journal ofPhotoenergy
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear InstallationsScience and Technology of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Solar EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Wind EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear EnergyInternational Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
High Energy PhysicsAdvances in
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Science and Technology of Nuclear Installations 17
[31] J Bergstra and Y Bengio ldquoRandom search for hyper-parameteroptimizationrdquo Journal of Machine Learning Research vol 13 pp281ndash305 2012
[32] J W Hines and D R Garvey ldquoDevelopment and applicationof fault detectability performance metrics for instrument cali-bration verification and anomaly detectionrdquo Journal of PatternRecognition Research vol 1 no 1 pp 2ndash15 2006
[33] A Usynin and J W Hines ldquoOn-line monitoring robustnessmeasures and comparisonsrdquo in Proceedings of the Interna-tional Atomic Energy Agency Technical Meeting on ldquoIncreasingInstrument Calibration Interval through On-Line CalibrationTechnologyrdquo OECD Halden Reactor Project Halden NorwaySeptember 2004
[34] A Wald ldquoSequential tests of statistical hypothesesrdquo Annals ofMathematical Statistics vol 16 no 2 pp 117ndash186 1945
[35] F Di Maio P Baraldi E Zio and R Seraoui ldquoFault detection innuclear power plants components by a combination of statisticalmethodsrdquo IEEE Transactions on Reliability vol 62 no 4 pp833ndash845 2013
[36] S Cheng and M Pecht ldquoUsing cross-validation for modelparameter selection of sequential probability ratio testrdquo ExpertSystems with Applications vol 39 no 9 pp 8467ndash8473 2012
TribologyAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
FuelsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal ofPetroleum Engineering
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Power ElectronicsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
CombustionJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Renewable Energy
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
StructuresJournal of
International Journal of
RotatingMachinery
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporation httpwwwhindawicom
Journal ofEngineeringVolume 2014
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
International Journal ofPhotoenergy
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear InstallationsScience and Technology of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Solar EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Wind EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear EnergyInternational Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
High Energy PhysicsAdvances in
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
TribologyAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
FuelsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal ofPetroleum Engineering
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Power ElectronicsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
CombustionJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Renewable Energy
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
StructuresJournal of
International Journal of
RotatingMachinery
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporation httpwwwhindawicom
Journal ofEngineeringVolume 2014
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
International Journal ofPhotoenergy
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear InstallationsScience and Technology of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Solar EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Wind EnergyJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Nuclear EnergyInternational Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
High Energy PhysicsAdvances in
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014