Bayesian Emulator Approach forComplex Dynamical Systems
F.A. Dı́az De la O and S. Adhikari
School of Engineering, Swansea University, Swansea, UK
Email: [email protected]
URL: http://engweb.swan.ac.uk/∼adhikaris
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.1/44
Outline
Introduction
Emulators
Linear Structural Dynamics
Applications3DOF exampleLarge (1200 DOF) simulation exampleExperimental example
Conclusions
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.2/44
Introduction
Complex engineering dynamical systems areoften investigated running computer codes, alsoknown as simulators (O’Hagan, 2006).
A simulator is a function η(·) that, given an inputx, it produces an output y.
Sophisticated simulators can have a high costof execution, measured in terms of:
CPU time employed
Floating point operations performed
Computer capability required
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.3/44
Introduction
A possible solution is to build an emulator of theexpensive simulator.
An emulator is a statistical approximation to thesimulator, i.e., it provides a probabilitydistribution for η(·).
Emulators have already been implemented in anumber of fields, which include:
Environmental science (Challenor et al., 2006)
Climate modeling (Rougier, 2007)
Medical science (Haylock and O’Hagan, 1996)
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.4/44
Introduction
In Structural Dynamics, an example of anexpensive simulator is a high-resolution finiteelement model, which can be difficult to runeven for obtaining a dynamic response at fewfrequency points.
Emulation can thus be a useful computationaltool to be implemented in a Structural Dynamicscontext.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.5/44
Introduction
To test the convenience of the use of emulatorsfor studying engineering dynamical systems,the following problems will be addressed:
1. Computational cost. Can the output of a computercode be approximated using only a few trial runs?
2. Efficiency. Can the number of floating pointoperations in an expensive code be reduced but stillproduce a satisfactory output?
3. Interpolation of experimental data. Can experimentaldata be confidently interpolated to cope with the lackof a mathematical/computer model?
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.6/44
Emulators
An emulator is built by first choosing n designpoints in the input domain of the simulator andobtaining the training set {η(x1), . . . , η(xn)}.
After that initial choice is made, an emulatorshould:
Reproduce the known output at any design point.
At any untried input, provide a distribution whosemean value constitutes a plausible interpolation of thetraining data. The probability distribution around thismean value should also express the uncertaintyabout how the emulator might interpolate.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.7/44
Emulators
To illustrate what do the above criteria mean, anemulator was constructed to approximate thesimple simulator y = cos(x).
In the following figures, the solid line is the trueoutput of the simulator. The circles representthe training runs, and the dots are the mean ofthe emulator, which provides the approximation.
Note how the approximation improves whenmore design points are chosen.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.8/44
Emulators
−5 −4 −3 −2 −1 0 1 2 3 4 5−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
Simulator Input, x
Sim
ulat
or O
utpu
t, y
Figure 1: Approximation using 5 design points.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.9/44
Emulators
−5 −4 −3 −2 −1 0 1 2 3 4 5−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
Simulator Input, x
Sim
ulat
or O
utpu
t, y
Figure 2: Approximation using 7 design points.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.10/44
Emulators
In the same way, the following figures showupper and lower probability bounds of twostandard deviations for the mean of theemulator. The solid line is the true output of thesimulator. The circles represent the trainingruns, and the dots are the bounds.
Note how the uncertainty about theapproximation is reduces as more design pointsare chosen.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.11/44
Emulators
−5 −4 −3 −2 −1 0 1 2 3 4 5−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
Simulator Input, x
Sim
ulat
or O
utpu
t, y
Figure 3: Uncertainty using 5 design points.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.12/44
Emulators
−5 −4 −3 −2 −1 0 1 2 3 4 5−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
Simulator Input, x
Sim
ulat
or O
utpu
t, y
Figure 4: Uncertainty using 7 design points.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.13/44
Emulators
From the perspective of Bayesian Statistics, η(·)is a random variable in the sense that it isunknown until the simulator is run.
Assume that η(·) deviates from the mean of itsdistribution in the following way
η(x) =n∑
j=1
βjhj(x) + Z(x) (1)
where for all j, hj(x) is a known function and βj
is an unknown coefficient.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.14/44
Emulators
The function Z(·) in Eq.(1) is assumed to be aGaussian stochastic process (GP) with meanzero and covariance given by
Cov(η(x), η(x′
)) = σ2e−(x−x′
)T B(x−x′
) (2)
where B is a positive definite diagonal matrixthat contains smoothness parameters.
If the mean of η(·) is of the form m(·) = h(·)Tβ
then η(·) has a GP distribution with mean m(·)and covariance given by Eq.(2).
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.15/44
Emulators
The latter is symbolized as
η(·)|β, σ2 ∼ N(h(·)Tβ, σ2C(·, ·)) (3)
This prior distribution contains subjectiveinformation about the relation of the input andthe unknown output . The next step is to updatethis belief by adding objective information,represented by the vector of observationsy = [y1 = η(x1), . . . , yn = η(xn)]
T .
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.16/44
Emulators
Using standard integration techniques, it can beshown (Haylock and O’Hagan, 1996) that suchupdate is
η(·)|y, σ2 ∼ N(m∗∗(·), σ2C∗∗(·, ·)) (4)
where m∗∗(·) constitutes the fast approximationof η(x) for any x in the input domain.
Moreover, it can be shown that
η(x) − m∗∗(x)
σ̂
√(n−q−2)C∗∗(x)
n−q
∼ tn−q (5)
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.17/44
Structural Dynamics
Consider the problem of modeling the responseof a structural system to different frequencyranges of vibration.
To obtain the corresponding FrequencyResponse Function (FRF), the followingequation of motion must be solved:
Mq̈(t) + Cq̇(t) + Kq(t) = f(t) (6)
where K, C and M ∈ RN×N are respectively the
stiffness, damping and mass matrices, f(t) isthe forcing vector and q(t) the response vector.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.18/44
Structural Dynamics
Equation(6) can be solved in terms of theexcitation frequency level, ω ∈ [0, ...,∞), as
q(ω) = [−ω2M + iωC + K]−1f(ω) (7)
where q(ω) and f(ω) are the Fourier transformsof q(t) and f(t). Since it is a complex-valuedfunction, the relevant simulator becomes:
η(ω) =∣∣∣[−ω2M + iωC + K]−1f(ω)
∣∣∣ (8)
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.19/44
Structural Dynamics
To emulate the FRF expressed in Eq.(8), thefollowing algorithm is proposed.1. Select n initial frequency values ω1, . . . , ωn.
2. Obtain the vector of observationsy = [y1 = η(ω1), . . . , y
n= η(ωn)]T .
3. Update the prior distribution (3), which containssubjective information, by adding the objectiveinformation y. This will enable the calculation ofm∗∗(·), the mean of the updated posterior distribution(4) given the data y. As already mentioned, suchmean constitutes an approximation of η(ω) for any ω.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.20/44
Structural Dynamics
For a damped three-degree-of-freedomspring-mass system, it can be shown that theFRF for k fixed is
η(ω) =3∑
j=1
x̃T
j f̄(iω)√
(ω2j − ωj)2 + (2ωωjζj)2
x̃kj (9)
where k = 1, . . . , 3 and for all j, ωj are thenatural frequencies, x̃j are the normal modesand ζj are the damping ratios.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.21/44
Applications
In the figure below, the mass of each block is 1kg and the stiffness of each spring is 1 N/m.The viscous damping constant of the damperassociated with each block is 0.2 Ns/m.Suppose he dynamic response when only thefirst mass is subjected to unit initialdisplacement is to be obtained.
m m m
k k
u 1 u 2 u 3
k k
c c c Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.22/44
Applications
Twenty-one equally-spaced design points wereselected and the smoothness parameterscalculated. The above algorithm was thenapplied to construct the corresponding emulatorfor the FRF.
The results for k = 1 are shown below. Asbefore, the circles represent the training runs,the dots represent the mean emulator and theuncertainty bounds respectively, and the solidline is the true value of the simulator.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.23/44
Applications
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
Frequency,Hz
Freq
uenc
y R
espo
nse
Figure 5: Approximation using 21 design points.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.24/44
Applications
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
Frequency,Hz
Freq
uenc
y R
espo
nse
Figure 6: Uncertainty using 21 design points.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.25/44
Applications
Unfortunately, Eq.(8) cannot always be solvedanalytically. Furthermore, a simulator can bevery expensive to run for real-life engineeringapplications, v.g., modeling the response of anaircraft, or parts of it, to vibration.
The potential use of intensive computation is asuitable scenario to apply emulators.
The following figures refer to a realization of asimulator (Adhikari et al., 2007) of the frequencyresponse of a steel plate with a fixed edge.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.26/44
Applications
0 500 1000 1500 2000 2500 3000 3500 4000−40
−30
−20
−10
0
10
20
30
40
50
Frequency,Hz
Log
ampl
itude
, dB
Figure 7: Simulator of the FRF of a steel plate subject to vibration.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.27/44
Applications
00.2
0.40.6
0.81
0
0.2
0.4
0.6
0.8−0.5
0
0.5
1
6
4
X direction (length)
5
Outputs
2
3
Input
1
Y direction (width)
Fixed edge
Figure 8: The plate is divided into 25 × 15 elements, resulting in 1200 DOF.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.28/44
Applications
To test the potential application of emulation toengineering dynamical systems, the inputdomain of the previously shown simulator wasdivided in low (0-1 kHz), medium (1-2.5 kHz)and high (2.5-4 kHz) frequency ranges.
The following figure shows a comparisonbetween selecting a different number of designpoints to approximate the response of node 1 tovibration in the medium-frequency range. Notehow the approximation is improved, the moredesign points are used.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.29/44
Applications
1000 1500 2000 2500−20
−10
0
10
20
30
40
50
Frequency,Hz
Log
ampl
itude
, dB
1000 1500 2000 2500−20
−10
0
10
20
30
40
50
Frequency,Hz
Log
ampl
itude
, dB
1000 1500 2000 2500−20
−10
0
10
20
30
40
50
Frequency,Hz
Log
ampl
itude
, dB
1000 1500 2000 2500−20
−10
0
10
20
30
40
50
Frequency,Hz
Log
ampl
itude
, dB
Figure 9: Emulation with 25, 50, 75 and 100 design points, mid-freq range.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.30/44
Applications
One hundred equally-spaced design pointswere selected in each range, upon which anemulator was constructed to infer the value ofthe output at untried inputs. The results for themean of the emulator in the different frequencyranges and the corresponding probabilitybounds are shown in the following figures.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.31/44
Applications
0 100 200 300 400 500 600 700 800 900 1000−20
−10
0
10
20
30
40
50
Frequency,Hz
Log
ampl
itude
, dB
0 100 200 300 400 500 600 700 800 900 1000−20
−10
0
10
20
30
40
50
Frequency,HzLo
g am
plitu
de, d
B
Figure 10: Emulation of the response of node 1, low-frequency range.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.32/44
Applications
1000 1500 2000 2500−20
−10
0
10
20
30
40
50
Frequency,Hz
Log
ampl
itude
, dB
1000 1500 2000 2500−20
−10
0
10
20
30
40
50
Frequency,HzLo
g am
plitu
de, d
B
Figure 11: Emulation of the response of node 1, medium-frequency range.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.33/44
Applications
2500 3000 3500 4000−20
−10
0
10
20
30
40
50
Frequency,Hz
Log
ampl
itude
, dB
2500 3000 3500 4000−20
−10
0
10
20
30
40
50
Frequency,HzLo
g am
plitu
de, d
B
Figure 12: Emulation of the response of node 1, high-frequency range.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.34/44
Applications
A setup in which there is no simulator available,due perhaps to the lack of knowledge of thephysics of the system, was also considered.
To approximate the response function of a platesubject to vibration, an experiment (Adhikari etal., 2007) was performed.
The physical and geometrical properties of theplate are shown in the following table.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.35/44
Applications
Figure 13: Experimental setup
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.36/44
Applications
Plate Properties Numerical valuesLength 998 mmWidth 530 mmThickness 3.0 mmMass density 7860 kg/m3
Young’s modulus 2.0 × 105 MPaPoisson’s ratio 0.3Total weight 12.47 kg
Table 1: Material and geometric properties of the plate considered for the experiment.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.37/44
Applications
In an experimental context like the onedescribed, there would only be realmeasurements available.
These measurements would act as the trainingruns and emulation should be applied the sameway as in the previous cases. This wouldreduce the number of experimental runsnecessary to obtain an empirical FRF.
The results of doing so for the referredexperiment are shown below, for everyfrequency range.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.38/44
Applications
0 100 200 300 400 500 600 700 800 900 1000−20
−10
0
10
20
30
40
50
Frequency,Hz
Log
ampl
itude
, dB
0 100 200 300 400 500 600 700 800 900 1000−20
−10
0
10
20
30
40
50
Frequency,HzLo
g am
plitu
de, d
B
Figure 14: Emulation of the response of node 1, low-frequency range. The initial design
is based on experimental measurements.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.39/44
Applications
1000 1500 2000 2500−20
−10
0
10
20
30
40
50
Frequency,Hz
Log
ampl
itude
, dB
1000 1500 2000 2500−20
−10
0
10
20
30
40
50
Frequency,HzLo
g am
plitu
de, d
B
Figure 15: Emulation of the response of node 1, medium-frequency range. The initial
design is based on experimental measurements.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.40/44
Applications
2500 3000 3500 4000−20
−10
0
10
20
30
40
50
Frequency,Hz
Log
ampl
itude
, dB
2500 3000 3500 4000−20
−10
0
10
20
30
40
50
Frequency,HzLo
g am
plitu
de, d
B
Figure 16: Emulation of the response of node 1, high-frequency range. The initial design
is based on experimental measurements.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.41/44
Conclusions
Computer codes for structural dynamics canbecome very expensive to run, as the numberof degrees of freedom in the system increases.
The use of emulators has been proposed, sincethey provide a fast approximation to the outputof the original code using in only a few trainingruns.
With regards to computational cost, an emulatorfor a simple spring-mass system with threedegrees of freedom was constructed toillustrate how the mean of the emulatorapproximates the FRF.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.42/44
Conclusions
Regarding efficiency, the dynamic response ofa system with 1200 degrees of freedom wasemulated using only 300 training runs, thusreducing the number of necessary floating pointoperations. The results were particularlyappealing for the medium and high-frequencyranges.
To interpolate experimental data, realmeasurements were used as the set of trainingruns necessary to construct an emulator andthe real experimental output was compared withthe corresponding approximation.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.43/44
References
O’Hagan, A. (2006) Bayesian Analysis of Computer Code Outputs: A tutorial,Reliability Engineering & System Safety, 91, 1290-1300.
Challenor, P. G., Hankin, R. K. S., Marsh, R. (2006) Towards the Probability of RapidClimate Change, in Scellnhuber, H. J., Cramer, W., Nakicenovic, N., Wigley, T., Yohe,G. (eds), Avoiding Dangerous Climate Change, Cambridge University Press,Cambridge.
Rougier, J. (1996) Probabilistic Inference for Future Climate Using an Ensemble ofClimate Model Evaluations, Climatic Change, forthcoming.
Haylock, R. G. and O’Hagan, A. (1996) On Inference for Outputs of ComputationallyExpensive Algorithms with Uncertainty on the Inputs, Bayesian Statistics 5, OxfordUniversity Press, Oxford.
Adhikari, S., Friswell, M. I., and Lonkar, K. (2007) Uncertainty in structural dynamics:Experimental case studies on beams and plates, Proceedings of the ComputationalMethods in Structural Dynamics and Earthquake Engineering, Crete, Greece.
Schaumburg, Illinois, 10 April 2008 Bayesian Emulators for Dynamical Systems – p.44/44