estimation of precursors for extreme events using the
TRANSCRIPT
Estimation of precursors for extreme events using
the adjoint based optimization approach
by
Rishabh Ishar
B.E., Punjab Engineering College (2018)
Submitted to the Department of Mechanical Engineeringin partial fulfillment of the requirements for the degree of
Master of Science in Mechanical Engineering
at the
MASSACHUSETTS INSTITUTE OF TECHNOLOGY
February 2020
c Massachusetts Institute of Technology 2020. All rights reserved.
Author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Department of Mechanical Engineering
January 14, 2020
Certified by. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Themistoklis P. Sapsis
Doherty Associate Professor in Ocean UtilizationThesis Supervisor
Accepted by . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Nicolas Hadjiconstantinou
Chairman, Committee on Graduate Students
2
Estimation of precursors for extreme events using the adjoint
based optimization approach
by
Rishabh Ishar
Submitted to the Department of Mechanical Engineeringon January 14, 2020, in partial fulfillment of the
requirements for the degree ofMaster of Science in Mechanical Engineering
Abstract
We formulate a generalized optimization problem for a non-linear dynamical systemgoverned by a set of differential equations. The plant under focus is the 2-D Kol-mogorov flow, as this flow has inherent turbulence which would give rise to chaosand intermittent bursts in a selected observable. As a first step, an observable withpotential extreme events in its time series is selected. In our case, we choose thekinetic energy of the flow field as the observable under study. The next step is toderive the adjoint equations for the kinetic energy that is the quantity of interestwith the velocity field as the optimizing variable. This obtained velocity field formsthe precursor for extreme events in the kinetic energy. The prediction capabilities forthis precursor are then explored in more detail. The goal is to select the precursorsuch that it predicts the extreme events in a given time horizon which can generatewarning signals effectively. We also present a coupled flow solver in Nek5000 andadjoint solver in MATLAB, the latter can be applied to any dynamical system tostudy the extreme events and obtain the relevant precursor. In a consecutive section,the results for extreme events in the kinetic energy and the lift coefficient for theflow over a 2-D airfoil are presented. As part of future work, the implementation andapplication of the solver for the flow past the airfoil and over a 3-D Ahmed body areproposed.
Thesis Supervisor: Themistoklis P. SapsisTitle: Doherty Associate Professor in Ocean Utilization
3
4
Acknowledgments
First and foremost, I would like to thank my supervisor, Professor Themis Sapsis,
for his constant guidance during the course of my research. This work wouldnβt have
been possible without his support, that he made sure I had enough, even during his
sabbatical.
I would like to express my gratitude to Dr. Antoine Blanchard, for his simplifying
explanations and great help during my research. I thank him for all his time. His
expertise in fluid mechanics and dynamical system analysis gave me a lot of insight
into these topics.
I want to thank Prof. Bernd R. Noack, who has been a great collaborator since
my third year of undergrad. He introduced me to topics like model reduction and
machine learning that were really important during my studies at MIT. I also want
to thank all my collaborators from USA, Germany, France, and Poland, including
Dr. Eurika Kaiser, Prof. Marek Morzynski, Prof. Steve Brunton, Prof. Wolfgang
Schroder, Prof. Richard Semaan, Mr. Daniel Fernex, Mr. Marian Albers, Dr. Pascal
Meysonnat, Ms. Camila Chovet, Mr. Guy Maceda, and Prof. Nathan Kutz.
I would like to thank my labmates Dr. Mohammad Farazmand, Mr. Alexis-
Tzianni Charalampopoulos, Mr. Zhong Yi Wan, Mr. Stephen Guth, and Dr. Hassan
Arbabi, for the stimulating discussions regarding coursework and research. I also
want to thank the Army Research Office for supporting me.
Finally, I want to thank my family and friends for their support during the course
of my degree.
5
6
Contents
1 Introduction 13
1.1 Prediction and control of extreme events . . . . . . . . . . . . . . . . 14
1.2 Motivation and contributions . . . . . . . . . . . . . . . . . . . . . . 15
2 Adjoint based optimization approach 17
2.1 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.1.1 The generalized optimization problem . . . . . . . . . . . . . . 18
2.1.2 Derivation of the adjoint equations . . . . . . . . . . . . . . . 18
2.1.3 Gradient descent . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.1.4 Layout for one complete loop/iteration of the adjoint solver . 25
2.2 Statistical analysis and prediction of extreme events . . . . . . . . . . 25
2.2.1 Joint and conditional statistics of the extreme events . . . . . 26
2.3 An overview of Nek5000 (flow solver) . . . . . . . . . . . . . . . . . . 28
3 Applications and results 29
3.1 Kolmogorov flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.1.1 Flow solver, setup, and simulation . . . . . . . . . . . . . . . . 30
3.1.2 POD and extreme event analysis results . . . . . . . . . . . . 31
3.2 2-D Airfoil flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.2.1 Flow solver, setup, and simulation . . . . . . . . . . . . . . . . 43
3.2.2 Lift coefficient over time analysis to determine the critical Reynolds
number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
7
3.2.3 Generating the stochastic inlet velocity and mode reduction
using Karhunen-Loeve expansion . . . . . . . . . . . . . . . . 45
3.2.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
4 Conclusions and future work 57
4.1 Summary of the results and discussion . . . . . . . . . . . . . . . . . 57
4.2 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
8
List of Figures
2-1 Layout of the loop to be completed in each iteration of the adjoint
problem till the solution converges . . . . . . . . . . . . . . . . . . . . 25
3-1 2-D mesh for the Kolmogorov flow. . . . . . . . . . . . . . . . . . . . 31
3-2 Time series plot of the kinetic energy. . . . . . . . . . . . . . . . . . . 32
3-3 Vorticity (the scale is -2 to 2) of the POD modes π = β Γ π(),
π = 1, . . . , 10 for the Kolmogorov flow. . . . . . . . . . . . . . . . . . . 33
3-4 The first 10 POD coefficients for the Kolmogorov flow. . . . . . . . . 34
3-5 Initial guess of the velocity field. . . . . . . . . . . . . . . . . . . . . . 35
3-6 Precursor obtained after 1500 iterations of the adjoint problem. . . . 35
3-7 π and Kinetic energy (KE) vs. snapshot number (the time lag between
π and KE is 1 time unit). . . . . . . . . . . . . . . . . . . . . . . . . 36
3-8 Zoomed peak near t = 2040 for the π and Kinetic energy vs. snapshot
number plot. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
3-9 Plot showing correlation between π and π·π (the time lag between π
and KE is 1 time unit). . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3-10 Joint pdf of π and π·π (the time lag between π and KE is 1 time unit). 37
3-11 Conditional pdf of π and π·π (the time lag between π and KE is 1 time
unit). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3-12 π and Kinetic energy vs. snapshot number (10000 time units simula-
tion, the time lag between π and KE is 1 time unit). . . . . . . . . . 39
3-13 Plot showing correlation between π and π·π (10000 time units simula-
tion, the time lag between π and KE is 1 time unit). . . . . . . . . . 39
9
3-14 Joint pdf of π and π·π (10000 time units simulation, the time lag
between π and KE is 1 time unit). . . . . . . . . . . . . . . . . . . . . 40
3-15 Conditional pdf of π and π·π (10000 time units simulation, the time
lag between π and KE is 1 time unit). . . . . . . . . . . . . . . . . . . 40
3-16 π and Dissipative energy (DE) vs. snapshot number (10000 time units
simulation, the time lag between π and DE is 1 time unit). . . . . . . 41
3-17 Plot showing correlation between π and π·π (DE) (10000 time units
simulation, the time lag between π and DE is 1 time unit). . . . . . . 41
3-18 Joint pdf of π and π·π (DE) (10000 time units simulation, the time lag
between π and DE is 1 time unit). . . . . . . . . . . . . . . . . . . . . 42
3-19 Conditional pdf of π and π·π (DE) (10000 time units simulation, the
time lag between π and DE is 1 time unit). . . . . . . . . . . . . . . . 42
3-20 2-D mesh of the flow around the airfoil. . . . . . . . . . . . . . . . . . 44
3-21 Vorticity snapshots at T=100 for Reynolds numbers near π ππ: (a) π π
= 90 (b) π π = 105 (c) π π = 110. . . . . . . . . . . . . . . . . . . . . 48
3-22 The kinetic energy plot for 3000 time units. . . . . . . . . . . . . . . 49
3-23 The inlet velocity input signal. . . . . . . . . . . . . . . . . . . . . . . 49
3-24 (a) Re = 90 for t = 15 to 200 (b) Re = 90, zoomed peaks (c) Re =
105, for t = 15 to 300 (d) Re = 105, zoomed peaks (e) Re = 110, for t
= 15 to 300 (f) Re = 110, zoomed peaks. . . . . . . . . . . . . . . . 50
3-25 Vorticity (the scale is -2 to 2) of the POD modes π = β Γ π(),
π = 1, . . . , 10 for the airfoil flow. . . . . . . . . . . . . . . . . . . . . . 51
3-26 The first 10 POD coefficients for the airfoil flow. . . . . . . . . . . . 52
3-27 (a) πΆπΏ from t = 20 to 3000 (b) The averaged curve (c) fluctuations in
πΆπΏ calculated by removing the moving average (5 time units moving
window). π is taken as the inlet KLE velocity with zero mean. From
t = 20 to 3000. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
3-28 π₯ component of the velocities at the three probes (a) ππ₯1 (b) ππ₯2 (c)
ππ₯3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3-29 π¦ component of the velocities at the three probes (a) ππ¦1 (b) ππ¦2 (c) ππ¦3. 55
10
List of Tables
3.1 Location of the three probes. . . . . . . . . . . . . . . . . . . . . . . . 47
4.1 Summary of the extreme events prediction results for the Kolmogorov
flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
11
12
Chapter 1
Introduction
Extreme events [1] form an integral part of complex dynamical systems. Also called
rare events, they have a really low probability of occurrence but can have significant
outcomes. Their relevance stems from the fact that they occur in systems like the
ocean (rogue waves ([26]), freak waves ([63], [40], [23], [58], [84])), climate (hurricanes
([56]), stock market ([31], [30], [59]), etc. which can be highly influential. Though
rare, they can have catastrophic economic and social consequences which makes their
study even more important. A prominent feature of extreme events is that they
do not follow Gaussian statistics, and the number of their occurrences cannot be
accurately predicted by the Gaussian statistics. The variables involved follow non-
Gaussian statistics in the probability density function (pdf), an example can be a
heavy-tailed pdf where the values of the observed variables can be many standard
deviations away from the mean. Another example can be of a Poisson [13] or weibull
distribution [32]. Some examples of non-Gaussian statistics can be seen in [83], [48],
[57], [20], [47], [45]. This poses a key challenge for efficiently describing the behavior
in the pdf of a dynamical system, which could be non-linear, highly dimensional, or
have intermittent events in the response.
13
1.1 Prediction and control of extreme events
An informal definition of an extreme event is that they - (a) are rare (b) occur in ran-
dom intervals (c) there is a system variable that has intermittent bursts in its observed
values and takes on extreme values (d) they are the characteristic of a system, not
necessarily forced. In a mathematical perspective, an extreme event is said to occur
when the value of a variable crosses a threshold, within the upper and lower bounds
of the range of the data set. The properties of dynamical systems that cause extreme
events are stochasticity and non-linearities inhibited within the system. Undesirable
random processes that are inherently part of the system contribute to the stochastic
behavior. Hence, due to these random processes, extreme events occur spontaneously
with few warning signals which makes their quantification and prediction a difficult
task.
A lot of work being done on the prediction of extreme events is based on the
extreme value theory (EVT) and can be found in [41], [16],[19], [2], and more enclosed
references. The first EVT states that the maxima of the sample asymptotically follow
one of the three distributions, namely Gumbel (Type I), Frechet (Type II) or Weibull
(Type III). This is for the univariate case. For the multivariate case, [79], [21],
[22], [12], give a solid foundation on analyzing the extreme event statistics using
prescribed-size blocks or the observed values of the variable that exceed a certain
level of magnitude. J. H. J. Einmahl and others ([27]) proposed a nonparametric
model that generates estimators while satisfying the moment constraints and applied
it to fit an angular density to a bivariate distribution.
Time series analysis [35] is another approach towards the quantification of extreme
events. It has a strong connection with chaos theory ([33], [42], [54], [24], [25], [86]).
Degrees of freedom (DOF) of the system played an important role as it was discovered
that an infinite number of DOF (infinitely many dimensions) can be determined from
a finite number of DOF (low dimensional) in a non-linear system like the Lorenz
attractor [49]. Stochasticity in such systems was established using DOF. If the DOF
were finite then the system would be deterministic otherwise it would be stochastic
14
(infinitely many DOF). The underlying dynamics of the attractor were evaluated and
reconstructed by a βmethod of delaysβ which was further developed ([71] and [78]).
Due to noise in the practical/experimental data, only a base structure of the attractor
can be described, without all the intricacies.
A better physical understanding is provided by dynamical and stochastic modeling
of the system.
1.2 Motivation and contributions
The study of turbulent flows using invariant solutions of the Navier-Stokes equa-
tions has been an active area of research ([38]). There isnβt much information about
the analytical behavior of these invariant solutions due to their complex properties
in space and time. M. Farazmand [28] derived the adjoint equations to compute
the steady-state and traveling-wave solutions for a flow with incompressible Navier-
Stokes equations with periodic boundary conditions and a space-dependent forcing.
A steady-state solution is established as a precursor that gives rise to extreme events
in the dissipative energy of the flow.
O. Soto and others ([76]) used a scheme formulated for the continuous adjoint
problem and applied it to optimize the shape in fluid flows, examples include lift
control over a cylinder and the flow over a 3-D hypersonic wing (angle of attack =
5 degrees). The approach included obtaining the gradient of the target objective
function with respect to a selected number of design parameters, this was further im-
proved by a pseudo-shell approach. The purpose of shape optimization in an airfoil
is to improve its performance. M. Schramm and others ([73]) used the adjoint ap-
proach to maximize the lift to drag ratio in a flow over a 2-D airfoil with and without
constraints.
In this work, we develop a finite-time horizon approach to the adjoint problem
and build on the instantaneous problem as provided in [29] with a few modifications.
Also, P. J. Blonigan and others ([9]) studied the turbulent channel flow and identified
the precursors for extreme events from a high probability manifold of the state. A
15
very small time-horizon is used so that the results can be compared with the instan-
taneous problem. We use the kinetic energy of the flow as our observable for extreme
events, derive the associated adjoint equations for an arbitrary fluid flow with given
constraints, and solve the equations to obtain the precursor. This obtained precursor
is used with a time lag to estimate the prediction capabilities before the occurrence of
the extreme events. We also present a unified framework to solve the coupled adjoint
problem using Nek5000 and MATLAB. Hence, this can be applied to any fluid flow
system which has inherent extreme events and establish the precursors that lead to
intermittent bursts in the space of the targeted variable. Here, we present the results
as obtained for the Kolmogorov flow and the flow over a 2-D airfoil at an angle of
attack of 30 degrees.
16
Chapter 2
Adjoint based optimization approach
The goal of this chapter is to describe the generalized optimization problem for a
dynamical system that is governed by a set of differential equations which can be
potentially non-linear. Some examples can be seen in [53], [17], [68], [43], [36], [44],
[52], [65], [75], [15]. If there are constraints associated with the governing equations,
then it becomes a bounded and constrained optimization problem. An observable is
then chosen for which we optimize the adjoint equations. After this, we formulate the
Lagrangian (β). The gradient of this Lagrangian is set to zero for all the variables
involved in it, which is done to converge to the minima of the observable in the
problem. Standard gradient descent is then used to iterate and reach the updated
initial condition, that would be used as the new initial condition for the next iteration.
This process is continued until the norm difference between the new and old initial
condition falls below a certain threshold.
17
2.1 Methodology
2.1.1 The generalized optimization problem
In this section, we define the governing equations of a general dynamical system as
ππ‘π’ = π© (π’) (2.1a)
π¦(π’) = 0 (2.1b)
π’(π₯, π‘0) = π’0(π₯), (2.1c)
where π’ determines the complete state of the system, π© and π¦ are differential oper-
ators that are based on the physical model of the system. They govern the evolution
of the system in space and time, and can have non-linear terms associated with them.
The focus of this section is to predict the occurrence of extreme events, via the inter-
mittent bursts in an observable. This observable is denoted by πΌ and it would have a
time series with rare extreme events after certain intervals. π’(π‘) is the driver for these
extreme occurrences and there are certain instances of π’(π‘) that would cause sudden
bursts in the observable πΌ. We can imagine the attractor with regions of instability,
and as soon as the state π’(π‘) reaches this instability region, there would be a large
deviation from the general trajectory, which would cause an extreme event in the
time-series of the observable. We examine these instability regions and the associated
states. Also, these states should have a non-zero probability of occurrence, which can
be enforced by using a constraint. This prevents the adjoint solver from converging to
states that have negligible probability of occurrence and would not occur in practical
simulations/experiments.
2.1.2 Derivation of the adjoint equations
In this section, we derive the adjoint equations for a dynamical system that is gov-
erned by the Navier-Stokes equations. The flows considered here are assumed to be
18
incompressible. These set of equations can be applied to both 2-D and 3-D fluid flow
systems. The system can also be subjected to a number of constraints which enforce
the state of the system to stay close to the attractor. This means that during any
iteration of the solver, the state does not assume impractical states and the ones
which have really low probabilities of occurrence. The observable in which we are
interested in to study the extreme event statistics is the kinetic energy of the system.
This is given by
πΌ() =1
2
β«Ξ©
. πΞ©
where is the velocity field and Ξ© is the flow domain. The velocity field is integrated
over each element in the whole domain, and then summed up to get the total kinetic
energy, πΌ().
For a fluid flow, the constrained optimization problem becomes
max (πΌ((π )) β πΌ((0))) (2.2)
Through this, the deviation in the kinetic energy is being maximized, that is we aim
to solve for the initial condition (0) that gives rise to the state (π ) having the
maximum growth in the kinetic energy in the time horizon π .
Here is subject to the following two governing equations and a feasibility constraint
given by
π© () = ππ‘ = β.βββπ + πβ + π (2.3)
where π© () is the Navier-stokes incompressible momentum equation and contains
non-linear terms and
π¦() = β. = 0 (2.4)
where π¦() is the Navier-stokes incompressibility condition.
Before we get to the feasibility constraint, we use proper orthogonal decomposition
19
(POD) to get the POD coefficients. Some studies on POD can be seen in [5], [18], [60],
[69], [85], [46]. Dynamic mode decomposition is another method that can be used
to extract coherent structures and dynamic information from flow fields as global
stability modes ([72], [61], [81]). Other areas of reduced order modeling have been
explored in [3], [7], [8], [6], [77]. When POD is used as an approximation method,
then any state on the attractor can be written as
(π₯, π‘) = (π₯) +πβ
π=1
ππ(π‘) π£π(π₯) (2.5)
where (π₯) is the mean flow, π£π are the POD mdoes, and ππ are the POD coefficients.
This is an approximation of a state (π₯, π‘) using the first π POD modes. π can be
chosen based on how much energy is desired in the state-space. More POD modes
provide better approximation of the state and capture more energy.
There is a feasibility constraint to ensure the initial states stay close to the attractor,
given byπβ
π=1
π2πππ
β€ π0 (2.6)
where ππ is the π-th POD coefficient, ππ is the π-th eigenvalue, π0 is a constant that
can be obtained as an average of the values from all the snapshots, π are the number
of modes to be used that contain more than 98% of the energy.
We define the Lagrangian, β(, π, *, π*, (0), (π )) as
β = πΌ((π )) β πΌ((0)) ββ« π
0
(*, ππ‘βπ())ππ
ββ« π
0
(π*,β.)ππ β π(((0),π (0)) β π0) (2.7)
where π is the pressure, * is the Lagrangian multiplier for the velocity field, π* is
the Lagrangian multiplier for the pressure field, π is the element-wise inverse of the
eigenvalue matrix, (0) is the initial velocity field, (π ) is the velocity field at time
π .
20
To evaluate the adjoint equations, we need to set the partial derivative of β with
respect to each of the six parameters equal to zero, so that the minimization problem
can reach a minima in the state-space that corresponds to the maximum growth in
the kinetic energy of the system.
1.) First, we set the partial derivative of the Lagrangian with respect to the ve-
locity field at any point in space and time equal to zero. Hence, all the other 5
parameters can be considered as constants.
πβπ
= ββ« π
0
(*, ππ‘βπ())ππ ββ« π
0
(π*,β.)ππ = 0
β« π
0
π
π
β«Ξ©
*(ππ‘βπ()) πΞ© ππ +
β« π
0
π
π
β«Ξ©
π*β. πΞ© ππ = 0
β« π
0
π
π
β«Ξ©
*(ππ‘ + .β + βπβ πββ π) πΞ© ππ +
β« π
0
π
π
β«Ξ©
π*β. πΞ© ππ = 0
β« π
0
β«Ξ©
(βππ‘* β .β* + *.βπ + πβ*) πΞ© ππ +
β« π
0
π
π
β«Ξ©
(ββπ*) πΞ© ππ = 0
where we have used integration by parts and the divergence theorem
β« π
0
β«Ξ©
(βππ‘* β .β* + *.βπ + πβ* ββπ*) πΞ© ππ = 0
which gives us the first adjoint Navier-stokes equation
ππ‘* + .β* β *.βπ β πβ* + βπ* = 0 (2.8)
This equation is not the same as the one we have for π© (π’), this is the adjoint version
of the incompressible Navier-stokes equation.
2.) Now we set the partial derivative of the Lagrangian with respect to the pres-
sure, π, equal to zero.
21
πβππ
=
β« π
0
π
ππ
β«Ξ©
*. βπ πΞ© ππ = 0
β« π
0
π
ππ
β«Ξ©
(ββ.*)π πΞ© ππ = 0
which gives us the second adjoint Navier-stokes equation
β.* = 0 (2.9)
3.) Here, the partial derivative of the Lagrangian multiplier of is set to zero. That
is with respect to *
πβπ* =
β« π
0
π
π*
β«Ξ©
*(ππ‘ + .β + βπβ πββ π) πΞ© ππ = 0
β« π
0
β«Ξ©
(ππ‘ + .β + βπβ πββ π) πΞ© ππ = 0
which gives us the first Navier-Stokes equation
ππ‘ = β.βββπ + πβ + π (2.10)
This equation is the same as the one we have for π© (π’), that is the incompressible
momentum Navier-stokes equation.
4.) Here, the partial derivative of the Lagrangian multiplier of π is set to zero. That
is with respect to π*
πβππ*
=
β« π
0
π
ππ*
β«Ξ©
π*β. πΞ© ππ = 0
β« π
0
β«Ξ©
β. πΞ© ππ = 0
22
which gives us the second Navier-Stokes equation (the incompressibility condition)
β. = 0 (2.11)
5.) With respect to the initial condition of the velocity field, (0), the partial deriva-
tive of the Lagrangian is set to zero.
πβπ(0)
= βππΌ((0))
π(0)+ *(0) β π
π
π(0)
β«Ξ©
(0)π π (0) πΞ© = 0
which gives
2(πΌ + ππ )(0) β *(0) = 0 (2.12)
The equation above plays a major role in calculating the gradient descent for the
initial velocity field, the parameter with respect to which the kinetic energy growth
is being maximized. This is described in the next section.
6.) With respect to the initial condition of the velocity field, (π ), the partial deriva-
tive of the Lagrangian is set to zero.
πβπ(π )
=ππΌ((π ))
π(π )β *(π ) = 0
where *(π ) comes from the integration by parts done in part 1.)
π
π(π )
β«Ξ©
(π ) . (π ) πΞ© β *(π ) = 0
which gives
2(π ) β *(π ) = 0 (2.13)
This equation provides the initial condition *(π ) for the adjoint problem, as it is
solved backwards in time to get to the final state *(0).
23
2.1.3 Gradient descent
This section aims to give a brief overview of the gradient descent optimization method.
Gradient descent ([14], [10], [51], [67], [70]) is used to find the minima of a function by
moving in the direction of the steepest descent. This is achieved through a number of
iterations, with the parameter in focus getting updated at each iteration. The direc-
tion of the steepest descent is given by the negative of the gradient of the functional
with respect to the parameter.
Gradient descent finds its use in various machine learning applications where a
model can have many parameters being updated at once. The size of the step which
dictates by how much we need to move in the direction of the steepest descent is
called the learning rate. This is denoted by πΌ in this section. When the learning rate
is high, convergence can be reached faster, as we take bigger steps. But there is a
risk of overshooting the minima in each iteration, which can make the solver to get
stuck in an infinite loop where it never reaches the minima. A safe option would be
choosing a low learning rate, but it would be slow and time consuming. The gradient
calculation is that of the function to be optimized with respect to a parameter.
In this thesis, the function would be given by equation 2.2 and the parameter is
the initial velocity field. The gradient of this function, πππ(0)
, is the same as the gra-
dient of the Lagrangian with respect to the initial velocity. Hence, it can be replaced
by πβπ(0)
Now, we use gradient descent to update the initial guess πππ(0)
πππ€(0) = πππ(0) β πΌπβ
π(0)
which gives
πππ€(0) = πππ(0) β πΌ(2(πΌ + ππ )πππ(0) β *(0)) (2.14)
where πΌ is the learning rate.
This process is repeated until the initial condition solution reaches convergence, that
is the value of the gradient πβπ(0)
tends to zero.
24
2.1.4 Layout for one complete loop/iteration of the adjoint
solver
In this section, we summarize the steps involved in one iteration of the adjoint prob-
lem. First, a random velocity field is initialized in MATLAB while satisfying the
feasibility constraints. This initial condition is used to solve the direct problem for-
ward in time for a given time window using Nek5000. The final state of the direct
problem can used to solve for the initial state of the adjoint problem using equation
2.13. The adjoint problem can be solved backward in time to reach the final state.
This gives all the parameters required to calculate the gradient of the Lagrangian
with respect to the initial velocity field. We update the initial velocity using equation
2.14 and this process can be continued until the solution converges.
Figure 2-1: Layout of the loop to be completed in each iteration of the adjoint problemtill the solution converges
2.2 Statistical analysis and prediction of extreme events
In this section, we define the extreme event indicator π, given as
π(π‘) =β¨β , π’*
0 β β©β
π’*0 β
(2.15)
25
where is the velocity field snapshot at time π‘, is the mean of the velocity field
over the entire time domain, and π’*0 is the velocity field of the precursor.
Here, we can see that the indicator value will be large when the velocity field snapshot
approaches the precursor velocity field. As the precursor is the cause of the extreme
events, the kinetic energy should also take a large value when the indicator does.
Hence, both the indicator and the kinetic energy would have extreme events almost
simultaneously. But given that we take a time window, the extreme event in the
indicator would precede the extreme event in the kinetic energy and that is how there
will be an early warning signal before the extreme event in the kinetic energy takes
place.
2.2.1 Joint and conditional statistics of the extreme events
The goal of this subsection is to quantify the prediction capabilities of the obtained
precursor from the adjoint-based optimization. For the quantitative analysis, we use
the joint and conditional statistics between the indicator π and the extreme event
observable i.e. the kinetic energy.
Here, for a given random variable ππ‘, the goal is to find an indicator variable ππ‘
(random variable), which can predict or generate a signal for an extreme event about
to occur in ππ‘ in a given time frame. An extreme event in ππ‘ is defined as instant
where the value of ππ‘ crosses a given extreme event threshold π₯π. For the random
variable ππ‘, the maximum value over a time interval [π‘ + π‘0, π‘ + π‘0 + βπ‘] for some π‘0,
and βπ‘ β₯ 0, is
π‘(π‘0,βπ‘) = max ππ π β[π‘+π‘0,π‘+π‘0+Ξπ‘]
(2.16)
Here, π‘(π‘0,βπ‘) is a new random variable and is a function of π‘0 and βπ‘. This variable
is a measure of the maximum value observed in the variable ππ‘ over a future time
interval defined by π . The joint probability distribution of (π‘, ππ‘) can be defined as
-
πΉππ‘,ππ‘(π₯, π¦) = π (π‘ β€ π₯, ππ‘ β€ π¦) =
β« π₯
ββ
β« π¦
ββπππ‘,ππ‘
(π₯, π¦) ππ₯ ππ¦ (2.17)
26
In the equation above, πππ‘,ππ‘is the probability density function (pdf) that is associated
with the probability distribution πΉππ‘,ππ‘. The instant where ππ‘ = π¦ and π‘ = π₯ at time
π‘, is measured by the probability density as a function of π₯ and π¦. The conditional
probability of π‘ = π₯ given that ππ‘ = π¦ can be defined using the Bayesβ formula given
as
πππ‘|ππ‘=
πππ‘,ππ‘
πππ‘
(2.18)
where πππ‘ is the probability density of ππ‘. In the time interval π , the conditional
probability density function πππ‘|ππ‘indicates the predictive capability of the indicator
ππ‘ for extreme events in ππ‘. The probability of upcoming extreme events in ππ‘ for a
given threshold π₯π given the current value of ππ‘ at π¦
πππ(π¦) =
β« β
π₯
πππ‘,ππ‘(π₯, π¦) ππ₯ (2.19)
Here, a large value of the indicator should indicate a higher probability of the oc-
curence of an extreme event in ππ‘. Hence, the function πππ should have monotonic
characteristics. For a very low value of the indicator, the probability of an extreme
event should be very low as well (close to zero), and for very high values of the indica-
tor, close to one. For a threshold on the probability, we classify an event as extreme
only if πππ(π¦π) = 0.5. We can define various scenarios based on the thresholds π₯π and
π¦π as
Correct Rejections (CR): π‘ < π₯π given ππ‘ < π¦π
Correct Predictions (CP): π‘ > π₯π given ππ‘ > π¦π
False Negatives (FN): π‘ > π₯π given ππ‘ < π¦π
False Positives (FP): π‘ < π₯π given ππ‘ > π¦π
From these prediction parameters, the indicator accuracy or prediction capabilities
can be defined as
27
Rate of successful predictions = πΆππΆπ+πΉπ
=
β« β
π₯π
β« β
π¦π
πππ‘,ππ‘(π₯, π¦) ππ₯ ππ¦β« β
π₯π
β« β
ββπππ‘,ππ‘
(π₯, π¦) ππ₯ ππ¦
Rate of successful rejections = πΆπ πΆπ +πΉπ
=
β« π₯π
ββ
β« π¦π
ββπππ‘,ππ‘
(π₯, π¦) ππ₯ ππ¦β« π₯π
ββ
β« β
ββπππ‘,ππ‘
(π₯, π¦) ππ₯ ππ¦
A good indicator would be one that has a higher number of correct predictions and
correct rejections, and a low amount of false positives and false negatives. Hence,
these are the desirable characteristics of a good indicator.
2.3 An overview of Nek5000 (flow solver)
As our fluid flow solver, we use Nek5000 which is a FORTRAN and C based solver
that can handle two- and three-dimensional domains that can be discretized into quad
or hex elements. It is an open source spectral element solver for computational fluid
dynamics simulations. It can be used to simulate unsteady incompressible fluid flows
with thermal and passive scalar transport. It is a code that is based on time-stepping
and supports steady Navier Stokes and heat conduction based problems.
It consists of three principal modules: the pre-processor ππππ₯, the solver πππ5000,
and the post-processor πππ π‘π₯. Nek5000 uses MPI for message passing and some
LAPACK routines for the computation of eigenvalues. Some studies using Nek5000
can be seen in [74], [50], [62], [39], [34], [66].
28
Chapter 3
Applications and results
The goal of this chapter is to apply the adjoint framework as described in chapter
2 to various fluid flow systems. In general, the framework can be applied to any
dynamical system where a set of governing equations and constraints are known. An
observable is then chosen to begin the analysis for the extreme event prediction. Here,
we first consider a 2-D flow namely the Kolmogorov flow. The governing equations
are first described with the boundary conditions. A simulation is performed for 3000
time steps and the proper orthogonal decomposition is performed which provides us
with the POD modes. A specific number of modes are chosen which capture more
than 90% of the energy of the system. This helps us in reducing the dimensionality
of the problem as well as retain a major portion of the information. An analysis is
performed to determine the order of the constraint π0 based on the snapshots that
were obtained from the simulation.
The adjoint setup with the governing and the constraints to be satisfied can then
be run. The solver iterates until it converges to a precursor and the order of the
magnitude of the Lagrangian tends to zero. The probability distributions for the cho-
sen parameters π (the indicator) and π·π (the extreme event in the observable in a
time horizon) are plotted with a time lag to see if the indicator jump above a certain
threshold precedes the extreme event in the observable. The rate of successful predic-
tions and successful rejections are presented which show the prediction capabilities of
the indicator π.
29
A similar analysis is repeated for the 2-D airfoil at an angle of attack of 30 de-
grees case and results are presented. A slight difference in this case is that the inlet
velocity is used as a source of stochasticity. Choosing a critical Reynolds number
after which the flow becomes unstable is described. This is done by zooming in at
the lift coefficient time series plots. Once the peaks start growing, an unstable flow is
indicated and this would be the critical Reynolds number of the flow. To generate the
stochastic inlet velocity, the Karhunen-Loeve expansion on the signal is performed
for smoothing purposes and reducing the order of the signal. The mean is taken as
the critical Reynolds number and the standard deviation is based on the amplitude
of the variation around the mean Reynolds number.
3.1 Kolmogorov flow
The Kolmogorov flow is a two-dimensional turbulent flow and has been an important
plant to study the nature of turbulent flows in general. The flow state is governed by
the 2-D Navier-Stokes equations which have a monochromatic forcing. This forcing
is sinusoidal in nature and is assumed to be stationary. The source of stochasticity
in the Kolmogorov flow comes from the inherent turbulence. We analyze the results
for a Reynolds number such that the flow has turbulence. Stochasticity in the system
causes intermittent bursts in the kinetic energy that we will see later in the section.
3.1.1 Flow solver, setup, and simulation
The governing equations for the Kolmogorov flow are given below
ππ‘ = β.βββπ + πβ + π (3.1)
and
β. = 0 (3.2)
There is a forcing term involved in the π₯ direction given by π = π ππ(πππ¦)ππ₯ where
ππ is the wave forcing number. ππ is chosen to be 4, as with this and π π = 40, the
30
flow will have inherent chaos and intermittent bursts in the kinetic energy. This is
shown in figure 3-2. The boundary conditions for the flow are assumed to be periodic
both in the π₯ and π¦ directions. The mesh (figure 3-1) for the Kolmogorov flow is
shown below. It contains 256 elements and order of the polynomial for the flow solver
is 7. The Reynolds number - π π is taken to be 40 which ensures turbulence. The
direct numerical simulation is performed for a total of 3000 convective time units
using Nek5000. Figure 3-2 shows the kinetic energy of the system plot vs. time.
The intermittent bursts can be seen which would be considered as the extreme events
in the observable πΌ. The goal of the precursor and the extreme event indicator π
would be to predict these bursts before a certain time frame so that we can have
some warning signals.
Figure 3-1: 2-D mesh for the Kolmogorov flow.
3.1.2 POD and extreme event analysis results
The goal of this subsection is to show the proper orthogonal decompositions results,
particularly the POD modes and the associated coefficients, the obtained precursor
and the prediction capabilities of the extreme event indicator π. For the analysis, the
first 500 snapshots of the simulation are neglected so that the transient states can be
removed as they contain random spikes that need to die down. The initial velocity
31
0 500 1000 1500 2000 2500 3000t
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9K
E
Figure 3-2: Time series plot of the kinetic energy.
field is initialized randomly based on the number π of the POD modes subspace. This
initial condition should lie on the attractor and that is ensured by making it satisfy the
feasibility constraint that makes the guess lie on the attractor. 10 degrees of freedom
are used for the POD coefficients as they capture more than 90% of the energy. The
vorticity snapshots of the POD modes are shown in figure 3-3 and the associated POD
coefficients are shown in figure 3-4. The first mode shows the predominant feature
of the flow and captures the most energy. The modes are arranged in an order of
decreasing energy, and the first 10 modes cumulatively sum up to 90% energy.
The randomly initialized velocity field is shown in figure 3-5, which is used as
the input for the solver for the first iteration. As the solver iterates, this field gets
updated based on the lagrangian until it converges to the precursor. After around
1500 iterations, the field converges to the required local minima velocity field which
would be the precursor as shown in figure 3-6. The time horizon used to evolve the
adjoint problem per iteration is 2e-5 time units (time step size is 2e-6) as this is the
instantaneous time horizon to make π‘ tend to 0. The learning rate is chosen to be
πΌ = 0.05.
32
π1 π6
π2 π7
π3 π8
π4 π9
π5 π10
Figure 3-3: Vorticity (the scale is -2 to 2) of the POD modes π = β Γ π(),π = 1, . . . , 10 for the Kolmogorov flow.
Figure 3-7 shows the indicator π and π·π (the max of the KE in each 1 time unit
window) on the same figure. It can be seen that the bursts in both the variables are
concurrent. Also, due to the time lag between π and π·π, the extreme events in π·π
can be predicted beforehand, by seeking states when π crosses a certain threshold.
It can be observed that the extreme event in π occurs before the extreme event in
KE, hence giving an early warning signal. See figure 3-8. A required property of
the extreme event indicator π is to maximize the number of correct predictions and
correct rejections. The results for these based on the formulas as derived earlier are
33
500 1000 1500 2000 2500t
-5
0
5a 1
500 1000 1500 2000 2500t
-3
-2
-1
0
1
2
a 6
500 1000 1500 2000 2500t
-5
0
5
a 2
500 1000 1500 2000 2500t
-2
-1
0
1
a 7
500 1000 1500 2000 2500t
-6
-4
-2
0
2
4
a 3
500 1000 1500 2000 2500t
-2
-1
0
1
2
3
a 8
500 1000 1500 2000 2500t
-4
-2
0
2
4
6
a 4
500 1000 1500 2000 2500t
-2
-1
0
1
2
a 9
500 1000 1500 2000 2500t
-1
-0.5
0
0.5
1
1.5
a 5
500 1000 1500 2000 2500t
-3
-2
-1
0
1
2
a 10
Figure 3-4: The first 10 POD coefficients for the Kolmogorov flow.
34
Figure 3-5: Initial guess of the velocity field.
Figure 3-6: Precursor obtained after 1500 iterations of the adjoint problem.
as follows -
Rate of successful predictions = 90.59%
Rate of successful rejections = 96.93%
We take the extreme event threshold π₯π to be = mean(KE) + 2*std(KE)
ππ is computed by setting πππ(ππ) = 0.5
The probability distribution function plots as shown in Figures 3-9 and 3-10, describe
the appromixate linearity between the values of π and π·π. Small values of π·π and
π are highly populated, which show the local minimums being predicted successfully.
35
500 1000 1500 2000 2500Snapshot #
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
KE
Figure 3-7: π and Kinetic energy (KE) vs. snapshot number (the time lag between πand KE is 1 time unit).
2025 2030 2035 2040 2045 2050 2055 2060Snapshot #
0.21
0.22
0.23
0.24
0.25
0.26
0.27
0.28
0.29
0.3
0.31
Figure 3-8: Zoomed peak near t = 2040 for the π and Kinetic energy vs. snapshotnumber plot.
36
0 0.1 0.2 0.3 0.4D
m
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
Figure 3-9: Plot showing correlation between π and π·π (the time lag between π andKE is 1 time unit).
Figure 3-10: Joint pdf of π and π·π (the time lag between π and KE is 1 time unit).
High values of π·π and π are low in population, which show the global minimums or
the extreme events being predicted successfully. A bad indicator would be one which
37
Figure 3-11: Conditional pdf of π and π·π (the time lag between π and KE is 1 timeunit).
has data points all over the joint or conditional probability distribution function plots.
A simulation for 10,000 time units is also performed and the calculations are made
using the precursor obtained previously. From calculations, the extreme events re-
sults are as follows -
Rate of successful predictions = 96.14%
Rate of successful rejections = 94.86%
The extreme events in the dissipative energy (DE) can also be predicted with a
good accuracy by using the same precursor. The results are as follows -
Rate of successful predictions = 80.71%
Rate of successful rejections = 83.93%
38
1000 2000 3000 4000 5000 6000 7000 8000 9000Snapshot #
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
KE
Figure 3-12: π and Kinetic energy vs. snapshot number (10000 time units simulation,the time lag between π and KE is 1 time unit).
0 0.1 0.2 0.3 0.4 0.5D
m
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
Figure 3-13: Plot showing correlation between π andπ·π (10000 time units simulation,the time lag between π and KE is 1 time unit).
39
Figure 3-14: Joint pdf of π andπ·π (10000 time units simulation, the time lag betweenπ and KE is 1 time unit).
Figure 3-15: Conditional pdf of π and π·π (10000 time units simulation, the time lagbetween π and KE is 1 time unit).
40
1000 2000 3000 4000 5000 6000 7000 8000 9000Snapshot #
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
KE
Figure 3-16: π and Dissipative energy (DE) vs. snapshot number (10000 time unitssimulation, the time lag between π and DE is 1 time unit).
0.4 0.5 0.6 0.7 0.8 0.9 1D
m
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
Figure 3-17: Plot showing correlation between π and π·π (DE) (10000 time unitssimulation, the time lag between π and DE is 1 time unit).
41
Figure 3-18: Joint pdf of π and π·π (DE) (10000 time units simulation, the time lagbetween π and DE is 1 time unit).
Figure 3-19: Conditional pdf of π and π·π (DE) (10000 time units simulation, thetime lag between π and DE is 1 time unit).
42
3.2 2-D Airfoil flow
In this section, we take the fluid flow over a 2-D airfoil at a certain angle of attack.
The source of stochasticity in this case is the stochastic inlet velocity that is generated
using a stochastic differential equation. The inlet velocity signal is further smoothed
out using the Karhuhen-Loeve Expansion (KLE). This is done to preserve the dynam-
ics and also capture the most important features in the signal. A critical Reynolds
number is established on which the inlet velocity is based using a certain amplitude.
For the original signal and the one generated by KLE to have similar statistics, we
keep the mean and standard deviation of the KLE signal close enough to the original
signal through running multiple iterations of random signals. The random signal that
has similar statistics under a small margin of error is chosen as the inlet velocity.
3.2.1 Flow solver, setup, and simulation
For the setup, we consider the NACA-0012 airfoil with an angle of attack of 30β.
The mesh (figure 3-20) for the given geometry is shown below (618 elements and
polynomial order of 9). A constant velocity profile (ππ₯ = 1) characterized by the
Reynolds number is given at the inlet. We initialize the flow field with a low amplitude
disturbance (ππ₯ = 1 + 10β5π¦). The Reynolds number in terms of the dimensional
parameters is π π = π ππΏπ
.
We perform a DNS simulation for the 2D airfoil flow using Nek5000. The simu-
lation is performed for a total of 3000 convective time units. The critical Reynolds
number (π ππ) is defined to be the one from which the flow starts becoming un-
steady i.e. doesnβt reach a fixed steady state solution. Vorticity snapshots for various
Reynolds numbers near the π ππ at 100 time units instance are given in figure 3-21
below, part (a) shows the converged steady state solution at a Reynolds number be-
low the critical Reynolds number, part (b) shows the beginning of the unsteady state
of the flow, above this Reynolds number, all solutions would be unsteady (see part(c)
as an example).
43
Figure 3-20: 2-D mesh of the flow around the airfoil.
3.2.2 Lift coefficient over time analysis to determine the crit-
ical Reynolds number
To study the unsteadiness in the flow and establish a critical Reynolds number, we
consider the lift coefficient over time. Figure 3-24 displays the lift coefficient over
time plots for the three Reynolds numbers as in figure 3-21. π π = 90, decays at a very
rapid rate and the oscillations become negligible in magnitude after 100 convective
time units. On zooming the regions, it can be observed that the peaks are decreasing
with time which means there will be a point where steady state would be established.
For π π = 105, the decay is at a very slow rate over time (as seen when zooming the
peaks in the plots). From T=280, it can be seen that the peaks start to stay where
they are in magnitude. This shows there will be oscillations with some magnitude
44
that wonβt die out over time. Whereas, for π π = 110, a chaotic behavior can be
observed which marks the onset of a transition. The peaks start to show oscillations
in themselves, meaning that there will be continued transience in the flow state for
all time. The critical Reynolds number (π ππ) can hence be said to be between 105
and 110 (π ππ = 107.5 Β± 2.5).
3.2.3 Generating the stochastic inlet velocity and mode re-
duction using Karhunen-Loeve expansion
The inlet velocity signal (π) that will be the source of stochasticity in the fluid flow
dynamics is generated by simulating the stochastic differential equation as given below
-
ππ = βπ π ππ‘ + π ππ (3.3)
with the parameters given as
π0 = π(1) = 0
π = 2000000 : Number of time steps in our fluid simulation
ππ‘ = 1/π
πππ = π§πβππ‘ :
where π§π is a random variable chosen from a normal distribution with mean = 0 and
std = 1
Hence, to simulate it numerically we write it as simply a finite time difference and
propagating forward in time
ππ+1 = ππ β π ππ ππ‘ + π πππ
If this signal is used by itself, then it gives rise to a noisy lift coefficient time se-
ries. This is due to the fact that the flow is non-symmetric and changing the inlet
45
velocity rapidly over time would make the flow field chaotic. Hence, there is a need
to reduce the order of the stochastic inlet velocity signal.
We generate the long time scale fluctuations and reduce the number of modes to
smooth out the signal using KLE. In essence, the Karhunen-Loeve expansion theorm
(KLE) represents a stochastic process as a linear combination of an infinite number
of orthogonal functions. This is similar to the principal component analysis (PCA)
and the fourier series expansion of a given function. A more detailed explanation can
be seen in [82]. The fluctuating inlet velocity using KLE is given by -
π(π) =βπβ₯1
βππ π’π(π) π
where π is a normally distributed random variable. To obtain the eigenvalues (ππ)
and eigenvectors (π’π), we solve the eigenvalue problem given by
β«Ξ©
πΆ(π, π β²) π’π(πβ²) ππ = ππ π’π(π)
where πΆ(π, π β²) is the correlation function of the process π(π)
We choose the number of modes by satisfying the minimum k condition -
πβπ=1
ππ β₯ 0.99πβ
π=1
ππ
To formulate the correlation matrix, we use the MATLAB snippet -
for i=1:length(ππ)
for j=1:length(ππ)
diff = abs((i-j)*0.5);
C(i,j) = variance * exp(-diff/b);
end
end
46
Probe # x y zProbe 1 1.5 0.5 0.0Probe 2 3.0 0.2 0.0Probe 3 5.0 0.0 0.0
Table 3.1: Location of the three probes.
where π is the correlation length taken to be 200 (large enough than the strouhal
period of 5). This has been chosen out of b=20, 50, 100 and 200. The inlet velocity
signal is generated with a mean of Reynolds number = 100 and amplitude = 20. The
signal can be seen in figure 3-23.
3.2.4 Results
The goal of this section is to show the proper orthogonal decomposition results and
the extreme events happening in the kinetic energy and the lift coefficient. The first
ten POD modes and the corresponding POD coefficients are displayed in the figures
3-25 and 3-26 respectively. They show the modes containing the most dominant
features in the flow simulation in decreasing order of energy.
The extreme events in the lift coefficient (πΆπΏ) can be seen in figure 3-27. The
fluctuations in the lift coefficient has a strong correlation with the inlet velocity as
can be seen in part(c), extreme events in the inlet velocity lead to extreme events
in the lift coefficient most of the times. Here, we could refer to an extreme event
as a significant jump or drop from the mean of the lift coefficient time series. We
also consider the x (figure 3-28) and y (figure 3-29) components of the velocity at 3
probes with their locations given as (see table 3.1)
47
(a)
(b)
(c)
Figure 3-21: Vorticity snapshots at T=100 for Reynolds numbers near π ππ: (a) π π= 90 (b) π π = 105 (c) π π = 110.
48
0 500 1000 1500 2000 2500 3000t
30
35
40
45
50
55
60K
E
Figure 3-22: The kinetic energy plot for 3000 time units.
0 500 1000 1500 2000 2500 3000t
80
85
90
95
100
105
110
115
120
Re
KLE, b = 200, std = 6.7261, std-formula=6.67, mean=101.7132, mean-base=100
Figure 3-23: The inlet velocity input signal.
49
(a) (b)
0 50 100 150 200t
0.9
0.95
1
1.05
1.1
1.15
1.2C
L
140 150 160 170 180 190 200t
0.91955
0.9196
0.91965
0.9197
0.91975
CL
(c) (d)
0 50 100 150 200 250 300t
0.85
0.9
0.95
1
1.05
1.1
1.15
CL
220 240 260 280 300t
0.935686302
0.935686304
0.935686306
0.935686308
0.93568631
0.935686312
0.935686314
0.935686316
0.935686318
0.93568632
(e) (f)
0 50 100 150 200 250 300t
0.85
0.9
0.95
1
1.05
1.1
1.15
CL
200 220 240 260 280t
0.948027234
0.948027236
0.948027238
0.94802724
0.948027242
0.948027244
Figure 3-24: (a) Re = 90 for t = 15 to 200 (b) Re = 90, zoomed peaks (c) Re =105, for t = 15 to 300 (d) Re = 105, zoomed peaks (e) Re = 110, for t = 15 to 300(f) Re = 110, zoomed peaks.
50
π1 π6
π2 π7
π3 π8
π4 π9
π5 π10
Figure 3-25: Vorticity (the scale is -2 to 2) of the POD modes π = β Γ π(),π = 1, . . . , 10 for the airfoil flow.
51
500 1000 1500 2000 2500t
-1
0
1
a 1
500 1000 1500 2000 2500t
-0.1
-0.05
0
0.05
0.1
a 6
500 1000 1500 2000 2500t
-0.5
0
0.5
a 2
500 1000 1500 2000 2500t
-0.1
-0.05
0
0.05
0.1
a 7
500 1000 1500 2000 2500t
-0.5
0
0.5
a 3
500 1000 1500 2000 2500t
-0.05
0
0.05
a 8
500 1000 1500 2000 2500t
-0.1
0
0.1
0.2
a 4
500 1000 1500 2000 2500t
-0.02
0
0.02
0.04
a 9
500 1000 1500 2000 2500t
-0.1
-0.05
0
0.05
0.1
a 5
500 1000 1500 2000 2500t
-0.04
-0.02
0
0.02
0.04
a 10
Figure 3-26: The first 10 POD coefficients for the airfoil flow.
52
(a)
500 1000 1500 2000 2500t
0.6
0.7
0.8
0.9
1
1.1
1.2
1.3
1.4
CL
CL
mean(CL
)
(b)
0 500 1000 1500 2000 2500 3000t
0.6
0.7
0.8
0.9
1
1.1
1.2
1.3
mea
n(C
L)
(c)
500 1000 1500 2000 2500t
-0.2
-0.15
-0.1
-0.05
0
0.05
0.1
0.15
0.2
CL
fluc(CL
)
U
Figure 3-27: (a) πΆπΏ from t = 20 to 3000 (b) The averaged curve (c) fluctuations inπΆπΏ calculated by removing the moving average (5 time units moving window). π istaken as the inlet KLE velocity with zero mean. From t = 20 to 3000.
53
(a)
500 1000 1500 2000 2500t
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
Vx1
(b)
500 1000 1500 2000 2500t
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.55
Vx2
(c)
500 1000 1500 2000 2500t
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Vx3
Figure 3-28: π₯ component of the velocities at the three probes (a) ππ₯1 (b) ππ₯2 (c) ππ₯3.
54
(a)
500 1000 1500 2000 2500t
-0.2
-0.15
-0.1
-0.05
0
0.05
0.1
Vy1
(b)
500 1000 1500 2000 2500t
-0.5
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.5
Vy2
(c)
500 1000 1500 2000 2500t
-0.5
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
Vy3
Figure 3-29: π¦ component of the velocities at the three probes (a) ππ¦1 (b) ππ¦2 (c) ππ¦3.
55
56
Chapter 4
Conclusions and future work
4.1 Summary of the results and discussion
In this thesis, we have derived the adjoint equations for the Navier-stokes equations of
a 2-D fluid flow. The extreme events are considered in the observable taken as kinetic
energy of the flow field. We have presented a coupled fluid (Nek5000) and adjoint
solver (MATLAB) that can be applied to any dynamical system with non-linear terms
in the flow map and subjected to constraints. The prediction capabilities of the solver
are demonstrated for the 2-D Kolmogorov flow, with the results summarized in the
table below 4.1.
The extreme events for the kinetic energy and the lift coefficient for a 2-D airfoil
flow at an angle of attack of 30 degrees are also presented. The extreme events bear
a strong correlation with the extreme events in the stochastic inlet velocity, which is
considered to be the source of the extreme events in this case.
Simulation Rate of successful predictions Rate of successful rejections3000 time units 90.59% 96.93%10000 time units 96.14% 94.86%
Table 4.1: Summary of the extreme events prediction results for the Kolmogorov flow
57
4.2 Future work
Future work would include the application of this coupled solver on the 2-D airfoil
flow presented in chapter 3. The goal would be to evaluate the prediction capabilities
of the obtained precursor and the time window in which it can predict extreme events
in the kinetic energy effectively. Another interesting case would be the 3-D Ahmed
body flow ([64], [11], [80], [37], [55], [4]), where the source of stochasticity would
be turbulence. The prediction capabilities can also be challenged by increasing the
time window in which the extreme event has to be predicted. This would expose the
maximum time before which the extreme events could be predicted without a large
decline in the prediction accuracy of the precursor.
58
Bibliography
[1] S. Albeverio, V. Jentsch, and H. Kantz. Extreme events in nature and society.Springer Science & Business Media, 2006.
[2] S. Albeverio and V. Piterbarg. Mathematical methods and concepts for theanalysis of extreme events. In Extreme events in nature and society, pages 47β68. Springer, 2006.
[3] H Babaee. An observation-driven time-dependent basis for a reduced de-scription of transient stochastic systems. Proceedings of the Royal Society A,475(2231):20190506, 2019.
[4] I. Bayraktar, D. Landman, and O. Baysal. Experimental and computationalinvestigation of ahmed body for ground vehicle aerodynamics. Technical report,SAE Technical Paper, 2001.
[5] G. Berkooz, P. Holmes, and J. L. Lumley. The proper orthogonal decompositionin the analysis of turbulent flows. Annual review of fluid mechanics, 25(1):539β575, 1993.
[6] A. Blanchard, S. Mowlavi, and T. P. Sapsis. Control of linear instabilities by dy-namically consistent order reduction on optimally time-dependent modes. Non-linear Dynamics, 95(4):2745β2764, 2019.
[7] A. Blanchard and T. P. Sapsis. Analytical description of optimally time-dependent modes for reduced-order modeling of transient instabilities. SIAMJournal on Applied Dynamical Systems, 18(2):1143β1162, 2019.
[8] A. Blanchard and T. P. Sapsis. Stabilization of unsteady flows by reduced-order control with optimally time-dependent modes. Physical Review Fluids,4(5):053902, 2019.
[9] P. J. Blonigan, M. Farazmand, and T. P. Sapsis. Are extreme dissipation eventspredictable in turbulent fluid flows? Physical Review Fluids, 4(4):044606, 2019.
[10] L. Bottou. Large-scale machine learning with stochastic gradient descent. InProceedings of COMPSTATβ2010, pages 177β186. Springer, 2010.
59
[11] C. Bruneau, E. CreusΓ©, D. Depeyras, P. GilliΓ©ron, and I. Mortazavi. Couplingactive and passive techniques to control the flow past the square back ahmedbody. Computers & Fluids, 39(10):1875β1892, 2010.
[12] A. BΓΌcher and J. Segers. Extreme value copula estimation based on block maximaof a multivariate stationary time series. Extremes, 17(3):495β528, 2014.
[13] A. Bunde, J. F. Eichner, S. Havlin, and J. W. Kantelhardt. The effect of long-term correlations on the return periods of rare events. Physica A: StatisticalMechanics and its Applications, 330(1-2):1β7, 2003.
[14] C. Burges, T. Shaked, E. Renshaw, A. Lazier, M. Deeds, N. Hamilton, and G. N.Hullender. Learning to rank using gradient descent. In Proceedings of the 22ndInternational Conference on Machine learning (ICML-05), pages 89β96, 2005.
[15] Y. Cao, S. Li, L. Petzold, and R. Serban. Adjoint sensitivity analysis fordifferential-algebraic equations: The adjoint dae system and its numerical so-lution. SIAM journal on scientific computing, 24(3):1076β1089, 2003.
[16] E. Castillo. Extreme value theory in engineering. Elsevier, 2012.
[17] M. A. Celia, T. F. Russell, I. Herrera, and R. E. Ewing. An eulerian-lagrangianlocalized adjoint method for the advection-diffusion equation. Advances in WaterResources, 13(4):187β206, 1990.
[18] A. Chatterjee. An introduction to the proper orthogonal decomposition. Currentscience, pages 808β817, 2000.
[19] V. A. Cicirello and S. F. Smith. Heuristic selection for stochastic search opti-mization: Modeling solution quality by extreme value theory. In InternationalConference on Principles and Practice of Constraint Programming, pages 197β211. Springer, 2004.
[20] P. Coles and J. D. Barrow. Non-gaussian statistics and the microwave backgroundradiation. Monthly Notices of the Royal Astronomical Society, 228(2):407β426,1987.
[21] L. De Haan and A. Ferreira. Extreme value theory: an introduction. SpringerScience & Business Media, 2007.
[22] L. De Haan and S. Resnick. On regular variation of probability densities. Stochas-tic processes and their applications, 25:83β93, 1987.
[23] R. G. Dean. Freak waves: a possible explanation. In Water wave kinematics,pages 609β612. Springer, 1990.
[24] R. Devaney. An introduction to chaotic dynamical systems. CRC Press, 2018.
[25] R. L. Devaney. A first course in chaotic dynamical systems: theory and experi-ment. CRC Press, 2018.
60
[26] K. Dysthe, H. E. Krogstad, and P. MΓΌller. Oceanic rogue waves. Annu. Rev.Fluid Mech., 40:287β310, 2008.
[27] J. H. J. Einmahl, J. Segers, et al. Maximum empirical likelihood estimation ofthe spectral measure of an extreme-value distribution. The Annals of Statistics,37(5B):2953β2989, 2009.
[28] M. Farazmand. An adjoint-based approach for finding invariant solutions ofnavierβstokes equations. Journal of Fluid Mechanics, 795:278β312, 2016.
[29] M. Farazmand and T. P. Sapsis. A variational approach to probing extremeevents in turbulent dynamical systems. Science advances, 3(9):e1701533, 2017.
[30] V. FernΓ‘ndez. Risk management under extreme events. International Review ofFinancial Analysis, 14(2):113β148, 2005.
[31] P. H. Franses and D. Van Dijk. Forecasting stock market volatility using (non-linear) garch models. Journal of Forecasting, 15(3):229β235, 1996.
[32] E. M. Furrer and R. W. Katz. Improving the simulation of extreme precipitationevents by stochastic weather generators. Water Resources Research, 44(12), 2008.
[33] J. Gleick. Chaos: Making a new science. Open Road Media, 2011.
[34] J. Gong, S. Markidis, M. Schliephake, E. Laure, D. Henningson, P. Schlatter,A. Peplinski, A. Hart, J. Doleschal, D. Henty, et al. Nek5000 with openacc. InInternational Conference on Exascale Applications and Software, pages 57β68.Springer, 2014.
[35] J. D. Hamilton. Time series analysis, volume 2. Princeton university pressPrinceton, NJ, 1994.
[36] R. W. Healy and T. F. Russell. A finite-volume eulerian-lagrangian localized ad-joint method for solution of the advection-dispersion equation. Water ResourcesResearch, 29(7):2399β2413, 1993.
[37] C. Hinterberger, M. Garcia-Villalba, and W. Rodi. Large eddy simulation of flowaround the ahmed body. In The aerodynamics of heavy vehicles: trucks, buses,and trains, pages 77β87. Springer, 2004.
[38] E. Hopf. A mathematical example displaying features of turbulence. Communi-cations on Pure and Applied Mathematics, 1(4):303β322, 1948.
[39] I. Ivanov, J. Gong, D. Akhmetova, I. B. Peng, S. Markidis, E. Laure, R. Machado,M. Rahn, V. Bartsch, A. Hart, et al. Evaluation of parallel communicationmodels in nekbone, a nek5000 mini-application. In 2015 IEEE InternationalConference on Cluster Computing, pages 760β767. IEEE, 2015.
[40] P. A. E. M. Janssen. Nonlinear four-wave interactions and freak waves. Journalof Physical Oceanography, 33(4):863β884, 2003.
61
[41] R. W. Katz, M. B. Parlange, and P. Naveau. Statistics of extremes in hydrology.Advances in water resources, 25(8-12):1287β1304, 2002.
[42] S. H. Kellert. In the wake of chaos: Unpredictable order in dynamical systems.University of Chicago press, 1993.
[43] H. Kim, D. Sasaki, S. Obayashi, and K. Nakahashi. Aerodynamic optimizationof supersonic transport wing using unstructured adjoint method. AIAA journal,39(6):1011β1020, 2001.
[44] S. Kim, J. Alonso, and A. Jameson. Design optimization of high-lift configu-rations using a viscous continuous adjoint method. In 40th AIAA AerospaceSciences Meeting & Exhibit, page 844, 2002.
[45] A. Kogut, A. J. Banday, C. L. Bennett, K. M. Gorski, G. Hinshaw, G. F. Smoot,and E. L. Wright. Tests for non-gaussian statistics in the dmr four-year skymaps. The Astrophysical Journal Letters, 464(1):L29, 1996.
[46] O. Lehmann, M. Luchtenburg, B. R. Noack, R. King, M. Morzynski, and G. Tad-mor. Wake stabilization using pod galerkin models with interpolated modes. InProceedings of the 44th IEEE Conference on Decision and Control, pages 500β505. IEEE, 2005.
[47] Y. Li and C. Meneveau. Origin of non-gaussian statistics in hydrodynamic tur-bulence. Physical review letters, 95(16):164502, 2005.
[48] B. Liu and J. Goree. Superdiffusion and non-gaussian statistics in a driven-dissipative 2d dusty plasma. Physical review letters, 100(5):055003, 2008.
[49] E. N. Lorenz. Deterministic nonperiodic flow. Journal of the atmospheric sci-ences, 20(2):130β141, 1963.
[50] S. Markidis, J. Gong, M. Schliephake, E. Laure, A. Hart, D. Henty, K. Heisey,and P. Fischer. Openacc acceleration of the nek5000 spectral element code. TheInternational Journal of High Performance Computing Applications, 29(3):311β319, 2015.
[51] L. Mason, J. Baxter, P. L. Bartlett, and M. R. Frean. Boosting algorithms asgradient descent. In Advances in neural information processing systems, pages512β518, 2000.
[52] K. Maute, M. Nikbay, and C. Farhat. Sensitivity analysis and design optimizationof three-dimensional non-linear aeroelastic systems by the adjoint method. In-ternational Journal for Numerical Methods in Engineering, 56(6):911β933, 2003.
[53] A. McNamara, A. Treuille, Z. PopoviΔ, and J. Stam. Fluid control using theadjoint method. In ACM Transactions On Graphics (TOG), volume 23, pages449β456. ACM, 2004.
62
[54] A. Medio and G. Gallo. Chaotic dynamics. Technical report, Cambridge Uni-versity Press, 1995.
[55] M. Minguez, R. Pasquetti, and E. Serre. High-order large-eddy simulation offlow over the βahmed bodyβ car model. Physics of fluids, 20(9):095101, 2008.
[56] J. F. B. Mitchell, J. Lowe, R. A. Wood, and M. Vellinga. Extreme events due tohuman-induced climate change. Philosophical Transactions of the Royal SocietyA: Mathematical, Physical and Engineering Sciences, 364(1845):2117β2133, 2006.
[57] A. Montina, U. Bortolozzo, S. Residori, and F. T. Arecchi. Non-gaussian statis-tics and extreme waves in a nonlinear optical cavity. Physical review letters,103(17):173901, 2009.
[58] N. Mori and P. A. E. M. Janssen. On kurtosis and occurrence probability offreak waves. Journal of Physical Oceanography, 36(7):1471β1483, 2006.
[59] G. Mu, W. Zhou, W. Chen, and J. KertΓ©sz. Order flow dynamics around ex-treme price changes on an emerging stock market. New Journal of Physics,12(7):075037, 2010.
[60] B. R. Noack, K. Afanasiev, M. MorzyΕski, G. Tadmor, and F. Thiele. A hierarchyof low-dimensional models for the transient and post-transient cylinder wake.Journal of Fluid Mechanics, 497:335β363, 2003.
[61] B. R. Noack, W. Stankiewicz, M. MorzyΕski, and P. J. Schmid. Recursive dy-namic mode decomposition of transient and post-transient wake flows. Journalof Fluid Mechanics, 809:843β872, 2016.
[62] N. Offermans, O. Marin, M. Schanen, J. Gong, P. Fischer, P. Schlatter, A. Ob-abko, A. Peplinski, M. Hutchinson, and E. Merzari. On the strong scaling ofthe spectral element solver nek5000 on petascale systems. In Proceedings of theExascale Applications and Software Conference 2016, page 5. ACM, 2016.
[63] M. Onorato, A. R. Osborne, M. Serio, and S. Bertone. Freak waves in randomoceanic sea states. Physical review letters, 86(25):5831, 2001.
[64] J. Γsth, B. R. Noack, S. KrajnoviΔ, D. Barros, and J. BorΓ©e. On the needfor a nonlinear subscale turbulence term in pod models as exemplified for ahigh-reynolds-number flow over an ahmed body. Journal of Fluid Mechanics,747:518β544, 2014.
[65] D.I. Papadimitriou and K.C. Giannakoglou. A continuous adjoint method withobjective function derivatives based on boundary integrals, for inviscid and vis-cous flows. Computers & Fluids, 36(2):325β341, 2007.
[66] A. Peplinski, P. Schlatter, P. F. Fischer, and D. S. Henningson. Stability tools forthe spectral-element code nek5000: Application to jet-in-crossflow. In Spectraland High Order Methods for Partial Differential Equations-ICOSAHOM 2012,pages 349β359. Springer, 2014.
63
[67] N. Qian. On the momentum term in gradient descent learning algorithms. Neuralnetworks, 12(1):145β151, 1999.
[68] C. Qiu and Q. Xu. A simple adjoint method of wind analysis for single-dopplerdata. Journal of Atmospheric and Oceanic Technology, 9(5):588β598, 1992.
[69] C. W. Rowley. Model reduction for fluids, using balanced proper orthogonal de-composition. International Journal of Bifurcation and Chaos, 15(03):997β1013,2005.
[70] S. Ruder. An overview of gradient descent optimization algorithms. arXivpreprint arXiv:1609.04747, 2016.
[71] D. Ruelle. Small random perturbations of dynamical systems and the definitionof attractors. Communications in Mathematical Physics, 82(1):137β151, 1981.
[72] P. J. Schmid. Dynamic mode decomposition of numerical and experimental data.Journal of fluid mechanics, 656:5β28, 2010.
[73] M. Schramm, B. Stoevesandt, and J. Peinke. Adjoint optimization of 2d-airfoilsin incompressible flows. In 11th World Congress on Computational Mechanics,Wccm, pages 6200β6211, 2014.
[74] J. Shin, M. W. Hall, J. Chame, C. Chen, P. F. Fischer, and P. D. Hovland.Speeding up nek5000 with autotuning and specialization. In Proceedings of the24th ACM International Conference on Supercomputing, pages 253β262. ACM,2010.
[75] O. Soto and R. LΓΆhner. On the computation of flow sensitivities from boundaryintegrals. AIAA paper, 112:2004, 2004.
[76] O. Soto, R. LΓΆhner, and C. Yang. An adjoint-based design methodology for cfdproblems. International Journal of Numerical Methods for Heat & Fluid Flow,14(6):734β759, 2004.
[77] G. Tadmor, O. Lehmann, B. R. Noack, L. Cordier, J. Delville, J. Bonnet, andM. MorzyΕski. Reduced-order models for closed-loop wake control. PhilosophicalTransactions of the Royal Society A: Mathematical, Physical and EngineeringSciences, 369(1940):1513β1524, 2011.
[78] F. Takens. Detecting strange attractors in turbulence. In Dynamical systemsand turbulence, Warwick 1980, pages 366β381. Springer, 1981.
[79] J. A. Tawn. Modelling multivariate extreme value distributions. Biometrika,77(2):245β253, 1990.
[80] A. Thacker, S. Aubrun, A. Leroy, and P. Devinant. Effects of suppressing the3d separation on the rear slant on the flow structures around an ahmed body.Journal of Wind Engineering and Industrial Aerodynamics, 107:237β243, 2012.
64
[81] G. Tissot, L. Cordier, N. Benard, and B. R. Noack. Model reduction usingdynamic mode decomposition. Comptes Rendus MΓ©canique, 342(6-7):410β416,2014.
[82] J. Venkatramani, S. K. Kumar, S. Sarkar, and S. Gupta. Physical mechanismof intermittency route to aeroelastic flutter. Journal of Fluids and Structures,75:9β26, 2017.
[83] J. Wenger, R. Tualle-Brouri, and P. Grangier. Non-gaussian statistics from in-dividual pulses of squeezed light. Physical review letters, 92(15):153601, 2004.
[84] B. S. White and B. Fornberg. On the chance of freak waves at sea. Journal offluid mechanics, 355:113β138, 1998.
[85] K. Willcox and J. Peraire. Balanced model reduction via the proper orthogonaldecomposition. AIAA journal, 40(11):2323β2330, 2002.
[86] G. Williams. Chaos theory tamed. CRC Press, 1997.
65