![Page 1: Numerics of Stochastic Processes II (14.11.2008)page.mi.fu-berlin.de/horenko/download/NumerikIV/VL_5.pdf1) Numerical Methods from PDEs like Finite-Element-Methods (FEM) become available](https://reader033.vdocuments.us/reader033/viewer/2022043017/5f39a9146fb2f71bb34d9a8f/html5/thumbnails/1.jpg)
Research Group “Computational TimeSeries Analysis “Institute of MathematicsFreie
Universität
Berlin (FU)
Numerics of Stochastic Processes II(14.11.2008)Illia Horenko
DFG Research Center MATHEON „Mathematics
in key
technologies“
![Page 2: Numerics of Stochastic Processes II (14.11.2008)page.mi.fu-berlin.de/horenko/download/NumerikIV/VL_5.pdf1) Numerical Methods from PDEs like Finite-Element-Methods (FEM) become available](https://reader033.vdocuments.us/reader033/viewer/2022043017/5f39a9146fb2f71bb34d9a8f/html5/thumbnails/2.jpg)
Short Reminder
Stochastic Processes
Discrete State Space,Discrete Time:Markov Chain
Discrete State Space,Continuous Time:Markov Process
Continuous State Space,Discrete Time:Markov Process
Continuous State Space,Continuous Time:Stochastic Differential Equation
Last Time
![Page 3: Numerics of Stochastic Processes II (14.11.2008)page.mi.fu-berlin.de/horenko/download/NumerikIV/VL_5.pdf1) Numerical Methods from PDEs like Finite-Element-Methods (FEM) become available](https://reader033.vdocuments.us/reader033/viewer/2022043017/5f39a9146fb2f71bb34d9a8f/html5/thumbnails/3.jpg)
Short Reminder: Markov Processes
Infenitisimal Generator:
Markov Process Dynamics:
This Equation is Deterministic: ODE!
1) Numerical Methods from ODEs like Runge-Kutta-Method become available for stochatic processes
2) Monte-Carlo-Sampling just at the end!
Concepts from the Theory of Dynamical Systems are Applicable
![Page 4: Numerics of Stochastic Processes II (14.11.2008)page.mi.fu-berlin.de/horenko/download/NumerikIV/VL_5.pdf1) Numerical Methods from PDEs like Finite-Element-Methods (FEM) become available](https://reader033.vdocuments.us/reader033/viewer/2022043017/5f39a9146fb2f71bb34d9a8f/html5/thumbnails/4.jpg)
Short Reminder
Stochastic Processes
Discrete State Space,Discrete Time:Markov Chain
Discrete State Space,Continuous Time:Markov Process
Continuous State Space,Discrete Time:Markov Process
Continuous State Space,Continuous Time:Stochastic Differential Equation
Today
![Page 5: Numerics of Stochastic Processes II (14.11.2008)page.mi.fu-berlin.de/horenko/download/NumerikIV/VL_5.pdf1) Numerical Methods from PDEs like Finite-Element-Methods (FEM) become available](https://reader033.vdocuments.us/reader033/viewer/2022043017/5f39a9146fb2f71bb34d9a8f/html5/thumbnails/5.jpg)
Markov Process in R
Realizations of the process:
Process:
Next-Step Probability:
Next-Step Distribution:
This Equation is Deterministic!
![Page 6: Numerics of Stochastic Processes II (14.11.2008)page.mi.fu-berlin.de/horenko/download/NumerikIV/VL_5.pdf1) Numerical Methods from PDEs like Finite-Element-Methods (FEM) become available](https://reader033.vdocuments.us/reader033/viewer/2022043017/5f39a9146fb2f71bb34d9a8f/html5/thumbnails/6.jpg)
Markov Process in R
Realizations of the process:
Process:
Next-Step Probability:
Next-Step Distribution:
This Equation is Deterministic!
![Page 7: Numerics of Stochastic Processes II (14.11.2008)page.mi.fu-berlin.de/horenko/download/NumerikIV/VL_5.pdf1) Numerical Methods from PDEs like Finite-Element-Methods (FEM) become available](https://reader033.vdocuments.us/reader033/viewer/2022043017/5f39a9146fb2f71bb34d9a8f/html5/thumbnails/7.jpg)
Infenitisimal Generator:
Markov Process Dynamics:
This Equation is Deterministic: PDE!
1) Numerical Methods from PDEs like Finite-Element-Methods (FEM) become available for stochatic processes
2) Monte-Carlo-Sampling just at the end!
Concepts from the Theory of Dynamical Systems are Applicable
Markov Process in R
![Page 8: Numerics of Stochastic Processes II (14.11.2008)page.mi.fu-berlin.de/horenko/download/NumerikIV/VL_5.pdf1) Numerical Methods from PDEs like Finite-Element-Methods (FEM) become available](https://reader033.vdocuments.us/reader033/viewer/2022043017/5f39a9146fb2f71bb34d9a8f/html5/thumbnails/8.jpg)
Short Trip in the World ofDynamical Systems
![Page 9: Numerics of Stochastic Processes II (14.11.2008)page.mi.fu-berlin.de/horenko/download/NumerikIV/VL_5.pdf1) Numerical Methods from PDEs like Finite-Element-Methods (FEM) become available](https://reader033.vdocuments.us/reader033/viewer/2022043017/5f39a9146fb2f71bb34d9a8f/html5/thumbnails/9.jpg)
Essential Dynamics of many dynamical systems is defined by attractors.
Attractor stays invariant under the
flow operator
Separation of informative (attractor) and non-informative (rest)parts of the space leads to dimension reduction
Dynamical
systems
viewpoint
Figures from http:\\w
ww
.wikipedia.org
![Page 10: Numerics of Stochastic Processes II (14.11.2008)page.mi.fu-berlin.de/horenko/download/NumerikIV/VL_5.pdf1) Numerical Methods from PDEs like Finite-Element-Methods (FEM) become available](https://reader033.vdocuments.us/reader033/viewer/2022043017/5f39a9146fb2f71bb34d9a8f/html5/thumbnails/10.jpg)
Problem: Euclidean distance
cannot be used as a measure
for the
relative neighbourghood of attractor elements.
How
to localize
the
attractor?
Attractors can have very complex, even fractal geometry
Strategy: the data have to be “embedded“
into Eucllidean space
Whitney embedding theorem (Whitney, 1936) : sufficiently
smooth connected -dimensional manifolds can be smoothly
embedded in -dimensional Euclidean space.
![Page 11: Numerics of Stochastic Processes II (14.11.2008)page.mi.fu-berlin.de/horenko/download/NumerikIV/VL_5.pdf1) Numerical Methods from PDEs like Finite-Element-Methods (FEM) become available](https://reader033.vdocuments.us/reader033/viewer/2022043017/5f39a9146fb2f71bb34d9a8f/html5/thumbnails/11.jpg)
Takens‘ Embedding (Takens, 1981)
and is a smooth map.
Assume that has an attractor with dimension
. Let , be a “proper“
measurement process. Then
is a -dimensional embedding in Euclidean space
How
to construct
the
embedding?
![Page 12: Numerics of Stochastic Processes II (14.11.2008)page.mi.fu-berlin.de/horenko/download/NumerikIV/VL_5.pdf1) Numerical Methods from PDEs like Finite-Element-Methods (FEM) become available](https://reader033.vdocuments.us/reader033/viewer/2022043017/5f39a9146fb2f71bb34d9a8f/html5/thumbnails/12.jpg)
Illustration of Takens‘
EmbeddingThe image of is unfolded in
Problems:-
how to “filter out“
the attractive subspace?
-
how to connect the changes in attractive subspace
with hiddenphase?
Strategy:
Let . Define a
new variable
. Let the attractor for
each of the hidden phases be contained in a distinct linear
manifold defined via
![Page 13: Numerics of Stochastic Processes II (14.11.2008)page.mi.fu-berlin.de/horenko/download/NumerikIV/VL_5.pdf1) Numerical Methods from PDEs like Finite-Element-Methods (FEM) become available](https://reader033.vdocuments.us/reader033/viewer/2022043017/5f39a9146fb2f71bb34d9a8f/html5/thumbnails/13.jpg)
Topological
dimension
reduction
(H. 07): for a given time series we look for a
minimum of the reconstruction error
μ
T
![Page 14: Numerics of Stochastic Processes II (14.11.2008)page.mi.fu-berlin.de/horenko/download/NumerikIV/VL_5.pdf1) Numerical Methods from PDEs like Finite-Element-Methods (FEM) become available](https://reader033.vdocuments.us/reader033/viewer/2022043017/5f39a9146fb2f71bb34d9a8f/html5/thumbnails/14.jpg)
Topological
dimension
reduction
(H. 07): for a given time series we look for a
minimum of the reconstruction error
μ
(X-μ)
TT (X-μ)T
Txt
t
t
![Page 15: Numerics of Stochastic Processes II (14.11.2008)page.mi.fu-berlin.de/horenko/download/NumerikIV/VL_5.pdf1) Numerical Methods from PDEs like Finite-Element-Methods (FEM) become available](https://reader033.vdocuments.us/reader033/viewer/2022043017/5f39a9146fb2f71bb34d9a8f/html5/thumbnails/15.jpg)
Topological
dimension
reduction
(H. 07): for a given time series we look for a
minimum of the reconstruction error
μ
(X-μ)
TT (X-μ)T
Txt
t
t
![Page 16: Numerics of Stochastic Processes II (14.11.2008)page.mi.fu-berlin.de/horenko/download/NumerikIV/VL_5.pdf1) Numerical Methods from PDEs like Finite-Element-Methods (FEM) become available](https://reader033.vdocuments.us/reader033/viewer/2022043017/5f39a9146fb2f71bb34d9a8f/html5/thumbnails/16.jpg)
Analysis of embedded
data
PCA + Takens‘ Embedding (Broomhead&King, 1986)
Let . Define a new variable
. Let the attractor
from Takens‘
Theorem be in a linear manifold defined via
. Then
Reconstruction from m
essential coordinates:
![Page 17: Numerics of Stochastic Processes II (14.11.2008)page.mi.fu-berlin.de/horenko/download/NumerikIV/VL_5.pdf1) Numerical Methods from PDEs like Finite-Element-Methods (FEM) become available](https://reader033.vdocuments.us/reader033/viewer/2022043017/5f39a9146fb2f71bb34d9a8f/html5/thumbnails/17.jpg)
PCA+Takens: data
compression/reconstruction
1.
2.
3.
4.
5.
![Page 18: Numerics of Stochastic Processes II (14.11.2008)page.mi.fu-berlin.de/horenko/download/NumerikIV/VL_5.pdf1) Numerical Methods from PDEs like Finite-Element-Methods (FEM) become available](https://reader033.vdocuments.us/reader033/viewer/2022043017/5f39a9146fb2f71bb34d9a8f/html5/thumbnails/18.jpg)
0
10
20
30
40
50
−20−10
010
2030
−40
−20
0
20
40
Lorenz-Oszillator with measurement noise
Example: Lorenz
![Page 19: Numerics of Stochastic Processes II (14.11.2008)page.mi.fu-berlin.de/horenko/download/NumerikIV/VL_5.pdf1) Numerical Methods from PDEs like Finite-Element-Methods (FEM) become available](https://reader033.vdocuments.us/reader033/viewer/2022043017/5f39a9146fb2f71bb34d9a8f/html5/thumbnails/19.jpg)
Example: Lorenz
Reconstructed trajectory(red, for m=2)
![Page 20: Numerics of Stochastic Processes II (14.11.2008)page.mi.fu-berlin.de/horenko/download/NumerikIV/VL_5.pdf1) Numerical Methods from PDEs like Finite-Element-Methods (FEM) become available](https://reader033.vdocuments.us/reader033/viewer/2022043017/5f39a9146fb2f71bb34d9a8f/html5/thumbnails/20.jpg)
1) Markov-Prozesse (I): Dynamische Systeme und Identifikation der metastabilen Mengen (Maren)
2) Markov-Prozesse (II): Generatorschätzung (??)
3) Large Deviations Theory (Olga)
4) Dimensionsreduktion in Datenanalyse: graphentheoretische Zugänge I (Paul)
5) Dimensionsreduktion in Datenanalyse: graphentheoretische Zugänge II(Ilja)
6) Prozesse mit Errinerung (I): opt. Steuerung, Kalman-Filter und ARMAX-Model (??)
7) Prozesse mit Errinerung (II): Change-Point analysis (Peter)
8) Varianz von Parameterschätzungen (Bootstrap-Algorithmus) (Beatrice)
9) Modelvergleich in Finanzsektor (Lars)
Themen