suggested reading relevant journal articles -...

NONLINEAR DYNAMICS AND CHAOS

Patrick E McSharry

Systems Analysis, Modelling & Prediction Group

www.eng.ox.ac.uk/samp

[email protected]

Tel: +44 20 8123 1574

Trinity Term 2007, Weeks 3 and 4

Mondays, Wednesdays & Fridays 09:00 - 11:00

Seminar Room 2

Mathematical Institute

University of Oxford

Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.1

LECTURE 1: INTRODUCTION

1. Introduction

2. Maps

3. Flows

4. Fractals and Attractors

5. Bifurcations

6. Quantifying Chaos

7. Nonlinear Time Series Analysis

8. Nonlinear Modelling and Forecasting

9. Real-World Applications

10. Weather Forecasting

11. Biomedical Models

12. Time Series Analysis Workshop


Suggested Reading

Strogatz, S. H., Nonlinear Dynamics and Chaos: With Applications to Physics, Biology,Chemistry, and Engineering, Reading, MA: Addison-Wesley (1994)

Eubank, S., and D. Farmer, An introduction to chaos and randomness. In Jen, E. (Ed.), 1989Lectures in Complex Systems. Santa Fe Institute Studies in the Sciences of Complexity,Lecture Vol. II, pp. 75-190. Reading, MA: Addison-Wesley, (1990)

Ott, E. and Sauer, T. and Yorke, J. Coping with Chaos, J. A. John Wiley & Sons, New York(1984)

Ott, E., Chaos in Dynamical Systems, Cambridge: Cambridge University Press (1993)

Schuster, H.G., Deterministic Chaos: An Introduction, VCH, (1995).

Wiggins, S., Introduction to Applied Nonlinear Dynamical Systems and Chaos, Springer(1990)

Kantz, H. and Schreiber, T., Nonlinear Time Series Analysis, Cambridge Univ. Press (1997)

Abarbanel, H.D.I, Analysis of Observed Chaotic Data, Springer, (1996)


Relevant journal articles

Yorke, J. and Li, T. Y., Period Three Implies Chaos, American Mathematical Monthly82:985-992 (1975)

May, R., Simple mathematical models with very complicated dynamics. Nature 261:459-467 (1976)

Packard, N. and Crutchfield, J. and Farmer, J. D. and Shaw, R., Geometry from a timeseries, Phys. Rev. Lett. 45: 712-716 (1980)

Crutchfield, J. P, N. H. Packard, J. D. Farmer, and R. S. Shaw. (1986) Chaos, ScientificAmerican 255:46-57 (1996)


Sources of information

Journals:Physica DPhysical Review LettersPhysical Review EPhysics Letters A

International Journal of Bifurcation and Chaos

Online discussion groups:

sci.nonlinearwww.jiscmail.ac.uk/lists/allstat.html

www.jiscmail.ac.uk/lists/timeseries.html

Websites:www.societyforchaostheory.org (Society for chaos theory)

www.physionet.org (MIT-Harvard biomedical database and tools)

www.comdig.org (Complexity Digest)


Suggested special topics

Look for evidence of low-dimensionality in real time series

Investigate the predictability of a real time series using nonlinear methods

Available time series:electronic circuitslaserssunspot record

electricity data (demand, price, grid frequency)

weather data (temperature, precipitation, wind speed)

economic data (GDP, inflation, interest rates, unemployment)

financial data (stockmarket tick data, S&P500, Dow Jones, FTSE100)

biomedical signals (ECG, EEG, blood pressure, respiration, blood gases)

Construct and investigate a nonlinear model of a particular system

Compare nonlinear methods for forecasting, (e.g. RBFs vs. local linear)

Develop new methods for classification of health/disease using biomedical signals

TISEAN package

Matlab software


Special topic instructions

Should be one week of work

Marking scheme:-

content 20 ptspresentation 5 pts

Two copies of final report required


Linear analysis

Definition of linearity:

L(ax) = aL(x)

L(x + y) = L(x) + L(y)

Principle of superposition: If x and y are solutions, then z = ax + by is also a solution

Advantages of linear models:

Often have analytical solutions

A large body of historical knowledge for helping with model specification and estimation

Less parameters - smaller chance of overfitting

Can employ Fourier spectral analysis and associated techniques

Disadvantages of linear models:

Real-world systems are usually nonlinear

Linearity is a first order approximation and neglects higher orders

Stanislaw Ulam: nonlinear science is like non-elephant zoology

In practice, while underlying dynamics may be nonlinear, observed data may only providesufficient resolution for linear models

Need relevant null hypothesis tests for nonlinearity (surrogate data)


Normal distributions

Mean x0 and variance σ2:

p(x) =1√

2πσ2exp

»−(x − x0)2

2σ2

–

All higher order moments are given in terms of x0 and σ

Easily manipulated: if x ∼ N(0, σ2x) and y ∼ N(0, σ2

y), then x + y ∼ N(0, σ2x + σ2

y)

Central limit theorem: sum of a large number of IID random variables (with finite mean andvariance) is normally distributed

For linear models:Normal distributions are preserved by principle of superposition

Normally distributed forecast errors: Maximum likelihood gives least squares

Useful for calculating prediction intervals

Problems for nonlinear systems:

Use of normal distributions neglects possibility of asymmetric distributions

Fat tailed distributions imply larger probability of worse case scenarios (riskmanagement)


Dynamical systems I

Variables Linear Nonlinear

n=1 Growth, decay, equilibrium

Exponential growth Fixed points

RC circuit Bifurcations

Radioactive decay Overdamped systems, relaxational dynamics

Logistic equation for single species

n=2 Oscillations

Linear oscillator Pendulum

Mass and spring Anharmonic oscillator

RLC circuit Limit cycles

2-body problem Biological oscillators

Predator-prey cycles

Nonlinear electronics (van Der Pol)

n ≥ 3 Chaos

Civil engineering Strange attractors (Lorenz)

Electrical engineering 3-body problem (Poincar e)

Chemical kinetics

Iterated maps (Feigenbaum)

Fractals (Mandelbrot)

Forced nonlinear oscillators (Levison, Smale)


Dynamical systems II

Variables Linear Nonlinear

n À 1 Collective phenomena

Coupled harmonic oscillators Coupled nonlinear oscillators

Solid-state physics Lasers, nonlinear optics

Molecular dynamics Nonequilibrium statistical mechanics

Equilibrium statistical mechanics Nonlinear solid-state physics

Heart cell synchronisation

Neural networks

Economics

Continuum Waves and patterns Spatio-temporal complexity

Elasticity Nonlinear waves (shocks, solitons)

Wave equations Plasmas

Electromagnetism (Maxwell) Earthquakes

Quantum mechanics (Schr odinger, Heisenberg) General relativity (Einstein)

Heat and diffusion Quantum field theory

Acoustics Reaction-diffusion, biological waves

Viscous fluids Fibrillation

Epilepsy

Turbulent fluids

Life


Napolean’s Army’s Russian Campaign

Map drawn by the French engineer Charles Joseph Minard in 1861 to show the tremendouslosses of Napolean’s army during his Russian Campaign of 1812


Happiness time evolution

From Culture and Subjective Well-being, edited by Ed Diener & Eunkook M. Suh (2002)


Happiness versus GDP

From Culture and Subjective Well-being, edited by Ed Diener & Eunkook M. Suh (2002)Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.14

Time series from nonlinear systems

A

B

C

D

E

F

G


Mathematical characterisation

Time series: variables are recorded as a function of time

Steady state: a constant solution of a mathematical equatione.g. Homeostasis: relative constancy of the internal environment with respect to variablessuch as blood sugar, blood gases, blood pressure and pH. Control mechanisms constrainvariables to narrow limits. e.g. Following a hemorrhage, reflex mechanisms quickly restoreblood pressure to equilibrium values.

Oscillations: periodic solutions of mathematical equationse.g. Heartbeat, respiration, sleep-wake cycles and reproduction

Irregular activity: intrinsic fluctuations, can even be present when external parameters arerelatively constant. Two distinct mathematical descriptions: noise and chaos

Noise:Variability cannot be linked with any underlying stationary or periodic process

e.g. Fluctuating environment: eating, exercise, rest and posture affects heart rate, bloodpressure, blood-sugar levels and insulin levels

e.g. Respiratory Sinus Arrhythmia (RSA): heart rate increases during inspiration

Chaos:Irregularity that arises in a deterministic system

Chaos can exist without influence of external noise


Noise versus Chaos

Inter-event time sequences (e.g. heartbeat, neurons firing)

Believed to be a random process

Simplest model for such a random process is a Poisson process:

Probability of event to occur in a very short time increment dt is Rdt

The probability R is independent of the previous history

Probability of two or more events occurring during dt is negligible

Probability of k events in time interval t is Pk(t) =(Rt)k

k!e−Rt (Poisson distribution)

Probability that interval between event and (k + 1)st following event is

pk(t) =R(Rt)k

k!e−Rt (PDF of Poisson process)

Average time between events is 1/R and variance is 1/R2

Nonlinear map, ti+1 = −(1/R) ln |1 − 2 exp(−Rti)| also gives p(t) = Re−Rt

Observation of an exponential probability density is not sufficient to identify a Poissonprocess!

Use recurrence plots (e.g. ti+1 versus ti) to identify structural equations


An exponential distribution, but not a Poisson process

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.5

1

1.5

2

ln2/3 ti

ti+1

0 10 20 30 40 50 60 70 80 90 1000

0.2

0.4

0.6

0.8

1

1.2

1.4

i

ti

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.5

1

1.5

2

2.5

3

3.5

4

t

p(t)


Determinism and Predictability

After Newton people believed in a deterministic, and hence, predictable Universe“Given for one instant an intelligence which could comprehend all the forces by which natureis animated and the respective situation of the beings who compose it—an intelligence andsufficiently vast to submit these data to analysis—it would embrace in the same formula themovements of the greatest bodies of the universe and those of the lightest atom; for it,nothing would be uncertain and the future, as the past, would be present before its eyes.”[P.S. Laplace, 1814]

“Laplacian dream” excludes stochastic laws of physics

Laplace acknowledged that we would never achieve the “intelligence” required—a tacitappreciation that deterministic systems might not, in practice, be predictable

deterministic 6= predictable

Laplace saw probabilities as a way to describe our ignorance of a deterministic system

Analytic expediency means most mathematics revolves around linear systems


Poincaré

After tackling the 3-body problem Poincarè identified the phenomenon of sensitivedependence on initial conditions (SDIC), this provided a definition of “chaos”

“If we knew exactly the laws of nature and the situation of the universe at the initial moment,we could predict exactly the situation of that same universe at a succeeding moment. Buteven if it were the case that the natural laws had no longer any secret for us, we could stillonly know the initial situation approximately. If that enabled us to predict the succeedingsituation with the same approximation, that is all we require, and we should say that thephenomenon had been predicted, that is is governed by laws. But it is not always so; it mayhappen that small differences in the initial conditions produce very great ones in the finalphenomena. A small error in the former will produce an enormous error in the latter.Prediction becomes impossible, and we have the fortuitous phenomenon.”[H. Poincaré,1903]


Lorenz’s butterfly effect

0 1 2 3 4 5 6 7 8−20

−15

−10

−5

0

5

10

15

20

time [secs]

x(t)


Sensitive dependence on initial condition

−20 −15 −10 −5 0 5 10 15 200

5

10

15

20

25

30

35

40

45

50

x

z

Perfect model and perfect knowledge of observational uncertainty

Predictability varies with position


Chaos

Advent of digital computer allowed numerical investigation of nonlinear equations

Lorenz found SDIC in a numerical model of the atmosphere and constructed the “Lorenzsystem” to illustrate the effect in a simple system [1963]

Yorke and Li coined the word “chaos” in 1975

May demonstrates chaos in the one-dimensional Logistic map in 1976

Chaos becomes trendy, “Chaos” is published by James Gleick, 1987

Claims of chaos in the brain, heart, economy, stockmarket, ...

Investigations of nonlinear dynamical systems, claims of chaos are played down!


Two faces of Chaos

The word chaos refers to disorder and extreme confusion

To a scientist, it implies “deterministic disorder”

This might suggest that a chaotic system should be unpredictable

On the contrary, a chaotic deterministic system is, in principle, perfectly predictable

The sensitive dependence of the system dynamics to the initial conditions (SDIC) impliesthat, in reality, any error in specifying the initial condition will lead to an erroneous prediction

Laplace suggests using probabilistic predictions to overcome the problems, of divergingtrajectories, posed by chaotic systems

Chaos is sometimes used as a scapegoat: meteorologists blame chaos for inaccuratepredictions when it is often model inadequacy that is at fault

Important research: separating model inadequacy (structural and parametrical errors) fromeffects of observational uncertainty


Deterministic versus Stochastic

stochastic systemxt+1 = axt + εt εt : N (0, 1)

deterministic systemyt+1 = ayt + σzt

zt+1 = 4zt(1 − zt)


Deterministic versus Stochastic

Deterministic : Everything is described by a single point in state space. This descriptioncompletely determines the future.

Stochastic : Knowledge of present states does not determine the evolution of future states.

Can you tell if a system is stochastic or deterministic?

Should a system be modelled as stochastic or deterministic?

High dimensional deterministic chaotic system might be modelled as a stochastic system

State space trajectory of an autonomous, deterministic system never crosses itself

Sources of forecast uncertainty:Uncertainty in the initial condition

Model inadequacy (parametrical and structural)


Introduction to Dynamical Systems

A state is an array of numbers that provides sufficient information to describe the futureevolution of the system.

If m numbers are required, then these form an m-dimensional state vector x.

The collection of these state vectors defines an m-dimensional state space.

The rule for evolving from one state to another may be expressed as a discrete map or acontinuous flow:

map xt+1 = F (xt)

flow x(t) = f(x(t))

Fixed point of a map: x0 = F (x0)

Fixed point of a flow: x0 = f(x0) = 0

Non-autonomous system: x = f(x, t)

Autonomous system: x = f(x)

If the non-autonomous nature is due to periodic terms it can be made autonomous

Dissipative flow: ∇ · f < 0 implies contracting state space volume

Hamiltonian systems are non-dissipative or conservative, preserving state space volume(Liouville theorem)


Properties of dynamical systems

Determinism: trajectories should not diverge when going forward in time

Invertibility: A dynamical system is invertible if each state x(t) has a unique predecessorx(t − 1). This implies that trajectories should never merge.

Thus continuous deterministic flows are always invertible!

Maps derived from flows (Poincaré maps) are also invertible

Reversibility: if the dynamical system obtained by the transformation t → −t is equivalent tothe original one

Invariance under coordinate transforms: an invariant of a dynamical system represents afundamental property of that dynamical system, e.g. dimensions and Lyapunov exponents

System invariant offer a means of summarising the behaviour of a particular system: (e.g.health and disease)


Simple Pendulum

An example of a simple two-dimensional dynamical system

From Newtons’s second law, knowledge of the forces, position and velocity are sufficient todetermine future motion

Pendulum (constrained to move in the plane)

Dynamics fully specified by the displacement angle θ(t) and the angular velocity θ(t)

State vector given by x(t) = [θ(t), θ(t)]

Let m be the mass of the pendulum

g is the acceleration due to gravity

l is the length of the pendulum

Tangential restoring force due to gravity: −mg sin θ

Tangential force due to angular acceleration: mlθ

In the absence of friction, dynamics are governed by

d

dtθ = θ

d

dtθ = −g

lsin θ


Fixed Points

Consider the fixed point of a flow f(x0) = 0

Let x(t) = x0 + ε(t)

ε = f(x0 + ε)

ε = f(x0) + Dxf(x0)ε + O(||ε||2)

ε ≈ Dxf(x0)ε = Jε

J is Jacobian matrix of partial derivatives

The solution isε(t) = eJtε0

Let λi be (distinct) eigenvalues of J

P−1JP = Λ

where Λii = λi and Λij = 0 if i 6= j

Let ε = Py so

y = eΛty0


Classification of fixed points

<(λi) 6= 0 for all i: hyperbolic fixed point

Otherwise non-hyperbolic fixed point

Hyperbolic fixed points:

sinks: all <(λi) < 0

sources: all <(λi) > 0

saddle point: some positive and some negative

If =(λi) = 0: node

If =(λi) 6= 0: focus

Non-hyperbolic fixed points:

if <(λi) > 0 for some i: unstable

some <(λi) < 0 some = 0: neutrally stable

all <(λi) = 0: centre


suggested reading relevant journal articles -...

Documents