robert n. shorten hamilton institute modelling of dynamic systems: part 2 solving differential...
Post on 17-Dec-2015
213 Views
Preview:
TRANSCRIPT
Robert N. Shorten Hamilton Institute
Modelling of dynamic systems: Part 2
Solving differential equations using computers
Robert N. Shorten & Douglas LeithThe Hamilton Institute
NUI Maynooth
Hamilton Institute
Contacting me?
• How can you contact me? – email: robert.shorten@may.ie
– My office is in the Hamilton Institute
• When can you ask me questions?– During lectures
– Contact hours (any time)
• Books– Booklist on the web
(www.hamilton.may.ie/~course/)
Hamilton Institute
Overview• Lecture 1: Introduction
– What are ordinary differential equations?– Why are they important?– Why are they difficult to solve?
• Lecture 2: Solving differential equations– Euler’s method
• Lecture 3: Solving differential equations– Runge Kutta method
• Lecture 4: Systems of differential equations, stiff systems, and some examples
Introduction
Hamilton Institute
Mathematics!
Show no fear - Maths is just another language!!
Relax, don’t panic, and think about the equation.
Pay attention to detail and pay particular attention to Theorems
and to the proofs
Hamilton Institute
Building our first model
• Example: Malthus’s law of population growth
Government agencies use population models to plan.
What do you think be a good simple model for population growth?
Malthus’s law states that rate of an unperturbed population (Y) growth is proportional to the population present.
Introduction
kYdtdY
Hamilton Institute
1800 1820 1840 1860 1880 1900 1920 1940 1960 19800
50
100
150
200
250
YEAR
Pop
US Population Growth (millions) v. Year
Hamilton Institute
1800 1820 1840 1860 1880 1900 1920 1940 1960 19800
50
100
150
200
250
YEAR
Pop
US Population Growth (millions) v. Year
1800 1820 1840 1860 1880 1900 1920 1940 1960 19801.5
2
2.5
3
3.5
4
4.5
5
5.5
Slope = k
Intercept = ey0
YEAR
ln(Pop)
Hamilton Institute
1800 1820 1840 1860 1880 1900 1920 1940 1960 19800
50
100
150
200
250
300
350
YEAR
Pop
US Population Growth (millions) v. Year
MODEL
Hamilton Institute
Modelling
• Modelling is usually necessary for two reasons: to predict and to control. However to build models we need to do a lot of work.
– Postulate the model structure (most physical systems can be classified as belonging to the system classes that you have already seen)
– Identify the model parameters (later);
– Validate the parameters (later);
– Solve the equations to use the model for prediction and analysis (now);
Introduction
Hamilton Institute
Introduction: How do we predict?
• To predict we need to solve our equations:– continuous time systems; discrete time systems; and hybrid systems.
– Sometimes, when we are very lucky, we can find an analytic solution to these equations.
– More often, an analytic solution cannot be found, and we must use a computer to approximate the solution to the equation.
– This is usually not a problem for discrete time systems. However, for continuous systems, modelled using differential equations, life is not so simple. Solving differential equations is the subject of the next four lectures.
Introduction
Hamilton Institute
What are differential equations?
• Calculus, the science of change, was invented by Newton to help him in his investigations of natural phenomena.
• He discovered that lots of systems evolve according to rate equations.
• Example: Application of Newtons law of cooling
Experiments show that the time rate of change of temperature of an object and it’s surrounding is proportional to the difference between the temperature of the object and the surrounding medium:
Introduction
)TT(kdtdT
0
Hamilton Institute
What are differential equations?
• Calculus, the science of change, was invented by Newton to help him in his investigations of natural phenomena.
• Many other types of systems can be modelled by writing down an equation for the rate of change of some variable:
– bandwidth utilisation in TCP networks;
– acceleration of car;
– mortgage repayments;
– population increases;
– chemical change of some kind;
– flight of a football;
• All of the above behaviour can be captured by very simple differential equations
Introduction
Hamilton Institute
• The previous examples are examples of ordinary differential equations.
• Mathematically speaking, an ODE is an equation relating the derivatives of an unknown function of an dependent variable with known functions of an independent variable.
• ODE’s are used to model many physical systems and are therefore capable of predicting the behaviour of a system for all time into the future, only requiring an input and some activation.
• Consider a net constant longtitudinal force F acting on a body of mass M. From Newton’s
laws: F = Ma
What are ODE’s?
Introduction
dtMF
)t(vt
t0
Hamilton Institute
What is a differential equations telling us to do?
• Notice that the velocity at time t depends on the initial velocity.
• Sometimes we are asked to find the solution of an ODE that satisfies some initial value: this is an initial value problem.
• More often we are interested in the qualitative behaviour of the system for all initial conditions.
Introduction
tMF
V)t(v 0
Hamilton Institute
Can’t we solve them analytically?
• Many differential equations can be solved analytically. In fact, the theory underlying linear time-invariant systems is well-understood. Most linear systems can be modelled using linear differential equations of the following form
• Linearity means that the solution is specified by an equation involving only linear combinations derivatives of the dependent variable with respect to the independent variable. (the order of the system is given by the power of the highest derivative).
Introduction
M
k k
k
k
N
k k
k
k )t(xdtd
b)t(ydtd
a00
Hamilton Institute
Linear time-invariant ODE’s
Consider the RL circuit depicted below. The source is a
sinusoid of frequency w and of unit amplitude.
Introduction
)tsin(dtdi
LRi ωV(t)
R
i(t)
Hamilton Institute
Can’t we solve them analytically?
• It is convenient to express the output of a system, modelled by a differential equation, as the sum of two components: one associated with no input; the other associated with the input to the system.
• The first component is called the natural response of the system and is the system output when the input is zero.
• The second component is the component of the output due to the input only and is known as the forced response of the system.
Introduction
Hamilton Institute
The natural response
Consider the RL circuit depicted below. The source is a
sinusoid of frequency w and of unit amplitude.
The natural response is obtained by setting the input to zero.
Introduction
)tsin(dtdi
LRi ω
tRL
n ke)t(idtdi
LRi
0
V(t)
R
i(t)
Hamilton Institute
The forced response
The forced response is obtained by setting the input to zero.
Introduction
)tsin(dtdi
LRi ω
)tsin()tsin()t(i f ωβωα
V(t)
R
i(t)
Hamilton Institute
The combined response
What is the combined response if the current is 2 amps when t =0,
and if = R=L=1?
Introduction
)tsin(dtdi
LRi ω
tLR
ke)tsin()tsin(i
ωβωα
V(t)
R
i(t)
Hamilton Institute
The combined response
Introduction
tLR
ke)tsin()tsin()t(i
ωβωα
0 2 4 6 8 10 12 14 16 18 20-1
-0.5
0
0.5
1
1.5
2
t
in(t)
if(t)
i(t) i(t)
Hamilton Institute
What do we do when analytical solutions cannot be found?
• Mathematical analysis is not the grand thing it is made out to be!
• Maths solves only the simplest equations. As soon as the equations become a little more complicated, we are in trouble.
• Analytical solutions to differential equations arising in the real world can only rarely be found. Linear time-invariant systems are well understood, but most real systems are time-varying, non-linear and hybrid. In such cases we must use numerical techniques and a computer to investigate the system under study.
Introduction
Hamilton Institute
What do we do when we cannot find a soln?
• We begin our study of numerically approximating differential equations with
• What is this equation telling us to do?
• The equation is saying the following. Take any point in the x-t plane. If a solution to the above equation passes through this point then the tangent to the solution is given by f(t,y).
Introduction
yt)y,t(fdtdy
Hamilton Institute
Numerical analysis: A direction field
Introduction
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
t
y
Direction field
Hamilton Institute
Introduction
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
t
y
Direction field for example
Hamilton Institute
Introduction
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
t
y
Direction field for example
Hamilton Institute
Introduction
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
t
y
Direction field for example
Hamilton Institute
Introduction
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
t
y
Direction field for example
Hamilton Institute
The direction field
• Another equation:
• What is this equation telling us to do?
• The equation is saying the following. Take any point in the x-y plane. If a solution to the above equation passes through this point then the tangent to the solution is given by f(x,y).
Introduction
yxyx
)y,x(fdxdy
Hamilton Institute
Numerical analysis: A direction field
Introduction
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
X1
X 2
Direction field for Example 1
Hamilton Institute
Introduction
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
X1
X 2
Direction field for Example
Hamilton Institute
Introduction
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
X1
X 2
Direction field for Example
Hamilton Institute
Introduction
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
X1
X 2
Direction field for Example
Hamilton Institute
Introduction
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
X1
X 2
Direction field for Example
Hamilton Institute
Numerical analysis: Direction fields
• We have depicted graphically the directions in which solutions should proceed.
• Direction fields generate the flow associated with differential equations.
• For such a plot to make sense, the function f(x,y) should satisfy certain properties: it should be defined in some region of the plane; it should be single valued in this region; and solutions to the equation in this region should be differentiable.
Introduction
Hamilton Institute
Numerical analysis: Word of warning
• We have been a little bit hard on mathematics. However, we know from maths that a solution to a differential equation need not always exist. It need not even be unique.
Introduction
• Example: Does a solution to the following equation exist?
• Example: Is the solution to the following equation unique?
1002
2
)(y,y
dtdy
101 )(y,ydtdy
t
Hamilton Institute
Numerical analysis: Word of warning
• We have been a little bit hard on mathematics. However, we know from maths that a solution to a differential equation need not always exist. It need not even be unique.
• Computers don’t care. Always be careful about the output of a program. Generally, the computer will give you an answer; not necessarily the one that you want.
• Use mathematical theory to verify simulation. In particular, verify uniqueness and existence conditions are satisfied.
Introduction
Hamilton Institute
Existence and uniqueness
Introduction
• Existence and uniqueness theorems for ordinary differential equations. You should have done them.What are they?
• Roughly speaking: if f(t,y) is continuous in some region of the t-y plane then at least one solution to the differential equation exists; if the partial derivative of f w.r.t y exists and is continuous then the solution is unique.
• Revise the concept of Lipschitzness and the existence and uniqueness theorems for ODE’s from part one of the course (if it has been covered).
Robert N. Shorten Hamilton Institute
Modelling of dynamic systems: Part 2
Euler’s method
Robert N. Shorten & Douglas LeithThe Hamilton Institute
NUI Maynooth
Hamilton Institute
Overview• We have already spoken about the poor job that mathematics has done in
studying differential equations.
• Most engineering and scientific problems involve differential equations that cannot be solved analytically.
• We can use the direction field to get a solution if the order of the problem is not too high, or if there are not too many independent variables.
• What do we do in other cases?
Lecture 2
Hamilton Institute
Euler’s method• There are many methods for solving ODE’s numerically. We
begin with the simplest and most intuitive method: Euler’s method.
• Euler’s method is the easiest algorithm for numerically solving a differential equation.
• Unfortunately, it is useless. Nevertheless, it is useful to illustrate some pertinent points and introduce numerical methods. Let us assume that we are dealing with a differential equation that satisfies our existence and uniqueness conditions.
Lecture 2
)y,t(fdtdy
Hamilton Institute
Euler’s method
• Then, using Taylor’s theorem, one can write any function about t0 a
point as:
• Terms left out of the expansion are of the form
where n is greater than 2. Hence, for h = t-t0 small enough the
function can be approximated:
Lecture 2
......)tt(dtdy
)t(y)t(yt
00
0
n)tt( 0
hdtdy
)t(y)t(yt
0
0
Hamilton Institute
Euler’s method
• In other words, for small h, function y(t) can be approximated from knowledge of the function and derivative at t0.
• This observation suggests the following approximation to the solution to a differential equation. Suppose that we want to approximate the solution to a well posed initial value problem over the interval [a,b]. To construction this solution we divide the interval into a number of mesh points:
Lecture 2
Mab
h,M....k,khatk
0
Hamilton Institute
Euler’s method
• At the first mesh point we can approximate the solution as:
• The beauty of Euler’s method is that this procedure can be repeated at intermediate mesh points.
Lecture 2
)y,t(hf)t(y
dtdy
h)t(y)t(yt
000
01
0
)y,t(hf)t(y)t(y kkkk 1
Hamilton Institute
Example
• Use Eulers method to solve approximately the initial value problem
over the interval t in the interval [0,2] with y(t0) = 1. Use a step size
of 0.2.
Lecture 2
yt)y,t(fdtdy
k t y_eul y_act
0 0 1.0000 1.0000 1.0000 0.2000 0.8000 0.8375 2.0000 0.4000 0.6800 0.7406 3.0000 0.6000 0.6240 0.6976 4.0000 0.8000 0.6192 0.6987 5.0000 1.0000 0.6554 0.7358 6.0000 1.2000 0.7243 0.8024 7.0000 1.4000 0.8194 0.8932 8.0000 1.6000 0.9355 1.0038 9.0000 1.8000 1.0684 1.1306 10.0000 2.0000 1.2147 1.2707
Hamilton Institute
Lecture 2
k t y_eul y_act 0.0000 1.0000 2.0000 3.0000 4.0000 5.0000 6.0000 7.0000
8.0000 9.0000 10.0000 0.0000 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.0000 1.0000 1.1000 1.2100 1.3310 1.4641 1.6105 1.7716 1.9487 2.1436 2.3579 2.5937 1.0000 1.1002 1.2013 1.3042 1.4097 1.5185 1.6312 1.7484 1.8707 1.9984 2.1321
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
t
y
Hamilton Institute
Lecture 2
k t y_eul y_act 0.0000 1.0000 2.0000 3.0000 4.0000 5.0000 6.0000 7.0000
8.0000 9.0000 10.0000 0.0000 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.0000 1.0000 1.1000 1.2100 1.3310 1.4641 1.6105 1.7716 1.9487 2.1436 2.3579 2.5937 1.0000 1.1002 1.2013 1.3042 1.4097 1.5185 1.6312 1.7484 1.8707 1.9984 2.1321
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
t
y
Direction field for example
Hamilton Institute
Lecture 2
k t y_eul y_act 0.0000 1.0000 2.0000 3.0000 4.0000 5.0000 6.0000 7.0000
8.0000 9.0000 10.0000 0.0000 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.0000 1.0000 1.1000 1.2100 1.3310 1.4641 1.6105 1.7716 1.9487 2.1436 2.3579 2.5937 1.0000 1.1002 1.2013 1.3042 1.4097 1.5185 1.6312 1.7484 1.8707 1.9984 2.1321
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
t
y
Direction field for example
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
t
y
Direction field for example
Hamilton Institute
Euler’s method: Geometric interpretation
y
t
y(t)
h 2h 3h 4h
• Errors accumulate over time!!!!!
Hamilton Institute
Euler’s method
• What happens if the step size is decreased? What happens if the step size is increased?
0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
0.6
0.7
0.8
0.9
1
1.1
1.2
1.3
t
yDirection field for example
Hamilton Institute
Euler’s method
• What happens if the step size is decreased? What happens if the step size is increased?
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
t
y
Direction field for example
Hamilton Institute
Euler’s method: Errors
• We have seen that our approximation to the solution to the differential equation is not exact.
• Errors are introduced into the procedure as a result of two mechanisms
– Round-off
– Discretisation errors.
• Round-off errors arises from numbers being stored in computers using finite precision. This is usually not a concern as far more error is introduced as a result of truncation.
Hamilton Institute
Euler’s method: Local truncation error
• There are two types of truncation error: local truncation errors; and global truncation errors. The local truncation error is the error after just one step (show):
• Hence:
....dt
ydh)t(y)ht(y
ix
ii 2
22
1 21
)h(O
)t(y)ht(y iik
2
1ε
Hamilton Institute
Global discretisation error and convergence
• The global error at a certain value just what we would ordinarily call the error: the difference between the true value and the approximation. Roughly speaking, the global error can be written:
• So halving the step size should, roughly speaking, halve the global discretisation error.
)h(O
hdt
ydhe
ix
k
i
λ21
2
22
Hamilton Institute
Euler’s method: Errors
• For the previous example:
Step size Global truncation error10 0.062220 0.029040 0.014080 0.0069
Hamilton Institute
Improving Euler’s method
• The modified Euler’s method is based on including more terms in the Taylor expansion.
• We can approximate the first two terms in the expansion. How do we get the higher order term? We approximate this term as:
......dt
ydhdtdy
h)t(y)t(yt
2
22
01 20
h)y,t(f))t(y,t(f
h
dtdy
dtdy
dtyd
*
tt
00111
2
2
01
Hamilton Institute
Improving Euler’s method
• With this approximation for the second derivative we obtain the following approximation:
• It can be shown that for this method:
))y,t(f)y,t(f(h
yy kk
*
kkkk 111 2
)h(O
hdt
ydKhe
ix
k
i
2
2
3
33 λ
Hamilton Institute
Improving Euler’s method
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
t
y
Direction field for example
Hamilton Institute
Improving Euler’s method
• Investigate the effect of step-size on the accuracy of the solution.
• Show for the example that the global truncation error is approximately
• What order is the local truncation error?
)h(O 2
Robert N. Shorten Hamilton Institute
Modelling of dynamic systems: Part 2
Runge-Kutta method’s
Robert N. Shorten & Douglas LeithThe Hamilton Institute
NUI Maynooth
Hamilton Institute
Overview
• In the last lecture we have looked at the Euler method. Euler’s method is usually not good enough. The main reason is that the error is global O(h).
• An improvement in the Euler method was obtained through the modified Euler method.
• Let’s take a another approach to finding the numerical solution of the differential equation.
Lecture 2
Hamilton Institute
Second order RK method
Lecture 2
ht
tdt)t,y(f)t(y)t(y
0
0
01
t0 t0+h
f(t,y)
))y,t(f)y,t(f(h
yy kk
*
kkkk 111 2
Hamilton Institute
Fourth order RK method
• This results in the following algorithm:
Lecture 2
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
t
y
Direction field for example
))y,t(f)y,t(f(h
yy kk
*
kkkk 111 2
Hamilton Institute
Fourth order RK method
• This results in the following algorithm:
• This is the second order Runge-Kutta method. This algorithm is an improvement on what we have looked at already. However, it is based on simple trapezoidal integration. There are more sophisticated numerical integration algorithms. One of these is Simpson’s rule. Application of Simpson’s rule leads to the commonly used 4th order Runge-Kutta method.
Lecture 2
))y,t(f)y,t(f(h
yy kk
*
kkkk 111 2
Hamilton Institute
Fourth order RK method
• The fourth order Runge-Kutta yields:
• where
Lecture 2
)22(61
ˆˆ 43211 kkkkyy kk
)ˆ,(
)2
ˆ,2
(
)2
ˆ,2
(
)ˆ,(
34
23
12
1
kyhthfk
ky
hthfk
ky
hthfk
ythfk
kk
kk
kk
kk
Hamilton Institute
Example
• Use the RK method to solve approximately the initial value problem
over the interval t in the interval [0,1] with y(t0) = 1. Use a step size
of 0.1.
Lecture 2
yt)y,t(fdtdy
k t y_rk y_act
0 0 1.0000 1.0000 1.0000 0.2000 0.8375 0.8375 2.0000 0.4000 0.7406 0.7406 3.0000 0.6000 0.6976 0.6976 4.0000 0.8000 0.6987 0.6987 5.0000 1.0000 0.7358 0.7358 6.0000 1.2000 0.8024 0.8024 7.0000 1.4000 0.8932 0.8932 8.0000 1.6000 1.0038 1.0038 9.0000 1.8000 1.1306 1.1306 10.0000 2.0000 1.2707 1.2707
Hamilton Institute
Lecture 2
yt)y,t(fdtdy
k t y_eul y_act
0 0 1.0000 1.0000 1.0000 0.2000 0.8000 0.8375 2.0000 0.4000 0.6800 0.7406 3.0000 0.6000 0.6240 0.6976 4.0000 0.8000 0.6192 0.6987 5.0000 1.0000 0.6554 0.7358 6.0000 1.2000 0.7243 0.8024 7.0000 1.4000 0.8194 0.8932 8.0000 1.6000 0.9355 1.0038 9.0000 1.8000 1.0684 1.1306 10.0000 2.0000 1.2147 1.27070 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
t
y
Direction field for example
Hamilton Institute
Lecture 2
yt)y,t(fdtdy
k t y_eul y_act
0 0 1.0000 1.0000 1.0000 0.2000 0.8000 0.8375 2.0000 0.4000 0.6800 0.7406 3.0000 0.6000 0.6240 0.6976 4.0000 0.8000 0.6192 0.6987 5.0000 1.0000 0.6554 0.7358 6.0000 1.2000 0.7243 0.8024 7.0000 1.4000 0.8194 0.8932 8.0000 1.6000 0.9355 1.0038 9.0000 1.8000 1.0684 1.1306 10.0000 2.0000 1.2147 1.27070 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
t
y
Direction field for example
Hamilton Institute
Lecture 2
1.2 1.25 1.3 1.35 1.4 1.45 1.5 1.55 1.6
0.8
0.85
0.9
0.95
1
1.05
1.1
1.15
t
y
Direction field for example
Hamilton Institute
Runge-Kutta method
• What happens if the step size is decreased? What happens if the step size is increased?
Step size Global truncation error10 9.3771e-00620 5.1485e-00740 3.0130e-00880 1.8218e-009
Hamilton Institute
Fourth order Runge-Kutta method
• The fourth order Runge-Kutta is very popular for a variety of reasons.
1) It is simple to program.
2) It is very accurate. The global truncation error is O(h4). The local truncation error is
O(h5).
3) It is very efficient.
Hamilton Institute
A few words on stability
• All of the methods that we have considered were of the form:
• This is a first order difference equation and will be unstable (when)?
• If we keep the step size small enough then we can usually ensure stability.
somethingyKy kk
Hamilton Institute
Efficiency of the RK schemes
• We have said that the 4th order RK method is more efficient than other techniques that we have considered.
• How do me measure efficiency? There are two components to consider.
– Computational effort;
– Truncation errors.
• We get a rough measure of computational effort by counting how many times the function f(t,y) is evaluated as we make a prediction to a desired accuracy.
Hamilton Institute
Efficiency of the RK schemes
• The 4th order Runge Kutta method is much more efficient than the Euler method as we usually require much less computational effort to obtain the same accuracy.
• For our running example, the error of the RK4 method with a step size of 0.1 is 9.3771e-006. There total number of computations to obtain this estimate is 40. To obtain a similar error with Euler we need a step-size of 1e-5. The total computational requirement is therefor 1e5.
Scheme Computations/stepEuler 1
M Euler 2RK4 4
Hamilton Institute
Runge-Kutta-Fehlberg
• The big problem when using any method is to estimate the accuracy of the approximation (the global truncation error).
• Normally we do not have access to the solution so the problem is not straight forward.
• One approach to estimate the accuracy of the solution was proposed by Fehlberg.
Hamilton Institute
Runge-Kutta-Fehlberg
• Fehlberg’s basic idea was to run two RK methods of different order. For example, using a 4th order RK method, the error introduced by truncation may be estimated as:
• If we use a 5th order RK method:
• Hence an approximation to the truncation error (small h):
....BhAh)t(y)t(y kk 65
11
....Ch)t(y)t(y kk 6
11
5
11 )()(ˆ Ahtyty kk
Hamilton Institute
Runge-Kutta-Fehlberg
• At each time step we can estimate the truncation error. If this is too large, we simple decrease the step size. If it is too small, we can increase the step size.
• In fact, given a tolerance C, the new step size can be estimated as (show):
51
11
1 ))t(y)t(y
Tol(hh
kk
Hamilton Institute
Multi-step methods: A qualitative discussion
• All of the methods that we have considered have been of the form:
• It is of course possible to develop methods where the prediction depends on more than one previous value of prediction. These are called multi-step methods.
)y,t(gyKy kkk 1
Hamilton Institute
Adams-Bashforth Moulton method
• The ABM predictor-corrector method is a multi-step method. It is obtained by approximating the integral in the formula
by an interpolation polynomial of third degree. This polynomial is chosen so that is passes through the points
This produces an ABM prediction
1
1
k
k
t
tkk dt))t(y,t(f)t(y)t(y
)t(p k 1
210 ,,n),....,f,t( nknk
Hamilton Institute
Adams-Bashforth Moulton method
• Having obtained the prediction, a second polynomial is constructed to fit the points
• and the point
• This polynomial is then integrated to obtain the final prediction at time index k+1.
210 ,,n),....,f,t( nknk
))t(p,t( kk 11
Hamilton Institute
• The ABM formula’s are:
• The advantage of this technique is that the difference between the predictor and corrector gives an estimate of the truncation error.
Adams-Bashforth Moulton method
)ffff(h
)t(y)t(p kkkkkk 555937924 1231
)ffff(h
)t(y)t(y kkkkkk 1121 195924
Hamilton Institute
Adams-Bashforth-Moulton method
• Disadvantages of the technique include
(1) The first 4 predictions must be obtained using some other method.
(2)The technique results in a high order difference equation. Care must be taken to ensure that this
equation remains stable (zeros inside the unit circle).
Hamilton Institute
Adams-Bashforth-Moulton method
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
t
yDirection field for example
Hamilton Institute
Adams-Bashforth-Moulton method
0 2 4 6 8 10 12 14 16 18 200
2
4
6
8
10
12
14
16
18
20
t
y
Direction field for example
Hamilton Institute
Examples
• Eulers method
• Modified Euler method
• 4th order Runge-Kutta method
Robert N. Shorten Hamilton Institute
Modelling of dynamic systems: Part 2
Systems of differential equations
Robert N. Shorten & Douglas LeithThe Hamilton Institute
NUI Maynooth
Hamilton Institute
Is the world first order?
• Up to now we have considered approximating the solution to first order ordinary differential equations using numerical methods. Our methods work for non-linear systems and for time-varying systems.
• What about second order systems?
• What about higher order systems in general?
• What about systems of ODE’s?
Hamilton Institute
A system of differential equations
• The predator-prey model of Volterra (Foxes and Rabbits)
• Let x denote the rabbit population and let y denote the fox population. Volterra postulated the following. In the absence of foxes we have
• and in the absence of rabbits we have
parameter)constant some isa(,axdtdx
parameter)constant some isb(,bydtdy
Hamilton Institute
A system of differential equations
• Now let the populations of rabbits and foxes intermingle with one another. We assume that the number of contacts is proportional to the total population:
• We now have a system of differential equations.
)parametersconstant arec,a(,cxyaxdtdx
parameter)constant areb,d(,dxybydtdy
Hamilton Institute
The predator-prey model of Volterra
• Let’s assume that all parameters are unity. Assume that the initial condition is (2,4).
0 2 4 6 8 10 120
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
FOXES (y)
RABBITS (x)
VOLTERR'S MODEL
Hamilton Institute
Higher order equations
VCi
dtdi
LRi V(t)
R
i(t)
C
dtdV
Ci
dtdi
Rdt
idL
2
2
Hamilton Institute
How do we simulate complicated systems?
• Fortunately, systems of differential equations can be treated in the same way. To illustrate the concepts consider the initial value problem:
00
00
x)t(y
x)t(xwith
)t,y,x(gdtdy
)t,y,x(fdtdx
Hamilton Institute
How do we simulate complicated systems?
• Formally speaking, a solution to the system of differential equations is a pair of function f(x,y,t) and g(x,y,t) that satisfy:
• How do we numerically approximate such solutions?
00
00
x)t(y
x)t(xwith
)t,y,x(gdtdy
)t,y,x(fdtdx
Hamilton Institute
How do we simulate complicated systems?
• A numerical solution to the system of differential equations over some a<=t<=b interval is found by considering the differentials:
• Can you guess Euler’s method for systems of differential equations?
dt)y,x,t(gdy
dt)y,x,t(fdx
Hamilton Institute
Euler’s method for systems of ODE’s
• We approximate the differentials as:
• By subdividing the time interval into M sub-intervals we obtain the recursive formulae:
)tt)(y,x,t(gyy
)tt)(y,x,t(fxx
kkkk
kkkk
11
11
h)y,x,t(gyy
h)y,x,t(fxxM
abh
kk
kk
1
1
Hamilton Institute
The predator-prey model of Volterra
• Let’s assume that all parameters are unity. Assume that the initial condition is (2,4).
0 2 4 6 8 10 120
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
FOXES (y)
RABBITS (x)
VOLTERR'S MODEL
Hamilton Institute
The predator-prey model of Volterra
• We can also plot x against y - giving us the phase plane (more about this later).
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 50
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
FOXES (y)
RABBITS (x)
VOLTERRA'S MODEL
y
x
Hamilton Institute
Runge-Kutta for systems of ODE’s
• Virtually all of the ODE solvers that we have looked at can be used to solve systems of ODE’s.
• The Runge-Kutta solver for a pair of ODE’s is analagous to the solver for a first order ODE.
)gggg(h
yy
)ffff(h
xx
kk
kk
43211
43211
226
226
Hamilton Institute
Runge-Kutta for systems of ODE’s
)hgy,hfx,ht(gg),hgy,hfx,ht(ff
)gh
y,fh
x,h
t(gg),gh
y,fh
x,h
t(ff
)gh
y,fh
x,h
t(gg),gh
y,fh
x,h
t(ff
)y,x,t(gg),y,x,t(ff
kkkkkk
kkkkkk
kkkkkk
kkkkkk
332332
222222
112112
11
222222
222222
)gggg(h
yy
)ffff(h
xx
kk
kk
43211
43211
226
226
Hamilton Institute
Systems of differential equations
• Virtually all ODE solvers can be extended to solve systems of differential equations.
• We have considered (briefly) systems of two equations. Exactly the same methodology applies to systems of more than two equations.
• When comparing ODE solvers, the issues that are relevant for first order equations are also relevant for systems of equations. Can you think of any issues that are special to systems of equations?
Hamilton Institute
Higher order equations
• We have dealt with systems of equations. How do we deal with higher order differential equations?
• The equation
can also be written
i.e. as a system of differential equations.
UCi
dtdi
Rdt
idL
2
2
Udtdii
RCdt
dii
dtd 0
110
Hamilton Institute
Some more examples
• Systems of two first order equations (second order systems) are very special. Consider again:
• The plane with x and y as co-ordinates is known as the phase plane. The phase plane can be derived from the direction field of:
)y,x(gdtdy
),y,x(fdtdx
)y,x(g)y,x(f
dydx
Hamilton Institute
The phase plane
• Consider the system:
• This system is unstable.
y
x
y
x
dtd
21
12
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
X1
X 2
Phase plane for Example 1
Hamilton Institute
The phase plane
• Consider the system:
• This system is stable and has a fixed point at the origin.
y
x
y
x
dtd
12
41
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
X1
X 2
Phase plane for Example 2
Hamilton Institute
A word of warning: Stiff systems
• We have already encountered stability problems when looking at first order ODE’s.
• Roughly speaking, stability is maintained by choosing a very small step size.
• However, very small step-sizes lead to large computational burdens.
Hamilton Institute
A word of warning: Stiff systems
• Consider the differential equation:
• We can write
• This decays very fast and therefore
y
x.
y
x
dtd
100000
410
tke)t(y 1000
t.
xek)t(x 10
Hamilton Institute
A word of warning: Stiff systems
• So, given that y(t) is not that important, we should choose the step-size to approximate:
• WRONG!!!!! The stability of the numerical integration is determined by the largest eigenvalue of the system.Always choose a step-size to guarantee stability.
• This is an example of a stiff system;a number of variables evolving at different time scales. There are special solvers for such systems.
t.
xek)t(x 10
Hamilton Institute
Examples
• To finish this part of the course we give two examples where numerical solutions to systems of ODE’s are required.
1) [Physics] Modelling a pendulum
A non-linear continuous time system
2) [Computer Science] Network congestion control
A system with continuous/discrete stateA hybrid systems model (Hespanha)
Hamilton Institute
Modeling pendulums
• Pendulums appear everywhere!
mlddt
mg2
2
sin
O
mg
θ
l
Hamilton Institute
Modeling pendulums
• This is a second order differential equation. Hence, it can be written in the form.
ddt
ddt
ddt
mg
LNMM
OQPP
LNMM
OQPPsin
O
mg
θ
l
Hamilton Institute
Modeling pendulums
ddt
ddt
ddt
mg
LNMM
OQPP
LNMM
OQPPsin
O
mg
θ
l
0 5 10 15 20 25-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
Angle
Speed
Hamilton Institute
TCP congestion control
• Mathematical modelling of the internet is a hot topic!! Hopefully, we can use these models to understand what is going on and avoid congestion. Typically we model and analyse simple scenarios!
Data source 1
Data source n
Data source 2
Bottleneck link l
Router Router
Hamilton Institute
TCP congestion control
• Roughly speaking, TCP operates as follows:
– Data packets reaching a destination are acknowledged by sending an appropriate message to the sender.
– Upon receipt of the acknowledgement, data sources increase their send rate, thereby probing the network for available bandwidth, until congestion is encountered.
– Network congestion is deduced through the loss of data packets (receipt of duplicate ACK’s or non receipt of ACK’s), and results in sources reducing their send rate drastically (by half).
Hamilton Institute
TCP congestion control
• Congestion control is necessary for a number of reasons, so that:
– catastrophic collapse of the network is avoided under heavy loads;
– each data source receives a fair share of the available bandwidth;
– the available bandwidth B is utilised in an optimal fashion.
– interactions of the network sources should not cause destabilising network side effects such as oscillations or instability
Hamilton Institute
TCP congestion control
• Hespanha’s hybrid model of TCP traffic.
– Loss of packets caused by queues filling at the bottleneck link.
– TCP sources have two modes of operation
• Additive increase
• Multiplicative decrease
– Packet-loss detected at sources one RTT after loss of packet.
Data source 1
Data source n
Data source 2
Bottleneck link l
Router Router
Hamilton Institute
TCP congestion controlData source 1
Data source n
Data source 2
Bottleneck link l
Router Router
Packet notbeing
dropped
Packetsdropped
Packet dropdetected
Half source rate
Hamilton Institute
TCP congestion controlData source 1
Data source n
Data source 2
Bottleneck link l
Router Router
Queuenotfull
Queuefull
Packet dropdetected
Half source rate
Hamilton Institute
Modelling the ‘queue not full’ state
• The rate at which the queue grows is easy to determine.
• While the queue is not full:
BQ
TRTT
BRTT
wdtdQ
p
i
RTTdtdwi 1
Hamilton Institute
Modelling the ‘queue full’ state
• When the queue is full
• One RTT later the sources are informed of congestion
RTTdtdwdtdQ
1
0
Hamilton Institute
TCP congestion control
Queue fills
ONE RTT LATER
QUEUE FULL
RTTdt
dwdt
dQ
1
0
BRTT
w
dt
dQ i
RTTdt
dwi 1
ii w.w 50
Hamilton Institute
TCP congestion control: Example (Hespanha)
Seconds 40T
packets 250Q
cpackets/se1250
p
max
.
B
0 100 200 300 400 500 6000
50
100
150
200
250
300
350
400
Hamilton Institute
TCP congestion control: Example (Fairness)
Seconds 40T
packets 250Q
cpackets/se1250
p
max
.
B
0 200 400 600 800 1000 12000
100
200
300
400
500
600
700
top related