numerical analysis - roalnd pulch

8/13/2019 Numerical Analysis - Roalnd Pulch

1/187

Numerical Analysis and Simulation of

Ordinary Differential Equations

Roland Pulch

Lecture in Winter Term 2011/12

University of Wuppertal

Applied Mathematics/Numerical Analysis

Contents:

1. ODE models in science2. Short synopsis on theory of ODEs3. One-step methods4. Multi-step methods5. Integration of stiff systems6. Methods for differential algebraic equations7. Two-point boundary value problems

Literature:

J. Stoer, R. Bulirsch: Introduction to Numerical Analysis. Springer, Berlin 2002. (Chapter 7)

J. Stoer, R. Bulirsch: Numerische Mathematik 2. Springer, Berlin 2005. (Kapitel 7)

A. Quarteroni, R. Sacco, F. Saleri: Numerical Mathematics. Springer, New York 2007. (Chapter 11)


2/187

Contents

1 ODE Models in Science 1

1.1 Chemical reaction kinetics . . . . . . . . . . . . . . . . . . . 3

1.2 Electric circuits . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.3 Multibody problem . . . . . . . . . . . . . . . . . . . . . . . 8

1.4 Further models . . . . . . . . . . . . . . . . . . . . . . . . . 10

2 Short Synopsis on Theory of ODEs 12

2.1 Linear differential equations . . . . . . . . . . . . . . . . . . 12

2.2 Existence and uniqueness . . . . . . . . . . . . . . . . . . . . 13

2.3 Perturbation analysis . . . . . . . . . . . . . . . . . . . . . . 18

3 One-Step Methods 22

3.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . 22

3.2 Elementary integration schemes . . . . . . . . . . . . . . . . 23

3.3 Consistency and convergence . . . . . . . . . . . . . . . . . . 26

3.4 Taylor methods for ODEs . . . . . . . . . . . . . . . . . . . 34

3.5 Runge-Kutta methods . . . . . . . . . . . . . . . . . . . . . 36

3.6 Dense output . . . . . . . . . . . . . . . . . . . . . . . . . . 42

3.7 Step-Size Control . . . . . . . . . . . . . . . . . . . . . . . . 46

ii


3/187

4 Multistep Methods 50

4.1 Techniques based on numerical quadrature . . . . . . . . . . 504.2 Linear difference schemes . . . . . . . . . . . . . . . . . . . . 55

4.3 Consistency, stability and convergence . . . . . . . . . . . . 63

4.4 Analysis of multistep methods . . . . . . . . . . . . . . . . . 70

4.5 Techniques based on numerical differentiation . . . . . . . . 77

4.6 Predictor-Corrector-Methods . . . . . . . . . . . . . . . . . . 81

4.7 Order control . . . . . . . . . . . . . . . . . . . . . . . . . . 85

5 Integration of Stiff Systems 87

5.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

5.2 Test equations . . . . . . . . . . . . . . . . . . . . . . . . . . 90

5.3 A-stability for one-step methods . . . . . . . . . . . . . . . . 955.4 Implicit Runge-Kutta methods . . . . . . . . . . . . . . . . . 103

5.5 Rosenbrock-Wanner methods . . . . . . . . . . . . . . . . . 112

5.6 A-stability for multistep methods . . . . . . . . . . . . . . . 116

5.7 B-stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

5.8 Comparison of methods . . . . . . . . . . . . . . . . . . . . 123

6 Methods for Differential Algebraic Equations 126

6.1 Implicit ODEs . . . . . . . . . . . . . . . . . . . . . . . . . . 126

6.2 Linear DAEs . . . . . . . . . . . . . . . . . . . . . . . . . . 129

6.3 Index Concepts . . . . . . . . . . . . . . . . . . . . . . . . . 132

iii


4/187

6.4 Methods for General Systems . . . . . . . . . . . . . . . . . 139

6.5 Methods for Semi-Explicit Systems . . . . . . . . . . . . . . 1426.6 Illustrative Example: Mathematical Pendulum . . . . . . . . 148

7 Boundary Value Problems 153

7.1 Problem Definition . . . . . . . . . . . . . . . . . . . . . . . 153

7.2 Single Shooting Method . . . . . . . . . . . . . . . . . . . . 158

7.3 Multiple Shooting Method . . . . . . . . . . . . . . . . . . . 162

7.4 Finite Difference Methods . . . . . . . . . . . . . . . . . . . 167

7.5 Techniques with Trial Functions . . . . . . . . . . . . . . . . 176

iv


5/187

Chapter 1

ODE Models in Science

1

This lecture deals with the numerical solution of systems ofordinary differ-ential equations(ODEs), i.e.,

y(x) =f(x, y(x)),

or written component-wise

y1(x) = f1(x, y1(x), . . . , yn(x))y2(x) = f2(x, y1(x), . . . , yn(x))

...yn(x) = fn(x, y1(x), . . . , yn(x)).

Additional conditions are required to achieve a unique solution. On the onehand, initial value problems(IVPs) demand

y(x0) =y0

at a specific initial point x0 together with a predetermined value y0 n.Figure 1 outlines the task. On the other hand, boundary value problems(BVPs) impose a condition on an initial state as well as a final state, i.e.,

r(y(a), y(b)) = 0

with a given functionr : n n n and an interval [a, b]. For example,periodic boundary conditions read y(a) y(b) = 0.

1


6/187

y

xx0

0y

Figure 1: Initial value problem of an ODE.

An ODE ofnth order reads

z(n)(x) =g(x, z(x), z(x), z(x), . . . , z (n1)(x)).

We obtain an equivalent system of first order by arranging

y1:=z, y2:= z, y3:=z, . . . , yn:=z(n1).

It follows the system

y1=y2, y2=y3, . . . , y

n1=yn, y

n= g(x, y1, . . . , yn).

Thus we consider without loss of generality systems of first order only inthis lecture.

We start with some examples of ODE models resulting in various appli-cations ranging from science (chemical reaction kinetics) to classical me-

chanics (N-body problem) and electrical engineering (electric circuits). Inall cases, mathematical models are used to describe (approximatively) realprocesses. Due to simplifications and model assumptions, the exact solu-tion of the ODE models represents an approximation of the real process. Inmost cases, the independent variablexrepresents the time.

2


7/187

1.1 Chemical reaction kinetics

The radioactive decay represents a process depending on time. For example,the decay of a radon isotope occurs via

Rn-222 Po-218 + He-4 (-particle)with the rate T1/2 = 3.825 days. Let n be the number of particles of theisotope. The corresponding ODE model reads

n(t) = kn(t), n(0) =n0,where an initial value problem is formulated. The involved constant isk = l n 2/T1/2. In this simple example, the solution of the ODE can bedetermined analytically, i.e.,

n(t) =n0ekt.

Although the number of particles is an integer in reality, it is reasonable toapply real numbers in the model. The radioactive decay can be seen as aunimolecular reaction.

Chemical processes typically include bimolecular reactions

A + B C + D.The special case B = C represents a catalysis. Let cSbe the concentration ofthe substance S. The corresponding system of ordinary differential equationsreads

cA(t) = k cA(t)cB(t)

cB(t) = k cA(t)cB(t)cC(t) = +k cA(t)cB(t)cD(t) = +k cA(t)cB(t).

(1.1)

The reaction rate coefficientk >0 characterises the probability of the chem-ical reaction in case of a collision between the molecules A and B. The co-efficient k can also be seen as velocity of the reaction. The physical unitof the parameter k is litre/(s mol). According initial conditions have to bespecified for the system (1.1).

3


8/187

Now we consider a set ofm general chemical reactions involvingndifferentspecies A1, . . . , An (molecules/atoms) in total

1jA1+2jA2+ +njAn kj 1jA1+2jA2+ +njAnforj = 1, . . . , mor, equivalently,

ni=1

ijAikj

ni=1

ijAi for j = 1, . . . , m . (1.2)

The parameters ij, ij 0 represent the stoichiometric constants. Thejth reaction exhibits the rate coefficientk

j

+. Consequently, the result-ing mathematical model reads

dcAidt

=m

j=1

(ij ij)kjn

l=1

cAllj for i= 1, . . . , n ,

which represents a system ofn ordinary differential equations for the un-known concentrations. The evaluation of the right-hand side can be doneautomatically, if the corresponding chemical reactions (1.2) are specified.

The hydrolysis of urea represents an example of a more complex chemicalreaction. Thereby, urea is combining with water and results to ammoniumcarbonate. To achieve a sufficiently fast reaction, the help of the enzymeurease is necessary, since it decreases the energy of activation, i.e., theenzyme acts as a catalyser. The complete chemical reaction is given by theformula

(NH2)2CO + 2 H2O + urease (NH4)2CO3+ urease. (1.3)

This relation represents a complex reaction, since it consists of three simplerreactions. In the following, we use the abbreviations: U: urea, E: urease(enzyme), UE: combination of urea and urease, A: ammonium carbonate.The reaction (1.3) includes the three parts

U + E k1 UE

UE k2 U + E

UE + 2 H2O k3 A + E.

(1.4)

4


9/187

The parameters k1, k2, k3 specify the velocities of the reactions. The threeparts are complex reactions itself, i.e., they proceed as chains of simple

reactions, which are not considered here.

We construct a mathematical model for this system of reactions. Let cSbe the concentration of the substance S with unit mol/litre (mol/l). Thetransient behaviour of the concentrations shall be determined. Since thereaction takes place in water and the concentrations of the other substancesis relatively low, we assume the concentration of water to be constant intime (55.56 mol/l). The velocities of the reactions are

k1= 3.01 lmols, k2= 0.02 1s, k3= 0.1 1s. (1.5)

Consequently, we obtain a system of four ODEs for the four unknown con-centrations

cU = k1cUcE +k2cUEcE = k1cUcE +k2cUE +k3cUEcUE = k1cUcE k2cUE k3cUEcA = k3cUE.

(1.6)

This system exhibits a unique solution for predetermined initial values. Weapply the initial conditions

cU= 0.1mol

l , cE= 0.02moll , cUE

=cA= 0. (1.7)

Like in many other applications, an analytical solution of this system ofODEs is not feasible, i.e., we do not achieve an explicit formula for theunknown solution. Thus we employ a numerical simulation to determine asolution approximately. Figure 2 illustrates the results.

On the one hand, the concentration of urea decays to zero, since this sub-stance is decomposed in the hydrolysis. On the other hand, the productammonium carbonate is generated until no more urea is present. The con-centration of the enzyme urease decreases at the beginning. According toan enzyme, the initial amount of urease is recovered at the end.

5


10/187

0 20 40 60 80 100 120 140 160 180 2000

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

time [s]

concentrations[mol/l]

[U]

[E]

[UE]

[A]

Figure 2: Simulation of the hydrolysis of urea.

1.2 Electric circuits

As a simple example of an electric circuit, we consider an electromagneticoscillator, which consists of a capacitance Cand inductance Land a resis-tance R, see Figure 3 (left). Kirchhoffs current law yiels the relation

IC+IL+IR= 0.

Kirchhoffs voltage law implies U :=UC=UL= UR. Each basic element of

the circuit exhibits a voltage-current relation

CUC=IC, LIL=UL, R=

URIR

.

It follows a linear system of two ODEs

U = 1CIL 1RCUIL =

1LU

(1.8)

6


11/187

CLRI I I

U

CLRI I I

U

IS

Figure 3: Electromagnetic oscillator with (right) and without (left) current source.

for the two unknown functionsUandIL. Further calculations yield an ODE

of second order for the unknown voltage

U+ 1RCU+ 1LCU= 0.

If the resistance is sufficiently large, the solution becomes a damped oscil-lation

U(t) = e 12RC

t

A sin

1LC

t

+B cos

1LC

t

.

The constantsA andB are determined by initial conditions.

The system (1.8) of ODEs is autonomous. We obtain a time-dependentsystem by adding an independent current source to the circuit, see Figure 3(right). We apply the input

Iin(t) =I0 sin(0t) .

The corresponding ODE model becomes

U = 1CIL 1RCU 1CIin(t)I

L = 1

LU.

(1.9)

The system (1.9) exhibits periodic solutions with the rateT = 2/0. Hencewe can impose boundary conditions U(0) =U(T) and IL(0) =IL(T). Res-onance occurs in the case 0 = 1/

LC. Figure 4 shows the solutions of

initial value problems corresponding to (1.8) and (1.9), respectively.

To demonstrate the model of a more complex electric circuit, we considerthe Colpitts oscillator depicted in Figure 5. Mathematical modelling yields

7


12/187

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 21

0.8

0.6

0.4

0.2

0

0.2

0.4

0.6

0.8

1

time [ms]

voltage

[V]

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 210

8

6

4

2

0

2

4

6

8

10

time [ms]

voltage

[V]

Figure 4: Solution Uof ODE (1.8) (left) and ODE (1.9) (right).

an implicit system of four ODEs including four unknown node voltages:1 0 0 00 C1+C3 C3 C10 C3 C2+C3+C4 C20 C1 C2 C1+C2

U1U2U3U4

+

R2L(U1

U2)

R2U

op

1R2

(U1 Uop) Is+ Isbc g(U4 U2) +Isg(U4 U3)1R4

U3

Is+ Isbe

g(U4 U3) +Isg(U4 U2)

1R3

U4+ 1R1

(U4 Uop) + Isbeg(U4 U3) + Isbc

g(U4 U2)

= 000

0

.Several technical parameters appear in the system. The current-voltagerelation corresponding to the bipolar transistor reads

g(U) := exp UUth 1.Thus the system is nonlinear.

1.3 Multibody problem

We consider the two-body problem for two particles with masses m1, m2.Let Xi = (xi, yi, zi) be the location of the ith mass. The locations and the

8


13/187

R

1

C L

C

C

2

UR

3C

4

R

R

2

op

3

4

2

1

1

3 4

Figure 5: Electric circuit of the Colpitts oscillator.

velocities of the particles depend on time. The gravitation generates forcesbetween the masses. Newtons laws of motion yield the ODEs of second

orderm1X

1 (t) = G

m1m2| X1(t) X2(t)|3 (

X2(t) X1(t))

m2X2 (t) = G

m1m2| X1(t) X2(t)|3 (

X1(t) X2(t))with the gravitational constant G >0. Introducing the velocities Vi := X

i

implies a system of first order

X1 = V1

V1 = G m2

| X1 X2|3 ( X2 X1)X2 = V2V2 = G

m1| X1 X2|3 (

X1 X2)

including twelve ODEs. The system is autonomous. Initial conditions forXi(0),Vi(0) have to be specified. Figure 6 depicts the trajectories of a two-body problem with different masses m1 > m2. The movement typicallyproceeds approximatively along ellipses.

9


14/187

00.5

11.5

22.5

33.5

0

0.2

0.4

0.6

0.8

1

0

0.2

0.4

0.6

0.8

1

1.2

1.4

xy

z

0

2

4

00.10.20.30.40.50.60.70.80.91

0

0.2

0.4

0.6

0.8

1

1.2

1.4

x

y

z

Figure 6: Trajectories (locations) of a two-body problem with masses m1 > m2 from twodifferent viewpoints (solid line: first body, points: second body).

Moreover, the two-body problem can be solved analytically. In constrast,we arrange the N-body problem now, where N masses m1, . . . , mN areinvolved. Let Fij be the gravitational force on the ith mass caused by the

jth mass. Newtons laws of motion imply

miXi

=N

j=1,j=iFij =N

j=1,j=i G mimj

| Xj Xi|3( Xj

Xi)

fori= 1, . . . , N . It follows a system of 6NODEs of first order

Xi = Vi

Vi = GN

j=1,j=i

mj

| Xj Xi|3( Xj Xi) for i= 1, . . . , N .

The N-body problem cannot be solved analytically. Thus we require nu-merical methods to solve the problem.

1.4 Further models

In the previous sections, we have considered problems in the fields of chem-ical reactions, electrical engineering and mechanics. Systems of ODEs alsoappear in the following applications:

10


15/187

biology (predator-prey models, epidemic models, etc.),

simulation of war battles (Lanchesters combat models), semi-discretisation of partial differential equations, and others.

In financial mathematics, for example, modelling yields stochastic (ordi-nary) differential equations. Numerical methods for the stochastic differen-tial equations represent improvements of the techniques for ODEs. Hence

knowledge on ODE methods is necessary to deal with stochastic systems.

Further reading on ODE models:

P. Deuflhard, F. Bornemann: Scientific Computing with Ordinary Differen-tial Equations. Springer, New York 2002.

M. Braun: Differential Equations and Their Applications. (4th edition)Springer, Berlin 1993.

11


16/187

Chapter 2

Short Synopsis on Theory of ODEs

2

In this chapter, we review some basic results on existence and uniquenesscorresponding to solutions of ODEs. Further interesting properties are alsoconsidered.

2.1 Linear differential equations

An initial value problem of a linear (inhomogeneous) ODE reads

y(x) =a(x)y(x) +b(x), y(x0) =y0.

The corresponding solution exhibits the formula

y(x) = exp

xx0

a(s) ds

y0+

xx0

exp

sx0

a(t) dt

b(s) ds

,

which can be verified straightforward. A more explicit formula of the solu-tion is only obtained if the involved integrals can be solved analytically.

In case of linear (inhomogeneous) systems of ODEs, the initial value problembecomes

y(x) =A(x)y(x) +b(x), y(x0) =y0

with predetermined functions A :

nn andb :

n. Numericalmethods are required to solve the system. We obtain a formula of the

12


17/187

solution in case of constant coefficients A

nn, i.e.,

y(x) = exp(A(x x0)) y0+ xx0

exp(A(s x0))b(s) ds .The involved integral over a vector is evaluated component-wise. The matrixexponential is defined by

exp(At) :=k=0

tk

k!Ak,

where the sum converges for eacht

with respect to an arbitrary matrixnorm. In general, the matrix exponential cannot be evaluated analytically.Thus a numerical scheme is necessary. Further investigations show thatnumerical techniques avoiding the matrix exponential have to be preferredfor solving the linear system of ODEs.

In conclusion, an analytical solution of linear ODEs is not always feasible.Hence numerical methods yield the corresponding solutions. Of course, thisholds even more in case of nonlinear ODEs.

2.2 Existence and uniqueness

We consider initial value problems of systems of ODEs

y(x) =f(x, y(x)), y(x0) =y0 (2.1)

for functionsf :G

n with G

n and (x0, y0) G. A functiony

represents a solution of this problem if and only if

y(x) =y0+

xx0

f(s, y(s)) ds (2.2)

holds for all relevant x.

The theorem of Peano just requires a continuous right-hand side f. How-ever, this theorem yields only the existence and not the uniqueness of asolution. To apply numerical methods, we need both properties.

13


18/187

In the following, we assume the Lipschitz-condition

f(x, y) f(x, z) L y z (2.3)for allx, y, zlocated inG with a constantL >0. The involved vector normis arbitrary. Concerning the uniqueness of a solution to an initial valueproblem (2.1), it holds the following result.

Theorem 1 Let G n be an open set and let f : G n be acontinuous function satisfying the Lipschitz-condition (2.3). Consider twosolutions , : I

n of the ODE system y = f(x, y) on an intervalI . If(x0) =(x0)holds for somex0 I, then it follows(x) =(x)

for allx I.

Outline of the proof:

Let, :I n be two solutions ofy =f(x, y). We show that the condi-tion (x) =(x) for an arbitrary x I implies in a neighbourhoodof x. It holds

(x) =(x) + xx

f(s, (s)) ds, (x) =(x) + xx

f(s, (s)) ds.

The Lipschitz condition yields

(x) (x) x

x

f(s, (s)) f(s, (s)) ds

L xx

(s) (s) ds

.

We define

M(x) := sup{(s) (s) : |s x| |x x|}.It follows

(t) (t) L|t x|M(t) L|x x|M(x)for|t x| |x x| and thus

M(x) L|x x|M(x).14


19/187

For|x x| < 1/(2L), we obtain M(x) 12

M(x) and thus M(x) = 0 forthosex.

Now we consider the assumption (x0) =(x0). Let

x1:= sup{

s I :|[x0,s]=|[x0,s]}

.

Since both functions are continuous, it follows (x1) = (x1). Ifx1 is notequal to the right boundary of the interval, then a contradiction appearswith respect to the previous result, which states that the functions are equalin a complete neighbourhood ofx1. The same argumentation can be appliedto the left boundary x

x0.

The theorem of Picard-Lindelof yields a result on the existence.

Theorem 2 (Picard-Lindelof) LetG n be an open set and letf :G

n be a continuous function satisfying the Lipschitz-condition (2.3).Then for each (x0, y0) G it exists a real number > 0 and a solution: [x0 , x0+] n of the initial value problem (2.1).

Outline of the proof:

It existsr >0 such that the set

V := {(x, y) n : |x x0| r, y y0 r}satisfiesVG. Since f is continuous and V is compact, it exists M > 0such that

f(x, y) M for all (x, y) V.

We define:= min{r, r/M}andI := [x0 , x0+].A function satisfies the initial value problem if and only if

(x) =y0+

xx0

f(s, (s)) ds

holds for all x I. We define functions k:I n via the iteration

k+1(x) :=y0+

xx0

f(s, k(s)) ds

15


20/187

using the starting function 0(x) y0. Forx I, it follows

k+1(x) y0 xx0

f(s, k(s)) ds M|x x0| M rprovided thatk lies inV. By induction, the functions k are well-defined.

Furthermore, it follows by induction

k(x) k1(x) MLk1 |x x0|k

k! for each x I.

Hence it holds

k(x) k1(x) ML

(L)kk!

uniformly in I. The right-hand side exhibits terms of exponential seriesfor eL. It follows that (k)k is a Cauchy-sequence uniformly forx I.Consequently, the sequence (k)k converges uniformly to a continuousfunction . Moreover, we obtain

f(x, (x)) f(x, k(x)) L (x) k(x).

Thus the sequence (f(x, k(x))k converges uniformly to f(x, (x)). Itfollows

(x) = limk

k(x) = y0+ limk

xx0

f(s, k(s)) ds

= y0+

xx0

limk

f(s, k(s)) ds = y0+

xx0

f(s, (s)) ds

and the proof is completed.

The theorem of Picard-Lindelof also includes a method for the constructionof the solution by the iteration. We analyse this iteration further. We define

F() :=y0+

xx0

f(s, (s)) ds.

The fixed point = F() represents a solution of the initial value prob-lem (2.1). The corresponding iteration reads k+1=F(k). We obtain for

16


21/187

x0 x x1 :

F()(x) F()(x) xx0

f(s, (s)) f(s, (s)) ds

L xx0

(s) (s) ds

L x1x0

(s) (s) ds L(x1 x0) max

s[x0,x1](s) (s).

It followsmax

s[x0,x1]F()(s) F()(s) L(x1 x0) max

s[x0,x1](s) (s).

Hence the mapping Fis contractive with respect to the maximum norm ifx1 x0 < 1L holds. The theorem of Banach implies the convergence of thePicard-Lindelof iteration and the existence of a unique fixed point.

However, the iteration requires a subsequent solution of integrals, whichmakes it disadvantageous in practice. Furthermore, we may be forced touse small subintervals.

Finally, we cite a theorem concerning the maximum interval of the existenceof a solution to the initial value problem (2.1).

Theorem 3 Let the assumptions of Theorem 2 be fulfilled. Then it existsa maximum interval (, ) with < x0 < such that a unique solution : (, )

n of the initial value problem (2.1) exists. It holds either

= or < together with{(x, (x)) :x [x0, )} {(x, y) G: x = } = .

Analogue conditions follow for.

If a function f :

n

n is continuous everywhere, then


22/187

2.3 Perturbation analysis

We analyse the condition of initial value problems of ODEs, i.e., the sensi-tivity of the solutions in dependence on the data. The data are the initialvalues y0 and the right-hand side f. (Differences in the value x0 can bedescribed by different right-hand sides.)

We consider the solution y(x) of the initial value problem (2.1) and thesolution z(x) of the perturbed initial value problem

z(x) =f(x, z(x)) +(x), z(x0) =z0.

Let the function (x) be continuous. We estimate the resulting differencey(x) z(x) between the unperturbed and the perturbed solution in termsof the perturbations

:= y0 z0, := maxt[x0,x1]

(t)

using some vector norm.

The estimate is based on the following version of Gronwalls lemma.

Lemma 1 Assume that m(x) is a non-negative, continuous function andthat, 0, L >0. Then the integral inequality

m(x) +(x x0) +L xx0

m(s) ds (2.4)

implies the estimate

m(x) eL(xx0) + L

eL(xx0) 1 . (2.5)Proof:

At first we define the function

q(x) := eLx xx0

m(t) dt,

18


23/187

which is differentiable as m(x) is continuous and has the derivative

q(x) = LeLx xx0

m(t) dt+ eLxm(x).

Solving form(x), we get the relations

m(x) = eLxq(x) +L xx0

m(t) dt, (2.6)

= eLxq(x) +LeLxq(x) = (eLxq(x)). (2.7)

We now insert (2.6) in (2.4) and obtain

eLxq(x) +(x x0) (2.8)and solving for q(x) we get

q(x) ( x0)eLx +xeLx.Hence, performing integration, the inequality

q(x)

x0

L (eLx eLx0+ xx0 teLt dt (2.9)holds, where the integral can be calculated via integration by parts x

x0

teLt dt = 1L

teLtxx0

+ 1

L

xx0

eLt dt

= 1L

(xeLx x0eLx0

1L2(

eLx eLx0 .Finally, inserting (2.8),(2.9) into (2.7) we end up with

m(x) ( x0)

1 eL(xx0)

x+x0eL(xx0)

L

1 eL(xx0)

+ ( +(x x0))

= eL(xx0) +

L

eL(xx0) 1

,

which is the statement (2.5).

19


24/187

Using m(x) :=y(x) z(x), the assumptions of Gronwalls lemma arefulfilled, because it holds

y(x) z(x) =y0 z0 xx0

(s) ds + xx0

f(s, y(s)) f(s, z(s)) ds

and thus

y(x) z(x) y0 z0 + xx0

(s) ds

+

xx0

f(s, y(s)) f(s, z(s)) ds

y0 z0 + maxt[x0,x1](t) (x x0)+L

xx0

y(s) z(s) ds

forx0 x x1. Thus Gronwalls lemma yieldsy(x) z(x) eL(xx0) +

L

eL(xx0) 1

for x0 x x1. We recognise that the problem is well-posed, since itdepends continuously on the data. Nevertheless, the difference can increaseexponentially for increasingx. This is not always the case but may happen.

If the perturbation appears only in the initial values ( 0 = 0), thenthe corresponding estimate reads

y(x) z(x) y(x0) z(x0) eL(xx0) for each x x0. (2.10)This estimate implies again that the solution y(x) depends continuously onits initial value y(x0) =y0 for fixed x. Moreover, the dependence becomes

smooth for a smooth right-hand side f. We denote the dependence of thesolution on the inital values via y(x; y0).

Theorem 4 Suppose that f is continuous with respect to x and that thepartial derivatives offwith respect to y exist and are continuous. Then thesolutiony(x; y0) is smooth with respect to y0. The derivatives

(x) := y

y0(x; y0) nn

20


25/187

are the solution of the initial value problem of the matrix differential equa-tion

(x) = fy

(x, y(x; y0)) (x), (x0) =I (2.11)with the identityI

nn.

The proof can be found in: Hairer, Nrsett, Wanner: Solving OrdinaryDifferential Equations. Volume 1. Springer.

We just show the second statement of the theorem. Differentiating the

original system of ODEs

xy(x; y0) =f(x, y(x; y0))

with respect to the initial values yields

y0

xy(x; y0) =

y0f(x, y(x; y0))

x

y

y0(x; y0) =

f

y

(x, y(x; y0))

y

y0(x; y0)

x(x) =

f

y(x, y(x; y0)) (x).

The initial value y(x0; y0) =y0 implies the initial condition yy0

(x0; y0) =I.

The matrix differential equation consists of n separate systems of ODEs(with dimension n each). Moreover, the matrix differential equation ex-hibits a linear structure. The matrix differential equation can be solved

numerically in combination with the original system of ODEs (2.1). Alter-natively, numerical differentiation is feasible.

21


26/187

Chapter 3

One-Step Methods

3

We consider numerical methods for the initial value problems introduced inthe previous chapter. We start with one-step methods, whereas multi-stepmethods are discussed in a later chapter.

3.1 Preliminaries

We want to solve an initial value problem (2.1) of a system of ODEs nu-merically in some interval x[x0, xend]. All numerical methods for initialvalue problems, which we consider in this lecture, apply a finite set of gridpoints

x0< x1< x2< x3< < xN1< xN=xend.A feasible choice are equidistant grid points

xi:=x

0+ih with h:=

xend

x0

N for i= 0, 1, . . . , N .

Numerical solutions yi y(xi) are computed successively. In a one-stepmethod, the dependence of the values is just

y0 y1 y2 yN1 yN.In contrast, a multi-step method with k steps exhibits the dependence

yik, yik+1, . . . , yi2, yi1 yi for i= k, k+ 1, . . . , N .22


27/187

Thereby, the first valuesy1, . . . , yk1have to be provided by another schemein case ofk >1. Remark that a one-step method represents a special case

of a multi-step method with k= 1.

A general one-step method can be written in the form

yi+1=yi+hi(xi, yi, hi), (3.1)

where the function depends on the scheme as well as the right-hand sidefunction f.

3.2 Elementary integration schemes

Most of the methods for the initial value problem (2.1) are based on anapproximation of the corresponding integral equation (2.2). In the interval[x0, x0+h], we obtain

y(x0+h) = y0+

x0+hx0

f(s, y(s)) ds

= y0+h 10

f(x0+sh, y(x0+sh)) ds.

(3.2)

Now the integral on the right-hand side is replaced by a quadrature rule.The problem is that the function y, which appears in the integrand, isunknown a priori.

Since h is small, we consider simple quadrature rules. We discuss the fol-lowing four examples, see Figure 7:

(a) rectangle (left-hand):

The approximation becomes

y1= y0+hf(x0, y0).

This scheme is called the (explicit) Euler method. It is the most simplemethod, which is feasible. Given the initial valuey(x0) = y0, the approxi-mation y1 is computed directly by just a function evaluation off.

23


28/187

00000

00000

00000

00000

00000

00000

11111

11111

11111

11111

11111

11111

0

0

1

1

0

0

1

1

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

1111

1111

1111

1111

1111

1111

1111

1111

1111

1111

1111

0

0

1

1

0

0

1

1

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

1111

1111

1111

1111

1111

1111

1111

1111

1111

1111

1111

0

0

1

1

0

0

1

1

0

0

0

0

0

0

0

0

0

0

1

1

1

1

1

1

1

1

1

1

00000

00000

00000

00000

00000

00000

00000

00000

00000

00000

11111

11111

11111

11111

11111

11111

11111

11111

11111

11111

(a) (b) (c) (d)

Figure 7: Elementary quadrature rules: (a) rectangle (left-hand), (b) rectangle (right-

hand), (c) trapezoidal rule, (d) midpoint rule.

(b) rectangle (right-hand):

Now the scheme reads

y1=y0+hf(x0+h, y1). (3.3)

This technique is called the implicit Euler method. The unknown valuey1

appears on both sides of the relation. In general, we cannot achieve anexplicit formula for y1. The formula (3.3) represents a nonlinear system ofalgebraic equations for the unknowny1, i.e., the valuey1is defined implicitly.For example, a Newton iteration yields an approximative solution. Hencethe computational effort of one integration step becomes much larger thanin the explicit Euler method.

(c) trapezoidal rule:

If the integral is approximated by a trapezoid, the technique becomes

y1=y0+h

2(f(x0, y0) +f(x0+h, y1)) .

This approach results again in an implicit method. The computational effortof one integration step is nearly the same as in the implicit Euler method.However, the accuracy of the approximations is better in general, sincetrapezoids yield better approximations than rectangles in the quadrature.

24


29/187

(d) midpoint rule:

The midpoint rule applies a specific rectangle. It followsy1=y0+hf(x0+

12h, y(x0+

12h)). (3.4)

This scheme is not feasible yet, since both y1 andy(x0+ 12h) are unknown.

We require an additional equation. For example, an approximation of theintermediate valuey(x0+

12

h) can be computed by the explicit Euler method.The resulting technique reads

y1/2 = y0+ h2 f(x0, y0)

y1 = y0+hf(x0+ 12h, y1/2).

or, equivalently,

y1=y0+hf(x0+ h2 , y0+

h2 f(x0, y0)). (3.5)

The method is explicit, since we can compute successively y1/2andy1with-out solving nonlinear systems. Just two function evaluations of f are re-quired. This scheme is called the modified Euler method or the methodof Collatz. Although the method applies the Euler method first, the finalapproximation y1 is significantly more accurate than in the Euler methodin general.

The accuracy of the above methods follows from latter discussions.

Explicit Euler method (revisited)

We consider the explicit Euler method more detailed. This scheme can be

motivated by two other approaches. First, replacing the derivative in theODEy= f(x, y) by the common difference quotient (of first order) yields

y(x0+h) y(x0)h

.=f(x0, y(x0)) y1=y0+hf(x0, y0).

Second, we consider the tangent ofy(x) corresponding to the point (x0, y0)as approximation of the solution. The tangent is

t(x) =y(x0) + (x x0)y(x0) =y(x0) + (x x0)f(x0, y(x0)).25


30/187

It followsy1:=t(x0+h) =y0+hf(x0, y0),

i.e., the explicit Euler method.

For example, we solve the initial value problemy = 12y ,y(14) =

12 ,x [ 14, 2].

The solution is justy(x) =

x. Figure 8 illustrates the numerical solutionsfollowing form the Euler method. We recognise that the accuracy becomesbetter the more steps Nare applied.

3.3 Consistency and convergence

We consider a general explicit one-step method of the form (3.1) with theincrement function .

Different notations are used to analyse the accuracy of the approximationsyi+1 in comparison to the exact solutiony(xi). On a local scale, we arrangethe following definition.

Definition 1 (local discretisation error) Lety(x) be the exact solutionof the ODE-IVPy=f(x, y), y(x0) =y0 andy1=y0+ h(x0, y0, h)denotethe numerical approximation of one step withh >0. The local discretisationerror is then defined as

(h) :=y(x0+h) y1

h . (3.6)

The definition (3.6) of the local error can be interpreted in three differentways:

the difference between the exact solution and the numerical approxima-tion (discretisation error after one step starting from the exact solution)scaled by the step size h.

26


31/187

N= 5

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20.4

0.6

0.8

1

1.2

1.4

1.6

x

y(x)

N= 10

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20.4

0.6

0.8

1

1.2

1.4

1.6

x

y(x)

N= 50

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20.4

0.6

0.8

1

1.2

1.4

1.6

x

y(x)

Figure 8: Solution ofy = 12y

, y(14

) = 12

(solid line) and numerical approximation (points)resulting from the explicit Euler method using N steps.

27


32/187

y

x0

y0

x

y(x)

0x +h

secant of exact solution

secant ofapproximationy1

Figure 9: Secants of exact solution and numerical approximation.

the difference in the gradients of the respective secants

(h) =y(x0+h) y0

h exact solution

y1 y0h

approximation

.

The secants are illustrated in Fig. 9. If (h) 0 holds, then bothsecants become the tangent t(x) =y(x0) + (x

x0)y

(x0) in the limit.

the defect(h) =

y(x0+h) y0h

(x0, y0, h), (3.7)which results from inserting the exact solution into the formula of theapproximation.

Example 1: Local discretisation error of the explicit Euler method

Taylor expansion yields assumingy C2

y(x0+h) = y(x0) +hy(x0) + 12h

2y(x0+(h)h)

= y0+hf(x0, y0) + 12h

2y(x0+(h)h)

with 0< (h)< 1.

28


33/187

The local discretisation error becomes

(h) = 1

h

(y(x0+h)

y1)

= 1h(y(x0+h) y0 hf(x0, y0))= 12hy

(x0+(h)h).

It follows (h) = O(h).

Example 2: Local discretisation error of the implicit Euler method

For simplicity, we assume a bounded right-hand side, i.e.,f M. Onthe one hand, the implicit Euler method implies

y1= y0+hf(x0+h, y1) =y0+hf(x0+h, y0+hf(x0+h, y1)).

Multidimensional Taylor expansion yields

y1 = y0+h

f(x0, y0) + fx (x0, y0)h+

fy (x0, y0)hf(x0+h, y1) + O(h2)

= y0+hf(x0, y0) + O(h2).

On the other hand, the Taylor expansion of the exact solution from above

can be used. It follows(h) = 1h(y(x0+h) y1)

= 1h(y0+hf(x0, y0) + O(h2) (y0+hf(x0, y0) + O(h2))) = O(h).Again we obtain(h) = O(h) like in the explicit Euler method.

Based on the property of the local discretisation error, we define the con-sistency.

Definition 2 (consistency) A method (or its increment function ) iscalled consistent, if the local discretisation error tends to zero uniformly inx, y forh 0:

(h) (h) with limh0

(h) = 0.

The method is consistent of (at least) orderp, if

(h) = O(hp).

29


34/187

Consistency of one-step methods can be easily characterised by the followingproperty.

Lemma 2 Let the right-hand sidefof the ODEsy =f(x, y)be continuousin x and satisfy the Lipschitz-condition (2.3) with respect to y. Then it

follows the equivalence

is consistent limh0

(x,y,h) =f(x, y).

Proof:Letzbe the solution of the ODE-IVP z(x) =f(x, z(x)),z() =. Due tothe definition ofand the mean value theorem of differentiation

( , ,h) = z(+h) ( , , h)for some (0, 1). Since f andz are continuous in x, both functions areuniformly continuous in an interval [a, b]. It follows

limh0

z(+h) =z() =f(, )

uniformly and thus

limh0

( , ,h) = f(, ) limh0

( , , h).

This relation shows the statement.

The order of consistency describes the quality of the numerical approxima-tion after a single step. However, we are interested in the quality of theapproximation after N steps, where the final point xend is reached. Thismotivates the following definition.

Definition 3 (global discretisation error and convergence)The global discretisation error of a method using a gridx0< x1< < xNis defined by the difference

eN=y(xN) yN. (3.8)30


35/187

ForN , we assume|h| 0with|h| := maxi=0,...,N1

|xi+1xi|. The methodis called convergent, if for fixedx= x

N it holds

limN

eN= 0.

The method is convergent of (at least) orderp, if it holds

eN= O(|h|p).

Concerning consistency and convergence, we prove the following theorem.

Theorem 5 (convergence of one-step methods) Let f be continuousand satisfy the Lipschitz-condition (2.3). Consider a one-step scheme withincrement function, which is consistent of orderp, i.e.,

(h) = O(hp).Then the global discretisation error is bounded by

eN c |h|p

exp(L

|xN

x0

|)

1

L

with a constantc >0 and|h| = max{h0, h1, . . . , hN1} forhi:=xi+1 xi.

Proof:

The one-step scheme generates the sequencey1, . . . , yN. We define auxilaryODE-IVPs by

ui(x) =f(x, ui(x)), u(xi) =yi for i= 0, 1, . . . , N 1.The auxilary solutions are sketched in Figure 10. The global error can bewritten as

eN := y(xN) yN = u0(xN) yN

= uN1(xN) yN+N2

i=0ui(xN) ui+1(xN).

31


36/187

y

x0

y0

u

xx x1 2

0

xn

y

y

u2

u1

1

2

Figure 10: Lady Windermeres Fan.

We obtain the estimate

eN uN1(xN) yN +N2i=0

ui(xN) ui+1(xN).

Since the solutions ui satisfy the same system of ODEs for different initialvalues, we can apply the relation (2.10). It follows

eN uN1(xN) yN +N

2

i=0

ui(xi+1) ui+1(xi+1)eL|xNxi+1|

=N1i=0

ui(xi+1) yi+1eL|xNxi+1|.

The norms on the right-hand side correspond to the local errors after onestep. Since we assume a consistent method, it holds

ui(xi+1)

yi+1

c

hp+1

i c |

h|p

hi

uniformly with a constant c >0 andhi:=xi+1 xi. Thus we obtain

eN c |h|pN1i=0

hieL|xNxi+1| c |h|p

xNx0

eL|xNt|dt

= c |h|p eL|xNx0| 1

L ,

which completes the proof.

32


37/187

This theorem demonstrates that consistency is sufficient for convergence.Moreover, the order of consistency coincides with the order of convergence.

The consistency can be determined by analysing the increment function of the one-step method. Vice versa, convergent methods exist, which are notconsistent. Hence consistency is not necessary for convergence. However,inconsistent methods are not used in practice.

Comparison to numerical quadrature:

Assume that we want to compute the integral

I(g) := ba

g(x) dx

of a function g C2[a, b] approximately by trapezoidal rule. Let

M := maxaxb

|g(x)|.

We apply a grid xi = a+ ih with step size h = baN

. Let Txi+1xi (g) be the area of one

trapezoid constructed in the interval [xi, xi+1]. It holds

xi+hxi

g(x) dx Txi+1xi (g) = 112h3|g()| 112h3M=:R(h).The valueR(h) = O(h3) orR(h)/h= O(h2) can be seen as a local error of the trapezoidalrule. For the global error EN, we obtain

EN :=

ba

g(x) dx Ni=1

Txixi1(g)

=

Ni=1

xixi1

g(x) dx Txixi1(g)

N

i=1 xi

xi1

g(x) dx

Txixi1(g)

N

i=11

12

h3M = N 1

12

h3M = b a

12

h2M.

Thus it holds EN= O(h2). Remark that N= bah , i.e., ENcan be written in dependenceon the step size h. We recognise that the order of the global errorEN =O(h2) coincideswith the order of the local error R(h)/h= O(h2) (provided that we define the local erroras R(h)/h and not as R(h)).

33


38/187

3.4 Taylor methods for ODEs

The analysis of the order of consistency indicates an approach for numeri-cal techniques based on Taylor expansions. For simplicity, we consider anautonomous system of ODEs, i.e.,

y(x) =f(y(x)), y(x0) =y0.

An arbitrary intial value problem of ODEs y=f(x, y) can be transformedto an equivalent autonomous system via

Y(t) =F(Y(t)) with Y(t) := y(t)x(t)

, F(Y) := f(x, y)1

with initial conditions x(t0) =x0, y(t0) =y0.

Moreover, we discuss a scalar autonomous ODEy(x) =f(y(x)) now. Givena solutiony Cp+1, Taylor expansion yields

y(x0+h) = y(x0) +hy(x0) + h

2

2! y(x0) + + hpp! y(p)(x0)

+ hp+1(p+1)!y(p+1)(x0+(h)h)(3.9)

with 0 < (h) < 1. For a sufficiently smooth right-hand side f, we canreplace the derivatives of the unknown solution. It holds

y = f(y)

y = f(y)y = f(y)f(y)

y = (f(y)y)f(y) +f(y)(f(y)y) = f(y)f(y)2 +f(y)2f(y)

...

and thusy(x0) = f(y0)

y(x0) = f(y0)f(y0)

y(x0) = f(y0)f(y0)2 +f(y0)2f(y0)... .

34


39/187

Since the initial value y(x0) = y0 is given, we define one-step methodsy1=y0+h(y0, h) via

1(y, h) = f(y)

2(y, h) = f(y) + h2

f(y)f(y)

3(y, h) = f(y) + h2

f(y)f(y) + h2

6

f(y)f(y)2 +f(y)2f(y)

...

based on the Taylor expansion (3.9). The method specified by 1is just theexplicit Euler method. Due to this construction, the pth method exhibitsthe local discretisation error

p(h) =y(x0+h) y(x0)

h p(y0, h) = h

p

(p+ 1)!y(p+1)(x0+(h)h)

i.e.,p(h) = O(hp). It follows that the method is consistent of order p.However, the number of required derivatives increases rapidly in case ofsystems of ODEs:

f : ncomponentsfy : n

2 components2fy2 : n

3 components...

kfyk : n

k+1 components.

Hence the computational effort becomes large for higher orders. Moreover,the computation of derivatives of higher order via numerical differentiationbecomes more and more affected by roundoff errors.

In conclusion, Taylor methods of order p >1 are not used in practice.

35


40/187

3.5 Runge-Kutta methods

The most important class of one-step schemes are Runge-Kutta methods.The idea is to replace the integral in (2.2) by a quadrature rule with nodesc1, . . . , cs [0, 1] and (outer) weights b1, . . . , bs . Without loss of gener-ality, we assume c1 c2 cs. It follows a finite sum

y1=y0+hs

i=1

bif(x0+cih, y(x0+cih)).

The problem is that the intermediate values y(x0

+cih) are unknown a

priori. We achieve according approximations by an integral relation again

y(x0+cih) =y0+h

ci0

f(x0+sh, y(x0+sh)) ds.

The involved integrals are substituted by quadrature formulas. To avoid thegeneration of new unknowns, the same nodes c1, . . . , cs as before are used.Just new (inner) weights aij are introduced. We obtain the approximations

zi=y0+hs

j=1

aijf(x0+cjh, zj) (3.10)

fori= 1, . . . , s. The resulting final approximation becomes

y1= y0+hs

i=1

bif(x0+cih, zi).

The general relations (3.10) represent a nonlinear system for the unknowns

z1, . . . , z s. If these intermediate values have been determined, then we cancompute y1 directly via s evaluations of the function f.

Considering (3.10), a natural requirement is that a constant function f 1(y(x0+cih) =y0+cih) is resolved exactly. We obtain the conditions

ci=s

j=1

aij for each i= 1, . . . , s . (3.11)

36


41/187

This condition means that the sum of the weights is equal to the (relative)length of the corresponding subinterval.

A Runge-Kutta scheme is uniquely determined by its coefficients. The co-efficients can be written in a so-called Butcher-tableau:

c1 a11 a12 a1sc2 a21 a22 a2s...

... ... . . .

...cs as1 as2 ass

b1 b2

bs

resp. c A

b

withc

s,b

s,A

ss.

Examples: Schemes from Sect. 3.2

(a): expl. Euler method, (b): impl. Euler method, (c): trapezoidal rule,(d): method of Collatz (midpoint rule):

(a) 0 0

1 (b) 1 1

1 (c)

0 0 0

1 1

212

12

12

(d)

0 0 012

12 00 1

Example: Gauss-Runge-Kutta methods

For the nodes ci and the weights bi, we apply a Gaussian quadrature. TheGaussian quadrature exhibits the order 2s, i.e., it holds

si=1

bip(ci) = 10

p(x) dx for all p P2s1

(Pm: polynomials up to degree m). The weights aij are determined suchthat for each i= 1, . . . , sit holds

sj=1

aijp(cj) =

ci0

p(x) dx for all p Ps1.

37


42/187

In the simple cases = 1, it follows directly c1= 12

,b1= 1 anda11= 12

. Theresulting Runge-Kutta method is

z1 = y0+ h2

f(x0+ h2

, z1),

y1 = y0+hf(x0+ h2

, z1).(3.12)

This approach corresponds to the midpoint rule (3.4), where the approxi-mation z1

.=y(x0+

12

h) is determined by the implicit Euler method.

The Butcher tableau of the case s= 2 reads:

336

14

32312

3+36 3+2312 1412

12

If the matrix A = (aij) is full, then the Runge-Kutta method is implicit.A nonlinear system (3.10) ofs n algebraic equations has to be solved. Incontrast, we want to achieve an explicit method now. The correspondingcondition readsaij = 0 fori j. ThusAbecomes a strictly lower triangularmatrix. The Butcher-tableau exhibits the form:

0 0 0 0c2 a21 0

. . . ......

... . . . . . . . . . ...

... ... . . . 0 0

cs as1 as,s1 0b1 b2

bs1 bs

In particular, it follows c1 = 0 due to (3.11) and thus z1 = y0. Now thecomputation of the intermediate values reads

zi=y0+hi1j=1

aijf(x0+cjh, zj).

The computational effort for an explicit Runge-Kutta method just consistsin thes evaluations of the right-hand side f. Explicit methods correspond

38


43/187

to successive extrapolations using the intermediate values. Implicit methodscan be seen as an interpolation based on the intermediate values.

Examples: Some well-known explicit Runge-Kutta methods

Method of Heun (left), Kutta-Simpson rule (middle) and classical Runge-Kutta method (right):

013

13

2

3 0 2

314 0

34

012

12

1 1 216

46

16

012

12

12 0

12

1 0 0 116

26

26

16

An equivalent notation of Runge-Kutta schemes results from the definitionof the increments ki via

ki=f(x0+cih, zi) =f

x0+cih, y0+h

sj=1

aijkj

(3.13)

fori= 1, . . . , s. Now the Runge-Kutta method reads

ki = f

x0+cih, y0+h

sj=1

aijkj

, i= 1, . . . , s ,

y1 = y0+hs

i=1

biki.

(3.14)

Thereby, the increments ki are unknown a priori.

Order conditions

A Runge-Kutta method is determined by its coefficients ci, bi, aij. We deriveconditions on these coefficients such that the one-step method becomesconsistent of some orderp. We consider autonomous scalar ODEsy=f(y).It follows

y = fy = ff,

y = fyf+ffy = ff2 + (f)2f.

39


44/187

Taylor expansion of the exact solution yields

y(x0+h) = y(x0) +hy(x0) + h2

2y (x0) + h3

6y (x0) + O(h4

)= y0+hf(y0) +

h2

2f(y0)f(y0)

+ h3

6

f(y0)f(y0)2 +f(y0)2f(y0)

+ O(h4).

In the following, we use the abbreviations f = f(y0), f = f(y0), etc.

We assume that the Runge-Kutta method fulfills the fundamental condi-tion (3.11). A Taylor expansion of the functionf in the increments (3.13)implies fori= 1, . . . , s

ki = f+fh

sj=1

aijkj

+ 1

2fh2

sj=1

aijkj

2+ O(h3)

= f+fh

sj=1

aij

f+fh

sl=1

ajlkl

+ O(h2)

+ 12f

h2

s

j=1aij(f+ O(h))

2+ O(h3)

= f+fh s

j=1

aij

f+fh s

l=1

ajl(f+ O(h))

+ O(h2)

+ 12fh2 (f ci+ O(h))2 + O(h3)

= f+fh

sj=1

aij(

f+ff hcj+ O(h2)

+ 12f

f2h2c2i + O(h3)

= f+hff ci+h2(f)2f sj=1

aijcj+ 12h2ff2c2i + O(h3).

40


45/187

The approximation obtained by the Runge-Kutta method becomes

y1 = y0+hs

i=1

biki

= y0+hf

si=1

bi

+h2ff

si=1

bici

+h3(f)2f

si,j=1

biaijcj

+ 12h

3ff2

si=1

bic2i

+ O(h4).

A comparison to the Taylor expansion of the exact solution shows the con-ditions for consistency up to order p = 3. We also cite the conditions fororder p= 4:

p= 1 :s

i=1

bi = 1

p= 2 :s

i=1 bici = 12p= 3 :

si=1

bic2i =

13

si,j=1

biaijcj = 1

6

p= 4 :s

i=1

bic3i =

14

s

i,j=1biaijcicj = 18s

i,j=1

biaijc2j =

112

si,j,l=1

biaijajlcl = 1

24

The conditions for consistency can be derived up to an arbitrary order p. Incase of explicit Runge-Kutta methods, the sums just involve the non-zerocoefficients. For a desired order pof consistency, we like to apply a Runge-Kutta method with relatively low number of stages s. In case of implicitschemes, a method with s stages exhibits the maximum order p = 2s incase of Gauss-Runge-Kutta methods. In case of explicit schemes, Table 1gives corresponding informations.

41


46/187

stage number s 1 2 3 4 5 6 7 8 9 10 11 17maximum order p 1 2 3 4 4 5 6 6 7 7 8 10

orderp 1 2 3 4 5 6 7 8minimum stage number s 1 2 3 4 6 7 9 11number of order conditions 1 2 4 8 17 37 85 200

Table 1: Order and number of stages in explicit Runge-Kutta methods.

3.6 Dense output

The numerical method yields a sequence of approximations y0, y1, . . . , yNcorresponding to a grid x0 < x1


47/187

where the ODE y=f(x, y) has been applied. The basis polynomials read

p1() = 1 32

+ 23

, p2() = 32

23

,p3() = 22 +3, p4() = 2 +3,

for 0 1.

The evaluation of the Hermite interpolant y can be done online during theintegration. Moreover, the function evaluations f(xi, yi), f(xi+1, yi+1) areavailable from the (explicit) Runge-Kutta method.

We determine the accuracy of the approximation. On the one hand, it holds

|y(x) u(x)| 1

384 maxs[xi,xi+1] |y(4)(s)|h4i = O(h4i )for x [xi, xi+1]. On the other hand, we obtain using the Lipschitz-condition|f(xi, yi) f(xi, y(xi))| L |yi y(xi)|

|u(xi+hi) y(xi+hi)| |y(xi) yi| |p1()|+ |y(xi+1) yi+1| |p2()|+hi

L

|y(xi)

yi

| |p3()

|+hi L |y(xi+1) yi+1| |p4()|.The definition

pl := max[0,1]

|pl()| for l= 1, 2, 3, 4yields

|u(x) y(x)| |y(xi) yi| (p1+hiLp3) + |y(xi+1) yi+1| (p2+hiLp4).

for all x [xi, xi+1]. In case of a Runge-Kutta method with consistencyorder p, the convergence of the scheme implies|y(xi) yi| =O(|h|p) andthus|u(x) y(x)| = O(|h|p). It follows

|y(x) y(x)| = O(|h|4) + O(|h|p).Hence an orderpof consistency implies a dense output, which approximatesthe exact solution with order q= min{4, p}.

43


48/187

Second idea: Continuous Runge-Kutta method

The second strategy of dense output is to use a Runge-Kutta scheme asa basis for a continuous extension. Thereby, the constant weights bi arereplaced by polynomials bi() in (0, 1). Let the scheme be defined by

y(x0+h) = y0+hs

i=1

bi()ki, 0<


49/187

s

i,j=1bi()aij()cj() =

1

6

s

i,j=1bi()

aij

cj

=

1

6

s

i,j=1bi()aijcj =

3

6

The generalisation to higher orders is obvious. In general, the maximumorder for the dense output scheme will be lower than the maximum orderfor the pointwise (original) method.

Example:

We use the classical Runge-Kutta method with four stages. The scheme

is consistent of order 4. We determine the polynomialsbi() such that thedense output scheme features the order 3.

The order conditions imply the equations:

b1() +b2() +b3() +b4() =

12b2() +

12b3() +b4() =

2

2

14

b2() + 14

b3() +b4() = 3

3

14b3() + 12b4() =

3

6

Solving this linear system yields

b1() = 32

2 +

23

3 , b2() =b3() =

2 23

3 , b4() =

2

2 +

23

3 .

Now the dense output scheme (3.15) can be applied, since all involved coeffi-cients are determined. It holdsbi(0) = 0 andbi(1) =bifor eachi. Hence theapproximating function y is globally continuous and just piecewise smooth.

45


50/187

3.7 Step-Size Control

In a numerical integration, the approximations yk .= y(xk) are computedsuccessively by some numerical method. We would like to obtain an auto-matic selection of the step sizes hk:=xk+1 xksuch that the correspondingerror in the approximations remains sufficiently small.

Let y = (y1, . . . , yn) be the components of the solution. We assume that

a given numerical scheme exhibits a consistency order ofp, i.e., the corre-sponding approximationyh

.=y(x0+h) satisfies

yhi yi(x0+h) = O(hp+1) =Cihp+1 + O(hp+2) (3.16)with constants Ci= 0 for each component. A similar numerical techniqueis used to compute an approximation yh of a higher order

yhi yi(x0+h) = O(hp+2). (3.17)In Runge-Kutta methods, embedded schemes are usually employed. Sev-eral possibilities exist in case of multi-step methods. Richardson extrapola-

tion can be applied with respect to both one-step and multi-step methods.Thereby, an approximation is computed using step size h as usual and an-other approximation is calculated with two steps of size h2 , for example.Both values yield an approximation of a higher order in a correspondingextrapolation.

We want to estimate the error yh y(x0+ h) of the lower order method.Combining (3.16) and (3.17) yields

yhi yi(x0+ h) =y

hi y

hi (yi(x0+ h) y

hi) =y

hi y

hi + O(h

p+2

). (3.18)

Thus yh yh represents an estimator for the local error of order p+ 1.Applying (3.16) and (3.18), it follows

yhi yhi =Cihp+1 + O(hp+2). (3.19)We assume that we have already performed an integration step of thesize hused. Now we want to estimate an appropriate step size hopt to re-

46


51/187

peat the integration. The properties (3.16) and (3.19) imply approximately

yhusedi y

husedi

.= Cih

p+1

used,yhopti yi(x0+hopt) .= Cihp+1opt.

Eliminating the constant Ci yields

|yhopti yi(x0+hopt)||yhusedi yhusedi |

=

hopthused

p+1. (3.20)

The error estimate of the done step is given by

i:= |yhusedi yhusedi | (3.21)fori= 1, . . . , n. The error corresponding to the new step size shall satisfy

|yhopti yi(x0+hopt)| = TOL (3.22)for some given absolute tolerance TOL > 0 in all components. We donot want that the error is smaller than TOL, since a smaller error impliessmaller step sizes and thus more computational effort due to more steps.

Inserting the last two relations in equation (3.20) implies

hopt,i=hused p+1

TOL

i,

where each component exhibits a different step size. The size for the newstep is chosen as

hnew= mini=1,...,n

hopt,i

including some safety factor = 0.9, for example. To avoid oscillating stepsizes, the restriction

hused hnew husedis imposed with 0<


52/187

then the integration step is accepted and the next step is done using hnewas suggested step size.

Often the tolerance is defined relatively with respect to the magnitude of thesolution. Given some relative tolerance RTOL > 0 and absolute toleranceATOL> 0, we arrange

TOL = ATOL + RTOL |yhusedi |.The absolute part ATOL is required in case of|yhusedi | 0. Typical valuesare RTOL = 103 and ATOL = 106, for example.

Using the modulus || like above, i.e., some kind of maximum norm, exhibitsa lack of smoothness, which sometimes causes problems in the simulations.In practice, the scaled norm

ERR =

1n

ni=1

yhusedi yhusedi

ATOL + RTOL |yhusedi |

2(3.23)

ia applied, which represents a kind of weighted Euclidean norm. Note that

denominators are always positive in (3.23). The condition (3.22) corre-sponds to ERR = 1 now. The new step size becomes

hnew= hused 1p+1ERRusing some safety factor again.

The estimation is done for the error in the method of order p, whereas theresult of the method of order p + 1 is only used in the error estimation.

However, the approximation of order p+ 1 is often applied as the outputof the algorithm after each integration step. This is reasonable, since themethod of order p+ 1 is usally more accurate.

The above approach controls the local error in each integration step. How-ever, we like to select the step sizes such that the global error (3.8) satisfiesa predetermined accuracy. Yet there are no satisfactory strategies to controlthe global error. Hence numerical integrators of common software packages(e.g. MATLAB) just perform a step size selection based on the local error.

48


53/187

Embedded techniques

It remains to choose the two numerical methods in the estimation of the localerror. In case of Runge-Kutta methods, embedded schemes are applied,since the additional computational work for the second approximation isrelatively low.

The Butcher tableau of an embedded scheme reads

c1 a11 a12 a1sc2 a21 a22 a2s...

...

...

. ..

...cs as1 as2 ass

b1 b2 bsb1 b2 bs

with two sets of weights bi and bi, respectively. The corresponding approx-imations are

yh = y0+h(b1k1+ +bsks),yh = y0+h(b1k1+

+ bsks).

If the datak1, . . . , ks for computing the approximationyh is available, then

the second approximation yh can be calculated with nearly no additionaleffort.

In case of explicit Runge-Kutta methods, the class of the Runge-Kutta-Fehlberg methods represents embedded schemes.

Example: Runge-Kutta-Fehlberg 2(3)

014

14

2740

189800

729800

1 214891

133

650891

214891

133

650891 0

5332106 0

8001053 178

49


54/187

Chapter 4

Multistep Methods

4

In this chapter, we investigate multistep methods, i.e., several old approxi-mations are used to construct a new approximation. In contrast to one-steptechniques, consistency alone is not sufficient for the convergence of thesemethods.

4.1 Techniques based on numerical quadrature

We introduce an important class of multistep schemes now. The strategy isbased on the integral equation (2.2). A polynomial interpolation is arrangedand the exact integral of the polynomial yields an approximation.

We consider the initial value problemy=f(x, y), y(x0) =y0, see (2.1). Forthe following discussions, we assume a scalar ODE, since the strategy can beapplied in each component of a system separately. Let the approximations

(xik+1, yik+1), (xik+2, yik+2), . . . , (xi1, yi1), (xi, yi) (4.1)

be given for some integerk 1. We want to construct a new approximation(xi+1, yi+1). Choosing some integer l 1, the exact solution satisfies the

50


55/187

integral equation

y(xi+1) = y(xil+1) + xi+1xil+1

y(s) ds

= y(xil+1) + xi+1xil+1

f(s, y(s)) ds.(4.2)

Now we approximate the integrandf(x, y(x)). We arrange the polynomialpk,i Pk1, which interpolates the data

(xj, f(xj, yj)) for j =i k+ 1, i k+ 2, . . . , i 1, i.Consequently, it holds

pk,i(xj) =f(xj, yj) for j =i k+ 1, i k+ 2, . . . , i 1, i.The interpolating polynomial is unique. Using a Lagrange basis

Li,j(x) =k

=1,=j

x xi+1xij+1 xi+1 for j = 1, . . . , k ,

the polynomial becomes withfi:=f(xi, yi)

pk,i(x) =k

j=1

fij+1Li,j(x).

Due to the assumption pk,i(x) f(x, y(x)) in the considered domain, thenew approximation becomes due to (4.2)

yi+1= yil+1+k

j=1

fij+1 xi+1xil+1

Li,j(s) ds.

Since the Lagrange polynomials are known, the integral can be evaluatedexactly.

In most cases, it holds l k, i.e., the interval of the interpolation containsthe interval of the integration (to the left-hand side). Fig. 11 illustrates thisstrategy. We have achieved an explicit k-step method.

51


56/187

x xx

p(x)

x xik+1 il+1 i i+1

Figure 11: Construction of multistep method by quadrature.

In case of an equidistant grid xi = x0+ ih, the integrals of the Lagrangepolynomials are independent of the index i xi+1

xil+1

Li,j(s) ds =

xi+1xil+1

=j

s xi+1xij+1 xi+1 ds

= h 1

1l=jx0+ (i +u)h (x0+ (i + 1)h)

x0+ (i j+ 1)h (x0+ (i + 1)h)du

= h

11l

=j

u+ 1j du.

It follows a method

yi+1=yil+1+hk

j=1

jf(xij+1, yij+1)

with the constant coefficents

j :=

11l

k=1,=j

u + 1j du for j = 1, . . . , k .

An implicit multistep method results, if we include the unknown new ap-proximation (xi+1, yi+1) in the interpolation. Letqk,i Pk be the interpo-lating polynomial of the data

(xj, f(xj, yj)) for j =i k+ 1, i k+ 2, . . . , i 1, i , i+ 1.52


57/187

It follows

qk,i(xj) =f(xj, yj) for j =i k+ 1, i k+ 2, . . . , i 1, i , i+ 1.The corresponding Lagrange polynomials become

Li,j(x) =k

=0,=j

x xi+1xij+1 xi+1 for j = 0, 1, . . . , k

and thus

qk,i(x) =k

j=0 fij+1Li,j(x).We writeqk,i(x; yi+1) to emphasize that the polynomial depends on the newapproximation, which is unknown a priori. We obtain

yi+1=yil+1+ xi+1xil+1

qk,i(s; yi+1) ds.

This relation represents a nonlinear equation for the unknown yi+1. Hencethis approach yields an implicit method with k steps.

In case of equidistant step sizes, the method reads

yi+1=yil+1+hk

j=0

j f(xij+1, yij+1)

with corresponding coefficients

j := 1

1l

k

=0,=ju+ 1

j du for j = 0, 1, . . . , k .

Equivalently, we can write

yi+1 h0 f(xi+1, yi+1) =yil+1+hk

j=1

j f(xij+1, yij+1),

where the right-hand side involves known data and the left-hand side con-tains the unknown new approximation.

53


58/187


59/187

which is a two-step method. Alternatively, the implicit techniques are calledMilne methods. For equidistant step sizes, the case k = 1 results in the

explicit midpoint rule again, i.e., the termfi+1cancels out. The choicek= 2yields the Milne-Simpson rule

yi+1=yi1+h 13(f(xi1, yi1) + 4f(xi, yi) +f(xi+1, yi+1)) ,

which represents an implicit scheme. This method agrees to the Simpsonrule applied in numerical quadrature.

Remark that choicesl 3 in (4.2) are not important in practice. Moreover,the number of steps (max{k, l}) is often not larger than 5 in correspondingsoftware packages.

4.2 Linear difference schemes

We consider a scalar ODE and equidistant step sizes for simplicity. Themultistep methods from the previous section represent specific cases of linearmultistep schemes

kl=0

lyi+l=hk

l=0

lf(xi+l, yi+l). (4.6)

Remark that the ordering of the coefficients is opposite to Sect. 4.1, sinceyi+k represents the new approximation now. It holds k = 0, whereas0 = 0 is feasible, see the Adams methods with k > 1, for example. Ageneral (nonlinear) multistep scheme reads

nl=0

al yi+l=hF(xi, yim, . . . , yi+n) (4.7)

with a function F depending also on the right-hand side fof the systemof ODEs (2.1). We assume a0, an = 0 in (4.7). The integers n, m aredetermined by the method (n is not the dimension of the ODE systemhere). To analyse the stability of a multistep method, it is sufficient toinvestigate the linear difference scheme in the left-hand side of (4.7).

55


60/187

We apply complex numbers in the following. A linear difference equationof ordernreads

L(uj) :n

s=0

asuj+s=cj+n for j = 0, 1, 2, . . . , (4.8)

where the coefficients a0, . . . , an andci fori > n 1 are arbitraryexcept for the assumption a0, an= 0. The mapping Lis called a differenceoperator. We want to determine sequences (ui)i 0 , which satisfy thedifference equation (4.8). An initial condition

ui = vi for i= 0, . . . , n 1 (4.9)with predetermined values v0, v1, . . . , vn1 is required. The solution ofthe initial value problem (4.8),(4.9) results to

uj+n= 1

an

n1s=0

asuj+s+cj+n

for j = 0, 1, 2, . . . (4.10)

and can be computed successively.

The homogeneous difference equation corresponding to (4.8) is

L(uj) = 0 for j = 0, 1, 2, . . . . (4.11)

The solution of an initial value problem is given by (4.10) with ci = 0 for

alli. Since the operatorL is linear: L(u(1)j + u

(2)j ) =L(u

(1)j ) + L(u

(2)j ),

the solutions form a linear space.

Definition 4 The sequences(u()i )i 0 for= 1, . . . , r are called lin-

ear independent, if the relation

r=1

u()i = 0 for all i 0

implies= 0 for all= 1, . . . , r. A set ofn linear independent solutionsof the homogeneous difference equation (4.11) is a fundamental system.

56


61/187

Theorem 6 Let (u()i )i 0 for = 1, . . . , n be a fundamental system of

(4.11). Then each solution (vi)i

0of (4.11) exhibits a unique representa-

tion

vi=n

=1

u()i

with coefficients1, . . . , n .

Proof:

Since the elements of the fundamental system are linearly independent, it

follows that thenvectors (u()0 , . . . , u()n1) n for= 1, . . . , nare linearlyindependent. Thus the matrix

A:=

u(1)0 u(n)0...

...

u(1)n1 u(n)n1

nn (4.12)is regular. Letv = (v0, . . . , vn1) n. The linear systemAx = v withx = (1, . . . , n)

exhibits a unique solution, which represents the desiredcoefficients. Remark that initial value problems of (4.11) exhibit uniquesolutions.

Now we show the existence of a fundamental system for the homogeneousdifference equation (4.11) by construction.

Definition 5 The polynomial

pn(x) :=

ns=0

asxs =a0+a1x+ +an1xn1 +anxn (4.13)

is called the characteristic polynomial of the difference operatorL in (4.8).

Letx1, . . . , xm be the pairwise different roots (zeros) of the character-istic polynomial with the multiplicities r1, . . . , rm:

pn(x) =an(x x1)r1(x x2)r2 (x xm)rm.57


62/187


63/187

The Leipniz rule yields

dkdxk (xipn(x) = k

l=0

kl dkldxklxi dldxlpn(x) .

The property (4.14) yields L(u(k+1,)i ) = 0 for alli.

It remains to show that the n sequences are linearly independent. We ob-serve the square matrix formed by the values u

(k+1,)i fori= 0, 1, . . . , n 1.

The structure agrees to a Van-der-Monde matrix (cf. polynomial interpola-tion) and thus the matrix is regular. It follows that the system of sequences

is linearly independent.

We have shown the existence of a fundamental system (u()i ) for= 1, . . . , n.

We also achieve a standardised fundamental system (w()i ) for = 1, . . . , n

characterised by the initial conditions

w()i1=

1 if i= ,0 if i =,

for i, = 1, . . . , n .

We obtain the standardised system via

w()i =

nj=1

()j u

(j)i ,

where the coefficients x() = (()1 , . . . ,

()n ) follow from the linear system

Ax() = ewith the transformation matrix (4.12) and the th unit vectore= (0, . . . , 0, 1, 0, . . . , 0)

.

Lemma 3 Let(u

()

i )for= 0, 1, . . . , n1be the standardised fundamentalsystem of the homogeneous difference equation (4.11). Then the solution(ui)of the initial value problem (4.8), (4.9) of the inhomogeneous differenceequation is given by

ui=n1=0

vu()i +

1

an

ink=0

ck+nu(n1)ik1 for i= 0, 1, 2, . . . (4.15)

with the definitionsu(n1)j = 0 forj


64/187

Proof:

The first sum in (4.15) satisfies the homogeneous difference equation (4.8)as well as the initial conditions (4.9). We have to show that the secondsum fulfills the inhomogeneous difference equation (4.8) with initial valuesidentical to zero. Let

wi:= 1

an

ink=0

ck+nu(n1)ik1.

Due to the definition u(n1)j = 0 for j


65/187

A fundamental system (u()i ) for= 1, . . . , nis bounded, if it holds

|u()i | C for all i 0 and all = 1, . . . , nwith some constant C > 0. For the specific fundamental system fromTheorem 7, it follows C 1.

Lemma 4 A fundamental system of the linear difference scheme (4.8) isbounded if and only if all fundamental systems are bounded.

Proof:

Let (u()i ) for = 1, . . . , n be a bounded fundamental system. Given an

arbitrary fundamental system (v()i ) for= 1, . . . , n, it holds

v(j)i =

n=1

,ju()i for j = 1, . . . , n

with unique coefficients ,j

due to Theorem 6. It follows

|v(j)i | n

=1

|,j| |u()i | n

max,j

|,j|

max=1,...,n

|u()i | n

max,j

|,j|

C.

Thus the arbitrary fundamental system is bounded.

We have introduced the root condition, because we need the following prop-erty.

Theorem 8 A fundamental system of the linear difference scheme (4.8) isbounded if and only if the corresponding characteristic polynomial (4.13)

fulfills the root condition.

Proof:

Due to Lemma 4, it is sufficient to investigate the fundamental system (u()i )

for= 1, . . . , nfrom Theorem 7.

61


66/187

Let the root condition be satisfied. For simple roots with |x| 1, it follows

u(1,)i = |xi| = |x|i 1for eachi. For multiple roots|x| 1, then it followsu(1,)i = |xi| = |x|i .In case of a multiple root (r 2) with|x| = 1, we obtain

u

(2,)i

= |ixi| =i |x|i =i .

In both cases, the fundamental system becomes unbounded.

Stability often means the Lipschitz-continuous dependence on perturbationsin the initial data. In case of a homogeneous linear difference equation (4.11)and initial values (4.9) zero, the solution becomes identical to zero. Initialvalues not equal to zero can be seen as a perturbation of this solution. Let(u

()i ) for = 0, 1, . . . , n 1 be the standardised fundamental system. If

and only if the root condition is satisfied, then this system is bounded, i.e.,

|u

()i

| Cwith a constant C > 0. For initial values v0, v1, . . . , vn

1

,

the corresponding solution becomes

vi=n1=0

vu()i .

It follows

|vi| n1=0

|v| |u()i | Cn1=0

|v|.

62


67/187

Thus the solution (vi) depends Lipschitz-continuously on the perturbationsv0, v1, . . . , vn

1.

Now consider two solutions (vi) and (wi) of the inhomogeneous linear dif-ference equation (4.8). It holds

L(vi wi) =L(vi) L(wi) =ci+n ci+n= 0,i.e., the difference solves the homogeneous equation (4.11). Thus we canrepresent the difference by the standardised fundamental system

vi wi=n1=0

(v w)u()i .

It follows

|vi wi| Cn1=0

|v w|

for eachi. We recognise the Lipschitz-continuous dependence on the initialdata again.

4.3 Consistency, stability and convergence

We consider an initial value problem (2.1) of a scalar ODE. We apply anequidistant grid

xi=x0+ih for i= 0, 1, . . . , N with h:=xend x0

N .

Letyi :=y(xi) be the values of the exact solution, whereas ui denotes thenumerical approximations. Now we define a local discretisation error of amultistep method. The scheme (4.7) can be written in the form

1

h

ns=0

asui+s F(xi, uim, . . . , ui+n) = 0.

Inserting the exact solution y(x) in this formula yields a defect, which isthe local discretisation error.

63


68/187

Definition 7 (local discretisation error of multistep methods)Let y(x) be the exact solution of the ODE-IVP y = f(x, y), y(x0) = y0.The local discretisation error of the multistep method (4.7) is defined as thedefect

(h) :=1

h

ns=0

asy(xi+s) F(xi, y(xim), . . . , y(xi+n)). (4.16)

This definition agrees to the local error of one-step methods, cf. (3.7).

For example, we consider an explicit linear multistep method (4.6). The approximationbecomes (k = 0)

kui+k+k1l=0

lui+l =hk1l=0

lf(xi+l, ui+l).

If the involved initial values are exact (ui+l=y(xi+l) for l= 0, . . . , k 1), then it holds

kui+k+k1l=0

ly(xi+l) =hk1l=0

lf(xi+l, y(xi+l)).

The exact solution satisfies

ky(xi+k) +k1l=0

ly(xi+l) =h

k1l=0

lf(xi+l, y(xi+l)) +h (h).

It follows(h) =

kh

(y(xi+k) ui+k).The linear multistep method (4.6) can be normalized by setting k:= 1.

Again we define a consistency according to Def. 2.

Definition 8 (consistency of a multistep method)The multistep method (4.7) is consistent if the local discretisation error

from (4.16) satisfieslimh0

(h) = 0

uniformly inx, y. The method is consistent of (at least) orderp, if it holds(h) = O(hp).

64


69/187

In our discussion, we include errors in the initial values as well as roundofferrors in each step of the method now. We always have these errors in

practice. Thus we ask if the final approximation is still convergent in thepresence of the errors. We consider an interval [x0, xend] and equidistantstep sizes h= xendx0N . The multistep method (4.7) becomes

ui = yi+i for i= 0, 1, . . . , m+n 1,n

s=0

asui+s = hF(xi, uim, . . . , ui+n) +hi+n

for i= m, m+ 1, . . . , N

n

(4.17)

with the errors 0, . . . , Nand the exact solution yi=y(xi).

According to (4.16), the local discretisation error exhibits the form

i+n= 1

h

ns=0

asyi+s F(xi, yim, . . . , yi+n) for i= m, . . . , N n.

We make the following assumptions:

(i) It exists a function (h) 0 forh 0 such that|i| (h) for all i= 0, . . . , N . (4.18)

(ii) If the right-hand side of the ODE becomes f(x, y) 0, then it followsF(xi, uim, . . . , ui+n) 0 for all i= m, . . . , N n

and all h

0.

(iii) The functionF is Lipschitz-continuous: For allu, v m+n+1, it holds

|F(xi, vim, . . . , vi+n) F(xi, uim, . . . , ui+n)| Kn

=m|vi+ ui+|

(4.19)for eachi= m, . . . , N n, where the constant K 0 depends just onthe right-hand side f(and its derivatives).

65


70/187

(iv) It exists a function (h) 0 forh 0 with

|i+n

| (h) for all i= m, . . . , N

n. (4.20)

In case of a linear multistep method (4.6), the assumption (iii) is satisfied if the right-handside fexhibits a Lipschitz-condition (2.3):

|F(xi, v) F(xi, u)| =

n=m

f(xi+, vi+) n

=m

f(xi+, ui+)

n=m

|| |f(xi+, vi+) f(xi+, ui+)|

n

=m|| L |vi+ ui+| L

max

j=m,...,n|j| n

=m

|vi+ ui+|

= L

max

j=m,...,n|j|

v u1.

According to Def. 8, the consistency of a multistep scheme implies the ex-istence of a function(h) from assumption (iv) with

limh0

(h) = 0.

The convergence of the method is defined as follows. (The same definitioncan be done for one-step methods, if the influence of errors 0, . . . , N isconsidered.)

Definition 9 (convergence of multistep method)Assume that the function(h) from (4.18) satisfies

limh0

(h) = 0.

The multistep method (4.17) is convergent, if it holds (h= xendx0N )

limh0

maxi=0,...,N

|ui y(xi)|

= 0.

The method is convergent of (at least) orderp if

maxi=0,...,N

|ui y(xi)| = O(hp)holds provided that(h) = O(hp) is satisfied.

66


71/187

The following theorem (Dahlquist 1956) connects consistency and conver-gence. However, the stability of the involved difference scheme is required.

Theorem 9 (convergence of multistep methods)Let the multistep method (4.7) be consistent with respect to the ODE-IVPy =f(x, y), y(x0) =y0. The method (4.7) is convergent if and only if thecorresponding linear difference scheme is stable.

Proof:

1.) We assume that the root condition from Def. 6 is violated. We constructan example, which does not converge. Considerf(x, y) 0, which impliesthe solution y(x) 0 for the IVP y(x0) = 0. It followsF 0 due to ourassumption (iii).

Let be a simple root ofpn(x) with|| > 1 or a multiple root with

|| 1. We define the perturbations in (4.17) via

i:= h(i +i) if

|

|>1

hi(i +i) if || = 1 for i= 0, . . . , n 1and i := 0 for i = n, . . . , N . Since n > 0 is a fixed integer, it holds(h) = O(h). The multistep method (4.17) becomes

ui=i for i= 0, . . . , n 1,n

s=0

asui+s= 0 for i= n, . . . , N n.

Due to our construction, the solution of this difference scheme is just

ui=

h(i +

i) if || >1

hi(i +i) if || = 1 for i= 0, . . . , N .

Remark thatui for alli. It follows

uN= (xend x0)

1N(

N +N

) if || >1,(N +

N) if || = 1.

67


72/187

Due to= ||ei,j = ||jeij,j + j = 2||j cos(j), it follows for the finalapproximation limh

0 uN

= 0. Hence the convergence is violated.

2.) Vice versa, we assume that the root condition from Def. 6 is satisfiednow. The global errors are ei:=ui yi. We define

ci+n:=h(F(xi, uim, . . . , ui+n) F(xi, yim, . . . , yi+n)) +hi+n hi+n.Subtraction of (4.7) and the relation of the exact solution (see (4.16)) yields

em+k = m+k for k= 0, 1, . . . , n 1,n

s=0

asei+s = ci+n for i= m, m+ 1, . . . , N n . (4.21)

According to Lemma 3, the solution of this difference scheme exhibits theform

ei+m=n1k=0

em+ku(k)i +

1

an

ink=0

ck+m+nu(n1)ik1 (4.22)

fori = 0, . . . , N m, where (u(k)i ) fork = 0, 1, . . . , n 1 is the standardisedfundamental system corresponding to L(uj) = 0. Since the root conditionis assumed, the fundamental system is bounded:

|u(k)i | Q for k= 0, 1, . . . , n 1 and all i 0with a constant Q 1. Due to the assumptions (4.18),(4.19) and (4.20), itfollows

|ck+m+n| h

Km+n=0

|ek+| +(h) +(h)

.

Now we estimate (4.22)|ei+m| Qn max

k=0,...,

numerical analysis - roalnd pulch

Documents