Numerical Methods for Ordinary Differential Equations

P. Grohs

July 27, 2015

Contents

1 Preliminaries 21.1 Initial Value Problems (IVPs) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2.1 Resource Constrained Population Growth . . . . . . . . . . . . . . . . . . . . . . . 31.2.2 Predator-Prey Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.2.3 Mathematical Pendulum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.2.4 The Goal of Numerical Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.3 Basic Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71.3.1 Existence of Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71.3.2 Linear IVPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121.3.3 Sensitivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2 Single Step Methods (SSMs) 152.1 Basic Definitions and Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.1.1 Abstract Single Step Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152.1.2 Consistency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162.1.3 Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.2 Collocation and Runge Kutta schemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.2.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.2.2 Well Definedness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212.2.3 Autonomization Invariance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232.2.4 (*) Solvability of Stage Equations Revisited . . . . . . . . . . . . . . . . . . . . . . 24

2.3 Construction of High Order RK-SSMs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262.3.1 Order Barriers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262.3.2 Collocation Schemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282.3.3 Extrapolation Schemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322.3.4 Splitting Schemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

2.4 (*) Adaptive Timestepping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

3 Stability 403.1 Model Problem Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413.2 Inheritance of Asymptotic Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

3.2.1 Attractive Fixedpoints of ODEs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453.2.2 Attractive Fixedpoints of SSMs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

3.3 Nonexpansiveness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483.4 Uniform Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503.5 (*) Differential Algebraic Equations (DAEs) . . . . . . . . . . . . . . . . . . . . . . . . . . 53

4 Structure Preservation 544.1 Polynomial Invariants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 544.2 Volume Preservation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 574.3 Symplectic Integration of Hamiltonian Systems . . . . . . . . . . . . . . . . . . . . . . . . 59

1

Chapter 1

Preliminaries

1.1 Initial Value Problems (IVPs)

Initial value problems are ubiquitous in science and technology. In this section we introduce basic termi-nology.

Definition 1.1.1 (Ordinary Differential Equation). A first order ordinary differential equation (ODE)is given by a formal expression

y = f(t, y), (1.1)

where

• f : I ×D → Rd (d ∈ N) continuous, the ‘right hand side’,

• I ⊂ R, the ‘time interval’,

• D ⊂ Rd, the ‘state space’, or ‘phase-space’, and

• Ω := I ×D, the ‘augmented state space’.

The vector y ∈ Rd describes the state of a system in terms of d real numbers. The expression (1.1)describes the dynamical evolution of this state with time.

Definition 1.1.2 (Solution of ODE). A function y ∈ C1(J,Rd), where J ⊂ I is a subinterval, is calledsolution of (1.1) if

y(t) = f(t, y(t)) for all t ∈ J. (1.2)

The right hand side of the ODE (1.1) can be interpreted as a vector field, defined on extended phasespace Ω, by mapping each point (t, y) ∈ Ω to the tangent vector (1, f(t, y)), attached to (t, y). Solutions(1.2) are precisely the integral curves of this vector field, see Figure 1.1.

Looking at Figure 1.1 it becomes evident that we have to supplement the ODE (1.1) with suitableinitial conditions to arrive at a unique solution.

Definition 1.1.3 (Initial Value Problem). An initial value problem (IVP) is described by an ODE (1.1),together with initial data (t0, y0) ∈ Ω. A solution of (1.1) is called a solution of the IVP if y(t0) = y0.

Even though (1.1) only represents a first order ODE (in the sense that only first order derivatives ofy are involved) one can transform any ODE to a first order system of the form (1.1), defined on a higherdimensional phase space.

Remark 1.1.4 (Reduction of higher order ODEs to first order ODEs). An n-th order ODE is definedas a formal expression

y(n) = f(t, y, y, . . . , y(n−1)

)(1.3)

for n ≥ 1, t ∈ I, y, y, . . . , y(n−1), y(n) ∈ Rd, f : I × Rnd → Rd. In order to ensure uniqueness of

solutions it has to be augmented with initial conditions(y(t0), y(t0), . . . , y(n−1)(t0)

)T. By defining z(t) :=

2

Figure 1.1: Left: Right hand side f interpreted as vector-field on phase space. Right: Solutions of (1.1)are integral curves of this vector field

(y(t), y(t), . . . , y(n−1)(t)

)T ∈ Rnd and F (t, z) := (z2, z3, . . . , zn, f(t, z1, . . . , zn))T

for z = (z1, . . . , zn)T ∈Rnd and zi ∈ Rd for all i = 1, . . . , n, we see that (1.3) is equivalent to the first order ODE

z = F (t, z),

defined on an nd-dimensional phase space.

A particularly important class of ODEs are so-called autonomous systems.

Definition 1.1.5 (Autonomous ODE). An ODE is called autonomous if the right hand side f does notdepend on t, i.e. f(t, y) = f(y)

Remark 1.1.6 (Autonomization). Any ODE can be rewritten as an autonomous ODE on a higher-dimensional phase space by considering time t ∈ I as a state variable. Writing z := (t, y) we see that(1.1) is equivalent to the autonomous ODE z = F (z), where F (z) := (1, f(t, y)).

1.2 Examples

Let us now look at some instructive examples of ODEs.

1.2.1 Resource Constrained Population Growth

Suppose that y(t) models a population density at time t. A very simple model for the dynamics ofpopulation growth (or interest rates) is given by the ODE

y = ry,

which means that the growth rate at a time t is proportional to the current population, leading to aninfinitely sustained exponential growth.

Suppose that the population has to live off a finite amount of resources which get scarcer as thepopulation increases. In that case, the growth factor r may depend on y in a monotonically decreasingfashion.

This leads to the following simple model equation, known as the logistic equation:

y = (α− βy)y, α, β > 0.

3

0 0.5 1 1.50

0.5

1

1.5

t

y

Figure 1.2: Phase plot of logistic growth ODE. We see that solutions approach quickly the stationarystate.

The solution to the corresponding IVP with y(0) = y0 can be given explicitly as

y(t) =αy0

βy0 + (α− βy0) exp(−αt).

The logistic equation has the interesting property that it posesses two states where there is no growthor decline in the population: Suppose that at some time t we have y(t) = 0 or y(t) = α/β. Then thelogistics equation yields y(t) = 0 and the system doesn’t change. Such states are called stationary points.

Definition 1.2.1 (Stationary Point). A point (t, y) ∈ Ω is called a stationary point of the ODE (1.1) iff(t, y) = 0.

The two stationary points 0 and α/β behave quite differently. Solutions which start near 0 quicklymove away from 0 while general solutions approach the second stationary point α/β very quickly, seeFigure 1.2. In the case of y = α/β we speak of an attractive stationary point.

1.2.2 Predator-Prey Model

Now let us suppose that we have a population with two different species whose population densities aremodeled by u and v. v is supposed to consist of predators while u reprensents prey. Here, the dynamicsis as follows: if there is a lot of prey, then the predator population has a lot to eat and will grow, whilethe prey will be dimished, which in turn also leads to a slower growth of the predator population. Oncethe predator population becomes small enough, the prey population can recover and grow.

A mathematical model is given by the Lotka-Velterra system

u = (α− βv)u

v = (δu− γ)v, (1.4)

for coefficients α, β, γ, δ > 0. This is a two-dimensional system of ODEs. It possesses the stationarypoint (u, v) = (γ/δ, α/β). The Lotka-Volterra system possesses another important qualitative property.Indeed, the Lotka-Volterra ODE implies that

0 = (δ − γ/u)u− (α/v − β)v

4

0 0.5 1 1.5 2 2.5 3 3.5 40

1

2

3

4

5

6

X

u = y1

v =

y2

Figure 1.3: Solution Trajectories of the Lotka Volterra Equation.

for all solution trajectories. Putting

I(u, v) := δu− γ log(u)− α log(v) + βv (1.5)

We getd

dtI(u(t), v(t)) = 0

for all solution trajectories (u(t), v(t)) satisfying (1.4). This implies that the value I(u, v) remains constantalong (u(t), v(t)), or in other words, I is an invariant of the ODE (1.4):

Definition 1.2.2 (Invariant). Consider an ODE (1.1). If there exists a function I : D → R such thatfor all solutions y(t) of (1.1) we have

I(y(t)) ≡ const

then I is called an invariant of (1.1).

Since I as defined in (1.5) is an invariant of (1.4), we know that all solution trajectories (u(t), v(t))can be described as level curve of the function I. Since the function I is strictly concave, all such levelcurves are closed, see Figure 1.3. All solution trajectories with u(0), v(0) > 0 are periodic.

1.2.3 Mathematical Pendulum

Consider a pendulum modeled as a stiff rod of length l which is fixed at one endpoint and allowed torotate freely with a masspoint of mass m at the other end. The state of the pendulum can be uniquelydescribed by the rotation angle α ∈ [0, 2π). Suppose that a gravitational force acts on the pendulum.The angular force acting on the pendulum is given by −mg sin(α(t)), where g is the earth’s gravitationalconstant, see Figure 1.4. Then Newton’s law states that the motion of the system is governed by

mlα(t) = −mg sin(α(t)).

We have here a second order ODE but By Remark 1.1.4 we know how to transform this to a first orderODE. Writing q(t) = α(t) and p(t) = α(t) we get the equivalent system (where we set m = 1 for

5

mg

Z

x1

x2

g

0

Figure 1.4: Mathematical Pendulum.

simplicity)

p = −gl

sin(q)

q = p (1.6)

This equation possesses a lot of structure. Let us define the function

E(p, q) :=1

2l2p2 − lg cos(q)

which represents the total energy of the system (the first term the kinetic energy, the second term thepotential energy). It is easy to see that E is an invariant of (1.6), e.g. energy is conserved as it should be.Figure 1.5 plots the isolines of E in the (p, q)-plane. There exist both periodic and non periodic solutiontrajectories.

Equation (1.6) is a prototypical example of a Hamiltonian system.

Definition 1.2.3 (Hamiltonian System). Let M ⊂ Rn be a domain and suppose that H : Rn ×M →Mis a C1 function. Then the Hamiltonian system with Hamiltonian H is given by the first order system ofODEs

p = −∂H∂q

(p, q)

q =∂H

∂p(p, q). (1.7)

The Hamiltonian typically represents the total energy (often as a sum of kinetic and potential energy)of a mechanical system and Hamilton’s equations as defined above define its dynamics.

Lemma 1.2.4. The Hamiltonian H is an invariant of the associated Hamiltonian system.

6

q = α

p =

ve

loc

ity

−3 −2 −1 0 1 2 3−3

−2

−1

0

1

2

3

Figure 1.5: Solution Trajectories of Mathematical Pendulum.

Proof. We simply differentiate for a solution (p(t), q(t)) of (1.7)

d

dtH(p(t), q(t)) =

∂H

∂p(p(t), q(t))p(t) +

∂H

∂q(p(t), q(t))q(t)

(1.7)= −∂H

∂p(p(t), q(t))

∂H

∂q(p(t), q(t)) +

∂H

∂q(p(t), q(t))

∂H

∂p(p(t), q(t)) = 0.

1.2.4 The Goal of Numerical Methods

Now that we have seem some concrete examples we can ask how to simulate ODEs on a computer.Note that typically ODEs do not possess explicitly given solutions which makes a numerical solutionunavoidable. Since we only have finite computing capabilities at our disposal we will in general make anerror. The first question that we will study is how to design numerical schemes and quantify the tradeoffbetween efficiency (e.g. number of flops) and accuracy (error to the true solution). But besides that,most of the examples we have seen so far also possess distinct qualitative features (attractive stationarypoints, preservation of invariants, ...). Can we design numerical schemes which respect these qualitativefeatures? These questions will be addressed in this lecture.

1.3 Basic Theory

Before we go to study numerical methods we first need to gain a certain amount of theoretical understand-ing of ODEs. To this end, the present section contains some basic theoretical results for the mathematicalsolution of ODEs.

1.3.1 Existence of Solutions

First of all we study under which conditions an IVP possesses a unique maximally extended solution asdefined below.

7

Definition 1.3.1 (Maximally Continued Solution). A solution y ∈ C1([t0, t+), D) of the ODE (1.1) iscalled maximally continued if exactly one of the following holds:

(i) t+ =∞ (global existence),

(ii) t+ <∞ and lim supt→t+;t<t+ ‖y(t)‖ =∞ (blowup),

(iii) t+ <∞ and lim inft→t+; t<t+ dist ((t, y(t)), ∂Ω) = 0 (collapse).

Example 1.3.2 (Doomsday: Friday, 13 November 2026). In 1960 Von Foerster et. al. have proposed amodel for human population growth, according to the ODE

y = ry1+b

for some b ∼ 1. The motivation was that the human growth rate grows (almost) linearly with the popu-lation density due to the fact that a larger population leads to larger development. For b = 1 the explicitsolution to the ODE is given as

y(t) =1

1/y0 − rtwhich has a blowup at t = 1/ry0. If this model is correct our population will explode in finite time. Basedon statistical data, Von Foerster et. al. calculated the blowup date to be Friday, 13 November 2026. Westill have some time left...

Example 1.3.3 (Collapse). Consider the IVP

y = −y−1/2 y(0) = 1.

Its solution is given byy(t) = (1− 3t/2)2/3.

For t = 2/3 the function y is still well-defined but the right hand side is not.

Clearly, in order to prove satisfactory results we need to impose conditions on the right hand sidef . The following local Lipschitz condition turns out to be very fruitful in the sense that it is a weakcondition on f which nevertheless suffices to establish powerful mathematical results.

Definition 1.3.4 (Local Lipschitz Condition). f : Ω → Rd is locally Lipschitz continuous if for all(t, y) ∈ Ω there exists δ > 0 and a constant L > 0 such that for all z, w ∈ D with ‖z − y‖, ‖w − y‖ < δand for all τ ∈ I with |τ − t| < δ we have

‖f(τ, z)− f(τ, w)‖ ≤ L‖z − w‖. (1.8)

If f is not locally Lipschitz uniqueness may fail:

Example 1.3.5 (Non uniqueness). Consider the IVP

y = 2(|x|)1/2, y(0) = 0.

Then for any a ≥ 0 the function

ya(t) =

0 t ≤ a

(t− a)2 t ≥ a

solves the IVP.

Local Lipschitz continuity of f implies the existence of a unique solution to a given IVP.

Theorem 1.3.6 (Picard Lindelof). Assume that f : Ω → Rd is locally Lipschitz continuous. Then forall (t0, y0) ∈ Ω there exists a unique, maximally continued solution y ∈ C1 (J(t0, y0), D) with y(t0) = y0

and where J(t0, y0) is the maximal definition interval with t0 ∈ J(t0, y0).

8

Proof. First we show a local version of the statement, namely that there exists a time t2 > t0 such that aunique solution exists on the interval [t0, t2]. The general statement can be established by iterating thisargument. The basic idea is to rewrite (1.1) as an equivalent fixed point problem

y(t) = y0 +

∫ t

t0

f(s, y(s))ds. (1.9)

Now, for t > t0 and δ > 0 to be determined later we define the Banach space

F t,δ :=

z ∈ C([t0, t],R) : z(t0) = y0 and max

s∈[t0,t]‖z(t)− y0‖ < δ

with norm ‖z‖Ft,δ := maxs∈[t0,t] ‖z(s)‖. Define the operator

T :

F t,δ → C([t0, t],R)

z 7→(s 7→ y0 +

∫ st0f(τ, z(τ))dτ

)By continuity, for each t > t0 we have

M := maxf(s, y) : s ∈ [t0, t], ‖y − y0‖ < δ <∞.

Let us now pick t1 > t0 with(t1 − t0)M < δ. (1.10)

We claim that with this choice of t1 the operator T maps F t1,δ into itself. To establish this we need toverify that

sups∈[t0,t1]

‖Tz(s)− y0‖ < δ

for all z ∈ F t1,δ. To see this we estimate

‖Tz(s)− y0‖ = ‖∫ s

t0

f(s, z(s))ds‖ ≤ (s− t0)M ≤ (t1 − t0)M < δ

which is valid by the choice (1.10). Let L be the local Lipschitz constant of f in (1.8) for τ ∈ [t0, t1] and‖z − y0‖, ‖w − y0‖ < δ. Now we claim that the map T is a contraction for every t2 > t0 which satisfiest2 ≤ t1 and

ρ := (t2 − t0)L < 1 (1.11)

To see this we estimate for z, w ∈ F t2,δ and s ∈ [t0, t2]

‖Tz(s)− Tw(s)‖ ≤∫ s

t0

|f(τ, z(τ))− f(τ, w(τ))| dτ

≤ L(s− t0)‖z − w‖Ft2,δ ≤ ρ‖z − w‖Ft2,δ

It follows that‖Tz − Tw‖Ft2,δ ≤ ρ‖z − w‖Ft2,δ ,

which shows that T is a contraction on the Banach space F t2,δ. Now we can apply Banach’s fixed pointtheorem which yields the existence of a unique fixed point y ∈ F t2,δ with Ty = y. It follows that y is theunique solution of the IVP y = f(t, y), y(t0) = y0 on the interval [t0, t2]. The same argument shows thatthe unique solution y can be extended locally to the left of t0.

Now we will iterate this construction to obtain a maximally extended solution. Let

t+ := supt>t0

There exists a unique solution y(t) on the interval [t0, t).

The first possibility is that t+ = ∞ in which case we obtain global existence. Let us rule this out andassume that t+ <∞. For every t < t+ we have a unique solution and now we study what happens for

b := lim supt→t+; t<t+

y(t). (1.12)

9

First, it can happen that b =∞ in which case we have a blowup.Suppose that b <∞ and that lim inft→t+; t<t+ dist((t, y(t)), ∂Ω) > 0. Then the limes superior from the

left in (1.12) is actually a limit from the left: consider any sequence of (ti)i∈N ⊂ [t0, t+) with limi→∞ ti =t+. Then, since both y(t) and y(t) = f(t, y(t)) are bounded on [t0, t+) it follows that the sequence(y(ti))i∈N is a Cauchy sequence and therefore convergent. Denote y(t+) := limt→t+; t<t+ y(t) (whichexists by the previous paragraph). We can now apply our local arguments from above to conclude thatwe can find a unique solution of (1.1) in an interval around (t+, y(t+)) which contradicts the maximalityof t+. Therefore it must hold that limt→t+; t<t+ dist((t, y(t)), ∂Ω) = 0, which is a collapse.

An immediate corollary is the fact that a priori bounds on the solution yield global existence.

Corollary 1.3.7. Suppose that Ω = Rd+1 and there exists a nonnegative continuous function ψ : R→ R+

such that|y(t)| ≤ ψ(t)

for the solution y to the IVP (1.1) with y(t0) = y0. Then y can be globally extended, that is, J(t0, y0) = R.

Proof. Neither a finite time blowup or collapse can happen by the assumption. By Theorem 1.3.6 wemust have global existence.

Using Corollary 1.3.7 we can obtain global existence for right-hand sides f with sublinear growth.First we introduce the following fundamental lemma.

Lemma 1.3.8 (Gromwall). Let J ⊂ R be an interval, t0 ∈ J , a ∈ R+ and u, β ∈ C(J,R+) with

u(t) ≤ a+

∫ t

t0

β(τ)u(τ)dτ for all t ∈ J. (1.13)

Then

u(t) ≤ a exp

(∫ t

t0

β(τ)dτ

)for all t ∈ J. (1.14)

Proof. Define the function

Ψ(t) := a+

∫ t

t0

β(τ)u(τ)dτ.

It follows thatΨ(t) = β(t)u(t) ≤ β(t)Ψ(t),

by Assumption (1.13). Integrating this inequality yields

log (Ψ(t)) ≤ log(a) +

∫ t

t0

β(τ)dτ.

By Assumption (1.13) we have u(t) ≤ Ψ(t) and therefore the above inequality yields

log (u(t)) ≤ log(a) +

∫ t

t0

β(τ)dτ.

Exponentiating yields (1.14).

We will need Gromwall’s lemma several times later on. For now we use it to establish the followingglobal existence result.

Theorem 1.3.9. Suppose that Ω = Rd+1 and there exist continuous functions c1, c2 ∈ C(R,R+) suchthat

|f(t, y)| ≤ c1(t)|y|+ c2(t).

Then, for any (t0, y0) we have J(t0, y0) = R.

10

Proof. y is a solution to the IVP (1.1) if and only if y(t) = y0 +∫ tt0f(s, y(s))ds. It follows that

|y(t)| ≤ |y0|+∫ t

t0

|f(s, y(s))|ds

which, by our assumption on f gives the bound

|y(t)| ≤ |y0|+∫ t

t0

c2(s)ds+

∫ t

t0

c1(s)|y(s)|ds.

Now we apply Gromwall’s Lemma 1.3.8 with a = (|y0|+∫ tt0c2(s)ds) and u(s) = c1(s) and obtain

|y(t)| ≤(|y0|+

∫ t

t0

c2(s)ds

)exp

(∫ t

t0

c1(s)ds

)=: ψ(t).

Corollary 1.3.7 yields global existence.

Going back to the interpretation of y as a particle moving along with a flow (Figure 1.1) we introducethe following important concept.

Definition 1.3.10 (Flow Map). The family of maps Φt,s : y ∈ D : s ∈ J(t, y) ⊂ D → D is calledevolution or flow of the ODE (1.1) if the function y(s) := Φt,sy solves the IVP with y(t) = y.

In the following lemma we list a few simple properties of Φ.

Lemma 1.3.11 (Properties of the Flow Map). We have

(i)d

dsΦt,sy|s=t = f(t, y)

for all (t, y) ∈ Ω,

(ii)Φt,ty = y

for all y ∈ D, and

(iii)Φt,sy = Φr,s

(Φt,ry

)for all r ∈ [t, s].

Proof. These assertions follow directly from Definition 1.3.10 and Theorem 1.3.6.

It turns out that the three simple properties above already characterize Φ, as shown in the nextlemma.

Lemma 1.3.12. Properties (i), (ii) and (iii) of Lemma 1.3.11 characterize the flow map Φt,s.

Proof. Assume we have a family Ψt,s of maps which satisfy (i), (ii) and (iii) of Lemma 1.3.11. We needto show that the function s 7→ Ψt,sy satisfies the ODE (1.1). Therefore we compute

d

dsΨt,sy = lim

h→0

Ψt,s+hy −Ψt,sy

h

(iii)= lim

h→0

Ψs,s+h (Ψt,sy)−Ψt,sy

h

=d

drΨs,r

(Ψt,sy

)|r=s

(ii)= f(s,Ψt,sy).

This shows that s 7→ Ψt,sy solves (1.1). Finally, property (i) ensures that Ψt,ty = y which by Theorem1.3.6 implies that Ψt,s = Φt,s.

11

For autonomous problems (see Definition 1.1.5 the flow Φ assumes a special form, namely Φt,s onlydepends on s− t.Lemma 1.3.13. The flow of an autonomous ODE satisfies

Φt,sy = Φ0,s−ty.

Proof. We have to show that the map y(s) := s 7→ Φ0,s−t satisfies the autonomous ODE y = f(y). Tothis end let z(r) := Φ0,r. We have y(s) = z(s− t). By definition we have

z(r) = f(z(r))

and thereforey(s) = z(s− t) = f(z(s− t)) = f(y(s)).

Remark 1.3.14. Motivated by Lemma 1.3.13 we write Φt := Φ0,t for the flow associated to an au-tonomous ODE. The flow Φt satisfies the group property Φt+s = Φt Φs.

1.3.2 Linear IVPs

In this subsection we consider linear ODEs of the form

y = Ay + g(t) (1.15)

with A ∈ Rd×d, g(t) ∈ Rd. It turns out that the corresponding IVP possesses a solution which can becalculated explicitly in terms of the matrix exponential as defined below:

Definition 1.3.15 (Matrix Exponential). For A ∈ Rd×d we define the matrix exponential exp(A) ∈ Rd×dvia the absolutely convergent series expansion

exp(A) =

∞∑i=0

Ai

i!.

Lemma 1.3.16. The solution of the IVP y = Ay + g(t) with A ∈ Rd×d, g(t) ∈ Rd, y(t0) = y0 can bewritten as

y(t) = exp((t− t0)A)y0 +

∫ t

t0

exp((t− τ)A)g(τ)dτ. (1.16)

Proof. Simply differentiate (1.16)...

We also consider a more general type of linear IVP, namely the problem

y = A(t)y + g(t) (1.17)

with A(t) ∈ Rd×d, g(t) ∈ Rd. We first consider the homogenous case g ≡ 0 and observe that thecorresponding evolution can be written as a family of linear maps:

Φt,sy = E(t, s)y with E(t, s) ∈ Rd×d

andd

dsE(t, s) = A(s)E(t, s), E(t, t) = I. (1.18)

Lemma 1.3.17. The solution of the IVP y = A(t)y + g(t) with A(t) ∈ Rd×d, g(t) ∈ Rd, y(t0) = y0 canbe written as

y(t) = E(t0, t)y0 +

∫ t

t0

E(τ, t)g(τ)dτ, (1.19)

where E is defined by (1.18). If A(t) and g(t) are defined on R, then the solution y(t) is globally defined.

Proof. Simply differentiate (1.19)... The global existence of y follows from Theorem 1.3.9, together withthe estimate (for f(t, y) = A(t)y + g(t))

|f(t, y)| ≤ ‖A(t)‖|y(t)|+ |g(t)|.

12

1.3.3 Sensitivity

Here we address the question how the solution trajectory y(t) depends on the initial value y0.We shall require a slightly stronger requirement on the right hand side f than local Lipschitz continuity

as in (1.8).

Definition 1.3.18 (Global Lipschitz Condition). The function f : Ω → Rd fulfills a global Lipschitzcondition if for every t ∈ I there exists a constant L(t) > 0 such that

‖f(t, x)− f(t, y)‖ ≤ L(t)‖x− y‖ for all x, y ∈ D. (1.20)

Globally Lipschitz IVPs are always stable under perturbation of the initial values as the followingresult shows.

Theorem 1.3.19 (Stability of the Evolution under Perturbation of Initial Value). Assume that f satisfiesa global Lipschitz condition as in Definition 1.3.18. Let Φt,s be the flow (compare Definition 1.3.10)associated with the ODE (1.1). Then

‖Φt0,ty − Φt0,tz‖ ≤ ‖y − z‖ exp

(∫ t

t0

L(τ)dτ

). (1.21)

Proof. For simplicity puty(t) := Φt0,ty and z(t) := Φt0,tz.

We have

y(t)− z(t) = y − z +

∫ t

t0

(f(τ, y(τ))− f(τ, z(τ))) dτ

which, by the global Lipschitz condition, implies that

‖y(t)− z(t)‖ ≤ ‖y − z‖+

∫ t

t0

‖y(τ)− z(τ)‖L(τ)dτ.

Applying the Gromwall lemma 1.3.8 with a := ‖y − z‖, β(t) := L(t) and u(t) := ‖y(t)− z(t)‖ yields thedesired estimate.

The estimate in Theorem 1.3.19 is very pessimistic: the constant grows exponentially in (t − t0). Inorder to get a tighter grip on the local sensitivity behavior we can examine the first order variation ofthe solution trajectory with respect to initial data:

Definition 1.3.20 (Wronskian). The Wronskian W (t; t0, z) of the ODE (1.1) at (t0, z) ∈ Ω is definedas

W (t; t0, z) :=∂

∂yΦt0,ty|y=z ∈ Rd×d.

Once an IVP is solved (i.e. the flow s 7→ Φt,sy) it is possible to examine the sensitivity along thetrajectory by solving a matrix ODE.

Lemma 1.3.21. The Wronskian satisfies the matrix ODE

d

dtW (t; t0, z) =

∂f

∂y

(t,Φt0,tz

)W (t; t0, z) W (t0; t0, z) = I ∈ Rd×d. (1.22)

Proof. The result follows by direct calculation and the fact that partial derivatives commute:

d

dtW (t; t0, z) =

d

dt

∂

∂yΦt0,ty|y=z

=∂

∂y

d

dtΦt0,ty|y=z

=∂

∂yf(t,Φt0,ty)|y=z

=∂f

∂yf(t,Φt0,tz)

∂

∂yΦt0,ty|y=z

=∂f

∂y

(t,Φt0,tz

)W (t; t0, z).

13

14

Chapter 2

Single Step Methods (SSMs)

2.1 Basic Definitions and Results

Consider an ODE as in (1.1) with locally Lipschitz right hand side f , compare Definition 1.3.4. Our goalis to compute a numerical approximation of the flow t 7→ Φt0,ty0 for given initial data (t0, y0) ∈ Ω andt ∈ [t0, T ] for some end time T > t0.

2.1.1 Abstract Single Step Methods

The idea of single step methods (SSMs) is to approximate the (in general elusive) flow Φ by a numericalflow Ψ which can be computed from the right hand side f of the ODE (1.1). This approximate flow Ψ isthen applied iteratively for a discrete time grid, defined as follows, until a final time T is reached.

Definition 2.1.1. A discrete grid is defined as

G := t0 < t1 < · · · < tN = T. (2.1)

The meshwidth of the grid G is defined as

hG := max0≤k≤N−1

(tk+1 − tk).

Now we can define formally the notion of single step methods.

Definition 2.1.2 (Single Step Methods (SSM)). Given a discrete evolution map Ψt,s and a time GridG as in (2.1), a single step method for the IVP y = f(t, y), y(t0) = y0, is defined via the recursion

yk+1 := Ψtk,tk+1yk.

For h = hG we define the function yh : [t0, T ]→ Rd by

yh(tk) = yk for k = 0, . . . , N and yh is linear on (tk, tk+1) for k = 1, . . . , N − 1.

Example 2.1.3 (Explicit Euler).Ψt,sy = y + (t− s)f(t, y).

Example 2.1.4 (Implicit Euler).

Ψt,sy = y + (t− s)f(s,Ψt,sy

).

Example 2.1.5 (Implicit Midpoint).

Ψt,sy = y + (t− s)f(

1

2(t+ s),

1

2(y + Ψt,sy)

).

15

There is a qualitative difference among the three examples above. While the explicit Euler schemeis defined explicitly, the computation of the evolution associated to the implicit Euler and the implicitmidpoint scheme requires the solution of a nonlinear system of equations.

Definition 2.1.6 (Explicit vs. Implicit). A SSM is called explicit if the evolution Ψt,sy can be computedfrom finitely many evaluation of the right hand side f . Otherwise it is called implicit.

The goal of a single step method is certainly to provide a decent approximation to the true solutiony of the IVP. To this end we introduce the following notions.

Definition 2.1.7 (Discretization Error). For an SSM applied to an IVP we define the discretizationerror

εG := max0≤k≤N

‖y(tk)− yk‖

.

In what follows we will examine convergence properties of SSMs which will require us to quantify thedependence of the discretization error on the meshwidth.

2.1.2 Consistency

In order to have a chance for a SSM Ψ to give a decent approximation to Φ, the flow Φ has to be linkedto the ODE (1.1). This link is formalized in the following definition.

Definition 2.1.8 (Consistency). A SSM Ψ is called consistent (with the ODE (1.1)) if

(i)d

dsΨt,sy|s=t = f(t, y)

for all (t, y) ∈ Ω,

(ii)Φt,ty = y

for all y ∈ D.

Remark 2.1.9. Consistency means that the numerical flow Ψ satisfies properties (i) and (ii) from Lemma1.3.11. The reader might wonder whether one could also find a numerical flow Ψ satisfying all properties(i), (ii) and (iii) of Lemma 1.3.11. However by Lemma 1.3.12 this would already amount to computingthe exact flow Φ.

Lemma 2.1.10. Assume that for all (t, y) ∈ Ω the mapping h 7→ Ψt,t+hy is in C1([0, h∗],Rd) for someh∗ > 0. Then Ψ is consistent if and only if

Ψt,t+hy = y + hψ(t, y, h), where ψ ∈ C([0, h∗],Rd) and ψ(t, y, 0) = f(t, y). (2.2)

Proof. Assume that Ψ is consistent. By the assumption that h 7→ Ψt,t+hy is in C1([0, h∗],Rd) and (i) inDefinition 2.1.8 the function

ψ(t, y, h) =1

h

(Ψt,t+hy −Ψt,ty

)=

1

h

(Ψt,t+hy − y

)is continuous in a neighbourhood of h = 0. By (ii) in Definition 2.1.8 we have

ψ(t, y, 0) =d

dsΨt,t+sy|s=0 = f(t, y).

Now assume that (2.2) holds true. Then, clearly (i) and (ii) in Definition 2.1.8 hold and therefore Ψ isconsistent.

Definition 2.1.11 (Increment Function). The function ψ from Equation (2.2) is called increment func-tion.

16

Definition 2.1.12 (Consistency Error). The quantity

τ(t, y, h) := Φt,t+hy −Ψt,t+hy (2.3)

is called consistency error.

Definition 2.1.13 (Consistency Order). Ψ has consistency order p ∈ N if for all (t, y) ∈ Ω there existsδ > 0 such that for all (s, z) ∈ Ω with |t − s| < δ and ‖y − z‖ < δ there exists h∗ > 0 and a constantC > 0 such that

‖τ(s, z, h)‖ ≤ Chp+1 for all 0 < h < h∗.

2.1.3 Convergence

Now that we have linked a discrete evolution Ψ to the exact solution Φ we want to study how the solutionyh generated as in Definition 2.1.2 converges to the exact solution y as the meshwidth h tends to zero.

Definition 2.1.14 (Convergence Order). Consider an ODE (1.1) and a SSM Ψ. Then Ψ has convergenceorder p ∈ N if for all (t, y) ∈ Ω there exists δ > 0 and C > 0 such that for all (s, z) ∈ Ω with |t− s| < δand ‖y − z‖ < δ and for all grids G we have

Nmaxk=0‖yk − Φt0,tkz‖ ≤ ChpG ,

where yk = Ψtk−1,tkyk−1 and y0 = z.

In this subsection we show that consistency order p ∈ N as defined in Definition 2.1.13 impliesconvergence order p ∈ N as in Definition 2.1.14.

Before we go about this task we establish the following auxilliary and useful result.

Lemma 2.1.15 (Discrete Gromwall). Given numbers (ξk)k∈N0, ξk ≥ 0 for all k ∈ N0. Assume that with

constants p ∈ N and L,C, hk ≥ 0 the inequality

ξk+1 ≤ Chp+1k + (1 + Lhk)ξk for all k ∈ N0.

Then

ξN ≤ C(

maxk=0,...,N−1

hpk

)1

L

(exp

(L

N−1∑k=0

hk

)− 1

)+ exp

(L

N−1∑k=0

hk

)ξ0 for all N ∈ N0.

Proof. We use induction. The case N = 0 is trivially true. We assume that the statement holds for Nand we estimate

ξN+1 ≤ Chp+1N + (1 + LhN )ξN

induction hypothesis≤ Chp+1

N + (1 + LhN )

(C

(N−1maxk=0

hpk

)1

L

(exp

(L

N−1∑k=0

hk

)− 1

)+ exp

(L

N−1∑k=0

hk

)ξ0

)1+LhN≤exp(LhN )

≤ Chp+1N + C

(N−1maxk=0

hpk

)1

L

(exp

(L

N∑k=0

hk

)− 1− LhN

)+ exp

(L

N∑k=0

hk

)ξ0

≤ CN

maxk=0

hpk

(hN +

1

L

(exp

(L

N∑k=0

hk

)− 1− LhN

))+ exp

(L

N∑k=0

hk

)ξ0

= C

(max

k=0,...,Nhpk

)1

L

(exp

(L

N∑k=0

hk

)− 1

)+ exp

(L

N∑k=0

hk

)ξ0,

as desired.

Now we are ready to formulate the following fundamental convergence theorem.

17

Theorem 2.1.16 (Consistency implies Convergence). Let Ψ be a SSM for (1.1) such that h 7→ Ψt,t+hyis in C1([0, h∗],Rd) and such that the increment function ψ (which exists by Lemma 2.1.10) is locallyLipschitz continuous in y ∈ D. Assume that Ψ has consistency order p ∈ N. Then Ψ has convergenceorder p ∈ N.

Proof. We examine the discretization error ek := y(tk)− yk. As in Figure 2.1 we can split

ek+1 = y(tk+1)− ψtk,tk+1y(tk)︸ ︷︷ ︸=:eck+1 consistency error

+ψtk,tk+1y(tk)− ψtk,tk+1yk︸ ︷︷ ︸=:epk+1 propagated error

Let us assume for now that there exists D > 0 such that

‖yk − y(tk)‖ ≤ D for all k = 0, . . . , N, (2.4)

with D independent of h, i.e. yk always lies within a tube around the exact solution trajectory. Underthis condition all local Lipschitz constants can be taken as global constants. We will first gain recursiveestimates for the norms of eck and epk and apply the Gromwall Lemma 2.1.15. First of all, our assumptionon consistency of order p implies that

‖eck+1‖ ≤ Chp+1k ,

where hk := tk+1 − tk. On the other hand we get that

‖epk+1‖ ≤ ‖y(tk)− yk‖+ hkL‖y(tk)− yk‖ = (1 + hkL)‖ek‖,

where L is the Lipschitz constant of the increment function ψ. In summary, putting ξk := ‖ek‖ we haveexactly the equation which is used as assumption in Lemma 2.1.15 which implies that

ξN ≤ C(

maxk=0,...,N−1

hpk

)1

L

(exp

(L

N−1∑k=0

hk

)− 1

)for all N ∈ N0

which is the desired convergence property. We can now verify (2.4) a posteriori. Indeed, let hG < h∗ with

hp∗C1

L

(exp

(L

N−1∑k=0

hk

)− 1

)< D

Then by the same induction argument as above the tube constraint (2.4) remains valid.

Remark 2.1.17. By going through the proof of Theorem 2.1.16 we can observe that we only needed Ψ tobe consistent along the solution trajectory y(t). This will be important later on when we prove consistencyestimates for extrapolation methods.

2.2 Collocation and Runge Kutta schemes

We will now discover a family of SSMs, namely so-called collocation schemes, which possess an intuitiveconstruction principle and several desirable properties. Then we will embed these collocation scheme intoa much richer class of SSMs, the so-called Runge Kutta schemes (RK-SSMs).

Recall that a SSM is defined by a discrete flow map Ψ which is iteratively applied to the initial value.

2.2.1 Definitions

The idea behind collocation methods is to approximate the exact flow y(h) := Φt,t+hy by a polynomialyh ∈ Ps, the space of polynomials of degree ≤ s, and define the discrete flow Ψt,t+hy := yh(h).

Of course we do not know y explicitly so we cannot directly use a polynomial approximation methodfor y. What we do know however is that y satisfies the ODE (1.1), so we are looking for a polynomialyh ∈ Ps, approximately satisfying the ODE (1.1). More precisely, collocation methods require (1.1) tohold for yh ∈ Ps pointwise in s collocation nodes t ≤ τ1 < · · · < τs ≤ t+ h.

18

Figure 2.1: Splitting of discretization error ek = yk − y(tk) into consistency error and propagated error.

Definition 2.2.1 (Collocation Scheme). Given control points 0 ≤ c1 < · · · < cs ≤ 1 the correspondingcollocation scheme Ψ is defined by

Ψt,t+hy := yh(h),

where yh ∈ Ps satisfying

yh(0) = y and yh(τj) = f(τj , yh(τj)) for all j = 1, . . . , s,

and τj = t+ hcj, j = 1, . . . , s.

At this point it is far from obvious that the definition of collocation schemes makes sense, i.e. thatyh is well defined.

Let us define now the general class of Runge Kutta SSMs.

Definition 2.2.2 (Runge Kutta SSM). Given coefficients bi, aij , ci ∈ R, i, j = 1, . . . , s. The correspond-ing s-stage Runge Kutta SSM (RK-SSM) Ψ for the ODE (1.1) is defined by

Ψt,t+hy = y + h

s∑i=1

biki, (2.5)

where the so-called increments ki ∈ Rd satisfy the increment equations

ki = f

t+ cih, y + h

s∑j=1

aijkj

for all i = 1, . . . , s. (2.6)

Again it is at this point not clear that the above definition of RK-SSMs makes sense, i.e. that thestage equations (2.6) possess a unique solution.

A popular way to describe a RK-SSM is via a Butcher tableau:

19

c1 a11 . . . a1s

......

...cs as1 . . . ass

b1 . . . bs

Example 2.2.3. The Butcher tableaux for explicit, implicit Euler and implicit midpoint rule are givenas

0 0

1

1 1

1

12

12

1

Remark 2.2.4. A RK-SSM is explicit if and only if the matrix A = (aij)si,j=1 is strictly lower triangular.

To see this observe that the stage equations (2.6) can be solved explicitly if and only if A is strictly lowertriangular.

Our next aim is to show that collocation schemes are RK-SSMs.To this end we require some preparation. Given interpolation nodes 0 ≤ c1 < · · · < cs ≤ 1, consider

the associated Lagrange polynomial basis Lisi=1 for the polynomial space Ps−1, where

Li(x) =∏j 6=i

x− cjci − cj

. (2.7)

Observe thatLi(cj) = δij (2.8)

which implies the linear independence of the Li’s. For every polynomial p ∈ Ps we have the representation

p(x) =

s∑i=1

p(ci)Li(x). (2.9)

Lemma 2.2.5. Given interpolation nodes 0 ≤ c1 < · · · < cs ≤ 1 the collocation scheme Ψ as defined in

Definition 2.2.1 is a RK-SSM as in Definition 2.2.2 with bi =∫ 1

0Li(τ)dτ and aij =

∫ ci0Lj(τ)dτ .

Proof. Since yh ∈ Ps we have yh ∈ Ps−1 and thus we can use (2.9) to get

yh(t+ hx) =

s∑i=1

yh(t+ hci)Li(x).

By the definition of collocation schemes we thus get

yh(t+ hx) =

s∑i=1

f(t+ hci, yh(t+ hci))︸ ︷︷ ︸=:ki

Li(x)

and, by integration

yh(t+ hx) = y + h

s∑i=1

∫ x

0

Li(τ)dτki.

The last equation implies the desired statement.

For later reference we record here the fact that the Butcher matrix A = (aij)si,j=1 of a collocation

scheme is always regular.

Lemma 2.2.6. For every collocation scheme with cs > 0 the Butcher matrix A = (aij)si,j=1 is regular.

20

Proof. By Lemma 2.2.5 the coefficients aij are given by aij =∫ ci

0Lj(τ)dτ . Assume that there exists

x = (x1, . . . , xs)T ∈ Rs with Ax = 0. Then

s∑j=1

∫ ci

0

xjLj(τ)dτ = 0 for all i = 1, . . . , s. (2.10)

Define q(τ) :=∑sj=1 xjLj(τ) ∈ Ps−1. Then (2.10) implies that∫ ci

ci−1

q(τ)dτ = 0 for all i = 1, . . . , s.

It follows that for all i = 1, . . . , s there exists at least one zero of q in the open interval (ci−1, ci). Butthis would mean that q has s zeros in (0, cs) which, together with the fact that q ∈ Ps−1, implies thatq = 0.

We list two popular choices of Collocation schemes.

Example 2.2.7 (Gauß Collocation Schemes). We outline the construction of s stage Gauß collocationschemes. Take c1, . . . , cs to be the zeros of the s-th Legendre polynomial

ds

dxs(xs(x− 1)s)

The corresponding quadrature rule has order 2s, which is maximal among all s-point quadrature schemes.By Theorem 2.3.14 below the corresponding collocation scheme also has order 2s.

Example 2.2.8 (Radau Collocation Schemes). We outline the construction of Radau collocation schemes.Take c1, . . . , cs to be the zeros of

ds−1

dxs−1

(xs−1(x− 1)s

).

It is easy to see that cs = 1 which will be an important property for some problem classes as we will seelater on. The corresponding quadrature rule has order 2s−1. By Theorem 2.3.14 below the correspondingcollocation scheme also has order 2s− 1.

2.2.2 Well Definedness

We can now turn to the question of well-definedness for general RK-SSMs. The key point is to guaranteesolvability of the stage equations (2.6). As an auxilliary tool we require the following result.

Theorem 2.2.9 (Parameter Dependent Banach Fixedpoint Theorem). Let V ⊂ Rd be closed, U ⊂ Rnopen and F : U × V → V m-times continuously differentiable. Assume that there exists 0 ≤ q < 1 suchthat

‖F (u, z)− F (u,w)‖ ≤ q‖u− w‖ for all z, w ∈ V, u ∈ U. (2.11)

Then there exists a unique m-times continuously differentiable function G : U → V with

F (u,G(u)) = G(u) for all u ∈ U. (2.12)

The next theorem extablishes the fundamental result that all RK-SSMs are well-defined, providedthat h is sufficiently small. Of course this includes also collocation schemes by Lemma 2.2.5.

Theorem 2.2.10 (Well-definedness of RK-SSMs). Let the right hand side f in (1.1) be locally Lipschitzcontinuous on the augmented state space Ω. Then, for every (t0, y0) ∈ Ω there exists a h∗ > 0 suchthat (2.6) is uniquely solvable for the increments for every h < h∗, and such that these are continuousfunctions in h. For f ∈ Cm(Ω,Rd), m ∈ N, these increments are m-times continuously differentiablefunctions of y0, t0, h.

The associated RK-SSM is consistent if and only if

s∑i=1

bi = 1. (2.13)

21

Proof. We start with the well-definedness. By our assumptions there exist constants h1, ρ, L,M > 0 suchthat

‖f(t, z)− f(t, w)‖ ≤ L‖z − w‖ for all (t, z), (t, w) ∈ (t0 − h1, t0 + h1)×Bρ(y0), (2.14)

where we write Bρ(y0) = y ∈ Rd : ‖y − y0‖ ≤ ρ, and

‖f(t, y)‖ ≤M for all (t, z), (t, w) ∈ (t0 − h1, t0 + h1)×Bρ(y0). (2.15)

We now pick an arbitrary θ ∈ (0, 1) and prove the unique solvability of the stage equations (2.6) for allh < h∗ with

h∗ := min

1

‖c‖∞,ρ(1− θ)M‖A‖∞

,θ

L‖A‖∞

. (2.16)

Here we have written c = (c1, . . . , cs)T ∈ Rs, A = (aij)

si,j=1 ∈ Rs×s, ‖c‖∞ = maxsi=1 |ci| and ‖A‖∞ the

operator norm of A, induced by ‖ · ‖∞.Let us now define

U := (−h∗, h∗) ⊂ R, and V :=g = (z1, . . . , zs) ∈ Rsd : zi ∈ Bρ(y0) for all i = 1, . . . , s

⊂ Rsd

and F : U × V → Rsd defined by

F (h, (z1, . . . , zs)) = (F1(h, (z1, . . . , zs)), . . . , Fs(h, (z1, . . . , zs)))T,

where

Fi(h, (z1, . . . , zs)) := y0 + h

s∑j=1

f(t0 + cjh, zj) ∈ Rd i = 1, . . . , s.

We show that F satisfies the assumptions of the parameter dependent Banach fixedpoint theorem 2.2.9.First we verify (2.11) with q = θ. Indeed

smaxi=1‖Fi(h, (z1, . . . , zs))− F (h, (z1, . . . , zs))‖ ≤ h∗‖A‖∞

smaxj=1‖f(t0 + hcj , zj)− f(t0 + hcj , zj)‖

(2.14)

≤ h∗‖A‖∞Ls

maxj=1‖zj − zj‖

(2.16)

≤ θs

maxj=1‖zj − zj‖. (2.17)

It follows that F is a uniform contraction on Rsd with respect to the norm ‖(z1, . . . , zs)T ‖ = maxsi=1 ‖zi‖.

Next we need to show that F maps into V . Consider (z1, . . . , zs) ∈ V and h ∈ U . We need to verify that

smaxi=1‖Fi(h, (z1, . . . , zs))− y0‖ ≤ ρ. (2.18)

First we estimates

maxi=1‖Fi(h, (y0, . . . , y0))− y0‖ ≤ h∗M‖A‖∞, (2.19)

by the definition of Fi and (2.15). Now we use the triangle inequality, (2.17), (2.19) and (2.16) to establishfor any i = 1, . . . , s that

‖Fi(h, (z1, . . . , zs))− y0‖ ≤ ‖Fi(h, (y0, . . . , y0))− y0‖+ ‖Fi(h, (z1, . . . , zs))− Fi(h, (y0, . . . , y0))‖≤ h∗M‖A‖∞ + θρ ≤ ρ,

which implies (2.19). By Theorem 2.2.9 there exists a continuous function G : (−h∗, h∗)→ V with

F (h,G(h)) = G(h) for all h ∈ U.

Writing G(h) = (g1(h), . . . , gs(h))T with gi(h) ∈ Rd, i = 1, . . . , s, we get that

ki(h) := f(t+ cih, gi(h)) i = 1, . . . , s

22

solves the stage equations (2.6). The smoothness properties of the functions ki can be shown by applyingthe implicit function theorem to the function G. Finally we examine consistency. By definition we have

Ψt0,t0+hy0 = y0 + h

s∑i=1

bif(t0 + hci, gi(h))

and therefored

dhΨt0,t0+hy0|h=0 = lim

h→0

s∑i=1

bif(t0 + hci, gi(h)) = f(t0, y0)

s∑i=1

bi

which implies that Ψ is consistent if and only if∑si=1 bi = 1.

Remark 2.2.11. Keep in mind that the proof of Theorem 2.2.10 requires the stepwidth h to be smallerthan (L‖A‖∞)−1, up to a constant. Note however that this bound is an artifact of the proof technique(Banach fixed point iteration). Solving the stage equations with Newton’s method allows for larger stepsizesfor the RK-SSM to be well-defined and specific right-hand sides f .

Corollary 2.2.12. All collocation schemes are consistent.

Proof. We have bi =∫ 1

0Li(τ)dτ . Observing that

∑si=1 Li(τ) = 1 for all τ ∈ R implies that

∑si=1 bi = 1

and by Theorem 2.2.10 we get consistency.

2.2.3 Autonomization Invariance

In the following we shall discuss an argument why we may assume in several cases that the ODE (1.1) isautonomous, leading to the notion of autonomization invariance. By Remark 1.1.6 we can transform a(nonautonomous) ODE y = f(t, y) to an autonomous one z = F (z) on a higher dimensional phase space.Given a RK-SSM we can apply it to the nonautonomous problem y = f(t, y) (with exact evolution mapΦf ), yielding a discrete flow Ψf on Rd or we can apply it to z = F (z) (with exact evolution map ΦF ),yielding a discrete flow ΨF on Rd+1. How are these flows in relation to each other? Clearly we have that

ΦhF (t, y)T = (t+ h,Φt,t+hf y)T . (2.20)

Motivated by this we introduce the following definition

Definition 2.2.13 (Autonomization Invariance). A SSM is called autonomization invariant if

ΨhF (t, y)T = (t+ h,Ψt,t+h

f y)T . (2.21)

For RK-SSMs we obtain the following characterization of autonomization invariance.

Lemma 2.2.14. A RK-SSM is autonomization invariant if and only if

ci =

s∑j=1

aij for all i = 1, . . . , s and

s∑i=1

bi = 1. (2.22)

Proof. To apply the RK-SSM ΨF we need to solve the stage equations (2.6) for F (z) = (1, f(t, y))T andz(t) = (t, y(t))T . Hence, the stage equations to initial values (t0, y0)T become

kti = 1 for all i = 1, . . . , s

for the t-component, and

kyi = f(t0 + h

s∑j=1

aijktj , y0 + h

s∑j=1

aijkyj ) ∈ Rd for all i = 1, . . . , s

for the y-component. Finally we have

ΨhF (t0, y0) =

(t0 + h

s∑i=1

bi, y0 + h

s∑i=1

bikyi

).

23

From this the statement follows immediately: We have t0 + h∑si=1 bi = t0 + h if and only if

∑si=1 bi = 1

and y0 + h∑si=1 bik

yi = Ψt0,t0+h

f y0 if and only if kyi = ki for all i = 1, . . . , s where ki are the incrementsof Ψf . Looking at the increment equations for ki this holds if and only if

ci =

s∑j=1

aij for all i = 1, . . . , s.

2.2.4 (*) Solvability of Stage Equations Revisited

By Remark 2.2.11 one has to place quite stringent restrictions on the stepwidth h for problems with alarge Lipschitz constant L. In general one can show that these restrictions cannot be improved uponqualitatively, if one insists on deriving bounds which are valid for any right-hand side f . For f satisfyinga one sided Lipschitz condition, one can show more.

Definition 2.2.15 (One sided Lipschitz Condition). A mapping f : D → Rd satisfies a one sidedLipschitz condition if there exists ν ∈ R+ such that

〈f(t, y)− f(t, z), y − z〉 ≤ ν|y − z|2 for all (t, y), (t, z) ∈ Ω. (2.23)

Remark 2.2.16. A one sided Lipschitz condition is actually a much weaker condition than a globalLipschitz condition. For instance, it is easy to see that for f : R → R monotonously decreasing a onesided Lipschitz condition holds with ν = 0. Consider for instance the ODE

y = −50(y − cos(t)).

Then we have∂f

∂y= −50 ≤ 0.

It follows that(f(t, y)− f(t, z)) · (y − z) ≤ 0 for all y, z ∈ R.

A one sided Lipschitz condition can be formulated as a condition on the derivative of f .

Lemma 2.2.17. Suppose that f satisfies a one sided Lipschitz condition. Then for all v ∈ Rd we have⟨v,∂f

∂y(t, z)v

⟩≤ ν|v|2 for all (t, z) ∈ Ω.

Proof. Suppose that y = z + εv with ε > 0 arbitrary. Then (2.23) implies that

〈f(t, z + εv)− f(t, z), εv〉 ≤ νε2|v|2,

which implies that ⟨f(t, z + εv)− f(t, z)

ε, v

⟩≤ ν|v|2

Now it remains to note that

limε→0

f(t, z + εv)− f(t, z)

ε=∂f

∂y(t, z)v.

Before we show the main result of this section we need to introduce the following concept.

Definition 2.2.18. For d1, . . . , ds > 0 and D = diag(d1, . . . , ds) consider the inner product 〈u, v〉D :=uTDv with induced norm | · |D. For a matrix B ∈ Rs×s define

αD(B) = supα>0〈v,Bv〉 ≥ α|v|2D for all v ∈ Rs

andα0(B) := supαD(B) : D = diag(d1, . . . , ds), d1, . . . , d2 > 0.

24

We are concerned with solving the stage equations (2.6) which is equivalent to solving the system

gi = y0 + h

s∑j=1

aijf(t0 + hcj , gj) (2.24)

via the identification ki = f(t0 + cih, gi). The main result of this section reads as follows.

Theorem 2.2.19. Suppose that f satisfies a one sided Lipschitz condition as in Definition 2.2.15. Con-sider a RK-SSM with invertible coefficient matrix A = (aij)

si,j=1. Then for any h < ν−1α0(A−1) the

stage equations (2.6) possess a unique solution.

Proof. For simplicity we assume that d = 1. For h > 0 consider the quantities gi = gi(τ) defined by thefixed-point equation

gi = y0 + h

s∑j=1

aijf(t0 + hcj , gj) + (τ − 1)h

s∑j=1

aijf(t0 + hcj , y0).

For τ = 0 we get gi = y0 for i = 1, . . . , s and for τ = 1 we get the solution to the stage equations (2.24).We derive an ODE for the gi’s by differentiating the above equation. We get

gi = h

s∑j=1

aij∂f

∂y(t0 + hcj , gj)gj + h

s∑j=1

aijf(t0 + hcj , y0)

Let us introduce some simplifying notation. We write G = (g1, . . . , gs)T , F0 = (f(t0 +hc1, y0), . . . , f(t0 +

hcs, y0))T and F = diag(∂f∂y (t0 + hc1, g1), . . . , ∂f∂y (t0 + hcs, gs)

). Using this notation we get

G = hAFG+ hAF0. (2.25)

Now pick a diagonal matrix D with hν ≤ αD(A−1). We multiply (2.25) by GTDA−1 and obtain

〈G, A−1G〉D︸ ︷︷ ︸(A)

−h〈G, F G〉D︸ ︷︷ ︸(B)

= h〈G, F0〉D︸ ︷︷ ︸(C)

. (2.26)

By the definition of αD(A−1) we have

(A) ≥ αD(A−1)|G|2D. (2.27)

By Lemma 2.2.17 we have that|(B)| ≤ hν|G|2D. (2.28)

Finally, Cauchy-Schwarz inequality implies that

|(C)| ≤ h|G|D|F0|D. (2.29)

Putting (2.27), (2.28) and (2.29) into (2.26) we obtain that

(αD(A−1)− hν)|G|2D ≤ h|G|D|F0|D

which implies

|G|D ≤h|F0|D

αD(A−1)− hν.

This implies that G remains bounded and in particular that G satisfies a differential equation of the formG = A(G) with A bounded, provided that h < ν−1α0(A−1). Theorem 1.3.9 implies that the solution Gexists globally for all τ ∈ R, in particular for τ = 1. This shows that for h < ν−1α0(A−1) a solution tothe stage equations exists.

25

Let us turn to uniqueness. Suppose there exist g1, . . . , gs which also solve the stage equations (2.24).Let us write ∆G := (g1−g1, . . . , gs−gs)T and ∆F := (f(t0 +hc1, g1)−f(t0 +hc1, g1), . . . , f(t0 +hc1, gs)−f(t0 + hc1, gs))

T . Then we have∆G = hA∆F.

We can again multiply this equation with ∆GTDA−1 and obtain

〈∆G,A−1∆G〉D = h〈∆G,∆F 〉D.

Now we use again the one sided Lipschitz condition (2.23) of f (together with the fact that D is a diagonalmatrix) to estimate

h〈∆G,∆F 〉D ≤ hν|∆G|2D.

Similar to above (with ∆G instead of G) we can also estimate

〈∆G,A−1∆G〉D ≥ αD(A−1)|∆G|2D.

Putting these estimates together implies that

(αD(A−1)− hν)|∆G|2D ≤ 0

which (due to the assumption that αD(A−1)−hν ≥ 0) can only hold if ∆G = 0 which proves uniqueness.

We have seen already in Lemma 2.2.6 that for collocation schemes the Butcher matrix A is alwaysregular. For Gauss collocation schemes the following result holds.

Lemma 2.2.20. Let A be the Butcher matrix of an s-stage Gauss collocation scheme. Then

α0(A−1) =s

mini=1

1

2ci(1− ci).

For A the Butcher matrix of an s stage Radau collocation scheme we have

α0(A−1) =

1 s = 11

1−c2 else

Proof. Exercise.

2.3 Construction of High Order RK-SSMs

In the last section we have seen that a high consistency order implies a high convergence rate. How canwe design RK-SSMs with high consistency order?

2.3.1 Order Barriers

What is the maximal consistency order that can be reached by an s-stage RK-SSM? The answer is givenby the following result.

Theorem 2.3.1. An s stage RK-SSM has consistency order p ≤ 2s.

Proof. Assume that Ψ is an s-stage RK-SSM with consistency order p ≥ 2s+1. Applying Ψ to the simpleautonomous linear ODE

y = λy, λ ∈ C, (2.30)

yieldsΨt,t+hy = S(hλ)y, (2.31)

where S(z) : C→ C is the stability function

S(z) = (1 + zbT (I − zA)−11) (2.32)

26

and 1 = (1, . . . , 1)T ∈ Rs. This can be seen by explicitly solving the stage equations (2.6) which turn outto be linear for (2.30). Cramer’s rule applied to (2.32) yields that

S(z) =p(z)

q(z), with p, q ∈ Ps.

The exact evolution Φ is given byΦt,t+hy = exp(hλ)y. (2.33)

It follows from (2.31), (2.33) and the assumption on the consistency of Ψ that there exists a rational

function S(z) = p(z)q(z) , p, q ∈ Ps such that

|S(z)− exp(z)| ≤ C|z|p+1. (2.34)

Without loss of generality we can assume that q(0) = 0 (why?). Then (2.34) implies that

|p(z)− q(z) exp(z)| ≤ C|z|p+1. (2.35)

Equation (2.35) implies that p(z) is the Taylorpolynomial of degree p (please excuse the abuse of no-tation...) for the function q(z) exp(z) at zero. Comparison of polynomial coefficients leads to a linearsystem of equations which has as only solution p = q = 0, whenever p ≥ 2s+ 1 which implies the desiredstatement.

For explicit schemes stricter order barriers apply.

Theorem 2.3.2. An explicit s stage RK-SSM has consistency order p ≤ s.

Proof. The proof is analogous to the proof of Theorem 2.3.1 and observing that in the case of an explicitRK-SSM we have S ∈ Ps.

As a corollary we can show the following result which is not directly relevant for the present course.After having examined some inherent barriers on the maximally achievable consistency order of an s stepRK-SSM we turn to the question of verifying the consistency order of a given RK-SSM Ψ. Since allthe information on Ψ is encoded in its Butcher tableau, we can formulate the conditions for consistencyorder in terms of the Butcher coefficients. We shall only work out these conditions for the case p = 1, 2, 3as we will quickly see that the number of conditions to be satisfied grows very quickly with the desiredconsistency order p; their derivation is analogous to the derivation for p = 1, 2, 3.

Definition 2.3.3 (Order Condition). We formulate the following order conditions for a RK-SSM definedby its Butcher tableau

c1 a11 . . . a1s

......

...cs as1 . . . ass

b1 . . . bs.

s∑i=1

bi = 1, (2.36)

s∑i=1

bici =1

2, (2.37)

s∑i=1

bic2i =

1

3and

s∑i=1

bi

s∑j=1

aijcj =1

6. (2.38)

Lemma 2.3.4. A RK-SSM has consistency order 2 if and only if (2.36) and (2.37) hold. A RK-SSMhas consistency order 3 if and only if (2.36), (2.37) and (2.38) hold.

27

Proof. TBA!

As already said, analogous order conditions can be written down for higher orders p ≥ 4. This isimpractical for moderate p; for instance the consistency conditions for p = 20 already require 20247374nonlinear equations to be stated. Clearly, attempting to solve these equations is impossible (and notinsightful).

We close this section with some popular explicit RK-SSMs of order 2, 3, 4.

Example 2.3.5 (Explicit Trapezoidal). The explicit trapezoidal scheme, defined by

0 0 01 1 0

12

12

has consistency order 2.

Example 2.3.6 (RK3). The RK3 scheme, defined by

0 0 0 012

12 0 0

1 −1 2 0

16

23

16 .

has consistency order 3.

Example 2.3.7 (RK4). The RK4 scheme, defined by

0 0 0 0 012

12 0 0 0

12 0 1

2 0 01 0 0 1 0

16

13

13

16

has consistency order 4.

2.3.2 Collocation Schemes

What is the consistency order of a collocation scheme? To answer this question we start with a preliminaryresult.

Lemma 2.3.8. A collocation scheme with s nodes has consistency order at least s.

To prove this lemma we need the following auxilliary result from approximation theory.

Lemma 2.3.9. For collocation points 0 ≤ c1 < · · · < cs ≤ 1 and the associated interpolation operatorL : C([0, 1],Rd) → Pds−1 , defined by Lf(x) =

∑si=1 f(ci)Li(x) there exists a constant C > 0, only

depending on the collocation nodes and on s such that we have the error estimate

‖f − Lf‖∞ ≤ C‖f (s)‖∞.

Proof. First we note that L is exact on Ps−1, e.g.

Lp = p for all p ∈ Ps−1.

Furthermore, the operator L is bounded:

‖Lf‖∞ ≤

∥∥∥∥∥s∑i=1

|Li(x)|

∥∥∥∥∥∞

‖f‖∞.

Now the statement follows by standard arguments from approximation theory:

28

Polynomial exactness and boundedness implies approximation order.

First, note that, by Taylor expansion, one can find a polynomial pf ∈ Ps−1 with

‖f − pf‖∞ ≤ C‖f (s)‖∞

with a constant C only depending on s. Hence we can estimate

‖f − Lf‖∞ ≤ ‖f − pf‖∞ + ‖Lf − pf‖∞(polynomial exactness)

= ‖f − pf‖∞ + ‖Lf − Lpf‖∞(boundedness of L)

≤ ‖f − pf‖∞ +

∥∥∥∥∥s∑i=1

|Li(x)|

∥∥∥∥∥∞

‖f − pf‖∞

(Taylor expansion of f)≤

(1 +

∥∥∥∥∥s∑i=1

|Li(x)|

∥∥∥∥∥∞

)C‖f (s)‖∞

which is the desired estimate.

Now we can continue with the

Proof of Lemma 2.3.8. Consider the interpolation polynomial yh ∈ Ps. We have

yh(t0 + τh) = y0 + h

∫ τ

0

s∑i=1

Li(η)dηf(t0 + cih, yh(t0 + cih)) (2.39)

and

y(t0 + τh) = y0 + h

∫ τ

0

f(t0 + ηh, y(t0 + ηh))dη. (2.40)

Define the function g : [0, 1] → Rd, by g(τ) := f(t0 + τh, y(t0 + τh)). A simple application of the chainrule yields that

‖g(s)‖∞ ≤ Chs. (2.41)

It follows that for τ ∈ [0, 1] we can estimate

‖yh(t0 + τh)− y(t0 + τh)‖ = h‖∫ τ

0

s∑i=1

Li(η)f(t0 + cih, yh(t0 + cih))− f(t0 + ηh, y(t0 + ηh))dη‖

≤ h‖∫ τ

0

s∑i=1

Li(η)f(t0 + cih, y(t0 + cih))− f(t0 + ηh, y(t0 + ηh))dη‖

+h‖∫ τ

0

s∑i=1

Li(η)dηf(t0 + cih, yh(t0 + cih))− f(t0 + cih, y(t0 + cih))‖

= h‖∫ τ

0

Lg(η)− g(η)dη‖

+h‖∫ τ

0

s∑i=1

Li(η)dηf(t0 + cih, yh(t0 + cih))− f(t0 + cih, y(t0 + cih))‖

f loc. Lip.≤ h‖

∫ τ

0

Lg(η)− g(η)dη‖+ hL‖s∑i=1

|Li(η)|‖∞‖y − yh‖∞

Lemma 2.3.9≤ hC‖g(s)‖∞ + hL‖

s∑i=1

|Li(η)|‖∞‖y − yh‖∞

(2.41)

≤ Chs+1 + hL‖s∑i=1

|Li(η)|‖∞‖y − yh‖∞.

29

It follows that

‖y − yh‖∞ ≤C

1− hL‖∑si=1 |Li(η)|‖∞

hs+1

which implies consistency order s.

Lemma 2.3.8 is suboptimal: The implicit midpoint rule from Example 2.1.5 is a 1-step collocationscheme with consistency order 2. We need to gain a finer understanding of collocation schemes whichcan be done by drawing the following useful analogy to numerical quadrature.

Example 2.3.10 (Quadrature as SSM). Consider a very simple right hand side f , namely f = f(t), e.g.it is independent of y ∈ D. Then certainly we get that

Φt0,t0+hy0 = y0 +

∫ h

0

f(τ)dτ.

Let us apply a collocation scheme to this simple ODE. We get that

Ψt0,t0+hy0 = y0 + h

s∑i=1

bif(t0 + hci).

The consistency error can be written as a quadrature error

τ(t0, y0, h) =

∫ h

0

f(τ)dτ − hs∑i=1

bif(t0 + hci).

Thus, for the simple case f = f(t) we need to study the approximation error between Ihf :=∫ h

0f(τ)dτ

and the quadrature rule Qhf := h∑si=1 bif(t0 + hci).

Definition 2.3.11. A quadrature rule has order p ∈ N if there exists a constant C > 0 such that theestimate

|Qhf − Ihf | ≤ Chp+1‖f (p)‖∞

The study of quadrature order is simple, following the general mantra of approximation theory ‘Polynomialexactness and boundedness implies approximation order’, cf. the proof of Lemma 2.3.9.

Lemma 2.3.12. Assume that Qh is exact on Pp−1, e.g.,

Qhp = Ihp for all p ∈ Pp−1.

Then Qh has order p.

Proof. Assume that Qh is exact on Pp−1. By Taylor expansion we can find pf ∈ Pp−1 with ‖f − pf‖∞ ≤C‖f (p)‖∞hp. Hence, using polynomial exactness we can estimate

|Qhf − Ihf | ≤ |Ih(f − pf )|+ |Qh(f − pf )|.

Now all that remains is to note the estimates

|Ihg|, |Qhg| ≤ Ch‖g‖∞

to arrive at the desired conclusion.

How to verify whether Qh is exact on polynomials?

Lemma 2.3.13. Qh is exact on Pp−1 if and only if Q1 is.

Proof. The proof immediately follows from the fact that Pp−1 is invariant under scaling p(·) 7→ p(h·).

30

So to find out whether a quadrature rule Qh is exact on Pp−1 we simply have to check whether

s∑i=1

bicri =

1

r + 1for all r = 0, . . . , p− 1.

This is simple enough. Wouldn’t it be nice to check the order of a collocation scheme is a similarly simplefashion?

The next theorem shows that a collocation scheme possesses the same consistency order as the corre-sponding quadrature scheme, compare Example 2.3.10.

Theorem 2.3.14. The consistency order of a collocation scheme with nodes 0 ≤ c1 < · · · < cs ≤ 1 isequal to the order of the corresponding quadrature rule.

Proof. Assume that the quadrature rule has order p. The idea is to consider the collocation polynomialyh as a solution to a perturbed ODE

yh = f(t, yh) + δ(t), (2.42)

whereδ(t) := yh(t)− f(t, yh(t)).

We linearize f(t, y) around y(t) and obtain the following evolution equation for the error e(t) := yh(t)−y(t).

eh(t) =∂f

∂y(t, y(t))eh(t) + δ(t) + r(t), (2.43)

where

‖r(t)‖ ≤ C‖e(t)‖2Lemma 2.3.8

≤ Ch2s+2. (2.44)

Now we use the formula (1.19) with A(t) = ∂f∂y (t, y(t)) and g(t) = δ(t)+r(t) to arrive at the representation

eh(t) = E(t0, t) eh(0)︸ ︷︷ ︸=0!

+

∫ t

t0

E(τ, t)g(τ)dτ, (2.45)

whered

dsE(t, s) = A(t)E(t, s), E(t, t) = I.

Note that A(t) and hence also E(t, s) are smooth and independent of h! Clearly the consistency error

τ(t0, y0, h) is equal to −eh(h). Therefore we need to estimate |∫ ht0E(τ, h)(δ(τ) + r(τ))dτ |. First we note

that by (2.44) we get ∣∣∣∣∣∫ h

t0

E(τ, h)r(τ)dτ

∣∣∣∣∣ ≤ Ch2s+3. (2.46)

It remains to estimate |∫ ht0E(τ, h)δ(τ)dτ |. Since the quadrature rule has order p we get an estimate∣∣∣∣∣

∫ h

t0

E(τ, h)δ(τ)dτ −s∑i=1

biE(t0 + cih, h)δ(t0 + cih)

∣∣∣∣∣ ≤ Chp+1 pmaxl=0‖y(l)h ‖∞.

Now comes the main insight in the proof: By the collocation conditions for yh we have that

δ(t0 + cih) = 0 for all i = 1, . . . , s,

which yields ∣∣∣∣∣∫ h

t0

E(τ, h)δ(τ)dτ

∣∣∣∣∣ ≤ Chp+1 pmaxl=0‖y(l)h ‖∞.

Finally we note thatp

maxl=0‖y(l)h ‖∞ ≤ C

31

for a constant C > 0, independent of h (Exercise!) and we finally get∣∣∣∣∣∫ h

t0

E(τ, h)δ(τ)dτ

∣∣∣∣∣ ≤ Chp+1. (2.47)

Putting together (2.45), (2.46) and (2.47) we arrive at

|eh(h)| ≤ Chmin(p+1,2s+3). (2.48)

It is well-known (we will actually prove this later in Theorem 2.3.15) that for an s-point quadraturescheme we always have p ≤ 2s and hence we get

|eh(h)| ≤ Chp+1,

which proves that the collocation scheme has consistency order p.

We actually obtain bounds on the maximal order of collocation schemes by the above result.

Theorem 2.3.15. An s-point quadrature scheme has maximal order 2s.

Proof. Assume that a quadrature rule with s nodes has order p = 2s + 1. Then, by going through theproof of Theorem 2.3.14 (in particular (2.48)) we get that the corresponding collocation method hasconsistency order ≥ 2s+ 1 which contradicts Theorem 2.3.1.

We have already seen two examples of collocation schemes in Examples 2.2.7 and 2.2.8. We can useTheorem 2.3.14 to deduce their consistency orders.

Theorem 2.3.16. An s stage Gauss collocation scheme has maximal consistency order 2s. An s stageRadau collocation scheme has consistency order 2s− 1.

Proof. This follows from Theorem 2.3.14 together with the corresponding order properties of Gauss resp.Radau quadrature schemes.

2.3.3 Extrapolation Schemes

The aim of this section is to give a family of explicit RK-SSMs with arbitrary consistency order. Bythe discussion at the end of the previous section we cannot simply formulate the corresponding orderconsitions and solve for them – it would be too complicated. To still reach our goal of desighning arbitraryorder explicit RK-SSMs we are again inspired by quadrature, this time Romberg quadrature.

Example 2.3.17 (Romberg Quadrature). Suppose we want to approximate Ihf :=∫ h

0f(τ)dτ. For τ := h

Ndefine the composite trapezoidal rule

Πτ :=τ

2f(0) + τ

N−1∑j=1

f

(jh

N

)+τ

2f(1).

Clearly limτ→0 Πτ = Ihf . Let us suppose that the function τ 7→ Πτ ‘looks like a polynomial’ near τ = 0.Then we can approximate Ihf as follows.

1. Compute Πτi for different values (τi)ki=1.

2. Compute p ∈ Pk−1 with p(τi) = Πτi for all i = 1, . . . , k.

3. Extrapolate Qhf := p(0).

In the same spirit as Example 2.3.17 we may now define extrapolated SSMs.

Definition 2.3.18 (Extrapolation Method). Given a SSM Ψ and pairwise disjoint natural numbersn1, . . . , nk ∈ N. The associated extrapolation method Ψk is defined as follows for the ODE (1.1), (t0, y0) ∈Ω and h > 0.

32

1. Compute Πτi := yτi(t0 + h) for τi := hni

, i = 1, . . . , k, where yτi denotes the approximate solutionof the ODE (1.1).

2. Compute p ∈ Pk−1 with p(τi) = Πτi for all i = 1, . . . , k.

3. Extrapolate Ψt0,t0+hk y0 := p(0).

Lemma 2.3.19. Assume that Ψ is an explicit s stage RK-SSM. Then Ψk is an explicit s∑ki=1 ni stage

RK-SSM.

Proof. The fact that Ψ is an explicit RK-SSM follows directly from the definition. The bound on thenumber of stages can be established by counting the number of nodes where the right hand side f needsto be evaluated.

Our aim is to show the following theorem.

Theorem 2.3.20 (Consistency Order of Extrapolation Schemes). Suppose that Ψ is a consistent SSM.Then the extrapolated SSM Ψk has consistency order k + 1.

At the heart of Theorem 2.3.20 lies the existence of an asymptotic expansion of the approximationerror. The following lemma shows that the existence of an asymptotic expansion implies consistency ofthe extrapolation method.

Lemma 2.3.21. Assume that for all l ∈ N an asymptotic expansion of the form

yτ (t) = y(t) + e0(t)τ + · · ·+ el−1(t)τ l +Rl(t, τ) (2.49)

holds with‖Rl(t, τ)‖ ≤ Cτ l+1 (2.50)

and continuous functions ei, i = 0, . . . , l− 1 which satisfy ei(t0) = 0 for all i = 0, . . . , l− 1. Then Ψk hasconsistency order k.

Proof. Let p ∈ Pk−1 be the interpolation polynomial which satisfies p(τi) = yτi(t0 +h) for all i = 1, . . . , k.Let q(τ) := y(t0 + h) + e0(t0 + h)τ + · · ·+ ek−2(t0 + h)τk−1 ∈ Pk−1. Then

p(τi)− q(τi) = Rk−1(t0 + h, τi) for all i = 1, . . . , k − 1. (2.51)

It follows that

p(0)− q(0) =

k∑i=1

Rk−1(t0 + h, τi)Li(0)

and consequently

|p(0)− y(t0 + h)| ≤ Λ(n1, . . . , nk)k

maxi=1|Rk−1(t0 + h, τi)|, (2.52)

and (Li)ki=1 denoting the Lagrange polynomials for the nodes (τi)

ki=1 and

Λ(n1, . . . , nk) :=

k∑i=1

|Li(0)|,

the Lebesgue constant which only depends on (n1, . . . , nk) but not on h (why?). Since p(0) = Ψt0,t0+hk y0,

by (2.52) it remains to estimate the quantity maxki=1 |Rk−1(t0 + h, τi)|. Performing the asymptoticexpansion up to l = k we see that

Rk−1(t0 + h, τi) = ek(t0 + h)τki +Rk(t0 + h, τi) for all i = 1, . . . , k. (2.53)

Since τi ≤ h we have by (2.50) that

|Rk(t0 + h, τi)| ≤ Chk+1. (2.54)

33

Furthermore, since ek(t0) = 0 we have, by continuity, that

|ek(t0 + h)| ≤ Ch. (2.55)

Putting Equations (2.54) and (2.55) into (2.53) we arrive at

|Rk−1(t0 + h, τi)| ≤ Chk+1 for all i = 1, . . . , k,

which, together with (2.52) implies consistency order k.

Before we can finally attack the proof of Theorem 2.3.20 we record the following auxilliary perturbationlemma.

Lemma 2.3.22. Assume that δy ∈ Rd. Then, for a consistent SSM Ψ and h > 0 we have the perturbationbound

Ψt0,t0+h(y + δy) = Ψt0,t0+h(y0) + δy + h∂f

∂y(t0, y0)δy +R

with|R| ≤ Ch2δh.

Proof. TBA!

Now we have everything ready to proceed with the proof of the main Theorem 2.3.20.

Proof of Theorem 2.3.20. By Lemma 2.3.21 we ‘only’ need to show that an asymptotic expansion (2.49)exists. We start by giving the main idea which consists of iteratively constructing continuous functionse0, e1, . . . with el(t0) = 0 for all l = 0, 1, . . . such that

yl(t+ τ)−Ψt,t+τyl(t) = R(t, τ) (2.56)

where|R(t, τ)| ≤ Cτ l+2 (2.57)

andy0(t) = y(t) and yl(t) = yl−1(t) + el−1(t)τ l. (2.58)

Note that yl(t0) = y0 for all l = 1, 2, . . . . Equation (2.56) is a consistency estimate between the SSMΨt,t+τ and the trajectory yl(t+ τ). Theorem 2.1.16 (or rather its proof, compare Remark 2.1.17) impliesthat

yτ (t0 + h) = yl(t0 + h) +Rl(t, τ)

with|Rl(t, τ)| ≤ Cτ l+1,

which is precisely the desired asymptotic expansion (2.49).It remains to construct the functions el, l = 0, 1, . . . . We proceed inductively and assume that yl (and

el−1) are known. By assumption and Taylor expansion we get that

yl(t+ τ)−Ψt,t+τyl(t) = dl(t)τl+2 + S1(t, τ), (2.59)

where|S1(t, τ)| ≤ Cτ l+3.

We now construct el by making the ansatz

yl+1(t) = yl(t) + el(t)τl+1

and|yl+1(t+ τ)−Ψt,t+τyl+1(t)| ≤ Cτ l+3. (2.60)

34

We have

yl+1(t+ τ)−Ψt,t+τyl+1(t) = yl(t) + el(t)τl+1 + ψt,t+τ (yl(t) + el(t)τ

l+1)Lemma2.3.22

= yl(t+ τ)−Ψt,t+τyl(t) + (el(t+ τ)− el(t))τ l+1

−∂f∂y

(t, yl(t))el(t)τl+2 + S2(t, τ)︸ ︷︷ ︸

|S2(t,τ)|≤Cτ l+3

(2.59)= dl(t)τ

l+2 + (el(t+ τ)− el(t))τ l+1

−∂f∂y

(t, yl(t))el(t)τl+2 + S2(t, τ)︸ ︷︷ ︸

|S3(t,τ)|≤Cτ l+3

= (dl(t) + el(t)−∂f

∂y(t, yl(t))el(t))τ

l+2

+ S4(t, τ)︸ ︷︷ ︸|S4(t,τ)|≤Cτ l+3

.

In order to arrive at (2.60) we have to eliminate all terms of lower order than τ l+3 by choosing el as theunique function with el(t0) = 0 and

el(t) =∂f

∂y(t, yl(t))el(t)− dl(t).

The existence of el is guaranteed by Theorem 1.3.6. We are finished with the proof.

Remark 2.3.23. Theorem 2.3.20 and Lemma 2.3.19 imply that it is possible to construct explicit RK-

SSMs of consistency order k with s = k(k+1)2 steps by choosing the weight sequence (1, . . . , k). It is not

know in general what the optimal ration between consistency order and number of steps is for explicitRK-SSMs.

2.3.4 Splitting Schemes

A number of ODEs are of the formy = f1(y) + f2(y), (2.61)

where the flows Φi associated to the ODE y = fi(y), i = 1, 2 can be computed much more easily thanthe flow Φ associated to (2.61).

Example 2.3.24 (Hamiltonian Systems with separable Hamiltonian). A popular example for problems ofthe aforementioned form are Hamiltonian systems with separable Hamiltonian. Consider a Hamiltonian

H(p, q) = T (p)︸︷︷︸kinetic energy

+ U(q)︸︷︷︸potential energy

and p, q ∈ Rd describing momentum and state, respectively. The associated equations of motion are(pq

)=

(−DU(q)DT (p)

).

Putting y = (p, q)T ∈ R2d we arrive at the ODE

y = f1(y) + f2(y)

where

f1(y) =

(−DU(q)

0

)and f1(y) =

(0

DT (p)

).

It follows that

Φt1y =

(p− tDU(q)

q

)and Φt2y

(p

q + tDT (p)

).

35

Definition 2.3.25 (Splitting Method). Given the ODE (2.61) and the flow maps Φ,Φ1,Φ2 as above.Every pair of sequences a1, . . . , am, b1, . . . , bm ∈ R defines a splitting scheme Ψ via

Ψh = Φbmh2 Φamh1 Φbm−1h2 Φ

am−1h1 · · · Φb1h2 Φa1h1 . (2.62)

Example 2.3.26 (Lie-Trotter Splitting, Strang Splitting). The Lie-Trotter splitting scheme is defined bycoefficients a1 = b1 = 1. The Strang splitting is defined by b1 = 1 and a1 = a2 = 1

2 . Applied to Example2.3.24 they are both frequently utilized and known under the name symplectic Euler scheme (Lie Trottersplitting) and Stormer-Verlet scheme (Strang splitting).

Order Conditions for Splitting Schemes

We now show how to derive order conditions for splitting schemes. To this end we require the followingdefinition.

Definition 2.3.27 (Differential Operator associated to a Vector Field). For a vector field f = (f1, . . . , fd)T :D → Rd we define the associated differential operator, operating on ϕ ∈ C∞(D,R) via

Dfϕ(y) :=

d∑i=1

f i(y)∂

∂yi.

For an ODE (2.61) we use the notation

Di := Dfi , i = 1, 2.

Let us record a few basic properties of Definition 2.3.27.

Lemma 2.3.28. Let f, g be vector fields on D and t ∈ R. Then

tDf = Dtf (2.63)

and[Df , Dg] := Df Dg −Dg Df = Df,g, (2.64)

where f, g : D → Rd denotes the Poisson bracket defined by f, g(y) = (h1(y), . . . , hd(y))T , where

hi(y) :=

d∑j=1

(f j(y)

∂gi

∂yj(y)− gj(y)

∂f i

∂yj(y)

). (2.65)

Proof. Equation (2.63) is obvious. To show (2.64) we simply utilize the fact that higher order partialderivatives commute. This property will cancel out all higher order terms in [Df , Dg]. First compute

Df Dgϕ(y) = Df

(d∑i=1

gi(y)∂ϕ

∂yi(y)

)product rule

=

d∑i=1

d∑j=1

(f j(y)

∂gi

∂yj(y)

∂ϕ

∂yi(y) + f j(y)gi(y)

∂2ϕ

∂yj∂yi(y)

).

Analogously we get

Dg Dfϕ(y) =

d∑i=1

d∑j=1

(gj(y)

∂f i

∂yj(y)

∂ϕ

∂yi(y) + gj(y)f i(y)

∂2ϕ

∂yj∂yi(y)

)commutation of partial derivatives

=

d∑i=1

d∑j=1

(gj(y)

∂f i

∂yj(y)

∂ϕ

∂yi(y) + gj(y)f i(y)

∂2ϕ

∂yi∂yj(y)

).

Subtracting the two equations yields the result.

36

For a vector field f and the associated differential operator Df we define the exponential map exp(Df )which operates on ϕ ∈ C∞(D,R) via

exp(Df )ϕ(y) :=

∞∑i=0

1

i!Difϕ(y) :=

∞∑i=0

1

i!(Df · · · Df )︸ ︷︷ ︸

i−times

ϕ(y). (2.66)

The exponential map computes the integral curves of f as shown below.

Lemma 2.3.29. For a vector field f : D → Rd and the associated differential operator Df and everyϕ ∈ C∞(D,R) it holds that

ϕ(Φtfy) = exp(tDf )ϕ(y). (2.67)

Proof. We simply compute the Taylor expansion of the function

ν(t) := ϕ(Φtfy)

at t = 0. A direct computation (using the fact that ddtΦ

tfy = f(Φtfy) and Φ0

fy = y) yields that

di

dtiν(0) = Di

fϕ(y),

which yields the desired result.

As an immediate corollary to Lemma 2.3.29 we obtain the following useful representation of thesplitting scheme (2.62).

Lemma 2.3.30. For the splitting scheme Ψh defined in (2.62) we have

Ψhy = exp(ha1D1) exp(hb1D2) exp(ha2D1) exp(hb2D2) . . . exp(hamD1) exp(hbmD2)Id(y), (2.68)

where Id(y) = y is the identity function.

Proof. We simply show the statement for m = 1, the rest follows by the same arguments. By Lemma2.3.30, applied to the mapping ϕ(y) = Φhb12 (y) we have that

Φhb12 (Φha11 y) = exp(ha1D1)Φhb12 y.

Applying Lemma 2.3.30 again yields

Φhb12 (Φha11 y) = exp(ha1D1) exp(hbaD2)Id(y).

The following fundamental result gives us a tool to compute Taylor expansions (in h) of the discreteflow Ψh.

Theorem 2.3.31 (BCH Formula). For two vector fields f, g : D → Rd we have

exp(tDf ) exp(tDg) = exp(Z(t)), (2.69)

whereZ(t) = tC1 + t2C2 + . . . , (2.70)

where

C1 = Df +Dg, C2 =1

2[Df , Dg], C3 =

1

12[Df , [Df , Dg]] +

1

12[Dg, [Dg, Df ]] (2.71)

and similar formulae (higher order iterated Lie brackets) can be found for the higher order terms Ci,i ≥ 4.

Proof. One can show this by direct computation and comparing coefficients in the Taylor expansion.

37

Remark 2.3.32. The equality (2.69) (and also the equality (2.68) for that matter) has to be understoodin a formal sense. It means that the Taylor expansion of the function

t 7→ exp(tDf ) exp(tDg)ϕ(y)

around t = 0 up to order p coincides with the Taylor expansion of the function

t 7→ exp(tC1 + t2C2 + · · ·+ T pCp)ϕ(y),

up to order p.

Rather than stating a general theorem on the derivation of order conditions for splitting schemes(which would be awfully technical) we show how to use Lemma 2.3.30 and Theorem 2.3.31 to calculatethe consistency order of the Lie-Trotter splitting scheme and the Strang splitting scheme, respectively.These two examples already convey the general recipe for calculating order conditions.

Theorem 2.3.33. A splitting scheme Ψh as in (2.62) with length m = 2 has consistency order 1 if

a1 + a2 = b1 + b2 = 1 (2.72)

holds true. If in addition we have

b1a2 =1

2, (2.73)

then Ψ has consistency order 2. In particular, the Lie-Trotter splitting scheme has consistency order 1and the Strang splitting scheme has consistency order 2.

Proof. The basic idea is as follows. We use the representation (2.68). Iterative application of the BCHformula (2.69) yields an expression of the Taylor expansion of Ψhy up to order p of the form

Ψhy = exp(hC1 + h2C2 + · · ·+ hpCp)Id(y) +R,

where|R| ≤ Chp+1.

On the other hand, Lemma 2.3.30 tells us that

Φhy = exp(h(D1 +D2))Id(y).

Hence, to show that Ψ has consistency order p we need to verify that

C1 = D1 +D2 and Ci = 0 for i = 2, . . . , p.

Let us now compute C1, C2 for m = 2. We have

ψhy = exp(ha1D1) exp(hb1D2) exp(ha2D1) exp(hb2D2)Id(y).

Applying the BCH-Formula to the expression exp(ha2D1) exp(hb2D2) yields that

ψhy = exp(ha1D1) exp(hb1D2) exp(h(a2D1 + b2D2 + h/2[a2D1, b2D2]))Id(y) +R1,

where |R1| ≤ Ch3 (due to Lipschitz continuity of the function t 7→ exp(tD)ϕ(y)). We can apply theBCH-Formula again (and use that fact that [D2, D2] = 0) and obtain

ψhy = exp(ha1D1) exp(h(b1D2 + a2D1 + b2D2 + h/2[a2D1, b2D2]) + h/2[b1D2, a2D1]))Id(y) +R2,

where |R2| ≤ Ch3.Applying the BCH-Formula a third time yields

ψhy =

exp(h(a1D1+b1D2+a2D1+b2D2)+h2/2([a2D1, b2D2])+[b1D2, a2D1]+[a1D1, b1D2]+[a1D1, b2D2]))Id(y)

+R3,

38

where |R3| ≤ Ch3.It follows that

C1 = (a1 + a2)D1 + (b1 + b2)D2

andC2 = (a2b2 − b1a1 + a1b1 + a1b2)[D1, D2].

Recall that we need C1 = D1 + D1 for consistency order 1. This holds precisely if (2.72) holds true.Consistency order 2 holds precisely if C2 = 0, that is, if

a2b2 − b1a1 + a1b1 + a1b2. (2.74)

It is easy to see that (2.74) reduces to (2.73) if in addition (2.72) holds true. This proves the result.

2.4 (*) Adaptive Timestepping

TBA: embedded RK-SSMs

39

Chapter 3

Stability

In the last chapter we have seen that we can construct both implicit and explicit RK-SSMs of arbitraryconsistency order. But then why would anyone prefer an implicit (collocation) RK-SSM against a (muchsimpler to implement) explicit RK-SSM?

There are several aspects to this question, one of the most important ones being stability, the topicof this chapter.

We start off with a numerical example.Consider the Logistic Equation

y = 50y(1− y) y(0) = 0.1.

The solution to this ODE can be given explicitly as

y(t) =y(0)

y(0) + (1− y(0) exp(−50t))

and it can be seen that the solution very quickly converges to the stationary state y = 1. Figure 3.1shows the numerical solutions computed with the explicit RK4 method (??) and implicit Euler (??) andh = 0.05.

We observe that, while implicit Euler gives reasonable results, RK4 suffers a blowup and can effectivelynot be used for this problem.

Why?To get a better understanding of the behavior of numerical methods on our problem we linearize the

RHS around the fixed point y = 1. We get the linearized ODE

ylin = −50(ylin − 1),

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

1.2

t

y

y(t)

Implicit Euler

RK4

10−4

10−3

10−2

10−1

100

10−8

10−7

10−6

10−5

10−4

10−3

10−2

10−1

100

101

timestep h

∞−

err

or

implicit Euler

RK4

blow−up

Figure 3.1: Left: Approximative solutions with N = 30. Right: error vs. stepwidth.

40

or, putting z := ylin − 1,z = −50z

with explicit solution z(t) = z(0) exp(−50t) or ylin(t) = 1 + (ylin(0)− 1) exp(−50t).The intuition behind this is that the solution approaches the fixed point at an exponential speed and

therefore the linearized equation should still preserve its main qualitative features. In particular, supposewe apply a RK-SSM to the linearized equation z = −50z we expect the numerical solution to convergealso to the attractive fixed point z = 0. Later this will be called asymptotic stability.

We can now examine what happens if we apply implicit Euler or RK4 to the linearized ODE above.Let us start with implicit Euler with stepwidth h and updating rule

zl+1 = zl + hf(zl+1) = zl − 50hzl+1, l = 0, . . . , N − 1

which implies the explicit formula

zl+1 =zl

1 + 50h, l = 0, . . . , N − 1

and consequently

zl =z(0)

(1 + 50h)l, l = 1, . . . , N.

We have the desired property that

zh(hN) = zN → 0, if N →∞,

i.e. also the numerical solution converges to the attractive fixedpoint z = 0, no matter how small or largeh is.

Now let us turn to the RK4 method. With considerations akin to the ones above we get the formula

zl =

(1− 50h+ 25h2 − 25

3h3 +

25

12h4

)lz(0).

Note that the factor 1−50h+25h2− 253 h

3 + 2512h

4 is a polynomial in h and as such tends to∞ for h→∞.It follows that there exists a critical stepsize h∗ such that

1− 50h+ 25h2 − 25

3h3 +

25

12h4 > 1 for h > h∗.

But this means thatlimN→∞

zh(Nh) =∞ for h > h∗.

The numerical solution blows up while the true solution tends to zero. Only for small stepsizes wecan achieve a qualitatively meaningful solution with RK4, while implicit Euler always produces such asolution, regardless of h. In this chapter we would like to understand this phenomenon in more depth.

3.1 Model Problem Analysis

In the introduction above we have heuristically linearized our ODE around a fixed point. By applying atranslation we may without loss of generality assume that the fixedpoint is zero and we wind up with ageneral linear ODE

y = Ay, A ∈ Rd×d, y ∈ Rd. (3.1)

In the introduction the linearized ODE is of this form with A = −50 and d = 1. We have already seenthat the action of RK4 and implicit Euler on this ODE can be calculated explicitly. This is actually thecase for general RK-SSMs, as the following lemma shows. We first define the notion of stability functionwhich we have already used in (2.32) above.

41

Definition 3.1.1 (Stability Function). The stability function S(z) : C→ C of an s-stage RK-SSM Ψ isdefined by

S(z) = (1 + zbT (I − zA)−11) (3.2)

and 1 = (1, . . . , 1)T ∈ Rs.

Remark 3.1.2. Using Cramer’s rule it is easy to see that we have the alternative expression

S(z) =det(I − zA+ z1bT )

det(I − zA).

This implies that the stability function of an s-stage RK-SSM is always a rational function of degree atmost s (ie. equal to p/q, where p, q ∈ Ps).

The stability function of a RK-SSM Ψ exactly describes the evolution of Ψ applied to linear ODEs asthe following lemma shows.

Lemma 3.1.3. Let Ψ be a RK-SSM applied to the linear ODE

·y = Ay,

with A ∈ Rd×d. Then, for y ∈ Rd we have

Ψhy = S(Ah)y,

where S is the stability function of Ψ as defined in Definition 3.1.1. Note that by Remark 3.1.2 we canapply S also to matrices.

Proof. The case d = 1 has been shown in the proof of Theorem 2.3.1. Now let us assume that A isdiagonalizable, e.g. A = T−1DT with D = diag(λ1, . . . , λd). Then we have

ΨhyExercise!

= T−1ΨhDTyCase d = 1

= T−1diag(S(λ1h), . . . , S(λdh))Ty = T−1S(hD)Ty = S(hA)y.

For A general (nondiagonalizable) we may use that fact that the set of diagonalizable matrices is densein Rd×d.

Now that we know how a RK-SSM operates on linear ODEs we can ask ourselves how attractive fixedpoints of the the linear ODE get preserved by RK-SSMs. We first show a condition which ensures that0 ∈ Rd is an attractive fixedpoint.

Definition 3.1.4. For A ∈ Rd×d we denote its spectrum by σ(A) ⊂ C and define

τ∗(A) := max<λ : λ ∈ σ(A).

Lemma 3.1.5. Suppose that Φty = exp(tA)y is the flow map of the linear ODE y = Ay, A ∈ Rd×d andy ∈ Rd. Then, for all ε > 0 there exists a constant C(ε) such that

‖ exp(tA)‖ ≤ C(ε) exp(t(τ∗ + ε))

and consequently‖Φty‖ ≤ C(ε)|y| exp(t(τ∗ + ε)).

In particular, if τ∗(A) < 0 every trajectory tends to zero at exponential speed.

Proof. The statement is easy for A diagonalizable: Suppose that A = T−1DT with a diagonal matrixD = diag(λ1, . . . , λd) and (λi)

di=1 the eigenvalues of A. Then exp(tA) = T−1 exp(tD)T . Since exp(tD) =

diag(exp(tλ1), . . . , exp(tλd)), the statement follows. The general case can be shown using the Jordandecomposition A = T−1DT with D a blockdiagonal matrix consisting of Jordan blocks

Jk(λ) =

λ 1 0 · · · · · · 00 λ 1 0 · · · 0

. . .. . .

λ 1λ

=: λI +Nk ∈ Rk×k

42

andD = blockdiag(Jk1(λ1), . . . , Jkm(λm)),

where λ1, . . . , λm are the eigenvalues of A with multiplicities k1, . . . , km. Observe that the matrices Nkare nilpotent, e.g.

Nkk = 0.

This implies that

exp(tNk) =

k−1∑i=1

tiN ik

i!

and consequently, for t ≥ 0

‖ exp(tNk)‖ ≤k−1∑i=1

ti‖Nk‖i

i!=: pk(t) ∈ Pk−1.

Now since the matrices λI,Nk commute it holds that

exp(tJk(λ)) = exp(tλI) exp(tNk)

and thus‖ exp(tJk(λ))‖ ≤ exp(t<λ)|pk(t)|.

Since the exponential map acts separately on the different Jordan blocks we get that

‖ exp(tD)‖ ≤ exp(tτ∗(A))m

maxl=1|pkl(t)|.

Since any polynomial grows slower than any exponential function we know that for any ε > 0 there existsa constant B(ε) which satisfies

mmaxl=1|pkl(t)| ≤ B(ε) exp(εt)

and we get that‖ exp(tD)‖ ≤ B(ε) exp(t(τ∗(A) + ε)).

Finally we note thatexp(tA) = T−1 exp(tD)T

to conclude that‖ exp(tA)‖ ≤ B(ε)‖T‖‖T−1‖︸ ︷︷ ︸

C(ε)

exp(t(τ∗(A) + ε))

which is the desired claim.

Remark 3.1.6. In Lemma 3.1.5 it is crucial to assume that the linear ODE is autonomous. Considerfor instance the non-autonomous linear ODE

y = A(t)y,

where

A(t) =

(−1 + 3

2 cos(t)2 1− 32 cos(t) sin(t)

−1− 32 cos(t) sin(t) −1 + 3

2 sin(t)2

).

A short computation shows that the spectrum of A(t) is invariant in time. We have

σ(A(t)) = (−1 + i√

7)/4, (−1− i√

7)/4

and hence

τ∗(A(t)) =1

4for all t.

Let us now start the evolution arbitrarily close to the FP 0, with y(0) = (−ε, 0)T , where ε > 0 is arbitrarilysmall. The solution can then be described explicitly as

y(t) = ε(− cos(t), sin(t))T exp(t/2),

which tends to ∞ as t tends to ∞.

43

Let us now consider an ODE y = Ay with τ∗(A) < 0. By Lemma 3.1.5 we know that 0 is an attractivefixedpoint. Does this also hold for the numerical solution?

Lemma 3.1.7. Suppose that Ψh is the numerical flow associated with a RK-SSM method with stabilityfunction S, applied to the ODE y = Ay, τ∗(A) < 0. Let (yl)

∞l=0 be the numerical approximation generated

by the recursion yl = Ψhyl−1. Then we have liml→∞ yl = 0 holds if and only if

maxλ∈σ(A)

|S(hλ)| < 1. (3.3)

If we havemaxλ∈σ(A)

|S(hλ)| > 1 (3.4)

then we have blowup for initial data y0 lying in the eigenspace of an eigenvalue λ with S(hλ)| > 1, e.g.liml→∞ yl =∞.

Proof. We start again by assuming that A is diagonalizable with A = T−1DT with a diagonal matrixD = diag(λ1, . . . , λd) where the λi’s are the eigenvalues of A. Then we know from the fact that S(z) isa rational function that

S(hA) = T−1S(hD)T = T−1diag(S(hλ1), . . . , S(hλd))T.

By Lemma 3.1.3 we thus have that

yl = T−1diag(S(hλ1)l, . . . , S(hλd)l)Ty0.

Clearly liml→∞ is equivalent to (3.3). For the case of A being non-diagonalizable we can invoke againthe Jordan decomposition. Suppose that (3.3) holds true. Then, since σ(S(hA)) = S(hσ(A)), we knowthat the spectral radius of S(hA) is less than 1. Consider a Jordan block Jk(λ) = λI +Nk of S(hA) with|λ| < 1, by our assumption. Then, using the fact that Nk

k = 0 together with the fact that λI and Nkcommute, we get that

Jk(λ)l =

k∑i=1

(l

i

)λl−iN i

k,

which clearly goes to zero for l→∞. But this implies that

yl = T−1DlTy0 → 0

for l→∞ and T−1DlT the Jordan decomposition of S(hA).

The previous result shows us that (3.3) is crucial for a RK-SSM to preserve attractive fixedpoints forlinear ODEs. If (3.4) holds, the numerical solution even blows up! How can we avoid this?

Definition 3.1.8 (Stability Region, A-stability). Let Ψ be a RK-SSM. Then its stability region SΨ ⊂ Cis defined by

SΨ := z ∈ C : S(z) < 1 .

Ψ is called A-stable ifC− ⊂ SΨ.

We have already seen that the stability region plays a crucial role for the preservation of asymptoticstability of fixed points for linear ODEs. The following corollary summarizes this.

Corollary 3.1.9. Consider a linear ODE of the form

y = Ay

with τ∗(A) < 0. Then, for a RK-SSM to preserve the attractive fixedpoint 0 it is necessary that thestepsize restriction

h ≤ h∗ := maxg : gσ(A) ⊂ SΨ (3.5)

holds true. If Ψ is A-stable, then condition (3.5) is always true, meaning that h∗ =∞.

44

Proof. The proof follows directly from Lemma 3.1.7.

It is easy to see that an explicit RK-SSM can never be A-stable, as the following lemma shows.

Lemma 3.1.10. Let Ψ be an explicit RK-SSM. Then the stability region SΨ is bounded.

Proof. Note that the stability function of an explicit RK-SSM is a polynomial and hence holomorphic.Then the statement follows from the maximum principle.

Example 3.1.11 (Explicit Euler). The stability function of the explicit Euler method is given by S(z) =1 + z. Its stability region is the disc

SΨ = z : |1 + z| < 1.In particular, explicit Euler is not A-stable.

Example 3.1.12 (Implicit Euler). The stability function of the implicit Euler method is given by S(z) =1

1−z . It is easy to see thatC− ⊂ SΨ

and hence implicit Euler is A-stable.

3.2 Inheritance of Asymptotic Stability

So far we have seen that A-stability of a RK-SSM implies that attractive fixedpoints are preserved forlinear ODEs. How about nonlinear ODEs? This is the question that we shall study in the present section.

3.2.1 Attractive Fixedpoints of ODEs

Let us start by defining attractive fixedpoints of ODEs.

Definition 3.2.1 (Attractive FP). Consider an autonomous ODE y = f(y). Then y∗ ∈ Rd is called afixedpoint (FP) if f(y∗) = 0. Furthermore, y∗ is called an asymptotically stable (attractive) FP if thereexists δ > 0 such that for all y ∈ Rd with |y − y∗| < δ it holds that R+ ⊂ J(0, y) and limt→∞Φty = y∗.

In general it is difficult to characterize when precisely a fixed point is asymptotically stable. But thenext theorem provides a sufficient (and almost necessary) condition.

Theorem 3.2.2. Suppose that y∗ is a FP of y = f(y) and τ∗(Df(y∗)) < 0. Then y∗ is an attractive FP.

Proof. We assume without loss of generality that y∗ = 0 and, observing that f(0) = 0, linearize theright-hand-side f around 0:

f(y) = Df(0)y + r(y),

where r(y) = o(|y|) as y → 0. Using the solution formula (1.16) for nonhomogenous linear ODEs we getthe implicit representation

y(t) = exp(tDf(0))y(0) +

∫ t

0

exp((t− τ)Df(0))r(y(τ))dτ, (3.6)

and consequently

|y(t)| ≤ ‖ exp(tDf(0))‖|y(0)|+∫ t

0

‖ exp((t− τ)Df(0))‖|r(y(τ))|dτ. (3.7)

Since by assumption τ∗(Df(0)) < 0 we may choose ε > 0 such that β := τ∗(Df(0)) + ε < 0. UsingLemma 3.1.5 we see that there exists a constant C > 0 such that

‖ exp(tDf(0))‖ ≤ C exp(tβ) for all t > 0.

Inserting this bound into (3.7) yields

|y(t)| ≤ C exp(tβ)|y(0)|+∫ t

0

C exp((t− τ)β)|r(y(τ))|dτ. (3.8)

45

Since r(y) = o(|y|) there exists δ > 0 such that

|r(y)| ≤ −β2C|y| for all y with |y| < δ. (3.9)

Start with any initial value y(0) satisfying

|y(0)| < δ

max1, C. (3.10)

Then by Theorem 1.3.6 there exists a time ν > 0 with

|y(t)| < δ for all t with 0 ≤ t < ν. (3.11)

Defineν∗ = maxν : (3.11) holds true for ν.

Equation (3.11) and (3.9) imply that

|r(y)| ≤ −β2C|y| for all t with 0 ≤ t < ν∗. (3.12)

Inserting (3.12) and (3.10) into (3.8) yields

|y(t)| ≤ exp(tβ)δ +−β2

∫ t

0

exp((t− τ)β)|y(τ)|dτ for all t with 0 ≤ t < ν∗.

Putting u(t) := exp(−βt)|y(t)|, Gromwall’s Lemma 1.3.8 yields that

u(t) ≤ δ exp(−β/2t) for all t with 0 ≤ t < ν∗

and hence,|y(t)| ≤ δ exp(tβ/2) for all t with 0 ≤ t < ν∗. (3.13)

It is now easy to show that ν∗ =∞. For, suppose that ν∗ <∞, then, since there is no blowup or collapsewe can, by Theorem 1.3.6, extend the solution y(t) to an interval [0, ν∗ + θ] for some θ > 0 such that|y(t)| ≤ δ on this larger interval. But this is a contradiction to the finiteness of ν∗. Therefore (3.13) holdsfor all t ∈ R+ which proves the theorem.

Remark 3.2.3. Theorem 3.2.2 provides a sufficient condition for a FP to be asymptotically stable. Unlikein the linear case this condition is not necessary as higher order effects can dominate the evolution.

3.2.2 Attractive Fixedpoints of SSMs

In the previous subsection we studied attractive FPs of ODEs. In order to solve these ODEs we usuallyapply a SSM Ψh. Now we want to study whether this SSM recognizes the FP and whether it will alsoconverge towards it. We start by defining asymptotically stable FP for general dynamical systems (andlater specialize to the dynamical system Ψh).

Definition 3.2.4 (Dynamical System, FPs). A mapping Γ : D ⊂ Rd → Rd is called a dynamical system.A point y∗ ∈ D is called FP of Γ if

Γ(y∗) = y∗.

A FP y∗ of Γ is called asymptotically stable or attractive if there exists δ > 0 such that for all y0 with|y − y0| ≤ δ we have

liml→∞

yl = y∗,

whereyl = Γ(yl−1), l ≥ 1.

The first positive result that we can achieve is that any RK-SSM automatically recognizes a FP of anODE and that FPs are always preserved in the numerical evolution.

46

Lemma 3.2.5. Suppose that y∗ is an attractive FP of an ODE y = f(y) and let Ψ be a SSM, and h besmall enough for the increment equations to be uniquely solvable. Then y∗ is a FP of Ψh.

Proof. Since y∗ is a FP, we have f(y∗) = 0. This implies that ki = 0 for all i = 1, . . . , s solves theincrement equations. This implies the statement.

We now want to study whether also asymptotic stability is preserved by RK-SSMs. To this end westart with the following result which may be viewed as a discrete version of Lemma 3.1.5.

Lemma 3.2.6. Suppose that Γ ∈ C1(Ω,Rd) with a FP y∗. Suppose that

ρ(DΓ(y∗)) < 1. (3.14)

Then y∗ is an attractive FP of Γ.

Proof. We use Taylor expansion of Γ around y∗:

Γ(y) = Γ(y∗)︸ ︷︷ ︸=y∗

+DΓ(y∗)(y − y∗) + r(y − y∗),

where r(y) = o(|y|). Then we have, for any norm | · | on Rd,

|yl+1 − y∗| = |Γ(yl)− Γ(y∗)| = |DΓ(y∗)(yl − y∗) + r(yl − y∗)| ≤ |DΓ(y∗)(yl − y∗)|+ |r(yl − y∗)|.

Since (3.14) holds true, there exists a matrix norm ‖ · ‖′, induced by a norm | · |′ on Rd such that

γ := ‖DΓ(y∗)‖′ < 1.

It follows that|yl+1 − y∗|′ ≤ γ|yl − y∗|′ + |r(yl − y∗)|′

Since r(y) = o(|y|) there exists δ > 0 such that for all yl with |yl − y∗| < δ we have

|r(yl − y∗)|′ ≤1

2(1− γ)|yl − y∗|′.

We then have

|yl+1 − y∗|′ ≤1 + γ

2|yl − y∗|′.

Since

µ :=1 + γ

2< 1

we get, by iteratively applying this argument that

|yl − y∗|′ ≤ µlδ,

which proves the result.

In order to apply Lemma 3.2.6 to a RK-SSM Ψh we need to be able to assess the matrix DΨh(y∗).Luckily this matrix can be easily described by Df(y∗) and the stability function.

Lemma 3.2.7. Suppose that Ψ is a RK-SSM applied to the ODE y = f(y) with FP y∗. Then

σ(DΨh(y∗)) = S(hσ(Df(y∗))). (3.15)

Proof. We start by computing the differentials of the increments, using the increment equations (2.6):

Dki(y) = Df(y + h

s∑j=1

aijkj(y))(I +

s∑j=1

aijDkj(y)).

47

Inserting y = y∗ and the fact that ki(y∗) = 0 for all i = 1, . . . , s we get that

Dki(y∗) = Df(y∗)(I + h

s∑j=1

aijDkj(y∗))

and

DΨh(y∗) = I + h

s∑i=1

biDki(y∗).

Observe that the matrices Dki(y∗) are now precisely the increments for the RK-SSM Ψ applied to the

matrix ODEY = Df(y∗)Y, Y0 = I ∈ Rd×d.

It follows that (observe that the action of a RK-SSM to a linear ODE can be described by its stabilityfunction, see Lemma 3.1.3)

DΨh(y∗) = S(hDf(y∗)),

which implies the desired claim.

Now we only have to put things together to see that asymptotically stable FPs are preserved ifhσ(Df(y∗)) lies within the stability region of the RK-SSM.

Theorem 3.2.8. Suppose that y∗ is a FP of y = f(y). Let Ψ be a RK-SSM with stability region SΨ.Suppose that hσ(Df(y∗)) ⊂ SΨ. Then y∗ is an attractive FP of Ψh for all h with Ψh well-defined.

Proof. This is now a direct consequence of Lemma 3.2.7 and Lemma 3.2.6.

Specializing to A-stable RK-SSMs we get the following immediate corollary.

Corollary 3.2.9. Suppose that Ψ is A-stable and let y∗ be a FP of y = f(y) with τ∗(Df(y∗)) < 0. Theny∗ is an attractive FP of Ψh for all h for which Ψh is well-defined.

Proof. This follows immediately from Theorem 3.2.8 and the definition of A-stability in Definition 3.1.8.

3.3 Nonexpansiveness

In the last subsections we have identified A-stability as a crucial qualitative property of SSMs. It allowsthe stable treatment of asymptotically stable FPs, without any restrictions on the stepsize. But do thereexist A-stable RK-SSMs of arbitrary order? The answer is ‘yes’: The goal of this section is the proof ofthe following theorem.

Theorem 3.3.1. All Gauss collocation schemes are A-stable.

But this requires some preparation. First we introduce the notion of non-expansivity.

Definition 3.3.2 (Non-expansivity). Let Φt : D ⊂ Rd → Rd, t ∈ [0, T ] be a family of maps with Φ0 = Id.Then Φt is called non-expansive if

|Φty − Φty| ≤ |y − y| for all y, y ∈ D, t ∈ [0, T ].

If Ψt happens to be a flow map of an outonomous ODE, one can characterize non-expansivity easilyin terms of the RHS f .

Lemma 3.3.3. Suppose that Φt is the flow map associated to the autonomous ODE y = f(y). Then Φt

is non-expansive if and only if

(y − y)T (f(y)− f(y)) ≤ 0 for all y, y ∈ D. (3.16)

48

Proof. Non-expansiveness means that the function

τ(t) := |Φty − Φty|2

is monotonically non-increasing, e.g.τ ′(t) ≤ 0.

The chain rule yields that

τ ′(t) = (Φty − Φty)T (d

dtΦty − d

dtΦty).

Since Φ is the flow map of the ODE y = f(y) it follows that

τ ′(t) = (Φty − Φty)T (f(Φty)− f(Φty))

for all t, y, y for which the above expression is defined. This clearly implies the desired statement.

Armed with Lemma 3.3.3 we can now show the first fundamental result of this section, namely thatany RK-SSM which inherits non-expansiveness is automatically A-stable.

Theorem 3.3.4. Suppose that Ψ is a RK-SSM with the property that the family Ψh is non-expansivewhenever applied to a nonexpansive flow Φt of an ODE y = f(y) (inheritance of non-expansiveness).Then Ψ is A-stable.

Proof. Consider the complex ODE z = λz with λ = α+ iβ and α < 0. Writing z = u+ iv we get a linearODE

y = Ay, y(0) = y0 ∈ R2 (3.17)

and

A =

(α −ββ α

).

We first show that the flow map of (3.17) is non-expansive. To this end we need to verify that (3.16)holds true. For this particular (linear) RHS (3.16) is equivalent to

yTAy ≤ 0 for all y ∈ R2.

A short calculation yields thatyTAy = α|y|2

which, together with the assumption α < 0, yields non-expansiveness. Now we invoke our assumption,namely that Ψh inherits the non-expansiveness of Φt. Since Ψh is a linear map it follows that

‖Φh‖|y| ≤ |y|

which, together with the fact that Φh = S(hλ), yields

|S(hλ)| ≤ 1.

Since in our argument λ ∈ C− was arbitrary, we get

|S(z)| ≤ 1 for all z ∈ C−.

By the maximum principle for holomorphic functions it now follows that

|S(z)| < 1 for all z ∈ C−

which is A-stability.

Now we can finally proceed with the proof of Theorem 3.3.1!

49

Proof of Theorem 3.3.1. By Theorem 3.3.4 we have to show that Gauss collocation methods inherit non-expansiveness. So let y = f(y) an ODE with RHS f satisfying (3.16). As in the definition (Definition2.2.1) the flow Ψh(y) = y(h) and Ψhy = y(h), where y, y ∈ Ps, the interpolation polynomials. Considernow the quantity

d(τ) := |y(τh)− y(τh)|2 ∈ P2s.

We need to show that d(1) ≤ d(0). To this end we write

d(1) = d(0) +

∫ 1

0

d′(τ)dτ.

The proof is finished once we can establish that∫ 1

0

d′(τ)dτ ≤ 0.

Now comes the crucial point: Since d′ ∈ Ps−1 is exact, we can write∫ 1

0

d′(τ)dτ =

s∑i=1

bid′(ci).

A short calculation yields that

d′(ci) = 2h(y(cih)− y(cih))T (y(cih)− ˙y(cih)).

Observe that the collocation conditions precisely say that

y(cih) = f(y(cih))

and an analogous expression for y. In summary we get

d′(ci) = 2h(y(cih)− y(cih))T (f(y(cih))− f(y(cih))) ≤ 0,

the last equation being due to the assumption that Φt is non-expansive and (3.16). Since the weights forGauss integration bi are positive for i = 1, . . . , s we thus get∫ 1

0

d′(τ)dτ =

s∑i=1

bid′(ci) ≤ 0

which is what we wanted to show.

3.4 Uniform Stability

We have shown that the concept of A-stability allows for the construction of RK-SSMs which preserveasymptotic stability of FPs. However, nothing is said regarding the convergence speed towards the FP.For instance, suppose that the ODE flow Φt converges to the FP very quickly, does the SSM also convergefast? Let us consider a simple example, the ODE

y = −λy ∈ R, y(0) = 1

with λ large, say, λ = 1000 and solution y(t) = exp(−1000t). Observe that here 0 is a FP and it isapproached very quickly. For instance at about t = 0.025 the value of y(t) is already below 10−12.

Let us now solve our ODE using a RK-SSM. Previously we have shown the A-stability of Gaussmethods, so we know that also the numerical solution will converge to 0. But how fast? Let us startwith the simplest Gauss method, the implicit midpoint rule with s = 1. Its stability function is given by

SIM (z) = 1−z/21+z/2 . So with stepwidth h we have that

yIMh (lh) = yIMl = (SIM (−1000h))l.

50

Figure 3.2: Solution of the logistic ODE y = −500y(1− y) with implicit midpoint and implicit Euler.

Let us start with a rather small stepwidth h = 0.01. Then

yIMh (0.03) = (SIM (−10))3 = (−2/3)3 ∼ −0.3,

which is still far from zero (and negative; also there are oscillations of the sign in the simulation!).Let us do the same experiment with implicit Euler and stability function SIE(z) = 1

1−z . Then

yIEh (0.03) = (SIE(−10))3 = (1/11)3 ∼ 10−3,

which is much better than what we get with the implicit midpoint rule (and in particular non negative).If we increase −λ this effect gets even more pronounced, and persists for nonlinear problems with stronglyattractive FPs, see Figure 3.2 which shows solutions to the logistic ODE

y = −500y(1− y), y(0) = 1

with the implicit midpoint and the implicit Euler method. The underlying reason that implicit Eulerperforms better than implicit Midpoint for strongly attractive FPs is simply that

limz=−∞

SIM (z) = −1,

whilelimz=−∞

SIE(z) = 0.

So, the larger −λ, the ‘closer’ SIE(−hλ) is at zero, while for IM the opposite is the case: SIM (hλ) ∼ −1.We mention the following result without proof.

Lemma 3.4.1. Let S be the stability function of the s-stage Gauss collocation scheme. Then

limz→−∞

S(z) = (−1)s.

So, unfortunately, all Gauss collocation suffer from the fact that they converge slower towards anasymptotically stable FP, the faster the exact solution does. On the other hand, the implicit Eulermethod converges faster towards an asymptotically stable FP, the faster the exact solution does. Canwe construct arbitrary order RK-SSMs with the same great properties?

51

Definition 3.4.2 (Uniform Stability). A RK-SSM is uniformly stable if it is A-stable and its stabilityfunction S satisfies

limz→−∞

S(z) = 0. (3.18)

We start with a useful condition for (3.18) to hold true.

Lemma 3.4.3. Suppose that Ψ =c A

bTis a RK-SSM and assume that A is invertible. Let aj be the

j-th row of A. If b = aj for some j = 1, . . . , s, then (3.18) holds true for the stability function of Ψ.

Proof. By (3.2) it holds that

S(z) = 1 + zbT (I − zA)−11 = 1 + bT (1

zI −A)−11

It follows thatlim

z→−∞S(z) = 1− bTA−11.

Now suppose that b = aj . Then bTA = (0, . . . , 1, . . . , 0)︸ ︷︷ ︸j th coordinate

and therefore

limz→−∞

S(z) = 1− bTA−11 = 1− 1 = 0.

In addition, recall from Lemma 2.2.6 that the collocation matrix of any collocation scheme (exceptexplicit Euler) is regular. So we want to construct uniformly stable collocation schemes. By Lemma 3.4.3we know that we need to pick the collocation nodes in a way that a row of A is equal to the weightsequence b.

Lemma 3.4.4. Suppose that Ψ is an s-stage collocation scheme with cs = 1. Then

limz→−∞

S(z) = 0.

Proof. By Lemma 2.2.5 we know that aij =∫ ci

0Lj(τ)dτ and bi =

∫ 1

0Lj(τ)dτ . This implies the statement.

Lemma 3.4.5. The family of Radau collocation methods as defined in Example 2.2.8 is of consistencyorder 2s− 1 (where s is the number of stages) and satisfies

limz→−∞

S(z) = 0.

Proof. The statement on the consistency order follows from the corresponding statement for Radauquadrature and Theorem 2.3.14. The definition of Radau schemes in Example 2.2.8 implies that cs = 1which, by Lemma 3.4.4 implies the desired result.

We can now prove the main result of this section.

Theorem 3.4.6. Radau methods are uniformly stable.

Proof. By Lemma 3.4.5 we only need to show that Radau methods are A-stable. The proof of this latterfact proceeds essentially in the same way as the corresponding proof for Gauss methods, e.g. Theorem3.3.1 and therefore we only present the necessary adjustments. As in the Gauss case we need to showthat ∫ 1

0

d′(τ)dτ ≤ 0.

where d(τ) := |y(τh) − y(τh)|2 ∈ P2s, compare the proof of Theorem 3.3.1 for more information. Weknow that d′ ∈ P2s−1 and in the proof of Theorem 3.3.1 we made crucial use of the fact that Gauss

52

collocation is exact on P2s−1. This is not the case for Radau quadrature! However we may use a generalformula for the representation of the quadrature error:∫ 1

0

q′(τ)dτ =

s∑i=1

biq′(ci)−

s((s− 1)!)4

1((2s− 1)!)2q(2s)(η),

for some η ∈ (0, 1) and all q ∈ C2s. We want to apply this formula to d ∈ P2s. Therefore we know thatq(2s)(η) = (2s)!α2s, where α2s denotes the leading coefficient of the polynomial d. Thus we only need toshow that α2s ≥ 0. But this follows from the fact that q is always nonnegative.

3.5 (*) Differential Algebraic Equations (DAEs)

TBA

53

Chapter 4

Structure Preservation

In this chapter we are interested in the study of preservation of mapping properties of the flow Φt to an(autonomous) ODE. As a motivating example, consider the mathematical pendulum studied in Section1.2.3. There, we have shown that the Hamiltonian

H(p, q) =1

2p2 − cos(q)

is an invariant of the Hamiltonian system

p = − sin(q)

q = p,

or in other words, the associated flow map Φt satisfies

H Φt = H Φ0.

As an exercise you can also show another mapping property of Φt (or rather its derivative). Writingy = (p, q)T we have that

det

(∂

∂yΦt|y

)= 1

for all t, y. This follows directly from the fact that the flow Φ is associated with a Hamiltonian system.In particular it implies that the flow Φt is volume preserving in R2 for all t.

In summary we can say that the mapping Φt often possesses a lot of structure, encoded in invarianceproperties of Φt or its derivatives. Suppose we try to solve the corresponding ODE with a SSM Ψh.The goal of this section is to study to which extent the numerical flow Ψh can inherit the correspondinginvariance properties1.

4.1 Polynomial Invariants

Recall Definition 1.2.2: A function I : D → R such that for all solutions y(t) of (1.1) we have

I(y(t)) ≡ const

is called an invariant of (1.1). In other words, if (1.1) is autonomous, I is an invariant iff I Φt = H Φ0.The following simple first order condition on a function I to constitute an invariant is very useful.

1We will only study very basic invariance properties and in particular we will only touch the tip of an iceberg: the fieldof structure preserving SSMs is a vast and interesting one. Maybe the most important area that we had to leave out, dueto time restrictions, is the symplectic integration of Hamiltonian systems. The interested reader can consult the article

E. Hairer, Ch. Lubich, G. Wanner. Geometric Numerical Integration illustrated by the Stormer-Verlet method.Acta Numerica 12, pp 399-450 (2003),

and the many references therein, in particular the great monograph Geometrical Numerical Integration by the same authors.

54

Lemma 4.1.1. A function I ∈ C1(D,R) is an invariant of the ODE y = f(y) if and only if

∇I(y) · f(y) = 0 for all y ∈ D.

Proof. This is simply due to the fact that the derivative of I(y(t)) must vanish for I being an invariant.

In this section we are interested in a particular class of invariants, namely polynomial invariants.

Definition 4.1.2. Suppose that I is an invariant of an autonomous ODE. I is called a polynomialinvariant of degree n if I ∈ Pn(Rd). For n = 1 we also call I a linear invariant, for n = 2 a quadraticinvariant and for n = 3 a cubic invariant, and so on. Suppose that Ψ is a RK-SSM. We say that Ψpreserves polynomial invariants of degree n if for any autonomous ODE with an invariant I ∈ Pn(Rd), thefunction I is also an invariant of the corresponding numerical flow map Ψh in the sense that IΨh = IΨ0

for all h > 0 such that this expression is well-defined.

Let us first look at the case n = 1.

Lemma 4.1.3. Any RK-SSM preserves linear invariants.

Proof. Suppose that y = f(y) is an autonomous ODE with a linear invariant I(y) = uT y + c, whereu ∈ Rd and c ∈ R. To compute the numerical flow Ψh we need to solve the stage equations (2.6), e.g.

ki = f(y0 + h

s∑j=1

aijkj), i, j ∈ 1, . . . , s.

By assumption, I is an invariant so, by Lemma 4.1.1 we get that

uT f(y) = 0 for all y ∈ D.

In particular, we get that

uT ki = uT f(y0 + h

s∑j=1

aijkj) = 0.

It follows that

I Ψh(y0) = uTΨh(y0) + c = uT

(y0 + h

s∑i=1

biki

)+ c = uT y0 + c = I Ψ0y0.

How about quadratic invariants?

Example 4.1.4. Let us first start with a simple example, the ODE

y = f(y) := c× y ∈ R3,

where c × y :=

c2y3 − c3y2

c3y1 − c1y3

c1y2 − c2y1

denotes the crossproduct and c ∈ R3 arbitrary. We now show that the

function I(y) := y21 + y2

2 + y23 = y · y ∈ P2(R3) is a quadratic invariant. To this end, notice that

∇I(y) = 2y.

Since y · (c × y) = 0 we get that ∇I(y) · f(y) = 0 which, by Lemma 4.1.1 implies that I is indeed aninvariant. We could try to solve this ODE using the explicit Euler method. Then Ψh(y) = y + hc × y.We have

I(Ψh(y)) = y · y + 2hy · (c× y) + h(c× y) · (c× y) =︸︷︷︸y·(c×y)=0

I(y) + hI(c× y).

Clearly, the invariant I is not preserved.

55

Let us now try the implicit midpoint rule for the same problem. There we have that

k1 = f(y + h/2k1) = c× y +h

2c× k1

andΨh(y) = y + hk1.

We get that

I(Ψh(y)) = I(y) + 2hy · k1 + h2k1 · k1 = I(y) + 2h(y + h/2k1 − h/2k1) · k1 + h2k1 · k1

Using thatk1 = f(y + h/2k1)

we get that

I(Ψh(y)) = I(y) + 2h(y + h/2k1) · f(y + h/2k1)− h2k2 · k1 + h2k1 · k1 = I(y),

where we have used that (y + h/2k1) · f(y + h/2k1) = 0.In summary I is not preserved by explicit Euler but it is preserved by the implicit midpoint rule.

The previous example has shown us that preservation of quadratic invariants depends on the schemewe use. The midpoint rule does preserve quadratic invariants (as we will show shortly) while the explicitEuler scheme does not (in fact, it can be shown that no explicit RK-SSM preserves quadratic invariants).Similar considerations show that the implicit Euler scheme does not preserve quadratic invariants, either.

Our next main result shows that Gauss collocation methods always preserve quadratic invariants.

Theorem 4.1.5. All Gauss collocation schemes preserve quadratic invariants.

Proof. Suppose that I ∈ P2(Rd) is an invariant of the autonomous ODE y = f(y). Let us apply the s-stage Gauss collocation scheme to this equations (see Definition 2.2.1 and Example 2.2.7). So let y ∈ Psbe the collocation polynomial such that y(h) = Ψh(y) (please excuse the obvious abuse of notation).Define

d(τ) := I(y(τh)) ∈ P2s, τ ∈ [0, 1].

Obviously we have d(0) = I(y) and d(1) = I(Ψhy), so we need to show that d(1) = d(0). To this end wewrite

d(1) = d(0) +

∫ 1

0

d′(τ)dτ.

Now comes the point where we use the special properties of Gauss collocation: Since d′ ∈ P2s−1, the

integral∫ 1

0d′(τ)dτ can be evaluated exactly using Gauss quadrature:∫ 1

0

d′(τ)dτ =

s∑i=1

bid′(ci).

We will show that d′(ci) = 0 for all i = 1, . . . , s which will finish the proof.Obviously,

d′(τ) = h∇I(y(τh)) · y(τh),

and therefored′(ci) = h∇I(y(cih)) · y(cih) = h∇I(y(cih)) · f(y(cih)),

by the very definition of collocation schemes (Definition 2.2.1). It remains to note that I is an invariantof y = f(y) which, by Lemma 4.1.1 implies that

∇I(y(cih)) · f(y(cih)) = 0

and we are done.

56

Gauss collocation schemes are remarkable; they possess optimal order and preserve quadratic in-variants. The latter fact has several important consequences that we cannot go into. For example,preservation of quadratic invariants implies symplecticity, a crucial property for the efficient long termintegration of Hamiltonian systems.

But how about higher order invariants? Can we find RK-SSMs which are capable to preserve arbitrarypolynomial invariants? Intuitively this might sound too good to be true and this intuition turns our tobe correct. In fact, there does not even exist a RK-SSM which preserves cubic invariants.

Theorem 4.1.6. There does not exist a RK-SSM which preserves cubic invariants.

Proof. We argue by contradiction. Suppose that the RK-SSM Ψ preserves cubic invariants. We willderive a contradiction from this statement.

Consider the linear matrix ODE

Y = AY, A, Y ∈ Rn×n.

As a first step we show that the function det : Y 7→ det(Y ) ∈ Pn is an invariant if trace(A) = 0. UsingJacobi’s formula for the derivative of the determinant we get that

d

dtdet(Y (t)) = det(Y (t))trace(Y (t)Y −1(t)) = det(Y (t))trace(A) = 0

if trace(A) = 0 (note that in the previous formula we needed to invert Y (t); if this were impossible wewould have det(Y (t)) = 0 which would also imply the desired statement).

Recall that we assumed that Ψ preserves cubic invariants. So, for n = 3 we would have that, for anyA ∈ R3×3 with trace(A) = 0 we have

det(Ψh)I = det(S(hA)) det(I) = det(I) = 1,

where S is the stability function of Ψ and I is the 3 × 3 identity matrix. In particular this has to holdfor all matrices

A = diag(ν, µ,−(ν + µ)), ν, µ ∈ R,

which implies thatS(ν)S(µ)S(−(ν + µ)) = 1 for all ν, µ ∈ R.

Putting ν = −µ and using the fact that S(0) = 1 we see that

S(µ)−1 = S(−µ) for all µ ∈ R

which implies thatS(ν)S(µ) = S(ν + µ) for all ν, µ ∈ R.

We know that this functional equation can only be satisfied by S(z) = exp(z) which contradicts the factthat the stability function of a RK-SSM must be rational.

4.2 Volume Preservation

Another important qualitative property of certain ODEs is that their flow map is volume-preserving.

Definition 4.2.1 (Volume preserving map). A mapping Φ : D ⊂ Rd → Rd is volume-preserving, if forall measurable sets V ⊂ D we have

volume(V ) = volume(Φ(V )).

An autonomous ODE y = f(y) is volume-preserving if the flow map Φt is volume-preserving for all twhere the latter is well-defined.

Lemma 4.2.2. Suppose that Φ ∈ C1(D,Rd). Then Φ is volume-preserving if and only if |det(DΦ|y)| = 1for all y ∈ D.

57

Proof. This follows from the transformation formula for integrals and the fact that volume(V ) =∫V

1dx.

Lemma 4.2.3. An autonomous ODE is volume-preserving if and only if div(f(y)) = 0 for all y ∈ D.

Proof. For t = 0, Φ0 = Id and thus volume-preserving with det(DΦ0|y) = 1. Therefore, an autonomousODE is volume-preserving if and only if d

dt det(DΦt|y) = 0 for all t and y. Again, by Jacobi’s formula wehave that

d

dtdet(DΦt|y) = det(DΦt|y)trace(

d

dtDΦt|y(DΦt|y)−1)).

Now note thatDΦt|y = W (y; t),

the Wronskian (see (1.22)), and in particular we have that

d

dtDΦt|y = Df |ΦtyDΦt|y,

which yieldsd

dtdet(DΦt|y) = det(DΦt|y)trace(Df |Φty) = det(DΦt|y)div(f)(Φty).

So if div(f(y)) = 0 for all y ∈ D if follows that the corresponding ODE is volume-preserving. Theconverse is left to the reader.

The question we want to ask ourselves is the following: Suppose that y = f(y) is a volume-preservingODE, e.g. that div(f(y)) = 0 for all y ∈ D. Can we derive SSMs which also preserve volume in thesense that for all h > 0, the maps Ψh are volume-preserving? Note that volume-preservation is not aninvariant in the sense of Definition 1.2.2; it is rather an invariant of the variational equation (1.22), or inother words, the determinant function is an invariant of DΦt, which satisfies (1.22).

In what follows we will require the following fundamental result which is best expressed in the formof commuting diagrams.

Lemma 4.2.4. The following diagram commutes.

y = f(y)RK-SSM−−−−−−−→ Ψhy0y d

dy

y ddy

y = f(y); Y = Df(y)YRK-SSM−−−−−−−→ (Ψhy0, DΨh|y0)

In other words, suppose we apply a RK-SSM to an ODE y = f(y). Then the differential DΨh|y0 can

be computed by applying the same RK-SSM to the variational ODE y = f(y); Y = Df |yY , y(0) =y0; Y (0) = I.

Proof. We first compute DΨh|y0 by differentiting the stage equations:

Dki = Df(y0 + h

s∑j=1

aijkj)(I + h

s∑j=1

aijDkj).

Note that the Dki’s satisfy precisely the stage equations for the matrix ODE Y = Df |yY . This provesthe statement.

Since, provided that d ≤ 2, the function det is a quadratic invariant of the variational equation,Lemma 4.2.4 implies that Gauss collocation schemes preserve volume in this case. However in higherdimensions no RK-SSM can preserve volume in general as the following lemma shows.

Lemma 4.2.5. Suppose that y = f(y) ∈ Rd is a volume-preserving ODE. Then, if d ≤ 2 and Ψ is aRK-SSM preserving quadratic invariants (such as Gauss collocation schemes, See 4.1.5), Ψ is volume-preserving. If d > 2 then no RK-SSM is volume-preserving for all volume-preserving ODEs.

58

Proof. Let us first consider the case d ≤ 2. We want to show that det(DΨh|y) = 1. By Lemma 4.2.4DΨh|y is the result of applying the RK-SSM for one step with stepwidth h to the IVP

Y = Df(Φty)Y,

which possesses the (quadratic) invariant det. Since this invariant is preserved by assumption, we get thatdet(DΨh|y) = 1 which proves the first part. The fact that no RK-SSM preserves volume in dimensionsd > 2 follows from the arguments in the proof of Theorem 4.1.6.

So in general it is not possible for any RK-SSM to preserve volume in dimensions higher than 2.There are several remedies for this problem, among them is to decompose a divergence-free vector fieldf : D ⊂ Rd → Rd into two-dimensional divergence-free vector fields.

Lemma 4.2.6. Let f : D ⊂ Rd → Rd is a divergence-free vector field. Then there exist divergence-freevector fields gi : D ⊂ Rd → Rd, i = 1, . . . , d− 1 with

f =

d−1∑i=1

gi

andgi(y) = (0, . . . , 0, g1

i (y)︸ ︷︷ ︸i-th coordinate

, 0, . . . , 0, g2i (y))T .

Proof. We write f(y) = (f1(y), . . . , fd(y))T and y = (y1, . . . , yd)T . Since f is divergence-free we get

fd(y) =

∫∂fd

∂yd(y)dym + C(y1, . . . , yd−1) = −

∫ d−1∑i=1

∂f i

∂yi(y)dym + C(y1, . . . , yd−1),

where C(y1, . . . , yd−1) is a suitable integration constant (which clearly may depend on the first d − 1variables). We will ignore this integration constant in what follows because it is irrelevant (why?). Wecan now put, for i = 1, . . . , d− 1 and j = 1, . . . , d,

gii(y) = fi(y), gdi (y) = −∫∂f i

∂yi(y)dym, gji (y) = 0, j /∈ i, d.

It remains to show that the vector fields gi are divergence-free. To this end we compute

div(gi(y)) =∂f i

∂yi(y)− ∂

∂ym

∫∂f i

∂yi(y)dym = 0.

Now suppose we are given a volume-preserving ODE y = f(y), where f =∑d−1i=1 gi according to

Lemma 4.2.6. Given a RK-SSM preserving quadratic invariants (for instance Gauss collocation) anddenote by Φhi this RK-SSM applied to the ODE y = gi(y) for i = 1, . . . , d−1. Since, according to Lemma4.2.6 gi is a two-dimensional, divergence-free vector field, the RK-SSM Ψh

i is volume-preserving. Sincethe composition of volume-preserving mappings is volume-preserving, we can design a volume-preservingsplitting scheme

Ψh := Ψhd−1 · · · Ψh

1

for the numerical solution of y = f(y). By designing more elaborate splitting schemes and sufficientlyhigh order of the underlying RK-SSM method, it is possible to design volume-preserving methods ofarbitrary order (see also Section 2.3.4). But note that the so-designed SSM is problem-dependent andonly works for the specific ODE y = f(y). Its design requires the knowledge of a decomposition accordingto Lemma 4.2.6.

4.3 Symplectic Integration of Hamiltonian Systems

TBA.

59