numerical methods for ordinary differential …bnikolic/teaching/phys660/lectures...numerical...

Numerical Methods for Ordinary Differential Equations

Branislav K. NikolićDepartment of Physics and Astronomy, University of Delaware, U.S.A.

PHYS 460/660: Computational Methods of Physics http://www.physics.udel.edu/~bnikolic/teaching/phys660/phys660.html

PHYS 460/660: Numerical Methods for ODE

Ordinary Differential Equations

2

2, ( ), ( ), ..., ( ) 0

n

n

d d dF y y t y t y t

dt dt dt

=

Ordinary: only one independent variable

Differential: unknown functions enter into the equation through its derivatives

Order: highest derivative in F

Degree: exponent of the highest derivative3

2

2Example: ( ) ( ) 0

dy t y t

dt

− =


What is Solution of ODE?

A problem involving ODE is not completely specified by its equation

ODE has to be supplemented with boundary conditions:

( )y y t=

•Initial value problem: is given at some starting value , and it is desired to find at some final points or at some discrete list of points (for example, at tabulated intervals).

•Two point bondary value problem: Boundary conditions are specified at more than one ; typically some of the conditions will be specified at and some at .

yit

y ft

tit

ft


What is Numerical Solution to the Initial Value Problem?

( ) 0 0

( ), ( ) ; ( )

dy tf t y t y t y

dt= =

A numerical solution to this problem generates sequence of values for the independent variable

and a corresponding sequence of values of the dependent variable

so that each approximates solution at :

1 2, , , nt t t…

1 2, , , ny y y…

ny nt

( ) , 0,1,n ny t y n≈ = …


Euler Metod

tru ey

t∆

y

t

E u lery

All finite difference methods start from the same conceptual idea: Add small increments to your function corresponding to derivatives (right-hand side of the equations) multiplied by the stepsize.

Euler method is an implementation of this idea in the simplest and most direct form.

Single-Step Forward

Propagation


Euler Algorithm for First-Order ODE

( , )dy

f t ydt

= ( , )y f t y t∆ = ∆

1 1 1

1

1

initialize , ( )

do while

( , )

end do

i i i

i i

t y y t

i n

y y f t y t

t t t

+

+

≡

≤

= + ∆

= + ∆


Step Size Effects in Radioactive Decay

Analytics: ( 0)t

U UU U

dN NN N t e

dtτ

τ

−

= − ⇒ = =

( )2

1

Numerics (Euler):

( ) (0) ( )UU U

ii i

dNN t N t O t

dt

NN N t

τ+

∆ = + ∆ + ∆

≈ − ∆

Numerical solution will depend Numerical solution will depend on the step sizeon the step size

t∆


Stability of Euler Algorithm

t∆

1

(0) 1,

After n Euler steps of size :

(1 )

at

n

n n n n

dyay y y e

dt

t

y y ay t y a t

−

+

= − ⇒ = =

∆

= − ∆ ⇒ = − ∆

Approximate solution will decay monotonically only if is small enough:

For a single decaying exponential-like solution (i.e. if there is only one first order equation) the existence of a stability criterion is not a problem because has to be small for the reasons of accuracy.t∆

Step size if often limited by the stability criterion:

max

1t t

a∆ ≤ ∆ ≡


Accuracy:Discretization and Roundoff Errors

Local:

Global:

1 1 1

( , )( )

( )

n n

n n n n

n n n

duf u t

LE y u tdt

u t y+ + +

=

⇒ = −=

( )n n nGE y y t= −

0

1

0

1

0

1

0

1

0

( ) ( ) ( ) ( )

( ) ( )

( ) ( )

N

n

n

N

Nt

n nt

n

t

n n nt

Nt

n n nt

n

N

n n

n

f f t y t f d h f t

LE h f t f d

GE h f t f d

GE LE

τ τ

τ τ

τ τ

+

−

=

−

=

−

=

= ⇒ = ≈

= −

= −

=

∑∫

∫

∑ ∫

∑

Method is of order n iff:1 1( )n n

n nLE O h LE Ch+ += ⇔ ≤

1n nh t t t+= − ≡ ∆

1

1

Number of steps for roundoff error to be comparable with the discretization error:

pCN L

Lε

+ ≈

Integrate over interval: Full Error: 0 p

f

LL t t Ch

h

ε= − ⇒ +


Global Discretization Error Example

2 3

2 3

2 3

2 2

2

( ) , 1

( ) ( )( ) 1

2! 3!

( 1) ( ) ( 1)( 2) ( )1

2! 3!

1 ( ) 3 ( ) 1( )

2! 3! 2

n

aT

n

n

aT

n

Ty T e y a

n

aT aTy T aT

n n aT n n n aTy aT

n n

aT aT a ty T y O aTe

n n n

−

−

= = −

= − + − +

− − −= − + − +

∆ − = − + +

…

…

… ∼

Suppose we want to find the solution over the interval Suppose we want to find the solution over the interval →→Divide Divide the interval into the interval into nn equal steps so that equal steps so that

[ ]0,Tt T n∆ =

This is a measure of theglobal truncation errorglobal truncation error,i.e., the error over a fixed range in t. It is proportional to the first power of the step size and hence the Euler the Euler method is a first order method is a first order methodmethod (do not confuse this with the fact that we are applying it in this case to a first order equation).


Reducing Higher Order ODE to System of First Order ODE

Solve higher order ODEs by splitting them into sets of first order equations:

2

2( ) ( ) ( )

( ) ( ) ( )

There is no unique way to do this:

( )( ) ( )

( )

( )

d y dyp t q t y g t

dt dt

dzg t p t z q t y

dy dtz

dydtz

dt

dz dp tg t q t y

dy dt dtz p t y

dt dyz p t y

dt

+ + =

= − −

= ⇒ =

= + − = + ⇒

= −


Example: Realistic Motion of Baseball

1 1

1

1

initialize , ( )

do while

( , )

end do

i i i i

i i

t y t

i n

y y f t y t

t t t

+

+

≤

= + ∆

= + ∆

v rω−2

2

2 02

d r vm mg B v S v

dt vω= − + ×

v rω+

d r a gF

v

ω

1

21

1

1

1

01

x

i i i

x x x

i i i

y

i i i

y y

i i

z

i i i

z z xi i

x x v t

Bv v vv t

m

y y v t

v v g t

z z v t

S vv v t

m

ω

+

+

+

+

+

+

= + ∆

= − ∆

= + ∆

= − ∆

= + ∆

= − ∆

⇔


More Realistic Modeling of Air Flow


ODE of Linear Harmonic Oscillator

2

2

2

2

sin 0

for small θ sin

0,

d g

dt l

d g g

dt l l

θθ

θ θ

θθ

+ =

⇒ ≈

+ = Ω =

2

2 21 1must be conserved!

2 2total

dE ml mgl

dt

θθ

= +


Euler Method for Linear Harmonic Oscillator

Switch to dimensionless quantities: 2

02

2

2

0 sin( )

1 1

2 2total

dt

dt

dE

dt

θθ θ θ φ

θθ

+ = ⇒ = Ω +

= +

( )

( )

2 211 1

12

1

1

2

1

n n ntotal n n

n n n

total nn n

tE

t

E E tt t t

ω ω θω θ

θ θ ω+

+ +

+

+

= − ∆ = +

= + ∆ ⇒ = + ∆= + ∆

Euler discretization algorithm:


Euler Fails on ( )tθ


Euler Fails on ( )tω


Euler Fails on Phase Space Trajectory


Can We Resurrect Euler by Using Smaller Step Size?


Cromer Fixed Euler Method for LHO

1

1 1 1

1

n n n

n n n n n

n n

t

t

t t t

ω ω θ

ω ω θ θ ω

+

+ + +

+

= − ∆

→ ⇒ = + ∆ = + ∆

Apparently trivial trick, but:

( ) ( )

2 2 20 0

2 2 2 3

1

0 0 0 0

cos2( ) 0

1

2

sin( ), cos( )

over a period

n n n n

t t

E E t O t

t t t t

ω θ θ

ω θ

θ θ ω θ

+

− = − =

= + − ∆ + ∆

= − = −


From Euler to Higher Order Algorithms

1

1

( , )n n n n

n n

y y f t y

t t h

+

+

= +

= + ( ) ( )m

exact

ty t t y t dy dt t+ ∆ = + ∆

vs.

Mean value theorem


Midpoint Method: Second Order Runge-Kutta

1

2 1

( , )

( , )2 2

n n

n n

s f t y

h hs f t y s

=

= + +

3

1 2

1

( )n n

n n

y y hs O h

t t h

+

+

= + +

= +


Classical Runge-Kutta

5

1 1 2 3 4

1

( 2 2 ) ( )6

n n

n n

hy y s s s s O h

t t h

+

+

= + + + + +

= +

Fourth-order method

1

2 1

3 2

4 3

( , )

( , )2 2

( , )2 2

( , )

n n

n n

n n

n n

s f t y

h hs f t y s

h hs f t y s

s f t h y hs

=

= + +

= + +

= + +


Classical Runge-Kutta F90 Subroutine


General Single-Step Methods

1

,

1

( , ), 1, ,i

i n i n i j j

j

s f t h y h s i kα β−

=

= + + =∑ …

Each of the k stages of the algorithm computes slope by evaluating for a particular value of and a value of obtained by taking linear combinations of the previous slopes:

is( , )f t y t y

The proposed step is also a linear combination of the slopes:

1

1

k

n n i i

i

y y h sγ+=

= + ∑Error is estimated from yet another linear combination of the slopes:

1

1

k

n i i

i

e h sδ+=

= ∑

The parameters are determined by matching terms in the Taylor series expansion of the slopes → the order of the method is the exponent of the smallest power of h that cannot be matched.

In MATLAB ODE numerical routines are named as odennxx, where nn indicates the order and xx is some special feature of the method.


Example: MATLAB ode23 Function(Bogacki and Shampine BS23 Algorithm)

1

2 1

3 2

( , )

( , )2 2

3 3( , )

4 4

n n

n n

n n

s f t y

h hs f t y s

s f t h y hs

=

= + +

= + +

1 1 2 3

1 4 1 1

(2 3 4 )9

; ( , )

n n

n n n n

hy y s s s

t t h s f t y

+

+ + +

= + + +

= + =

1 1 2 3 4( 5 6 8 9 )72

n

he s s s s+ = − + + −


Beyond Runge-Kutta Methods

Runge-Kutta methods propagates a solution over an interval by combining the information from several Euler-style steps (each involving one evaluation of the right-hand side f’s), and then using the information obtained to match Taylor series expansion up to some higher order.

Richardson extrapolation method used the powerful idea of extrapolating computed result to the value that would have been obtained if the stepsize had been very much smaller than it actually was. In particular, extrapolation to zero stepsize is the desired goal – implemented by Burlich-Stoer algorithm.

Predictor-corrector methods store the solution along the way, and use those results to extrapolate the solution one step advanced; they correct the extrapolation using derivative information at the new point.


Stiff Systems of Differential Equations

Stiffness arises in systems of ODE where there are two or more very different scales of the independent variable:

2998 1998 , 999 1999

(0) 1, (0) 0

du dvu y zu v u v

dt dtv y z

u v

= −= + = − −

⇒ = − += =

1000

1000

2 t t

t t

u e e

v e e

− −

− −

= −⇒

= − +

Follow the variation in the solution on the shortest length scale to maintain stability of the integration even though accuracy requirements allow for a much larger step size → use implicit methods:

explicit

1

implicit

1 1 1

, 0 (1 )

2 as

1

n n n n

n

nn n n n

y cy c y y ty c t y

t c y n

yy cy y y ty y

c t

+

+ + +

′ ′= − > ⇒ = +∆ = − ∆

∆ > ⇔ →∞ →∞

′ ′= − ⇒ = +∆ ⇒ =+ ∆


Solutions to Stiffness Beyond Implicit Euler

Generalizations of Runge-Kutta methods →Rosenbrock methods and Kaps-Rentrop methods.

Burlich-Stoer algorithm generalized to Bader-Deuflhard semi-implicit extrapolation method.

Predictor-corrector methods generalized toGear backward differentiation method.

To improve higher-order (than Euler, which is first-order) methods use:

numerical methods for ordinary differential …bnikolic/teaching/phys660/lectures...numerical...

Documents