random thoughts on numerical analysis

8/3/2019 Random Thoughts on Numerical Analysis

1/17

Numerical Analysis thoughts

Matthew OConnell

January 12, 2010

Regions of stability

Region of Stability: Heun

We wish to describe the region of stability forstiff problems of different numerical ODE meth-ods, particularly Heun, Crank-Nicelson, and theclassical Runge-Kutta. Beginning with Heun.

Heun is a one step method for solving CauchyODEs given by:

un+1 = un+h

2

f(tn, un)+f(tn+h, un+hf(tn, un))

Using the test problem:

P =

y(t) = yy(0) = 1

we can see that f(tn, un) = un.

un+1 = un +h

2

un + +[un + hun]

after some simplification

un+1 = un +

1 + h +

h2

2 2

We can now apply the initial condition, u0 = 1and see

u1 = [1 + h +h2

22]

u2 = u1[1 + h +h2

22] or [1 + h +

h2

22]2

furthermore, successive values of u will be givenby:

un+1 = [1 + h +h2

22]n.

We know from the analytical solution of thetest problem, that un+1 0, as n . Thiswill only occur if

[1 + h +h2

22] 1.

Thus Heun is stable within the region:

z C : [1 + hz +

z2

2 ] 1.

Graduate Student, Computational Engineering Department, University of Tennessee Chattanooga.

1


2/17

Random thoughts on Numerical Analysis 2

Figure 1: Region of stability for Heun

Region of Stability: Crank Nicelson

Crank Nicelson (CN) is a one step methodfor solving Cauchy problems given by:

un+1 = un +h

2(f(tn, un) + f(tn+1, un+1))

From the test problem, fn = un. Makingthis substitution:

un+1 = un +h

2[un + un+1]

And rewritten:

un+1(1 h2

) = un(1 +h

2)

un+1 = un(1 + h2 )

(1 h2 ).

Applying the initial condition:

u1 = u0(1 + h2 )

(1 h2 )= 1

(1 + h2 )

(1 h2 )and successively:

un+1 =

1 + h21 h2

n

And we know from the analytical solutionun+1 0 as n 0. This could only occur if

1 + h21 h2

1.

This will occur for any negative value of .Therefore

{z C :

1 + z21 z2

1}.

Region of Stability: Classic 4-stage Runge-Kutta

Classic 4-stage Runge-Kutta is a one stepmethod described as: un+1 = un + h

=1

6

K1 + 2K2 + 2K3 + K4


3/17


Figure 2: Crank Nicelson stability, stable for any negative real.

K1 = f(tn, un)

K2 = f(tn + h2, un + h2

K1)

K3 = f(tn +h

2, un +

h

2K2)

K3 = f(tn + h, un + hK3)

From the test problem, the stages become:

K1 = un

K2 = [un + hK1]

K3 = [un + hK2]

K3 = [un + hK3]h is now:

h = hun +h22

2un +

h33

6un +

h44

24un

and the RK method can be written as:

un+1 = un + hun +h22

2un +

h33

6un +

h44

24un

un+1 = un1 + h +h22

2+

h33

6+

h44

24

making the substitution z =h:

un+1 = un

1 + z +

z2

2+

z3

6+

z4

24

.

As we have done before, knowing that u0=1,successive values un+1 are

un+1 =

1 + z +z2

2+

z3

6+

z4

24

n.

From the analytic solution we know that asn , un+1 0. This requires that

1 + z +

z2

2+

z3

6+

z4

24

1.

Thus classic fourth order Runge-Kutta is sta-ble within the region

R = z C : 1 + z +z2

2+

z3

6+

z4

24 1


4/17


Figure 3: Region of stability for classic Runge-Kutta.

Higher Order Polynomial Interpolants and Condition Number

We wish to explore polynomial interpolationwith many interpolating nodes. We begin byconsiderin a function f(x) with known functionalvalues at nodes x = {0,1,2,3,4}. And we wish tointerpolate this function using a polynomial

p(x) = c4x4 + c3x

3 + c2x2 + c1x + c0.

Knowing that p must interpolate f at all nodes,we have five equations:

p(0) = f(0) (1)

p(1) = f(1) (2)

p(2) = f(2) (3)

p(3) = f(3) (4)

p(4) = f(4) (5)

f(0) = c0

f(1) = c0 + c1 + c2 + c3 + c4

f(2) = c0 + 2c1 + 4c2 + 8c3 + 16c4

f(3) = c0 + 3c1 + 9c2 + 27c3 + 81c4

f(4) = c0 + 4c1 + 16c2 + 64c3 + 256c4

And we have a linear system: Ac = b:

1 0 0 0 01 1 1 1 11 2 4 8 161 3 9 27 81

1 4 16 64 256

c0c1c2c3

c4

=

f(0)f(1)f(2)f(3)

f(4)

.A is an ill conditioned matrix, with a condi-

tion number K2 = 2592.9. This is an ill condi-tioned problem without regard to the functionf(x) as A does not depend on the functional val-ues being used to interpolate.


5/17


If we begin using more nodes, thus moreequations the condition number of the resultingA increases.

x= {0

,1

,2

,3

,4

,5} K2 = 5

.7689 10

4

x = {0, 1, 2, 3, 4, 5, 6} K2 = 1.5973 106So we can see from an increasing condi-

tion number that higher order polynomial inter-

polants cannot be expected to give a desirableinterpolation of a function. The reason for thiscan be thought of as if the vectors of the matrixA begin to point in the same direction. But we

can also see this just by looking at the polyno-mials being used. As shown in Figure ??, as thedegree of the polynomial increases, the distinc-tion between successive degrees decreases.

0 1 2 3 4 5 6 7

1

2

3

4

5

6

Figure 4: Polynomials of degree 1 6. As the degree of the polynomial increases, the distinctionbetween one polynomial and the next lessens.

Modified Euler and Heun methods as Runge-Kutta methods

Runge-Kutta Review

We can be shown that both Heun and Modi-fied Euler numerical methods for Cauchy ODEsare two stage Runge-Kutta methods. So quicklya review of Runge-Kutta methods. Runge-Kuttamethods are one step methods. All one step

methods have the form;

un+1 = un + h(tn, un; h)

For Runge-Kutta methods the step function is

given by:

(tn, un; h) =s

i=1

biki for an s stage method

where the ks are given by:

ki = f(tn + cih, un + h(s

j=1

aijkj))

and the as bs and cs are described in theButcher Tableau for the method in the form:


6/17


c1 a11 . . . a1s...

......

cs as1 . . . ass

b1 . . . bs

As a side note: if aij = 0 for all a such that j>ithen the method is explicit.

It can be ( and indeed will be) shown thatHeun and Modified Euler are two stage explicitRunge-Kutta methods by showing the coeffi-cients a,b and c for each method.

Modified Euler as a Runge-Kutta

Method

Beginning with the step function for ModifiedEuler:

= f(tn +1

2h, un +

1

2f(tn, un))

or the step function can be rewritten as:

= 0k1 + 1k2 thus b = 0, 1

where the ks are:

k1 =

f(

tn + 0

h, un + 0

h)

k2 = f(tn +1

2h, un +

1

2hk1)

In this form we can easily see that this is aRunge-Kutta method with coefficients in theButcher Tableau below.

0 0 012

12 0

0 1

Heun as a Runge-Kutta Method

Using the same methodology, Heuns step func-tion is

=1

2

f(tn, un) + f(tn + h, un + hf(tn, un))

which can be rewritten as

=

1

2 k1 +

1

2 k2 thus b =

1

2 ,

1

2 .

The stages are calculated by

k1 = f(tn + 0h, un + 0h)

k2 = f(tn + 1h, un + 1hk1)

And we have the Butcher Tableau:

0 0 01 1 0

12

12

Therefore both Heun and Modified Euler aretwo-stage explicit Runge-Kutta methods.


7/17


Example Lagrange and Cubic Spline Interpolation

As an example use of Lagrange and Natural Cubic Spline interpolation, we wish to find an inter-polating polynomial for the leftmost morning glory cloud photographed in figure ?? 1. To begin,

we take some data points on the front of the leftmost cloud

Table 1: Data set for the cloud.

x 73.7 161.2 226.1 282.1 322.4 372.3 413.9 488.4 545.1 610.7

f(x) 418.6 408.7 396.4 373.9 345.9 268.5 240.0 194.3 152.6 97.3

Figure 5: Morning Glory Clouds Shot over Australia,

Natural Cubic SplinesWe will use Natural Cubic Splines describe the function between interpolating nodes. The splinepolynomials on the intervals between the nodes given by

1APOD 2009 August 24, Picture by Mick Petroff, used under Creative Commons Attribution-Share Alike 3.0

Unported


8/17


Sj1 =Mj1(xj x)3

6hj+

Mj(x xj)36hj

+ Cj1(x xj) + Cj1

where C and Cj is given by

Cj1 = fj1 Mj1h2j

6

Cj1 =fj fj1

hj hj

6(Mj Mj1). (6)

The Ms are given by solving the linear system:

2 0 0 0 0 0 0 0 0 01 2 1 0 0 0 0 0 0 00 2 2 2 0 0 0 0 0 0

0 0 3 2 3 0 0 0 0 00 0 0 4 2 4 0 0 0 00 0 0 0 5 2 5 0 0 00 0 0 0 0 6 2 6 0 00 0 0 0 0 0 7 2 7 00 0 0 0 0 0 0 8 2 80 0 0 0 0 0 0 0 9 2

M0M1M2

M3M4M5M6M7M8M9

=

d0d1d2

d3d4d5d6d7d8d9

with , and d given by

j =hj

hj + hj+1

j =hj+1

hj + hj + 1

dj =6

hj + hj+1

fj+1 fj

hj+1 fj fj1

hj

(7)

for j = 1 (n 1)

Since we are using a cubic spline (the second derivative on the boundary nodes is fixed at zero)

0 = n = d0 = dn = 0.

With all required data at hand, we can now solve for all Sj s, yielding the cubic polynomials for


9/17


each interval xj xj+1.

S0 = 418.6 + 1.2189667 106(400315.5 + t) 0.12247557(73.7 + t)S1 = 409.1492

1.6434451

106(226.1

t)3

0.1525749(

161.2 + t)

0.000010415(

161.2 + t)3

S2 = 398.5198 0.0000120706(282.1 t)3 0.39774469(226.1 + t) 0.000013359260(226.1 + t)3S3 = 375.11501 0.000018563736(322.4 t)3 0.6138761(282.1 + t) 0.00006838420(282.1 + t)3S4 = 352.7621 0.00005522812(372.3 t)3 1.3596182(322.4 + t) + 0.000012738458(322.4 + t)3S5 = 285.39997 + 0.00001528002(413.9 t)3 1.1851325(372.3 + t) + 0.00005419465(372.3 + t)3S6 = 227.48697 + 0.000030261715(488.4 t)3 0.36623957(413.9 + t) 0.000014273799(413.9 + t)3S7 = 197.71870 0.000018754815(545.1 t)3 0.78479305(488.4 + t) 3.406446 106(488.4 + t)S8 = 153.4311 2.9442917 106(610.7 t)3 0.8556581(545.1 + t) + 0.(545.1 + t)3

The plot of the Sj s along with the original data points is in figure ?? and you can see how wellit matched the original function (cloud) in figure ??.

100 200 300 400 500 600

100

200

300

400

500

600

Figure 6: Cubic Spline Interpolant with interpolating nodes.

Lagrange Interpolation

Lagrange Interpolation is defined as

L(t) =k

j=0

yjlj(t) (8)


10/17


with lj(t)

lj(t) =k

i=0,i=j

t titj ti (9)

which will yield an order 9 polynomial:

L(x) = 97.3 + (0.598324 + (0.00106141 + (3.564172124982928 106+ (1.0552218733964638 108 + (7.933757878347993 1011+ (1.744071613713679 1013 + (7.947314825996885 1015+ (5.151899580586824 1017 4.410873647694004 1019

(282.1 + x))(413.9 + x))(226.1 + x))(545.1 + x))(161.2 + x))(488.4 + x))(322.4 + x))(73.7 + x))(610.7 + x)

The results of Lagrange interpolation can be seen in figure??

.

0 100 200 300 400 500 600

100

200

300

400

500

600

Figure 7: Lagrange Interpolating polynomial with the interpolating nodes.

Summary

The original function, data points, and cubic and Lagrange interpolation are all plotted in figureno4.On the interval [x0, xn], the spline interpolant was quite good, only deviating visibly from theoriginal function on the intervals: 2, 3, 6 and 7 (intervals where the function has the highestcurvature). Lagrange interpolation did a better than expected job on intervals 3 through 8 withserious deviations on intervals 1, 2 and 9. Given that the Lagrange polynomial was 9th order, wilddeviations are to be expected on intervals near the boundary.


11/17


Figure 8: Cubic Spline in red, Lagrange Interpolating polynomial in blue and the interpolationnodes.

Hermite Interpolating polynomials cannot exist for some functions

Given the set of data:

f0 = f(1) = 1f1 = f

(1) = 1f2 = f

(1) = 2f3 = f(2) = 1

we wish to show there there does not exist a Her-mite interpolating polynomial. Knowing that ifa polynomial H was the Hermite interpolatingpolynomial H(xi) = f(xi) and H

(xi) = f(xi).Therefore we assume that H(x) and H(x) are ofthe form of second and third degree polynomials:

H(x) = a0 + a1x + a2x2 + a3x3

H(x) = 0 + a1 + 2a2x + 3a3x2

Applying the conditions given by the data set wehave a linear system Ac = f:

1 1 1 10 1 2 30 1 2 31 2 4 8

c0c1c2c3

=

f(1)f(1)

f(1)f(2)

=

1121

. Where det(A) = 0 therefore A is singular andthere exists no unique solution for c therefore noHermite Interpolating polynomial for this set ofnodes regardless of their functional values.


12/17


Example Numerical Integration

Numerical Integration of a desired error

We wish to numerically integrate the func-tion f(x) 1

0exdx

between 0 and 1. The exact integral is

10

ex = ex|10 = e 1

We wish to evaluate this integral within anabsolute error 5x104 using Composite Trape-

zoid (ct) and Composite Simpsons (cs).

Composite Trapezoid

We know the error term for composite Trapezoidis:

E = (b a)12

h2f()

Knowing f(x) = ex, f(x) = ex. To guaranteethat we will be within the error we will choose such that f() is the maximum on the interval[0,1]. This occurs at the right end x = 1. Sof() = e. So the error term becomes

E = 112

h2e.

If we desire the error term to be at most 5x104

| 112

h2e| 5 104

Knowing that h = 1n

where n is the number ofsubintervals and solving for the smallest numberthat will give the desired error:

| 112n2

e| 5 104

| 1n2

| 12e

5 104

n =

e

(12)5 104

n = 21.2849 = 22Using Composite Trapezoid to integrate f(x)

between 0 and 1 with subintervals we get a valueof 1.718578 (using a C program written for a as-signment 4) for an absolute error of

Ect =

f(x)dx Ict(f) = 2.96172 104

So we have met our goal and we are under thedesired error. However, upon further experimen-tation we can see that we can lower the numberof subintervals.


13/17


Table 2: The error of Composite Trapezoid on f(x) with differing number of subintervals.

m error

22 2.96172 104

21 3.246820 10420 3.579605 10419 2.958372 10418 4.419222 10417 4.954391 10416 5593001 104

As we can see from Table ??, we can actu-ally go as low as n = 17 and still be within our

desired error. It should be noted that this typeof analysis would not be possible without beingable to evaluate the derivative of f(x) to gatherconcavity information.

Composite Simpsons

Following a similar method for determining theleast number of subintervals, we begin with theerror term for Composite Simpsons:

Ecs =

(b

a)

180h

24f(4)

(

)

f(4)(x) = ex

f(4)() = e

Ecs = h4

180 24 eMaking the substitution of h = 1

n

Ecs = 12880n4

e

We want n such that Ecs 5 104

e2880n4

5 104therefore n must be at least

n =

4

e

2880 5 104

= 1.034418 = 2.

Using Composite Simpsons to integrate f(x)between 0 and 1 with 2 subintervals we get avalue of 1.71886115 for an absolute error of

Ecs =

f(x)dx Ics(f) = 5.79323 104

which is actually larger than the desired er-ror. In order to get an error below 5 104 wehave to use 4 subintervals in Composite Simp-sons. Which yields an error of 3.701346 104.

Steffensens root finding example

We wish to numerically find the first positive root of f(x)

f(x) = ex sin(x)

We can see visually that the first positive root for this function exists and as have Lipschitz conti-nuity near the root. We will use two methods to calculate the root, the standard Newtons method


14/17


which requires derivative evaluations and Steffensens method that only function evaluations toestimate a derivative (average slope). We will see that both methods converge quadratically.

-1 1 2 3 4

1

2

3

4

5

6

Figure 9: f(x) = ex sin(x) on the interval -1.5 x 4.5.

Steffensens Method

Steffensens Method is a method for solving nonlinear equations which is implemented by

x(k+1) = x(k) f(x(k)

)(x(k))

(x(k)) =f(x(k) + f(x(k))) f(x(k))

f(x(k))

Using Steffensens Method we wish to solve for the first positive root of

f(x) = ex sin(x)

Using Octave, the following functions were used to calculated this root.A few words on the Octave functions. Some of the constants chosen in the functions (particularly

the upper bound on the for loop) were chosen after seeing how long it took for the method to

converge so that the program didnt error during runtime for a division by zero.

Steffens = 0f o r i = 1 : 4Steffens (i+1) = Steffens (i ) step ( Steffens (i ) ) ;en dSteffens


15/17


SError = 0 . 5 8 8 5 3 2 7 4 3 9 8 1 8 6 1 Steffens ;y = [ 0 , 1 , 2 , 3 , 4 ] ;s e m i l o g y (y , abs ( SError ) )x l a b e l ( ' I t e r a t i o n ' )

y l a b e l ('

E r r o r'

)

f u n c t i o n l = step (x )y = ff (x ) ;phi = ff (x + y ) y ;phi = phi/y ;l = y / phi ;

Using an initial guess of x = 0 Steffensens method converged in 4 iterations to yield a solution

x = 0.588532743981867.

The intermediate values calculated and their corresponding errors are in Table 3. The acceptedvalue, shown in the Steffensens Octave function, was the final value given by Newtons Method ona machine with machine

machine = 2.22044604925031 1016.

Table 3: Successive estimates for x given by Steffensens Method and their errors.

Iteration Value Error0 0.00

1 0.678614100575150 9.00813565932895 102

2 0.589658358068303 1.12561408644185 1033 0.588532939946140 1.95964279137151 1074 0.588532743981867 6.10622663543836 1015

Newtons Method

Newtons method also solves nonlinear equations, Newtons method is given by:

x(k+1) = x(k)

f(x(k))

f

(x(k)

)Newtons method was implemented in Octave code by thefunction given below.

Newtons ( 1 ) = 0 ;f o r i = 1 : 5Newtons (i+1) = Newtons (i ) ff ( Newtons (i ) ) /dff ( Newtons (i ) ) ;en d


16/17


Newtons

NError = 0 . 5 8 8 5 3 2 7 4 3 9 8 1 8 6 1 Newtonsy = [ 0 , 1 , 2 , 3 , 4 , 5 ] ;s e m i l o g y (y , abs ( NError ) )

x l a b e l ('

I t e r a t i o n'

)y l a b e l ( ' E r r o r ' )

Newtons Method converged in 5 iterations with an initial guess of x = 0, all intermediate valuesand their corresponding error is given in Table 4.

Table 4: Successive Estimates for x given by Newtons Method.

Iteration Value Error 0 0.00 1 0.500000000000000 8.85327439818610 1022 0.585643816966433 2.88892701542842

103

3 0.588529412626355 3.33135550611985 1064 0.588532743977419 4.44211334382771 10125 0.588532743981861 1.11022302462516 1016

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 510

16

1014

1012

1010

108

106

104

102

100

Iteration

Error

Figure 10: The convergence of Newtons Method

in 5 iterations.

0 0.5 1 1.5 2 2.5 3 3.5 410

15

1010

105

100

Iteration

Error

Figure 11: The convergence of Steffensens

Method in 4 iterations.

Summary

Both Steffensens and Newtons methods converged (within 1015) and both did so fairly quickly,using at most 5 iterations. And by looking at Tables 3 and 4 as well as Figures ?? and 8 we can see


17/17


that the errors of both dropped off quadratically. Steffensens method quite surprising to convergequadratically without directly calculating the derivative.

random thoughts on numerical analysis

Documents