lecture 05

5/24/2018 Lecture 05

1/44

ENGG 407

Numerical Methods in Engineering

P14L01

Lecture #5

Dr. Sameh Nassar

5/24/2018 Lecture 05

2/44

ENGG 407 L02 Winter 2012 Lecture #5 Dr. Sameh Nassar

Sameh Nassar

ENGG 407 L01 Spring 2014 Lecture #5 Dr. Sameh Nassar

Recall: ENGG 407 P14 Topics

1. Introduction and Mathematical Background (Ch. #1, #2)

2. Roots of Nonlinear Equations (Ch. #3)

3. Linear Equations and Systems (Ch. #4, #5)

4. Interpolation, Least-squares Estimation & Curve Fitting (Ch. #6)

5. Numerical Differentiation (Ch. #8)

6. Numerical Integration (Ch. #9)

7. Ordinary Differential Equations: Initial Value Problems (Ch. #10)

8. Ordinary Differential Equations: Boundary Value Problems (Ch. #11)

5/24/2018 Lecture 05

3/44


Sameh Nassar


Recall from Lecture #1: Taylor Series

f(xi+1)

f(xi)

h

The term Rn is called theRemainder

where is a value of x that

lies between xi and xi+1

For a function f(x)that depends on only one independent

variablex, the value at pointxi+1can be approximated bythe following Taylor Series:

5/24/2018 Lecture 05

4/44


Sameh Nassar



k

0k

i

)k(

1i h!k

)x(f)x(f

=

+ =

nni

)n(2i

''

i

'

i1i Rh!n

)x(f

....h!2

)x(f

h)x(f)x(f)x(f +++++=

+

)x(f)x(f i1i + zero order approximation

h)x(f)x(f)x(f i'

i1i ++ 1st order approximation

2i

''

i'

i1i h!2)x(fh)x(f)x(f)x(f +++ 2nd order approximation

M

ni

)n(

2i

''

i'

i1i h!n)x(f....h

!2)x(fh)x(f)x(f)x(f +++++ nth order approx.

5/24/2018 Lecture 05

5/44


Sameh Nassar



Sometimes, Taylor Series can be written using the following

format:

i.e. replacing the symbols xi+1by xand xi(point of expansion)by a.

For a function that is a function of two or more (independent)variables, Taylor Series can be applied in the same manner asfor one variable, however, the differentiation will involvepartial derivatives.

5/24/2018 Lecture 05

6/44


Sameh Nassar


Why Numerical Differentiation?

Thus, in the first case above (i.e. known function), tabulated datapoints are obtained first from the function and then numericaldifferentiation is performed.

In the second case above (i.e. given data points), numericaldifferentiation can be done using one of two approaches:

(1) Finite difference approximation.

(2) Function approximation (by fitting a curve first to the data using

any method from Lecture #4and then differentiate it analytically ).

Numerical differentiation is employed in the following cases:

The function is known but is difficult (or not possible) to bedifferentiated analytically.

A set of discrete points is provided and a differentiation is required.

5/24/2018 Lecture 05

7/44


Sameh Nassar


Numerical Differentiation

Finite DifferenceApproximation

FunctionApproximation

Discrete

Points

Fitted

Curve

5/24/2018 Lecture 05

8/44


Sameh Nassar



)i(xf'

NumericalDifferentiation

(finite differenceapproximation)

Differentiation

5/24/2018 Lecture 05

9/44


Sameh Nassar


Finite Difference Approximation

General Finite difference

approximation formulaefor derivatives areobtained using Taylorseries.

However, for 1st orderapproximations (i.e. firstderivatives), formulae can

be obtained directly usingsimple mathematics (orgeometry!)

5/24/2018 Lecture 05

10/44


Sameh Nassar



nni1i

i

)n(

2i1i

i

''

i1ii'

i1i R)xx(!n

)x(f....)xx(!2)x(f)xx)(x(f)x(f)x(f +++++=

++++

nni

)n(

2i

''

i

'

i1i Rh

!n

)x(f....h

!2

)x(fh)x(f)x(f)x(f +++++=

+

1

i

'

i1i Rh)x(f)x(f)x(f ++=

+

)h(h

f

h

R

h

)x(f)x(f)x(f i

1

i1ii

'O+

=

=

+

Forward Finite Divided Difference (Using Taylor Series):

)x(f 1i+

)x(f i

if 1st forward difference

h

fi 1st forward finitedivided difference

)x('f i 1st order forward finite difference

Truncation error

Assuming equidistant points

5/24/2018 Lecture 05

11/44


Sameh Nassar



nn

i1ii

)n(2

i1ii

''

i1ii

'

i1i R)xx(

!n)x(f....)xx(

!2)x(f)xx)(x(f)x(f)x(f +++++=

nni

)n(

2i

''

i

'

i1i Rh

!n

)x(f....h

!2

)x(fh)x(f)x(f)x(f ++++=

1

i

'

i1i Rh)x(f)x(f)x(f +=

)h(h

f

h

R

h

)x(f)x(f)x(f i

1

1iii

'O+

=

=

Backward Finite Divided Difference (Using Taylor Series):

)x(f1i

)x(fi

if 1st backward difference

h

fi 1st backward finitedivided difference

)x('f i 1st order backward finite difference

Truncation error


5/24/2018 Lecture 05

12/44


Sameh Nassar



.......h!3

)x(fh

!2

)x(fh)x(f)x(f)x(f 3i

)3(

2i

''

i

'

i1i ++=

.......h!3

)x(fh!2

)x(fh)x(f)x(f)x(f 3i

)3(2i''

i

'

i1i ++++=

+

...h!3

)x(f2h)x(f2)x(f)x(f

3i

)3(

i

'

1i1i ++= +

)h(h2

)x(f)x(f)x(f

h6

)x(f

h2

)x(f)x(f)x(f

21i1i

i

'

2i)3(

1i1ii

'

O+

=

=

+

+

Centered Finite Divided Difference (Using Taylor Series):

_ subtract

Truncation error



)x(f1i

)x(f 1i+

ix

5/24/2018 Lecture 05

13/44


Sameh Nassar


Simple Finite Differences for the First Derivative

forward finite

divided difference

centered finitedivided difference

backward finitedivided difference

h

)x(f)x(f)x(f i1ii

'

+

h

)x(f)x(f)x(f 1iii

'

h2

)x(f)x(f)x(f 1i1ii

' +

ii

iii

xx

xfxfxf

+

+

1

1' )()()(OR

OR

OR

1

1' )()()(

ii

iii

xx

xfxfxf

11

11' )()()(+

+

ii

iii

xx

xfxfxf

Note that h is used when the points are equally spaced.

For each of the given formulae above, only 2 points are used toestimate the 1st derivative, and hence they are called two-pointfinite difference formulae.

5/24/2018 Lecture 05

14/44


Sameh Nassar


Example #1

Use the forward, backward and centered differenceapproximation to estimate the first derivative of:

f(x) = sin(x)at x=0.8using a step size h=0.6.

Then, estimate the true relative error for each case.

Hint: all x values are in radians

5/24/2018 Lecture 05

15/44


Sameh Nassar


Example #1 (sol. 1/3)

4.1x,2.0x1i1i == +

18148960.446822736.0

)8.0sin()4.1sin(

h

)x(f)x(f)x(f i1ii

'

forw =

=

+

35074360.864477936.0

)2.0sin()8.0sin(h

)x(f)x(f)x(f 1iii'

back ==

f(x)=sin(x)

cos(0.8) = 0.696706709347165 (assumed true value)

6.0h,8.0xi

==

f(x)=cos(x)

26611660.65565033

6.0*2

)2.0sin()4.1sin(

h2

)x(f)x(f)x(f 1i1ii

'

cent =

=

+

5/24/2018 Lecture 05

17/44


Sameh Nassar



X (radians)

f(x)

f(x)=sin(x)

)8.0(f'

forw

f(0.8)=cos(0.8)

)8.0(f'

back

)8.0(f'

cent

5/24/2018 Lecture 05

18/44


Sameh Nassar


Finite Differences for the First Derivative

If higher accuracy is required for the estimated derivative, more than 2points should be used and also Taylor series should not be truncatedafter only the linear term. For example:

High-Accuracy Divided-Differences Formulae

It can be shown that:

forward Taylor series expansion

Three-point forwarddifference for 1st derivative

5/24/2018 Lecture 05

19/44


Sameh Nassar


Finite Differences for the First Derivative

Summary (for Equidistant Points)

5/24/2018 Lecture 05

20/44


Sameh Nassar


Example #2

Use the centered difference approximation to estimate the firstderivative of:

f(x) = sin(x)at x=0.8using a step size h=0.6.

Apply both the two-pointand the four-point formulae.

Then, estimate the true relative error for each case.

Hint: all x values are in radians

5/24/2018 Lecture 05

21/44


Sameh Nassar



f(x)=sin(x)

cos(0.8) = 0.696706709347165 (assumed true value)

f(x)=cos(x)

h2

)x(f)x(f)x(f 1i1ii

'

cent

+

h12

)x(f)x(f8)x(f8)x(f)x(f 2i1i1i2ii'

cent

++ ++Four-Point formula

Two-Point formula

0.2x,4.1x

4.0x,2.0x

2i1i

2i1i

==

==

++

6.0h,8.0x i ==

Centered divided difference formulas:

5/24/2018 Lecture 05

22/44


Sameh Nassar



h2

)x(f)x(f)x(f 1i1ii

'

cent+

h12

)x(f)x(f8)x(f8)x(f)x(f 2i1i1i2ii

'

cent++

++

26611660.655650336.0*2

)2.0sin()4.1sin()x(f i

'

cent =

%5.9%10093471650.69670670

26611660.65565033-93471650.69670670| ==

centt

|

33906750.693823256.0*12

)4.0sin()2.0sin(*8)4.1sin(*8)0.2sin()x(f i

'

cent =

++

%0.413%100

93471650.69670670

33906750.69382325-93471650.69670670| ==

centt|

Two-Point formula

Four-Point formula

5/24/2018 Lecture 05

23/44


Sameh Nassar


Finite Differences for the Second Derivative


5/24/2018 Lecture 05

24/44


Sameh Nassar


Finite Differences for the Third Derivative


5/24/2018 Lecture 05

25/44


Sameh Nassar


Finite Differences for the Fourth Derivative


5/24/2018 Lecture 05

26/44


Sameh Nassar


Finite Difference Numerical Differentiation Errors

Recall that his typically < 1.0.

For small hvalues, the values of

f(x-h), f(x), f(x+h)will be very closeto each other, and hence the use offinite differenceformulae mayresult in loss of precision due tosubtractive cancellation.

Recall from Lecture #1 thatsmaller hvalues will reducetruncation errors but may result in

large round-off errors.

Recall from Lecture #1:

Total Numerical Error = Truncation Errors + Round-Off Errors

5/24/2018 Lecture 05

27/44


Sameh Nassar


Example #3 (In Class Example!)

Use the two-pointcentered difference approximation toestimate the first derivative of:

f(x) = sin(x)at x=0.8using the following step sizes:

h = 0.6, 0.0375, 1.875x10-4and 2.5x10-7.

Then, estimate the true relative error for each case andcomment on the results.

Hints:

-All xvalues are in radians.

- Use 8 decimal points in your computations.

5/24/2018 Lecture 05

28/44


Sameh Nassar



(2) Function Approximation Techniques:

As mentioned earlier, this is done by fittinga curve first to the data using any methodfrom Lecture #4and then differentiateit (i.e. the fitted function) analytically.

This approach has the advantages of beingused with non-equidistant data and thatthe derivative can be estimated at any point between the given datapoints or the points themselves.

Usually polynomials are used for the curve fitting since they areeasy to differentiate.

Also, Lagrange polynomials are often used due to their simplicity.

Discrete

Points

FittedCurve

ENGG 407 L02 Wi 2012 L #5 D S h N

ENGG 407 L01 S i 2014 L #5 D S h N

5/24/2018 Lecture 05

29/44


Sameh Nassar



(2) Function Approximation Techniques:

Recall Lagrange Polynomials for n+1 data points from Lecture #4:

where:

Caseof 1st

order

Case of2nd

order

ENGG 407 L02 Wi t 2012 L t #5 D S h N

ENGG 407 L01 S i 2014 L t #5 D S h N

5/24/2018 Lecture 05

30/44


Sameh Nassar


Numerical Differentiation(2) Function Approximation Techniques:

Thus, the derivative of any order k, can be obtained at any point x as:

0

Case of n = 2 (i.e. 3 data points):

0

Home Work!:

For the above derivative, prove that in case of equidistant points it willsimplify to the three-point forward divided difference formula used

earlier.

ENGG 407 L02 Winter 2012 Lecture #5 Dr Sameh Nassar

ENGG 407 L01 Spring 2014 Lecture #5 Dr Sameh Nassar

5/24/2018 Lecture 05

31/44


Sameh Nassar


Richardsons Extrapolation Error-correctiontechniques are used to improve the results of

numerical differentiation on the basis of the derivative estimatesthemselves.

The methods using two estimates of a derivative to compute a third,more accurate approximation, are known as Richardsons

Extrapolation techniques. Hence, Richardsons Extrapolationis a general algorithm that

minimizes the error in the computation of a derivative, i.e. f(x)bycombining two less accurate approximations of the derivative.

This is done as steps, and based on the number of such steps, thetruncation error will decrease from O(h2)to O(h4), O(h6), O(h8),.

Richardsons Extrapolation will be used again with some numericalintegrationapproaches (stay tuned for discussion in Lecture #6!).



5/24/2018 Lecture 05

32/44


Sameh Nassar


Richardsons Extrapolation

Using finite difference for numerical differentiation to obtain the first

derivative f(x) with a step size h:

(h)(h) EDD(x)f' +== )( 1 ii xxh = +

Exact valueof the derivative

Approximate valueof the derivative

TruncationError

with:

)(h)(h)(h)(h 2211 EDEDD +=+=

1

ii1n

xxh

)( 1 =

+

with:2

ii2n

xxh

)( 1 =

+

)1212 hh(i.e.,nn and:



5/24/2018 Lecture 05

33/44


Sameh Nassar


Richardsons ExtrapolationRecall:

)](h)(h[h(h

)(hxf 1221

2 DDDD

+=1)/

1)('

2

2

2

2

1

2

1

h

h

)(h

)(h=

E

E

i.e. it is O(h2)estimation error

With O(h4)

estimation

error

Using the two-point centered divided difference:

)h(h2

)x(f)x(f)x(f

h6

)x(f

h2

)x(f)x(f)x(f

21i1i

i

'

2i)3(

1i1ii

'

O+

=

=

+

+

Truncation error

Then, if using two different step sizes (h1 andh2 ,the following relationship can be obtained:

Thus:

(h)D



5/24/2018 Lecture 05

34/44


Sameh Nassar



Special Case:

)(h)(h 12 DDD 3

1

3

4

2hh 12 /=

li.e. DDD

3

1

3

4a

More accurateestimate

Less accurateestimate

ImprovedDerivative

]4[3

1

or )(h)(h 12 DDD



5/24/2018 Lecture 05

35/44


Sameh Nassar



n h D0 D1 D2 D3

0 h1 D(h1) D(h2, h1) D(h3, h2, h1) D(h4, h3, h2, h1)

1 h2 D(h2 ) D(h3, h2) D(h4, h3, h2)

2 h3 D(h3 ) D(h4, h3)

3 h4 D(h4)

l001DDD3

1

3

4a

l112DDD

15

1

15

16

al223

DDD63

1

63

64

a

O(h4) O(h6)O(h2) O(h8)Error:

D0 is the derivative obtained using the two-point centered divided difference

h2 = h1/2, h3 = h1/4, h4 = h1/8, .



5/24/2018 Lecture 05

36/44


Sameh Nassar



Thus, the general formulae for Richardsons Extrapolation (for the

first derivative) can be written as:

For efficient application of Richardsons Extrapolation, the algorithm

is applied recursively using a specified number of Napproximationsfor successive values of Dms obtained using h/2

n.

In this case, a starting value for h is selected (n =0) and N+1 valuesare computed (0 n N), i.e.:

Then the recursive computation is done as:



5/24/2018 Lecture 05

37/44

Sameh Nassar

p g


Like any recursive (iterative method), the approximations can stop by

either specifying an or choosing a specific N. As discussed earlier with finite difference formulae (when very small his

selected), Richardsons Extrapolation may lead to round-off errors aswell, however, it can provide very high accuracy without using extremely

small values of h.

Error:



5/24/2018 Lecture 05

38/44

Sameh Nassar

p g

Example #4 (In Class Example)

Estimate the first derivative of f(x) = sin(x) at x=0.8using

Richardsons Extrapolation with an initial step size of 0.3.

Apply three approximation steps (i.e. N = 3)

Then, estimate the true relative error after each approximationand comment on the results.

Hints:

-All xvalues are in radians.- Use 8 decimal points in your computations.



5/24/2018 Lecture 05

39/44

Sameh Nassar

Numerical Partial Differentiation

Numerical partial differentiation is used to estimate the derivatives offunctions of more than one variable, i.e. for z = f(x, y)for example.

In this case, zis a function of 2 independentvariables (xand y) and the zvalue is given atn m points (xi, yj).

The derivatives are obtained for one variableat a time (i.e. considering all other variablesas constants).

Thus, the numerical differentiation formulae offunctions of one variable are applied to multi-

variable functions but for each variable at atime.

In the sequel, some of the finite difference formulae (used earlier) aregiven for data with constant spacing hxand hyin the x direction and y

direction, respectively.



5/24/2018 Lecture 05

40/44

Sameh Nassar


Two-point central divided difference for 1st derivatives

Three-point central divided difference for 2nd

derivatives:



5/24/2018 Lecture 05

41/44

Sameh Nassar


Four-point central divided difference for mixed 2nd derivatives:



5/24/2018 Lecture 05

42/44

Sameh Nassar

Example #5

The following data for the velocity component in the x-direction, u,are obtained as a function of the two coordinates x and y.

Use the four-point central divided difference formula for mixed 2nd

derivatives to evaluate such derivative at the point (2, 3).



5/24/2018 Lecture 05

43/44

Sameh Nassar

Example #5

hx = 1, hy =1

f(xi+1, yj+1) = u(3, 4) = 9 f(xi-1, yj+1) = u(1, 4) = 7

f(xi+1, yj-1) = u(3, 2) =22

f(xi-1, yj-1) = u(1, 2) = 8

xi = 2, yj =3



5/24/2018 Lecture 05

44/44

Sameh Nassar

Sections:

8.18.2

8.3

8.48.5

8.6

8.78.8

8.9

8.10

Textbook Readings

lecture 05

Documents