Chapter 19 Numerical DifferentiationChapter 19 Numerical Differentiation

Estimate the derivatives (slope, curvature, etc.) of a function by using the function values at only a set of discrete points

Ordinary differential equation (ODE)Partial differential equation (PDE)Represent the function by Taylor polynomials

or Lagrange interpolationEvaluate the derivatives of the interpolation

polynomial at selected (unevenly distributed) nodal points

0

( ) ( )Differentiation lim

( )

( ) ( ) Difference

( )

A smaller step (or h) results in a smaller error

x

y f(x)

dy f x x f x

dx x x x

y f x x f x dy

x x x x dx

x

xi-3 xi-2 xi-1 xi xi+1 xi+2 xi+3

hx

Evenly distributed points along the x-axis

x1 x2 x3

Unevenly distributed points along the x-axis

Distance between two neighboring points is the same, i.e. h.

Forward difference

Backward difference

Centered difference

Numerical DifferentiationNumerical Differentiation

Forward differenceForward difference

xi1 xi xi+1

xh

True derivative

Approximation

Backward differenceBackward difference

xi1 xi xi+1

xh

True derivative

Approxi

mat

ion

Centered differenceCentered difference

xi1 xi xi+1

x2h

True derivative

Approximatio

n

First DerivativesFirst Derivatives

Forward difference

Backward difference

Central difference

)x(f

i-2 i-1 i i+1 i+2

1i1i

1i1i

1i1i

1i1i

1ii

1ii

1ii

1ii

i1i

i1i

i1i

i1i

xx

yy

xx

)x(f)x(f)x(f

xx

yy

xx

)x(f)x(f)x(f

xx

yy

xx

)x(f)x(f)x(f

x

y

Truncation ErrorsTruncation ErrorsUniform grid spacing

)x(f!3

h)x(f

!2

h)x(fh)x(f)hx(f)x(f

)x(f!3

h)x(f

!2

h)x(fh)x(f)hx(f)x(f

i

3

i

2

iii1i

i

3

i

2

iii1i

))()()(

)(:

)()()(

)(:

)()()(

)(:

23

21i1i

i

21ii

i

1i1i

i

O(h f6

h

h2

xfxfxfcentral

O(h) f2

h

h

xfxfxfbackward

O(h) f2

h

h

xfxfxfforward

Example: First DerivativesExample: First Derivatives Use forward and backward difference approximations

to estimate the first derivative of

at x = 0.5 with h = 0.5 and 0.25 (exact sol. = -0.9125)Forward Difference

Backward Difference

2.1x25.0x5.0x15.0x1.0)x(f 234

%.,..

..

..

).().().(,.

%.,..

..

.

).()().(,.

526 1551250

9250636328130

50750

50f750f50f 250h

958 45150

925020

501

50f1f50f 50h

t

t

%.,..

..

..

).().().(,.

%.,..

..

.

)().().(,.

721 7140250

1035156319250

25050

250f50f50f 250h

739 55050

219250

050

0f50f50f 50h

t

t

Example: First DerivativeExample: First Derivative Use central difference approximation to estimate the

first derivative of

at x = 0.5 with h = 0.5 and 0.25 (exact sol. = -0.9125)

Central Difference

2.1x25.0x5.0x15.0x1.0)x(f 234

%4.2,934.05.0

10351563.163632813.0

25.075.0

)25.0(f)75.0(f)5.0(f,25.0h

%6.9,0.11

2.12.0

01

)0(f)1(f)5.0(f,5.0h

t

t

Second-DerivativesSecond-DerivativesTaylor-series expansionUniform grid spacing

Second-order accurate O(h2)

)x(f!4

h)x(f

!2

h)x(f2)x(f)x(f

)x(f!4

h)x(f

!3

h)x(f

!2

h)x(fh)x(f)x(f

)x(f!4

h)x(f

!3

h)x(f

!2

h)x(fh)x(f)x(f

i

4

i

2

i1i1i

i

4

i

3

i

2

ii1i

i

4

i

3

i

2

ii1i

)(!

)()()()( f

4

h

h

xfxf2xfxf

2

21ii1i

i

Centered Finite-Divided DifferencesCentered Finite-Divided Differences

Forward Finite-divided differencesForward Finite-divided differences

Backward finite-divided differencesBackward finite-divided differences

First DerivativesFirst Derivatives

3 -point Forward difference

3 -point Backward difference

)x(f

i-2 i-1 i i+1 i+2

2ii

2i1ii

i2i

i1i2i

xx

xf3xf4xf3xf

xx

xf3xf4xfxf

)()()()(

)()()()(

Parabolic curve

Example: First DerivativesExample: First Derivatives Use forward and backward difference approximations

of O(h2) to estimate the first derivative of

at x = 0.5 with h = 0.25 (exact sol. = -0.9125)Forward Difference

Backward Difference

2.1x25.0x5.0x15.0x1.0)x(f 234

%82.5,859375.05.0

)925.0(3)6363281.0(42.0

)25.0(2

)5.0(f3)75.0(f4)1(f)5.0(f

t

%77.3,878125.05.0

2.1)035156.1(4)925.0(3

)25.0(2

)0(f)25.0(f4)5.0(f3)5.0(f

t

Higher DerivativesHigher DerivativesAll second-order accurate O(h2)

More nodal points are needed for higher derivativesHigher order formula may be derived

42i1ii1i2i

i

32i1i1i2i

i

21ii1i

i

1i1ii

h

)x(f)x(f4)x(f6)x(f4)x(f)x(f

h

)x(f)x(f2)x(f2)x(f)x(f

h

)x(f)x(f2)x(f)x(f

h2

)x(f)x(f)x(f

19.3 Richardson Extrapolation2 4 2

1 2 1 4 1 1 1 1( ) ........ ( ) ( ) (1)D D h a h a h D h E h

D is the true value but unknown and D(h1) is an approximation based on the step size h1. Reducing the step size to half, h2 =h1/2, we obtained another approximation D(h2).

By properly combining the two approximations, D(h1) & D(h2), the error is reduced to O(h4).

2 4 22 2 2 4 2 2 2 2

241 1 2 1

3 122 2

( ) ( ) ( ) (2)

( ) 4 ( ) ( )4 Hence 4*(2) - (1), + ( )

3( )

D D h a h a h D h E h

E h D h D hD E h

E h

Example of using Richardson ExtrapolationCentral Difference Scheme

By combining the two approximations, D(h/2) & D(h), the errorof f’(h) is reduced to O(h4).

24

24

1'( ) [ ( ) ( )] '''( ) ( ) (1)

2 6

1'( ) / 2 [ ( / 2) ( / 2)] '''( ) ( ) (2)

24

hf x D h f x h f x h f x O h

h

hf x D h f x h f x h f x O h

h

44 ( / 2) ( )4*(2) - (1), ' ( )

3

D h D hf O h

Ex19.2: Richardson ExtrapolationEx19.2: Richardson Extrapolation Use central difference approximation to estimate the

first derivative of

at x = 0.5 with h = 0.5 and 0.25 (exact sol. = -0.9125)

2.1x25.0x5.0x15.0x1.0)x(f 234

1 t

2 t

2 1t

(1) (0) ( 0.5) 1.0, 9.6%

1 0

(0.75) (0.25)( 0.25) 0.934, 2.4%

0.75 0.25

40.9125, 0%

3

f fD h

f fD h

D DD

General Three-Point FormulaGeneral Three-Point FormulaLagrange interpolation polynomial for

unequally spaced data

)())((

))((

)())((

))((

)())((

))((

)()()()()()()(

1ii1i1i1i

i1i

i1ii1ii

1i1i

1i1i1ii1i

1ii

1i1iii1i1i

xfxxxx

xxxx

xfxxxx

xxxx

xfxxxx

xxxx

xfxLxfxLxfxLxf

Lagrange InterpolationLagrange Interpolation 1st-order Lagrange polynomial

Second-order Lagrange polynomial

)()()()()()( 101

00

10

111001 xf

xx

xxxf

xx

xxxfxLxfLxf

)x(f)xx)(xx(

)xx)(xx(

)x(f)xx)(xx(

)xx)(xx(

)x(f)xx)(xx(

)xx)(xx()x(f

21202

10

12101

20

02010

212

Lagrange InterpolationLagrange Interpolation Third-order Lagrange polynomial

)x(f)xx)(xx)(xx(

)xx)(xx)(xx(

)x(f)xx)(xx)(xx(

)xx)(xx)(xx(

)x(f)xx)(xx)(xx(

)xx)(xx)(xx(

)x(f)xx)(xx)(xx(

)xx)(xx)(xx()x(f

3231303

210

2321202

310

1312101

320

0302010

3213

Lagrange InterpolationLagrange Interpolation

L1(x)f(x1)

L2(x)f(x2)L0(x)f(x0)

x0 x1 x2

General Three-Point FormulaGeneral Three-Point Formula Lagrange interpolation polynomial for

unequally spaced data

First derivative

))((

))(()(

))((

))(()(

))((

))(()(

)()()()()()()(

i1i1i1i

i1i1i

1ii1ii

1i1ii

1i1ii1i

1ii1i

1i1iii1i1i

xxxx

xxxxxf

xxxx

xxxxxf

xxxx

xxxxxf

xfxLxfxLxfxLxf

))(()(

))(()(

))(()()(

i1i1i1i

i1i1i

1ii1ii

1i1ii

1i1ii1i

1ii1i

xxxx

xxx2xf

xxxx

xxx2xf

xxxx

xxx2xfxf

Second DerivativeSecond DerivativeFirst Derivative for unequally spaced data

Second Derivative for unequally spaced data

))(()(

))(()(

))(()()(

i1i1i1i

i1i1i

1ii1ii

1i1ii

1i1ii1i

1ii1i

xxxx

xxx2xf

xxxx

xxx2xf

xxxx

xxx2xfxf

))(()(

))(()(

))(()()(

i1i1i1i1i

1ii1iii

1i1ii1i1i

xxxx

2xf

xxxx

2xf

xxxx

2xfxf

Differentiation of Noisy DataDifferentiation of Noisy Data

MATLAB’s MethodsMATLAB’s MethodsDerivatives are sensitive to the noiseUse least square fit before taking derivatives

p = polyfit(x, y, n) - coefficients of Pn(x)

polyfit(p, x) - evaluation of Pn(x)

polyder(p) - differentiation

difference backward 1)x(i atdy/dx difference forward x(i) atdy/dx

iff(x)diff(y)./ddy

)1n(x)n(x ,),2(x)3(x ),1(x)2(x)x(diff)n(x,),2(x),1(xx