numerical analysis with python · 2021. 1. 20. · truncation errors • ‘truncation errors are...

Numerical Analysis with Python

Truncation errors and Tylor’s Series

LECTURE 03

• Truncation Errors

• Taylor’s Series

• Taylor’s Series remainder

• Maclaurin’s series

• Function approximation using Taylors Series

• Numerical differentiation

Truncation Errors

• ‘Truncation errors are those that result from using anapproximation in place of an exact mathematical procedure’.

• Example: Using finite-divided-difference equation to approximatethe derivative of function. E.g. velocity of a falling object undergravitational influence.

𝑑𝑣

𝑑𝑡≅Δ𝑣

Δ𝑡=𝑣 𝑡𝑖+1 − 𝑣 𝑡𝑖

𝑡𝑖+1 − 𝑡𝑖(1.1)

Truncation Errors

• A truncation error was introduced into the numerical solutionbecause the difference equation only approximates the true value ofthe derivative.

• One of the most widely used mathematical formulation in numericalmethods to express functions in an approximate fashion is Taylorseries.

• Taylor series provides a means to predict a function value at onepoint in terms of the function value and its derivatives at anotherpoint.

Tylor’s Series

Theorem:

“If the function f and its first (𝑛 + 1) derivatives are continuous on aninterval containing a and x, then the value of the function at x is givenby:”

Where 𝑅𝑛 is the remainder term is included to account for all termsfrom n+1 to infinity:

𝑓 𝑥 = 𝑓 𝑎 + 𝑓′ 𝑎 𝑥 − 𝑎 +𝑓′′ 𝑎

2!(𝑥 − 𝑎)2+

𝑓(3) 𝑎

3!(𝑥 − 𝑎)3. . . . +

𝑓(𝑛) 𝑎

𝑛!(𝑥 − 𝑎)𝑛+𝑅𝑛

Tylor’s Series Remainder

Remainder:

𝑅𝑛 is the remainder term used in Taylors series to account forTruncation Errors in all terms from (𝑛 + 1) to infinity

• Where 𝜉 is a point between a and x

𝑅𝑛 =𝑓(𝑛+1) 𝜉

(𝑛 + 1)!(𝑥 − 𝑎)𝑛+1

Maclaurin: special case of Taylor

Maclaurin series:

It is a special case of Taylor series with an exponential functionstarting at origin i.e. a=0

Proof:

• The Taylor series reduces to:

• Simplifying and truncating the remainder 𝑅𝑛

𝑓𝑜𝑟 𝑒𝑥𝑝𝑜𝑛𝑒𝑛𝑡𝑖𝑎𝑙, 𝑙𝑒𝑡 𝑓 𝑥 = 𝑒𝑥 𝑎𝑛𝑑 𝑎 = 0

𝑓 𝑥 = 𝑓 0 + 𝑓′ 0 𝑥 − 0 +𝑓′′ 0

2!(𝑥 − 0)2+

𝑓(3) 0

3!(𝑥 − 0)3. . . . +

𝑓(𝑛) 0

𝑛!(𝑥 − 0)𝑛+𝑅𝑛

𝑓 𝑥 = 𝑓 0 + 𝑓′ 0 𝑥 +𝑓′′ 0

2!(𝑥)2+

𝑓(3) 0

3!(𝑥)3. . . . +

𝑓(𝑛) 0

𝑛!(𝑥)𝑛


Proof:

• The Taylor series reduces to:

• Hence, 𝑓 𝑥 = 0 = 𝑒0 = 1 , 𝑓′ 0 = 𝑒0 = 1 , 𝑓′′ 0 = 𝑒0 = 1

• Therefore on substitution above:

𝑓𝑜𝑟 𝑒𝑥𝑝𝑜𝑛𝑒𝑛𝑡𝑖𝑎𝑙, 𝑙𝑒𝑡 𝑓 𝑥 = 𝑒𝑥

𝑓 𝑥 = 𝑓 0 + 𝑓′ 0 𝑥 +𝑓′′ 0

2!(𝑥)2+

𝑓(3) 0

3!(𝑥)3. . . . +

𝑓(𝑛) 0

𝑛!(𝑥)𝑛

𝑑𝑓(𝑥)

𝑑𝑥= 𝑓′ 𝑥 = 𝑒𝑥 𝑓′′ 𝑥 = 𝑒𝑥 𝑓′′′ 𝑥 = 𝑒𝑥 𝑎𝑛𝑑 𝑠𝑜 𝑜𝑛 …

𝑒𝑥 = 1 + 1 ∗ 𝑥 +1

2!(𝑥)2+

1

3!(𝑥)3. . . . +

1

𝑛!(𝑥)𝑛


Proof:

• Hence, Maclaurin Series is a special case of Tylor’s series forexponential function 𝒇 𝒙 = 𝒆𝒙 over the interval 𝟎 𝒙 𝒊. 𝒆. 𝒂𝒕 𝒂 =𝟎

𝒆𝒙 = 𝟏 + 𝒙 +𝒙𝟐

𝟐!+𝒙𝟑

𝟑!+𝒙𝟒

𝟒!+ . . . . +

𝒙𝒏

𝒏!

Maclaurin Series

Numerical form of Taylor series

Numerical Form:

The numerical equivalent of Taylor series expansion of a function isas follow

• Letting (𝑥𝑖+1−𝑥𝑖) = ℎ , which is the incremental difference knownas the step size, the series can be rewritten:

• And the remainder

𝑓 𝑥𝑖+1 = 𝑓 𝑥𝑖 + 𝑓′ 𝑥𝑖 𝑥𝑖+1 − 𝑥𝑖 +𝑓′′ 𝑥𝑖2!

(𝑥𝑖+1 − 𝑥𝑖)2+

𝑓(3) 𝑥𝑖

3!(𝑥𝑖+1 − 𝑥𝑖)

3. . . . +𝑓𝑛 𝑥𝑖

𝑛!(𝑥𝑖+1 − 𝑥𝑖)

𝑛+𝑅𝑛

𝑓 𝑥𝑖+1 = 𝑓 𝑥𝑖 + 𝑓′ 𝑥𝑖 ℎ +𝑓′′ 𝑥𝑖2!

ℎ2 +𝑓(3) 𝑥𝑖

3!ℎ3. . . . +

𝑓𝑛 𝑥𝑖𝑛!

ℎ𝑛 + 𝑅𝑛

𝑅𝑛 =𝑓(𝑛+1) 𝜉

(𝑛 + 1)!ℎ𝑛+1

Taylor series: Examples

Problem statement 1:

Use zero- through fourth-order Taylor series expansions toapproximate the function at𝑥𝑖+1 =1 starting at 𝑥𝑖 = 0

Solution:

𝑓 𝑥 = −0.1𝑥4 − 0.15𝑥3 − 0.5𝑥2 − 0.25𝑥 + 1.2

Step size ℎ = 𝑥𝑖+1 − 𝑥𝑖 = 1,

exact value at 𝑥𝑖+1 = 1, 𝑓 𝑥𝑖+1 = 1 =0.2

𝑇𝑎𝑦𝑙𝑜𝑟 𝑠𝑒𝑟𝑖𝑒𝑠 𝑎𝑡 𝑛 = 0;

𝑓 𝑥𝑖+1 = 𝑓 𝑥𝑖

𝑓 𝑥𝑖+1 = −0.1 ∗ 04 −0.15 ∗ 03 − 0.5 ∗ 02 − 0.25 ∗ 0 + 1.2 = 1.2


𝐹𝑜𝑟 𝑇𝑎𝑦𝑙𝑜𝑟 𝑠𝑒𝑟𝑖𝑒𝑠 𝑎𝑡 𝑛 = 1

𝑓 𝑥𝑖+1 = 𝑓 𝑥𝑖 + 𝑓′ 𝑥𝑖 ℎ

𝑓′ 𝑥𝑖 = −0.4𝑥3 − 0.45𝑥2 − 1 ∗ 𝑥 − 0.25

𝑓′ 𝑥𝑖 = 0 = −0.4 ∗ 03 −0.45 ∗ 02 − 1 ∗ 0 − 0.25 = −0.25

𝑓 𝑥𝑖+1 = 1.2 + −0.25 ∗ ℎ = 𝟎. 𝟗𝟓


𝑇𝑎𝑦𝑙𝑜𝑟 𝑠𝑒𝑟𝑖𝑒𝑠 𝑎𝑡 𝑛 = 2

𝑓 𝑥𝑖+1 = 𝑓 𝑥𝑖 + 𝑓′ 𝑥𝑖 ℎ +𝑓′′ 𝑥𝑖2!

ℎ2

𝑓′′ 𝑥𝑖 = −1.2𝑥2 − 0.9𝑥 − 1

𝑓′′ 𝑥𝑖 = 0 = −1.2 ∗ 02 −0.9 ∗ 0 − 1 = −1

𝑓 𝑥𝑖+1 = 1.2 − 0.25ℎ −1

2 ∗ 1ℎ2 = 𝟎. 𝟒𝟓



𝑓 𝑥𝑖+1 = 𝑓 𝑥𝑖 + 𝑓′ 𝑥𝑖 ℎ +𝑓′′ 𝑥𝑖2!

ℎ2 +𝑓(3) 𝑥𝑖

3!ℎ3

𝑓(3) 𝑥𝑖 = −2.4𝑥 − 0.9

𝑓(3) 𝑥𝑖 = 0 = −2.4 ∗ 0 − 0.9 = −0.9

𝑓 𝑥𝑖+1 = 1.2 − 0.25ℎ −1

2 ∗ 1ℎ2 −

0.9

3 ∗ 2 ∗ 1ℎ2 = 𝟎. 𝟑



𝑓 𝑥𝑖+1 = 𝑓 𝑥𝑖 + 𝑓′ 𝑥𝑖 ℎ +𝑓′′ 𝑥𝑖2!

ℎ2 +𝑓(3) 𝑥𝑖

3!ℎ3 +

𝑓(4) 𝑥𝑖4!

ℎ4

𝑓(4) 𝑥𝑖 = −2.4 𝑓 4 𝑥𝑖 = 0 = −2.4

𝑓 𝑥𝑖+1

= 1.2 − 0.25ℎ −1

2 ∗ 1ℎ2 −

0.9

3 ∗ 2 ∗ 1ℎ3 −

2.4

4 ∗ 3 ∗ 2 ∗ 1ℎ4

= 𝟎. 𝟐


𝐻𝑒𝑛𝑐𝑒 fourth-order Taylor series expansions

𝑓 𝑥 = −0.1𝑥4 − 0.15𝑥3 − 0.5𝑥2 − 0.25𝑥 + 1.2

𝑓 𝑥𝑖+1 = 1 = 1.2 − 0.25 ∗ 1 −1

2 ∗ 1∗ 12

−0.9

3∗2∗1∗ 13 −

2.4

4∗3∗2∗1∗ 14

𝑓 1 = 𝟎. 𝟐



Use Taylor series expansions with n=0 to 6 to approximate 𝑓 𝑥 =𝑐𝑜𝑠𝑥 𝑎𝑡 𝑥𝑖+1 =

𝜋

3on the basis of the value of f(x) and its derivatives at

𝑥𝑖 =𝜋

4.

Solution:

Step size ℎ = 𝑥𝑖+1 − 𝑥𝑖 =𝜋

3−

𝜋

4=

𝜋

12

exact value at 𝑥𝑖+1 =𝜋

3, 𝑓

𝜋

3= cos

𝜋

3= 0.5


𝑓 𝑥𝑖+1 ≅ 𝑓 𝑥𝑖

𝑓 𝑥𝑖+1 ≅ cos𝜋

4= 𝟎. 𝟕𝟎𝟕𝟏𝟎𝟔


Solution:


𝑓 𝑥𝑖+1 ≅ 𝑓 𝑥𝑖 + 𝑓′ 𝑥𝑖 ℎ

𝑓(1) 𝑥𝑖 = −sin 𝑥𝑖 , 𝑓 1 𝑥𝑖 =𝜋

4= −sin

𝜋

4= −0.707106

𝑓 𝑥𝑖+1 ≅ 0.707106 + −0.707106 ∗𝜋

12= 0.521986659


Solution:


𝑓 𝑥𝑖+1 ≅ 𝑓 𝑥𝑖 + 𝑓′ 𝑥𝑖 ℎ + 𝑓′ 𝑥𝑖 ℎ +𝑓′′ 𝑥𝑖2!

ℎ2

𝑓(2) 𝑥𝑖 = −cos 𝑥𝑖 , 𝑓 2 𝑥𝑖 =𝜋

4= −𝑐𝑜𝑠

𝜋

4= −0.707106

𝑓 𝑥𝑖+1 ≅ cos𝜋

4−sin

𝜋

4

𝜋

12−𝑐𝑜𝑠

𝜋4

2!

𝜋

12

2

= 0.497754491


Solution:𝐴𝑓𝑡𝑒𝑟 𝑐𝑜𝑚𝑝𝑢𝑡𝑖𝑛𝑔 𝑇𝑎𝑦𝑙𝑜𝑟 𝑠𝑒𝑟𝑖𝑒𝑠 𝑢𝑝 𝑡𝑜 𝑛 = 6 𝑡ℎ𝑒 𝑓𝑜𝑙𝑙𝑜𝑤𝑖𝑛𝑔 𝑟𝑒𝑠𝑢𝑙𝑡𝑠 𝑎𝑟𝑒 𝑜𝑏𝑡𝑎𝑖𝑛𝑒𝑑

Taylor series: Examples with python


Implement a program in python to solve the problem statement 1 and 2and display the results both in tabular and graphical form. Hint usethe symbolic expression modules (sympy) to compute the derivativesterms in the series.

Clue: to compute and evaluate derivative of

Truncation error in Taylor Series

• For a simple monomial with a finite number of derivatives. Thispermits a complete determination of the Taylor series remainder.


Employ the zero-order to first-order Taylor series to approximate thefunction 𝑓 𝑥 = 𝑥4 at 𝑥𝑖+1=2 when 𝑥𝑖 = 1. Compute the remainderof the Taylor series approximation.

Solution

Step size ℎ = 𝑥𝑖+1 − 𝑥𝑖 = 1,

exact value at 𝑥𝑖+1 = 1, 𝑓 𝑥𝑖+1 = 2 = 24 =16


𝑓 𝑥𝑖+1 ≅ 𝑓 𝑥𝑖 𝑓 𝑥𝑖+1 ≅ 14 = 𝟏


𝑎𝑡 𝑛 = 0

𝑅𝑒𝑚𝑎𝑖𝑛𝑑𝑒𝑟, 𝑅𝑛 = 𝑓′ 𝑥𝑖 ℎ +𝑓′′ 𝑥𝑖2!

ℎ2 +𝑓(3) 𝑥𝑖

3!ℎ3. . . . +

𝑓𝑛 𝑥𝑖𝑛!

ℎ𝑛

Since 𝑓′ 𝑥 = 4𝑥3, 𝑓(2) 𝑥 = 12𝑥2, 𝑓(3) 𝑥 = 24𝑥, 𝑓(4) 𝑥 = 24

Any higher derivatives are zero i.e. 𝑓(5) 𝑥 = 0 and so on---

𝑅𝑒𝑚𝑎𝑖𝑛𝑑𝑒𝑟, 𝑅𝑛 = 4𝑥3ℎ +12𝑥2

2!ℎ2 +

24𝑥

3!ℎ3 +

24

4!ℎ4 + 0 + 0 +⋯


Solution

At 𝑥 = 1

𝑅𝑒𝑚𝑎𝑖𝑛𝑑𝑒𝑟, 𝑅𝑛

= 4 ∗ 13 1 +12 ∗ 12

2!12 +

24 ∗ 1

3!13 +

24

4!14 + 0 + 0 = 𝟏𝟓


𝑓 𝑥𝑖+1 ≅ 𝑓 𝑥𝑖 + 𝑓′ 𝑥𝑖 ℎ= 14+ 4 ∗ 13=5

𝑅𝑒𝑚𝑎𝑖𝑛𝑑𝑒𝑟, 𝑅𝑛 =12 ∗ 12

2!12 +

24 ∗ 1

3!13 +

24

4!14 + 0 + 0 = 𝟏𝟏

Numerical Differentiation

• Taylor Series Expansion can be used to derive numericalapproximations of first-order derivatives of a function.

• Truncating the series after the first derivative term:

• On rearranging

• Where 𝑓′ 𝑥𝑖 is the first-order derivatives

𝑓 𝑥𝑖+1 = 𝑓 𝑥𝑖 + 𝑓′ 𝑥𝑖 ℎ + 𝑅𝑛 1.2

𝑓′ 𝑥𝑖 =𝑓 𝑥𝑖+1 − 𝑓 𝑥𝑖

ℎ−𝑅𝑛ℎ

𝑓 𝑥𝑖+1 − 𝑓 𝑥𝑖ℎ

= 𝑖𝑠 𝑡ℎ𝑒 𝑓𝑖𝑟𝑠𝑡 − 𝑜𝑟𝑑𝑒𝑟 𝐴𝑝𝑝𝑟𝑜𝑥𝑖𝑚𝑎𝑡𝑖𝑜𝑛

𝑅𝑛ℎ

= 𝑖𝑠 𝑡ℎ𝑒 𝑇𝑟𝑢𝑛𝑐𝑎𝑡𝑖𝑜𝑛 𝐸𝑟𝑟𝑜𝑟

Finite-Divided-Difference: Forward difference

• When Taylor series is expanded forward to calculate a next value onthe basis of a present value.

• Truncating the remainder and using first-order term only from theseries we have:

𝜵𝒇𝒊 is referred to as the first forward difference

𝑓 𝑥𝑖+1 ≅ 𝑓 𝑥𝑖 + 𝑓′ 𝑥𝑖 ℎ

𝑓′ 𝑥𝑖 ≅𝑓 𝑥𝑖+1 − 𝑓 𝑥𝑖

ℎ=∇𝑓𝑖ℎ

Finite-Divided-Difference: Backward difference

• When Taylor series is expanded backward to calculate a previousvalue on the basis of a present value.

• Truncating the remainder and using first-order term only from theseries we have:

𝜵𝒇𝒊 is referred to as the first backward difference

𝑓 𝑥𝑖 ≅ 𝑓 𝑥𝑖−1 + 𝑓′ 𝑥𝑖 ℎ

𝑓′ 𝑥𝑖 ≅𝑓 𝑥𝑖 − 𝑓 𝑥𝑖−1

ℎ=∇𝑓𝑖ℎ

Finite-Divided-Difference: Centered Difference

• A third way to approximate the first derivative is to subtract Eq.(4.19) from the forward Taylor series expansion.

• The result yields

• 𝜵𝒇𝒊 is referred to as the first centered difference

𝑓′ 𝑥𝑖 ≅𝑓 𝑥𝑖+1 − 𝑓 𝑥𝑖−1

2ℎ=∇𝑓𝑖2ℎ

Finite-Divided-Difference: examples


a) Use forward, backward and centered difference approximation toestimate the first derivative of the function below at x=0.5 using astep size h=0.5. Repeat the computation using h=0.25.

b) Estimate the truncation error in each case.

Solution:

𝑓 𝑥 = −0.1𝑥4 − 0.15𝑥3 − 0.5𝑥2 − 0.25𝑥 + 1.25

𝒖𝒔𝒊𝒏𝒈 𝒆𝒙𝒂𝒄𝒕 𝒎𝒆𝒕𝒉𝒐𝒅

𝑓′ 𝑥𝑖 = −0.4𝑥3 − 0.45𝑥2 − 1 ∗ 𝑥 − 0.25

𝑎𝑡 x=0.5

𝑓′ 0.5 = −0.4 ∗ 0.53 − 0.45 ∗ 0.52 − 1 ∗ 0.5 − 0.25 = −0.9125.


Solution:

𝑢𝑠𝑖𝑛𝑔 𝒇𝒐𝒓𝒘𝒂𝒓𝒅 𝑑𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒 𝑎𝑡 ℎ = 0.5, 𝑥𝑖 =0.5

𝑓′ 𝑥𝑖 ≅𝑓 𝑥𝑖+1 − 𝑓 𝑥𝑖

ℎ

𝑥𝑖+1 − 𝑥𝑖 = ℎ 𝑥𝑖+1 = 1

𝑓 0.5 = −0.1 ∗ 0.54 − 0.15 ∗ 0.53 − 0.5 ∗ 0.52 − 0.25 ∗ 0.5 + 1.25= 0.975𝑥𝑖+1=1𝑓 1 = 0.25

𝑓′ 0.5 =0.25 − 0.975

0.5= −1.45


𝑢𝑠𝑖𝑛𝑔 𝒇𝒐𝒓𝒘𝒂𝒓𝒅 𝑑𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒 𝑎𝑡 ℎ = 0.25, 𝑥𝑖+1 =0.5

𝑓 0.5 = 0.975

𝑓 0.75 = 0.686 𝑓′ 0.5 =0.686 − 0.975

0.25= −1.156

𝜀𝑡 =−0.9125− (−1.156)

−0.9125∗ 100 = 26.68%

𝜀𝑡 =−0.9125− (−1.45)

−0.9125∗ 100 = 58.9%


𝑢𝑠𝑖𝑛𝑔 𝒃𝒂𝒄𝒌𝒘𝒂𝒓𝒅 𝑑𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒 𝑎𝑡 ℎ = 0.5, 𝑥𝑖+1 =0.5

𝑓′ 𝑥𝑖 ≅𝑓 𝑥𝑖 − 𝑓 𝑥𝑖−1

ℎ

𝑥𝑖 − 𝑥𝑖−1 = ℎ 𝑥𝑖−1 = 0

𝑓(𝑥𝑖−1) = −0.1 ∗ 04 − 0.15 ∗ 03 − 0.5 ∗ 02 − 0.25 ∗ 0 + 1.2 = 𝟏. 𝟐𝟓

𝑓(𝑥𝑖) =𝑓 0.5 = −0.1 ∗ 0.54 − 0.15 ∗ 0.53 − 0.5 ∗ 0.52 − 0.25 ∗ 0.5 + 1.25= 0.975

𝑓′ 0.5 =0.975 − 1.25

0.5= −0.55


𝑢𝑠𝑖𝑛𝑔 𝒃𝒂𝒄𝒌𝒘𝒂𝒓𝒅 𝑑𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒 𝑎𝑡 ℎ = 0.25, 𝑥𝑖 =0.5

𝑓 0.25 = 1.153

𝑓 0.5 = 0.975

𝑓′ 0.5 =0.975 − 1.153

0.25= −0.712

𝜀𝑡 =−0.9125− (−0.712)

−0.9125∗ 100 = 22%

𝜀𝑡 =−0.9125− (−0.55)

−0.9125∗ 100 = 39.7%


𝑢𝑠𝑖𝑛𝑔 𝒄𝒆𝒏𝒕𝒆𝒓𝒆𝒅 𝑑𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒 𝑎𝑡 ℎ = 0.5, 𝑥𝑖 =0.5


2ℎ

𝑓 0 = 1.25 𝑓 𝑥𝑖−1

𝑓 0.5 = 0.975 𝑓 𝑥𝑖

𝑓 1 = 0.25 𝑓 𝑥𝑖+1

𝑓′ 0.5 =0.25 − 1.25

2 ∗ 0.5= −1

𝜀𝑡 =−0.9125− (−1)

−0.9125∗ 100 = 9.59%


𝑢𝑠𝑖𝑛𝑔 𝒄𝒆𝒏𝒕𝒆𝒓𝒆𝒅 𝑑𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒 𝑎𝑡 ℎ = 0.25, 𝑥𝑖+1 =0.5


2ℎ

𝑓 0.25 = 1.153 𝑓 𝑥𝑖−1

𝑓 0.5 = 0.975 𝑓 𝑥𝑖

𝑓 0.75 = 0.686 𝑓 𝑥𝑖+1

𝑓′ 0.5 =0.686 − 1.153

2 ∗ 0.25= −0.934

𝜀𝑡 =−0.9125− (−0.934)

−0.9125∗ 100 = 2.36%

numerical analysis with python · 2021. 1. 20. · truncation errors • ‘truncation errors are...

Documents