0761214_Topic1 1

0761214: Numerical AnalysisTopic 1:

Introduction to Numerical Methods and Taylor Series

Lectures 1-4:

0761214_Topic1 2

Lecture 1Introduction to Numerical Methods

What are NUMERICAL METHODS?

Why do we need them?

Topics covered in 0761214.

Reading Assignment: Pages 3-10 of textbook

0761214_Topic1 3

Numerical Methods

Numerical Methods:

Algorithms that are used to obtain numerical

solutions of a mathematical problem.

Why do we need them?

1. No analytical solution exists,

2. An analytical solution is difficult to obtain

or not practical.

0761214_Topic1 4

What do we need?

Basic Needs in the Numerical Methods:

Practical:

Can be computed in a reasonable amount of time.

Accurate:

Good approximate to the true value,

Information about the approximation error (Bounds, error order,… ).

0761214_Topic1 5

Outlines of the Course

Taylor Theorem

Number Representation

Solution of nonlinear Equations

Interpolation

Numerical Differentiation

Numerical Integration

Solution of linear Equations

Least Squares curve fitting

Solution of ordinary differential equations

Solution of Partial differential equations

0761214_Topic1 6

Solution of Nonlinear Equations

Some simple equations can be solved analytically:

Many other equations have no analytical solution:

31

)1(2

)3)(1(444solution Analytic

034

2

2

xandx

roots

xx

solution analytic No052 29

xex

xx

0761214_Topic1 7

Methods for Solving Nonlinear Equations

o Bisection Method

o Newton-Raphson Method

o Secant Method

0761214_Topic1 8

Solution of Systems of Linear Equations

unknowns. 1000in equations 1000

have weif do What to

123,2

523,3

:asit solvecan We

52

3

12

2221

21

21

xx

xxxx

xx

xx

0761214_Topic1 9

Cramer’s Rule is Not Practical

this.compute toyears 10 than more needscomputer super A

needed. are tionsmultiplica102.3 system, 30by 30 a solve To

tions.multiplica

1)N!1)(N(N need weunknowns, N with equations N solve To

problems. largefor practicalnot is Rule sCramer'But

2

21

11

51

31

,1

21

11

25

13

:system thesolve toused becan Rule sCramer'

20

35

21

xx

0761214_Topic1 10

Methods for Solving Systems of Linear

Equations

o Naive Gaussian Elimination

o Gaussian Elimination with Scaled Partial Pivoting

o Algorithm for Tri-diagonal Equations

0761214_Topic1 11

Curve Fitting

Given a set of data:

Select a curve that best fits the data. One choice is to find the curve so that the sum of the square of the error is minimized.

x 0 1 2

y 0.5 10.3 21.3

0761214_Topic1 12

Interpolation

Given a set of data:

Find a polynomial P(x) whose graph passes through all tabulated points.

xi 0 1 2

yi 0.5 10.3 15.3

tablein the is)( iii xifxPy

0761214_Topic1 13

Methods for Curve Fitting

o Least Squares

o Linear Regression

o Nonlinear Least Squares Problems

o Interpolation

o Newton Polynomial Interpolation

o Lagrange Interpolation

0761214_Topic1 14

Integration

Some functions can be integrated analytically:

?

:solutions analytical no have functionsmany But

42

1

2

9

2

1

0

3

1

2

3

1

2

dxe

xxdx

a

x

0761214_Topic1 15

Methods for Numerical Integration

o Upper and Lower Sums

o Trapezoid Method

o Romberg Method

o Gauss Quadrature

0761214_Topic1 16

Solution of Ordinary Differential Equations

only. cases special

for available are solutions Analytical *

equations. thesatisfies that function a is

0)0(;1)0('

0)(3)('3)(''

:equation aldifferenti theosolution tA

x(t)

xx

txtxtx

0761214_Topic1 17

Solution of Partial Differential Equations

Partial Differential Equations are more difficult to solve than ordinary differential equations:

)sin()0,(,0),1(),0(

022

2

2

2

xxututu

t

u

x

u

0761214_Topic1 18

Summary

Numerical Methods:

Algorithms that are used to obtain numerical solution of a mathematical problem.

We need them when

No analytical solution exists or it is difficult to obtain it.

Solution of Nonlinear Equations

Solution of Linear Equations

Curve Fitting

Least Squares

Interpolation

Numerical Integration

Numerical Differentiation

Solution of Ordinary Differential Equations

Solution of Partial Differential Equations

Topics Covered in the Course

0761214_Topic1 19

Number Representation Normalized Floating Point Representation Significant Digits Accuracy and Precision Rounding and Chopping

Reading Assignment: Chapter 3

Lecture 2

Number Representation and Accuracy

0761214_Topic1 20

Representing Real Numbers

You are familiar with the decimal system:

Decimal System: Base = 10 , Digits (0,1,…,9)

Standard Representations:

21012 10510410210110345.312

part part

fraction integralsign

54.213

0761214_Topic1 21

Normalized Floating Point Representation

Normalized Floating Point Representation:

Scientific Notation: Exactly one non-zero digit appears before decimal point.

Advantage: Efficient in representing very small or very

large numbers.

exponent signed:,0

exponent mantissa sign

104321.

nd

nffffd

0761214_Topic1 22

Binary System

Binary System: Base = 2, Digits {0,1}

exponent signed mantissa sign

2.1 4321nffff

10)625.1(10)3212201211(2)101.1(

0761214_Topic1 23

Fact

Numbers that have a finite expansion in one numbering system may have an infinite expansion in another numbering system:

You can never represent 1.1 exactly in binary system.

210 ...)011000001100110.1()1.1(

IEEE 754 Floating-Point Standard

Single Precision (32-bit representation)

1-bit Sign + 8-bit Exponent + 23-bit Fraction

Double Precision (64-bit representation)

1-bit Sign + 11-bit Exponent + 52-bit Fraction

0761214_Topic1 24

S Exponent8 Fraction23

S Exponent11 Fraction52

(continued)

0761214_Topic1 25

Significant Digits

Significant digits are those digits that can be

used with confidence.

Single-Precision: 7 Significant Digits

1.175494… × 10-38 to 3.402823… × 1038

Double-Precision: 15 Significant Digits

2.2250738… × 10-308 to 1.7976931… × 10308

0761214_Topic1 26

Remarks

Numbers that can be exactly represented are called machine numbers.

Difference between machine numbers is not uniform

Sum of machine numbers is not necessarily a machine number

0761214_Topic1 27

Calculator Example

Suppose you want to compute:

3.578 * 2.139

using a calculator with two-digit fractions

3.57 * 2.13 7.60=

7.653342True answer:

0761214_Topic1 28

48.9

Significant Digits - Example

0761214_Topic1 29

Accuracy and Precision

Accuracy is related to the closeness to the true value.

Precision is related to the closeness to other estimated values.

0761214_Topic1 30

0761214_Topic1 31

Rounding and Chopping

Rounding: Replace the number by the nearest

machine number.

Chopping: Throw all extra digits.

0761214_Topic1 32

Rounding and Chopping

0761214_Topic1 33

Can be computed if the true value is known:

100* valuetrue

ionapproximat valuetrue

Error RelativePercent Absolute

ionapproximat valuetrue

Error True Absolute

t

tE

Error Definitions – True Error

0761214_Topic1 34

When the true value is not known:

100*estimatecurrent

estimate previous estimatecurrent

Error RelativePercent Absolute Estimated

estimate previous estimatecurrent

Error Absolute Estimated

a

aE

Error Definitions – Estimated Error

0761214_Topic1 35

We say that the estimate is correct to ndecimal digits if:

We say that the estimate is correct to ndecimal digits rounded if:

n10Error

n 102

1Error

Notation

0761214_Topic1 36

Summary

Number RepresentationNumbers that have a finite expansion in one numbering system may have an infinite expansion in another numbering system.

Normalized Floating Point Representation Efficient in representing very small or very large numbers,

Difference between machine numbers is not uniform,

Representation error depends on the number of bits used in the mantissa.

0761214_Topic1 37

Lectures 3-4

Taylor Theorem

Motivation

Taylor Theorem

Examples

Reading assignment: Chapter 4

0761214_Topic1 38

Motivation

We can easily compute expressions like:

?)6.0sin(,4.1 computeyou do HowBut,

)4(2

103 2

x

way?practicala thisIs

sin(0.6)? compute to

definition theuse weCan

0.6

ab

0761214_Topic1 39

Remark

In this course, all angles are assumed to be in radian unless you are told otherwise.

0761214_Topic1 40

Taylor Series

∑∞

0

)(

0

)(

3)3(

2)2(

'

)()( !

1 )(

: writecan weconverge, series theIf

)()( !

1

...)(!3

)()(

!2

)()()()(

:about )( of expansion seriesTaylor The

k

kk

k

kk

axafk

xf

axafk

SeriesTaylor

or

axaf

axaf

axafaf

axf

0761214_Topic1 41

Maclaurin Series

Maclaurin series is a special case of Taylor series with the center of expansion a = 0.

∑∞

0

)(

3)3(

2)2(

'

)0( !

1 )(

: writecan weconverge, series theIf

...!3

)0(

!2

)0()0()0(

:)( of expansion series n MaclauriThe

k

kk xfk

xf

xf

xf

xff

xf

0761214_Topic1 42

Maclaurin Series – Example 1

∞.xfor converges series The

...!3!2

1!

)0(!

1

11)0()(

1)0()(

1)0(')('

1)0()(

32∞

0

∞

0

)(

)()(

)2()2(

∑∑

xxx

k

xxf

ke

kforfexf

fexf

fexf

fexf

k

k

k

kkx

kxk

x

x

x

xexf )( of expansion series n MaclauriObtain

0761214_Topic1 43

Taylor SeriesExample 1

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10

0.5

1

1.5

2

2.5

3

1

1+x

1+x+0.5x2

exp(x)

0761214_Topic1 44

Maclaurin Series – Example 2

∞.xfor converges series The

....!7!5!3!

)0()sin(

1)0()cos()(

0)0()sin()(

1)0(')cos()('

0)0()sin()(

753∞

0

)(

)3()3(

)2()2(

∑

xxxxx

k

fx

fxxf

fxxf

fxxf

fxxf

k

kk

: )sin()( of expansion series n MaclauriObtain xxf

0761214_Topic1 45

-4 -3 -2 -1 0 1 2 3 4-4

-3

-2

-1

0

1

2

3

4

x

x-x3/3!

x-x3/3!+x5/5!

sin(x)

0761214_Topic1 46

Maclaurin Series – Example 3

∞.for converges series The

....!6!4!2

1)(!

)0()cos(

0)0()sin()(

1)0()cos()(

0)0(')sin()('

1)0()cos()(

642∞

0

)(

)3()3(

)2()2(

∑

x

xxxx

k

fx

fxxf

fxxf

fxxf

fxxf

k

kk

)cos()( :of expansion series Maclaurin Obtain xxf

0761214_Topic1 47

Maclaurin Series – Example 4

1 ||for converges Series

...xxx1x1

1 :ofExpansion Series Maclaurin

6)0(1

6)(

2)0(1

2)(

1)0('1

1)('

1)0(1

1)(

of expansion series n MaclauriObtain

32

)3(

4

)3(

)2(

3

)2(

2

x

fx

xf

fx

xf

fx

xf

fx

xf

x1

1f(x)

0761214_Topic1 48

Example 4 - Remarks

Can we apply the series for x≥1??

How many terms are needed to get a good approximation???

These questions will be answered using Taylor’s Theorem.

0761214_Topic1 49

Taylor Series – Example 5

...)1()1()1(1 :)1(Expansion SeriesTaylor

6)1(6

)(

2)1(2

)(

1)1('1

)('

1)1(1

)(

1at of expansion seriesTaylor Obtain

32

)3(

4

)3(

)2(

3

)2(

2

xxxa

fx

xf

fx

xf

fx

xf

fx

xf

ax

1f(x)

0761214_Topic1 50

Taylor Series – Example 6

...)1(3

1)1(

2

1)1( :Expansion SeriesTaylor

2)1(1)1(,1)1(',0)1(

2)(,

1)(,

1)(',)ln()(

)1(at )ln( of expansion seriesTaylor Obtain

32

)3()2(

3

)3(

2

)2(

xxx

ffff

xxf

xxf

xxfxxf

axf(x)

0761214_Topic1 51

Convergence of Taylor Series

The Taylor series converges fast (few terms are needed) when x is near the point of expansion. If |x-a| is large then more terms are needed to get a good approximation.

0761214_Topic1 52

Taylor’s Theorem

. and between is)()!1(

)(

:where

)(!

)( )(

:by given is )( of value the then and containing interval an on

1)( ..., 2, 1, orders of sderivative possesses )( functiona If

1)1(

0

)(

∑

xaandaxn

fR

Raxk

afxf

xfxa

nxf

nn

n

n

n

k

kk

(n+1) terms Truncated

Taylor Series

Remainder

0761214_Topic1 53

Taylor’s Theorem

.applicablenot is TheoremTaylor

defined.not are sderivative

its and function thethen,1If

.1||if0 expansion ofpoint thewith1

1

:for theoremsTaylor'apply can We

x

xax

f(x)

0761214_Topic1 54

Error Term

. and between allfor

)()!1(

)(

:on boundupper an derive can we

error, ionapproximat about theidea anget To

1)1(

xaofvalues

axn

fR n

n

n

0761214_Topic1 55

Error Term - Example

0514268.82.0)!1(

)()!1(

)(

1≥≤)()(

31

2.0

1)1(

2.0)()(

ERn

eR

axn

fR

nforefexf

nn

nn

n

nxn

?2.00at

expansion seriesTaylor its of3)( terms4first the

by )( replaced weiferror theis largeHow

xwhena

n

exf x

0761214_Topic1 56

Alternative form of Taylor’s Theorem

hxxwherehn

fR

hRhk

xfhxf

hxx

nxfLet

nn

n

n

n

k

kk

and between is)!1(

)(

size) step ( !

)( )(

: then and containing interval an on

1)( ..., 2, 1, orders of sderivative have)(

1)1(

0

)(

0761214_Topic1 57

Taylor’s Theorem – Alternative forms

. and between is

)!1(

)(

!

)()(

,

. and between is

)()!1(

)()(

!

)()(

1)1(

0

)(

1)1(

0

)(

hxxwhere

hn

fh

k

xfhxf

hxxxa

xawhere

axn

fax

k

afxf

nnn

k

kk

nnn

k

kk

0761214_Topic1 58

Mean Value Theorem

)( )('

, ,0for Theorem sTaylor' Use:Proof

)('

),( exists therethen

),( interval open theon defined is derivative its and

],[ interval closeda on function continuousa is)( If

abξff(a)f(b)

bhxaxn

ab

f(a)f(b)ξf

baξ

ba

baxf

0761214_Topic1 59

Alternating Series Theorem

termomittedFirst :

n terms)first theof (sum sum Partial:

converges series The

then

0lim

If

S

:series galternatin heConsider t

1

1

4321

4321

n

n

nnnn

a

S

aSS

and

a

and

aaaa

aaaa

0761214_Topic1 60

Alternating Series – Example

!7

1

!5

1

!3

11)1(s

!5

1

!3

11)1(s

:Then

0lim

:since series galternatin convergent a is This

!7

1

!5

1

!3

11)1(s:usingcomputed becansin(1)

4321

in

in

aandaaaa

in

nn

0761214_Topic1 61

Example 7

? 1 with eapproximat to

used are terms1)( whenbeerror thecan largeHow

expansion) ofcenter (the5.0at)( of

expansion seriesTaylor theObtain

12

12

xe

n

aexf

x

x

0761214_Topic1 62

Example 7 – Taylor Series

...!

)5.0(2...

!2

)5.0(4)5.0(2

)5.0(!

)5.0(

2)5.0(2)(

4)5.0(4)(

2)5.0('2)('

)5.0()(

22

222

∞

0

)(12

2)(12)(

2)2(12)2(

212

212

∑

k

xe

xexee

xk

fe

efexf

efexf

efexf

efexf

kk

k

kk

x

kkxkk

x

x

x

5.0,)( of expansion seriesTaylor Obtain 12 aexf x

0761214_Topic1 63

Example 7 – Error Term

)!1(

max)!1(

)5.0(2

)!1(

)5.01(2

)5.0()!1(

)(

2)(

3

12

]1,5.0[

11

1121

1)1(

12)(

n

eError

en

Error

neError

xn

fError

exf

nn

nn

nn

xkk