part 3 chapter 12 - caunmtl.cau.ac.kr/lecture/numerical_analysis/chapter_3-12... · 2012. 5....

School of Mechanical EngineeringChung-Ang University

Iterative Methods

Prof. Seok-min Kim

[email protected]

Part 3Chapter 12

1

mailto:[email protected]�


Chapter Objectives

2

Understanding the difference between the Gauss-Seidel and Jacobi methods.

Knowing how to assess diagonal dominance and knowing what it means.

Recognizing how relaxation can be used to improve convergence of iterative methods.

Understanding how to solve systems of nonlinear equations with successive substitution and Newton-Raphson.


Gauss-Seidel Method

3

The Gauss-Seidel method is the most commonly used iterative method for solving linear algebraic equations [A]{x}={b}.

For a 3x3 system with nonzero elements along the diagonal, for example, the jth iteration values are found from the j-1th

iteration using:

x1j =

b1 − a12x2j−1 − a13x3

j−1

a11

x2j =

b2 − a21x1j − a23x3

j−1

a22

x3j =

b3 − a31x1j − a32x2

j

a33

11 12 13 1 1

21 22 23 2 2

31 23 33 3 3

a a a x ba a a x ba a a x b

=

11 1 1 12 2 13 3

22 1 2 21 1 23 3

33 1 3 31 1 32 2

a x b a x a xa x b a x a xa x b a x a x

= − −= − −

= − −


Jacobi Iteration

4

The Jacobi iteration is similar to the Gauss-Seidel method, except the j-1th information is used to update all variables in the jthiteration:a) Gauss-Seidelb) Jacobi

1 11 12 2 13 3

1111 1

2 21 1 23 32

221 1

3 31 1 32 23

33

j jj

j jj

j jj

b a x a xxa

b a x a xxa

b a x a xxa

− −

− −

− −

− −=

− −=

− −=


Convergence

5

The convergence of an iterative method can be calculated by determining the relative percent change of each element in {x}. For example, for the ith element in the jth iteration,

The method is ended when all elements have converged to a set tolerance.

εa,i =xi

j − xij−1

xij ×100%


Example 12.1

6

Q. Use the Gauss-Seidel Method to solve this set of equations.

4.71 10 .203.03.193.07 1.085.7 0.21.03

321

321

321

=+−−=−+

=−−

xxxxxxxxx

[ ]{ } 3 2.5 7Tx = −Note : True solution is

Assume that x2 and x3 are zero in the first computation.

32.01.085.7 32

1xxx ++

=7

3.01.03.19 312

xxx +−−=

102.03.04.71 21

3xxx +−

=

School of Mechanical EngineeringChung-Ang University7

1 12 3

17.85 0.1 0.2

3

j jj x xx

− −+ +=

11 3

219.3 0.1 0.3

7

j jj x xx

−− − +=

1 23

71.4 0.3 0.210

j jj x xx − +

=

616667.23

)0(2.0)0(1.085.71 =

++=x

794524.27

)0(3.0)616667.2(1.03.192 −=

+−−=x

005610.710

)794524.2(2.0)616667.2(3.04.713 =

−+−=x

1st iteration


990557.23

)005610.7(2.0)794524.2(1.085.71 =

+−+=x

499625.27

)005610.7(3.0)990557.2(1.03.192 −=

+−−=x

000291.710

)499625.2(2.0)990557.2(3.04.713 =

−+−=x

2nd iteration

%5.12%100990557.2

616667.2990557.21, =×

−=εa

εa,2 = 11.8%; εa,3 = 0.076%;


Diagonal Dominance

9

The Gauss-Seidel method may diverge, but if the system is diagonally dominant, it will definitely converge.

Diagonal dominance means:

Many engineering problems satisfy this requirement

aii > aijj=1j≠i

n

∑


MATLAB Program

10

new2

33

32new1

33

31

33

3new3

old3

22

23new1

22

21

22

2new2

old3

11

13old2

11

12

11

1new1

xaax

aa

abx

xaax

aa

abx

xaax

aa

abx

−−=

−−=

−−=

MATLAB M-file: GaussSeidel

}]{[}{}{ xCdx −=

=

333

222

111

///

}{ababab

d

=

0///0///0

][

33323331

22232221

11131112

aaaaaaaaaaaa

C


Relaxation

12

To enhance convergence, an iterative program can introduce relaxation where the value at a particular iteration is made up of a combination of the old value and the newly calculated value (update for the new one):

where λ is a weighting factor that is assigned a value between 0 and 2.

0< λ <1: underrelaxation λ =1: no relaxation 1< λ ≤2: overrelaxation

xinew = λxi

new + 1− λ( )xiold


Nonlinear Systems

13

Nonlinear systems can also be solved using the same strategy as the Gauss-Seidel method - solve each system for one of the unknowns and update each unknown using information from the previous iteration.

This is called successive substitution.


Example 12.2

14

Q. Use successive substitution to determine the roots ofthe following equation. A correct pair of roots is x1 = 2 and x2 = 3. Use the initial guesses of x1 = 1.5 and x2 = 3.5.

573

102212

2121

=+

=+

xxx

xxx

First iteration

2

21

110

xxx −

= 2212 357 xxx −=

21429.25.3

)5.1(10 2

1 =−

=x 37516.24)5.3)(21429.2(357 22 −=−=x

Second iteration

20910.037516.24

)21429.2(10 2

1 −=−−

=x 709.429)37516.24)(20910.0(357 22 =−−−=x

Seems to be diverging


Use the same equation but with a different format

211 10 xxx −=1

22 3

57x

xx −=

First iteration

Second iteration

17945.2)5.3(5.1101 =−=x 86051.2)17945.2(3

5.3572 =

−=x

94053.1)86051.2(17945.2101 =−=x04955.3

)94053.1(386051.257

2 =−

=x

The approach is converging on the true values.

The most serious shortcoming of substitution, which depends on the manner in which the equations are formulated.


Newton-Raphson

16

Nonlinear systems may also be solved using the Newton-Raphson method for multiple variables.

For a one-variable system, the Taylor series approximation and resulting Newton-Raphson equations are:

For a two-variable system, )()()()( 11 iiiii xfxxxfxf ′−+= ++ 1

( )( )

ii i

i

f xx xf x+ = −

′

f1,i+1 = f1,i + x1,i+1 − x1,i( )∂f1,i

∂x1

+ x2,i+1 − x2,i( )∂f1,i

∂x2

x1,i+1 = x1,i −f1,i

∂f2,i

∂x2

− f2,i∂f1,i

∂x2

∂f1,i

∂x1

∂f2,i

∂x2

−∂f1,i

∂x2

∂f2,i

∂x1

f2,i+1 = f2,i + x1,i+1 − x1,i( )∂f2,i

∂x1

+ x2,i+1 − x2,i( )∂f2,i

∂x2

x2,i+1 = x2,i −f2,i

∂f1,i

∂x1

− f1,i∂f2,i

∂x1

∂f1,i

∂x1

∂f2,i

∂x2

−∂f1,i

∂x2

∂f2,i

∂x1


1[ ]{ } { } i iZ x x f+ − = −

∂

∂

∂

∂

∂

∂

∂

∂

∂

∂

∂

∂∂

∂

∂

∂

∂

∂

=

n

ininin

n

iii

n

iii

xf

xf

xf

xf

xf

xf

xf

xf

xf

Z

,

2

,

1

,

,2

2

,2

1

,2

,1

2

,1

1

,1

][

1, 2, ,{ }Ti i i n ix x x x =

1 1, 1 2, 1 , 1{ }Ti i i n ix x x x+ + + + =

1, 2, ,{ }Ti i n if f f f =

( ) ( )

( ) ( )

1, 1,1, 1 1, 1, 1 1, 2, 1 2,

1 2

2, 2,2, 1 2, 1, 1 1, 2, 1 2,

1 2

i ii i i i i i

i ii i i i i i

f ff f x x x x

x xf f

f f x x x xx x

∂ ∂∂ ∂∂ ∂∂ ∂

+ + +

+ + +

= + − + −

= + − + −

1[ ]{ } { } [ ]{ }, where [ ] i iZ x f Z x Z Jacobian matrix+ = − + =


Example 12.3 (1/2)

19

Q. Use the Newton-Raphson method to determine theroots of the equations. Use the initial guesses of x1 =1.5 and x2 = 3.5.

573

102212

2121

=+

=+

xxx

xxx

5.32)5.3)(5.1(6161 75.36)5.3(33

5.1 5.65.3)5.1(22

212

0,2222

1

0,2

12

0,121

1

0,1

=+=+=∂

∂===

∂

∂

==∂

∂=+=+=

∂

∂

xxxf

xx

f

xxf

xxxf

6.5(32.5) 1.5(36.75) 156.125Jacobian = − =


Example 12.3 (2/2)

20

5.210)5.3(5.1)5.1( 20,1 −=−+=f

625.157)5.3)(5.1(35.3 20,2 =−+=f

The values of the functions can be evaluated at the initial guesses as

These values can be substituted to give

03603.2125.156

)5.1(625.1)5.32(5.25.11 =−−

−=x

84388.2125.156

)75.36)(5.2()5.6(625.15.32 =−−

−=x

The computation can be repeated until an acceptable accuracyis obtained.


MATLAB Program

21

part 3 chapter 12 - caunmtl.cau.ac.kr/lecture/numerical_analysis/chapter_3-12... · 2012. 5....

Documents