lecture 8 - iterative systems of equations

Lecture 8 - Iterative systems of Lecture 8 - Iterative systems of EquationsEquations

CVEN 302

June 19, 2002

Lecture’s GoalsLecture’s Goals

• Iterative Techniques– Jacobian method– Gauss-Siedel method– Relaxation technique

Iterative TechniquesIterative Techniques

• The method of solving simultaneous linear algebraic equations using Gaussian Elimination and the Gauss-Jordan Method. These techniques are known as direct methods. Problems can arise from round-off errors and zero on the diagonal.

• One means of obtaining an approximate solution to the equations is to use an “educated guess”.

Iterative MethodsIterative Methods

• We will look at three iterative methods:– Jacobi Method– Gauss-Seidel Method– Successive over Relaxation (SOR)

Convergence RestrictionsConvergence Restrictions

• There are two conditions for the iterative method to converge.– Necessary that 1 coefficient in each equation is

dominate.– The sufficient condition is that the diagonal is

dominate.

Jacobi IterationJacobi Iteration• If the diagonal is dominant, the matrix can be

rewritten in the following form

Jacobi IterationJacobi Iteration• The technique can be rewritten in a shorthand

fashion, where D is the diagonal, A” is the matrix without the diagonal and c is the right-hand side of the equations.

Jacobi IterationJacobi Iteration

• The technique solves for the entire set of x values for each iteration.

• The problem does not update the values until an iteration is completed.

Example (Jacobi Iteration)Example (Jacobi Iteration)

4X1 + 2X2 = 2

2X1 + 10X2 + 4X3 = 6

4X2 + 5X3 = 5

Solution: (X1 , X2 , X3 ) = (0.41379, 0.17241, 0.86206)

Jacobi ExampleJacobi Example

3

2

1

3

2

1

3

2

1

3

2

1

3

2

1

040

402

020

5

6

2

500

0100

004

5

6

2

040

402

020

500

0100

004

5

6

2

540

4102

024

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

Jacobi ExampleJacobi Example

3

2

1

3

2

1

3

2

1

3

2

1

040

402

020

2.000

01.00

0025.0

5

6

2

2.000

01.00

0025.0

500

0100

004

2.000

01.00

0025.0

2.000

01.00

0025.01

500

0100

004

040

402

020

5

6

2

500

0100

004

X

X

X

X

X

X

X

X

X

X

X

X

Jacobi ExampleJacobi Example

• Formulation of the matrix

Jacobi IterationJacobi IterationConvergence Graph

0

0.2

0.4

0.6

0.8

1

0 2 4 6 8 10

Number of Iterations

Fu

nc

tio

na

l V

alu

eX1

X2

X3

Iteration 1 2 3 4 5 6 7

X 1 0.5 0.2 0.45 0.324 0.429 0.376 0.42

X 2 0.6 0.1 0.352 0.142 0.248 0.16 0.204

X 3 1 0.52 0.92 0.718 0.886 0.802 0.872

Jacobi ProgramJacobi Program

• The computer program is setup to do the Jacobi method for any size square matrix:

Jacobi(A,b) • The program can has options for maximum

number of iterations, nmax, and tolerance, tol.

Jacobi(A,b,nmax,tol)

Gauss-Seidel IterationsGauss-Seidel Iterations

• The Gauss-Seidel / Seidel technique is similar to the Jacobi iteration technique with one difference.

• The method updates the results continuously. It uses the new information from the previous iteration to accelerate converge to a solution.

Gauss-Seidel ModelGauss-Seidel Model

• The Gauss-Seidel Algorithm:

• The combined vector is upgraded ever term.

Example (Gauss-Seidel Iteration)Example (Gauss-Seidel Iteration)

4X1 + 2X2 = 2

2X1 + 10X2 + 4X3 = 6

4X2 + 5X3 = 5

Solution: (X1 , X2 , X3 ) = (0.41379, 0.17241, 0.86206)

Gauss-Seidel ExampleGauss-Seidel Example

• Formulation of the matrix problem

Gauss-Seidel IterationGauss-Seidel Iteration

Iteration 1 2 3 4 5 6 7

X 1 0.5 0.25 0.345 0.384 0.401 0.408 0.411

X 2 0.5 0.31 0.231 0.197 0.183 0.177 0.175

X 3 0.6 0.75 0.815 0.842 0.854 0.858 0.858

Convergence Graph

0

0.2

0.4

0.6

0.8

1

0 2 4 6 8 10

Number of Iteration

Fu

nc

tio

na

l V

alu

es

X1

X2

X3

Gauss-Seidel ModelGauss-Seidel Model

• The computer program is setup to do the Gauss-Seidel method for any size square matrix:

Seidel(A,b)

• The program can has options for maximum number of iterations, nmax, and tolerance, tol.

Seidel(A,b,nmax,tol)

Example - Iteration with no Example - Iteration with no diagonal dominationdiagonal domination

3X1 - 3X2 + 5X3 = 4

X1 + 2X2 - 6X3 = 3

2X1 - X2 + 3X3 = 1

Solution: (X1 , X2 , X3 ) = (1.00, -2.00, -1.00)

Using the GS algorithmUsing the GS algorithm

Using the Gauss-Seidel Program with the following A

matrix and b vector.

Solution: (X1 , X2 , X3 ) = (1.00, -2.00, -1.00)

Gauss-Seidel ExampleGauss-Seidel Example

3

2

1

3

2

1

3

2

1

3

2

1

3

2

1

3

2

1

03333.06667.0

305.0

6667.110

3333.0

5.1

3333.1

012

601

530

3333.000

05.00

003333.0

1

3

4

3333.000

05.00

003333.0

012

601

530

1

3

4

300

020

003

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

Gauss-Seidel ExampleGauss-Seidel Example

Program will work with the equations:

3

2

1

3

2

1

03333.06667.0

305.0

6667.110

3333.0

5.1

3333.1

X

X

X

X

X

X

Using a Gauss-Seidel IterationUsing a Gauss-Seidel Iteration

Iteration 1 2 3 4 5 6 7

X 1 1.33 2.630 3.412 2.296 -1.375 6.078 -7.620

X 2 0.833 -0.648 -5.113 -10.584 -11.989 -3.698 14.769

X 3 -0.278 -1.635 -3.645 -4.725 -2.746 3.153 10.336

Convergence Graph

-15

-10

-5

0

5

10

15

20

0 2 4 6 8 10

Number of iterations

Fu

ncti

on

al V

alu

e

X1

X2

X3

Successive over RelaxationSuccessive over Relaxation

• The technique is a modification on the Gauss-Seidel method with an additional parameter, that may accelerate the convergence of the iterations.

• The weighting parameter, has two ranges 0 < <1, and 1< <2. If = 1, then the problem is the Gauss-Seidel technique.

SOR MethodSOR Method

• The SOR algorithm is defined as:

• The difference is the weighting parameter,

Weighting ParameterWeighting Parameter

• If the parameter, is under 1, the residuals will be under-relaxed.

• If the parameter, = 1, the residuals are equal to a Gauss-Seidel model.

• If 1< < 2 the residuals will be over-relaxed and will general help accelerate the convergence of the solution.

Example of SORExample of SOR

4X1 + 2X2 = 2

2X1 + 10X2 + 4X3 = 6

4X2 + 5X3 = 5

Solution: (X1 , X2 , X3 ) = (0.41379, 0.17241, 0.86206)

SOR ExampleSOR Example

• Formulation of the SOR Algorithm

Effects of Effects of ParameterParameter

Number of Number of

Iterations Iterations0.7 33 1.25 120.8 27 1.3 140.9 22 1.4 171 17 1.5 22

1.1 13 1.6 301.15 10 1.7 431.175 101.2 10

• Using the SOR program SOR(A,bnmax,tol) with nmax=50 and tol = 0.000001

SummarySummary

• Convergence conditions need to be met in order for iterative techniques to converge

• Jacobi method upgrades the values after each iteration.

• Gauss-Seidel upgrades continuously through method.

• SOR (Successive over Relaxation) uses the residuals to accelerate the convergence.

HomeworkHomework

• Check the Homework webpage