Jacobi Method

The Jacobi method is a method of solving a matrix equation on a matrix that has no zeros

along its main diagonal (Bronshtein and Semendyayev 1997, p. 892). Each diagonal element

is solved for, and an approximate value plugged in. The process is then iterated until it

converges. This algorithm is a stripped-down version of the Jacobi transformation method of

matrix diagonalization.

The Jacobi method is easily derived by examining each of the equations in the linear system

of equations in isolation. If, in the th equation

(1

)

solve for the value of while assuming the other entries of remain fixed. This gives

(2

)

which is the Jacobi method.

In this method, the order in which the equations are examined is irrelevant, since the Jacobi

method treats them independently. The definition of the Jacobi method can be expressed with

matrices as

(3

)

where the matrices , , and represent thediagonal, strictly lower triangular, and strictly

upper triangular parts of , respectively.

SEE ALSO: Gauss-Seidel Method, Nonstationary Iterative Method, Stationary Iterative

Method, Successive Overrelaxation Method, Symmetric Successive Overrelaxation Method

Portions of this entry contributed by Noel Black and Shirley Moore, adapted from Barrett et al.

(1994) (author's link)

REFERENCES:

Acton, F. S. Numerical Methods That Work, 2nd printing. Washington, DC: Math. Assoc. Amer., pp. 161-163, 1990.

Barrett, R.; Berry, M.; Chan, T. F.; Demmel, J.; Donato, J.; Dongarra, J.; Eijkhout, V.; Pozo, R.; Romine, C.; and van

der Vorst, H. Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods, 2nd ed.

Philadelphia, PA: SIAM, 1994. http://www.netlib.org/linalg/html_templates/Templates.html.

Bronshtein, I. N. and Semendyayev, K. A. Handbook of Mathematics, 3rd ed. New York: Springer-Verlag, p. 892,

1997.

Hageman, L. and Young, D. Applied Iterative Methods. New York: Academic Press, 1981.

Press, W. H.; Flannery, B. P.; Teukolsky, S. A.; and Vetterling, W. T. Numerical Recipes in FORTRAN: The Art of

Scientific Computing, 2nd ed. Cambridge, England: Cambridge University Press, pp. 864-866, 1992.

Varga, R. Matrix Iterative Analysis. Englewood Cliffs, NJ: Prentice-Hall, 1962.

Young, D. Iterative Solutions of Large Linear Systems. New York: Academic Press, 1971.

Module

for

Jacobi and Gauss-Seidel Iteration

     

Background

    Iterative schemes require time to achieve sufficient accuracy and are reserved for large

systems of equations where there are a majority of zero elements in the matrix. Often times the

algorithms are taylor-made to take advantage of the special structure such as band

matrices.  Practical uses include applications in circuit analysis, boundary value problems and

partial differential equations.

    

    Iteration is a popular technique finding roots of equations.  Generalization of fixed point

iteration can be applied to systems of linear equations to produce accurate results.  The

method Jacobi iteration is attributed to Carl Jacobi (1804-1851) and Gauss-Seidel iteration is

attributed to  Johann Carl Friedrich Gauss (1777-1855) and Philipp Ludwig von Seidel (1821-

1896).

 

    Consider that the n×n square matrix A is split into three parts, the main diagonal D, below

diagonal L and above diagonal U.  We have  A = D - L - U.

=

-

-

  

 

Definition (Diagonally Dominant).  The matrix     is strictly diagonally dominant if

            for    .  

 

Theorem (Jacobi Iteration).  The solution to the linear system     can be obtained

starting with   , and using iteration scheme

    

           

where  

           and   .  

If   is carefully chosen a sequence is generated which converges to the

solution  P,  i.e.   .  

A sufficient condition for the method to be applicable is that A is strictly diagonally dominant or

diagonally dominant and irreducible.  

Proof  Jacobi and Gauss-Seidel Iteration  Jacobi and Gauss-Seidel Iteration  

 

Theorem (Gauss-Seidel Iteration).  The solution to the linear system     can be

obtained starting with   , and using iteration scheme

    

           

where  

           and   .  

If   is carefully chosen a sequence is generated which converges to the

solution  P,  i.e.   .  

A sufficient condition for the method to be applicable is that A is strictly diagonally dominant or

diagonally dominant and irreducible.

Proof  Jacobi and Gauss-Seidel Iteration  Jacobi and Gauss-Seidel Iteration  

 

Computer Programs  Jacobi and Gauss-Seidel Iteration  Jacobi and Gauss-Seidel

Iteration  

Mathematica Subroutine (Jacobi Iteration).

Mathematica Subroutine (Gauss-Seidel Iteration).

Example 1.  Use Jacobi iteration to solve the linear system   .  

Try 10, 20 and 30 iterations.  

Solution 1.

 

Example 2.  Use Jacobi iteration to attempt solving the linear

system   .

Try 10 iterations.  

Observe that something is not working.  In example 5 we will check to see if this matrix is

diagonally dominant.   

Solution 2.

 

Example 3.  Use Gauss-Seidel iteration to solve the linear

system   .  

Try 10, 20 iterations.  Compare the speed of convergence with Jacobi iteration.

Solution 3.

 

Example 4  Use Gauss-Seidel iteration to attempt solving the linear

system   .

Try 10 iterations.  

Observe that something is not working.  In example 5 we will check to see if this matrix is

diagonally dominant.   

Solution 4.

 

 

Warning.  

    Iteration does not always converge.  A sufficient condition for iteration to Jacobi iteration to

converge is that A is strictly diagonally  dominant. The following subroutine will check to see if

a matrix is strictly diagonally dominant.  It should be used before any call to Jacobi iteration or

Gauss-Seidel iteration is made.  There exists a counter-example for which Jacobi iteration

converges and Gauss-Seidel iteration does not converge.  The "true" sufficient condition for

Jacobi iteration to converge is that the "spectral radius" of      is less than 1,

where   is the diagonal of   .  That is, the magnitude of the largest eigenvalue of M must be

less than 1.  This condition seems harsh because numerical computation of eigenvalues is an

advanced topic compared to solution of a linear system.  

Example 5  Test the matrix A of Examples 2 and 4 to see if A is strictly diagonally dominant.

Solution 5.

 

 

More efficient subroutines

    A tolerance can be supplied to either the Jacobi or Gauss-Seidel method which will permit it

to exit the loop if convergence has been achieved.  

 

Computer Programs  Jacobi and Gauss-Seidel Iteration  Jacobi and Gauss-Seidel

Iteration  

Mathematica Subroutine (Jacobi Iteration).

Mathematica Subroutine (Gauss-Seidel Iteration).

Example 6.  Use Jacobi and Gauss-Seidel iteration to solve the linear

system   .  

Use a tolerance of     and a maximum of  50  iterations.

Solution 6.

 

 

Subroutines using matrix commands

    In the Jacobi subroutine we can use fix point iteration as suggested by the theory.  

Mathematica Subroutine (Jacobi Iteration).

Example 7.  Use the fixed point version of Jacobi iteration to solve the linear

system   .  

Solution 7.

 

Research Experience for Undergraduates

Jacobi and Gauss-Seidel Iteration  Jacobi and Gauss-Seidel Iteration  Internet hyperlinks

to web sites and a bibliography of articles.  

 

Download this Mathematica Notebook Jacobi and Gauss-Seidel Iteration

 

Return to Numerical Methods - Numerical Analysis

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

(c) John H. Mathews 2004