Lecture 17Lecture 17

Numerical Analysis

Numerical Analysis

Chapter 4Chapter 4

Eigen Value Eigen Value ProblemsProblems

Eigen Value Eigen Value ProblemsProblems

Let [Let [AA] be an ] be an n x nn x n square matrix. square matrix. Suppose, there exists a scalar Suppose, there exists a scalar and a vectorand a vector

such thatsuch that

1 2( )TnX x x x

[ ]( ) ( )A X X

Power MethodPower Method

Jacobi’s MethodJacobi’s Method

Power MethodPower Method

Jacobi’s MethodJacobi’s Method

Power Power MethodMethodPower Power MethodMethod

To compute the largest eigen To compute the largest eigen value and the corresponding value and the corresponding eigenvector of the systemeigenvector of the system

where [where [AA] is a real, symmetric ] is a real, symmetric or un-symmetric matrix, the or un-symmetric matrix, the power method is widely used power method is widely used in practice. in practice.

[ ]( ) ( )A X X

ProcedureProcedureStep 1: Choose the initial vector Step 1: Choose the initial vector such that the largest element is such that the largest element is unity.unity.

Step 2: The normalized vector Step 2: The normalized vector is pre-multiplied by the matrix is pre-multiplied by the matrix [[AA].].

(0)v

Step 3:The resultant vector is Step 3:The resultant vector is again normalized.again normalized.

Step 4: This process of Step 4: This process of iteration is continued and the iteration is continued and the new normalized vector is new normalized vector is repeatedly pre-multiplied by repeatedly pre-multiplied by the matrix [the matrix [AA] until the ] until the required accuracy is required accuracy is obtained.obtained.

We calculateWe calculate

21 1 1 2 2

1 1

r r

r r nn nA v c v c v c v

1 1

1 1 21 1 1 2 2

1 1

( )r r

r r nn nA v c v c v c v

Now, the eigen value Now, the eigen value

can be computed as the limit of can be computed as the limit of the ratio of the corresponding the ratio of the corresponding components of and components of and That is, That is,

Here, the index Here, the index pp stands for the p-th stands for the p-th component in the corresponding vectorcomponent in the corresponding vector

1

rA v 1 .rA v

111

11

( ), 1, 2, ,

( )

rrp

r rrp

A vLt p n

A v

Sometimes, we may be Sometimes, we may be interested in finding the least interested in finding the least eigen value and the eigen value and the corresponding eigenvector. corresponding eigenvector. In that case, we proceed as In that case, we proceed as follows. follows. We note that We note that

Pre-multiplying by , we getPre-multiplying by , we get

[ ]( ) ( ).A X X

1[ ]A

1 1 1[ ][ ]( ) [ ] ( ) [ ]( )A A X A X A X

The inverse matrix has a The inverse matrix has a set of eigen values which set of eigen values which are the reciprocals of the are the reciprocals of the eigen values of [eigen values of [AA]. ].

1 1[ ]( ) ( )A X X

Thus, for finding the Thus, for finding the eigen value of the eigen value of the least magnitude of least magnitude of the matrix [the matrix [AA], we ], we have to apply power have to apply power method to the inverse method to the inverse of [of [AA]. ].

Jacobi’s Jacobi’s MethodMethod

Jacobi’s Jacobi’s MethodMethod

DefinitionDefinitionAn An n x nn x n matrix [ matrix [AA] is said to ] is said to be orthogonal ifbe orthogonal if

1

[ ] [ ] [ ],

i.e.[ ] [ ]

T

T

A A I

A A

If [If [AA] is an ] is an n x nn x n real real symmetric matrix, its eigen symmetric matrix, its eigen values are real, and there values are real, and there exists an orthogonal matrix exists an orthogonal matrix [[SS] such that the diagonal ] such that the diagonal matrix D ismatrix D is

1[ ][ ][ ]S A S

This diagonalization can be This diagonalization can be carried out by applying a carried out by applying a series of orthogonal series of orthogonal transformationstransformations

1 2, ,..., ,nS S S

Let Let AA be an be an n x nn x n real real symmetric matrix. Suppose symmetric matrix. Suppose be numerically the largest be numerically the largest element amongst the off-element amongst the off-diagonal elements of diagonal elements of AA. We . We construct an orthogonal matrix construct an orthogonal matrix SS11 defined as defined as

ija

sin , sin ,

cos , cos

ij ji

ii jj

s s

s s

1

i-th column -th column

1 0 0 0 0

0 1 0 0 0

0 0 cos sin 0 i-th row

0 0 sin cos 0 -th row

0 0 0 0 1

j

S

j

where where

are inserted in are inserted in positions respectively, and positions respectively, and elsewhere it is identical with a elsewhere it is identical with a unit matrix. unit matrix.

Now, we computeNow, we compute

cos , sin ,sin cosand

( , ), ( , ), ( , ), ( , ) thi i i j j i j j

11 1 1 1 1

TD S AS S AS

Therefore, , only if,Therefore, , only if,

That is ifThat is if

0ijd

cos 2 sin 2 02

jj iiij

a aa

2tan 2 ij

ii jj

a

a a

Thus, we choose such that the Thus, we choose such that the above equation is satisfied, above equation is satisfied, thereby, the pair of off-diagonal thereby, the pair of off-diagonal elements elements ddijij and and ddjiji reduces to reduces to zero.zero.However, though it creates a new However, though it creates a new pair of zeros, it also introduces pair of zeros, it also introduces non-zero contributions at non-zero contributions at formerly zero positions. formerly zero positions.

Also, the above equation gives Also, the above equation gives four values of , but to get the four values of , but to get the least possible rotation, we least possible rotation, we choose choose

4 4.

As a next step, the numerically As a next step, the numerically largest off-diagonal element in largest off-diagonal element in the newly obtained rotated the newly obtained rotated matrix matrix DD11 is identified and the is identified and the

above procedure is repeated above procedure is repeated using another orthogonal matrix using another orthogonal matrix SS22 to get to get DD22. That is we obtain. That is we obtain

12 2 1 2 2 1 1 2( )T TD S D S S S AS S

Similarly, we perform a series of Similarly, we perform a series of such two-dimensional rotations such two-dimensional rotations or orthogonal transformations. or orthogonal transformations. After making After making rr transformations, transformations, we obtainwe obtain

1 1 1 11 2 1 1 2 1

11 2 1 1 2 1

1

( ) ( )

r r r r r

r r r r

D S S S S AS S S S

S S S S A S S S S

S AS

ExampleExampleFind all the eigen values and Find all the eigen values and the corresponding eigen the corresponding eigen vectors of the matrix by vectors of the matrix by Jacobi’s methodJacobi’s method

1 2 2

2 3 2

2 2 1

A

SolutionSolutionThe given matrix is real and The given matrix is real and symmetric. The largest off-symmetric. The largest off-diagonal element is found to be diagonal element is found to be

Now, we computeNow, we compute13 31 2.a a

13

11 33

2 2 4tan 2

0ij

ii jj

a a

a a a a

Which gives, Which gives,

Thus, we construct an Thus, we construct an orthogonal matrix orthogonal matrix SSii as as

4

1 12 24 4

1

1 14 4 2 2

0cos 0 sin

0 1 0 0 1 0

sin 0 cos 0

S

11 1 1

1 1 1 12 2 2 2

1 1 1 12 2 2 2

1 2 20 0

0 1 0 2 3 2 0 1 0

0 02 2 1

D S AS

The first rotation gives,The first rotation gives,

3 2 0

2 3 0

0 0 1

We observe that the elements We observe that the elements dd1313 and and dd31 31 got annihilated. To got annihilated. To

make sure that calculations make sure that calculations are correct up to this step, we are correct up to this step, we see that the sum of the see that the sum of the diagonal elements of diagonal elements of DD11 is is

same as the sum of the same as the sum of the diagonal elements of the diagonal elements of the original matrix original matrix AA..

As a second step, we choose As a second step, we choose the largest off-diagonal the largest off-diagonal element of element of DD11 and is found to and is found to

be be and computeand compute

12

11 22

2 4tan 2

0

d

d d

12 21 2,d d

which again gives which again gives Thus, we construct the Thus, we construct the second rotation matrix assecond rotation matrix as

4

1 12 2

1 12 2 2

0

0

0 0 1

S

12 2 1 2

1 1 1 12 2 2 2

1 1 1 12 2 2 2

0 03 2 0

0 2 3 0 0

0 0 10 0 1 0 0 1

D S D S

5 0 0

0 1 0

0 0 1

At the end of the second At the end of the second rotation, we getrotation, we get

Which turned out to be a Which turned out to be a diagonal matrix, so we stop the diagonal matrix, so we stop the computation. From here, we computation. From here, we notice that the eigen values of notice that the eigen values of the given matrix are 5,1 and –1. the given matrix are 5,1 and –1. The eigenvectors are the column The eigenvectors are the column vectors of vectors of

1 2S S S

1 1 1 12 2 2 2

1 12 2

1 12 2

1 1 12 2 2

1 12 2

1 1 12 2 2

0 0

0 1 0 0

0 0 0 1

0

S

ThereforeTherefore

ExampleExampleFind all the eigen values of the Find all the eigen values of the matrix by Jacobi’s method.matrix by Jacobi’s method.

2 1 0

1 2 1

0 1 2

A

SolutionSolutionHere all the off-diagonal Here all the off-diagonal elements are of the same order elements are of the same order of magnitude. Therefore, we of magnitude. Therefore, we can choose any one of them. can choose any one of them. Suppose, we choose Suppose, we choose aa1212 as the as the largest element and computelargest element and compute

1tan 2

0

Which gives, Which gives,

Then Then and we construct an orthogonal and we construct an orthogonal matrix matrix SS11 such that such that

4.

cos sin 1 2

1 12 2

1 12 2

0

0

0 0 1

S

11 1 1

1 1 1 12 2 2 2

1 1 1 12 2 2 2

0 02 1 0

0 1 2 1 0

0 1 20 0 1 0 0 1

D S AS

12

12

1 12 2

1 0

0 3

2

The first rotation givesThe first rotation gives

Now, we choose Now, we choose

as the largest element of Das the largest element of D1 1 and and computecompute

13 1 2d

13

11 33

2 2tan 2

1 2

d

d d

27 22 41 .o

2

0.888 0 0.459

0 1 0

0.459 0 0.888

S

Now we construct another Now we construct another

orthogonal matrix Sorthogonal matrix S22, such that, such that

At the end of second rotation, we At the end of second rotation, we obtainobtain

Now, the numerically largest off-Now, the numerically largest off-diagonal element of diagonal element of DD2 2 is found is found

to be to be and compute and compute

12 2 1 2

0.634 0.325 0

0.325 3 0.628

0 0.628 2.365

D S D S

23 0.628d

Thus, the orthogonal matrix isThus, the orthogonal matrix is31 35 24 .o

3

1 0 0

0 0.852 0.524

0 0.524 0.852

S

2 0.628tan 2

3 2.365

At the end of third rotation, At the end of third rotation, we getwe get

13 3 2 3

0.634 0.277 0

0.277 3.386 0

0 0 1.979

D S D S

To reduce To reduce DD33 to a diagonal form, to a diagonal form,

some more rotations are required. some more rotations are required. However, we may take 0.634, 3.386 However, we may take 0.634, 3.386 and 1.979 as eigen values of the and 1.979 as eigen values of the given matrix.given matrix.

Lecture 17Lecture 17

Numerical Analysis

Numerical Analysis