eigenvalue problems - cs.mcgill.cachang/teaching/cs540/loginpage/lecs540/... · is...

EIGENVALUE PROBLEMS

EIGENVALUE PROBLEMS – p. 1/45

Eigenvalues and eigenvectors

Let A ∈ Cn×n. SupposeAx = λx, x 6= 0, thenx is a (right)

eigenvector ofA, corresponding to the eigenvalueλ.




eigenvector ofA, corresponding to the eigenvalueλ.Q: Prove that ifx is an eigenvector, so isαx, for α 6= 0.





So we often take‖x‖ = 1.






For an eigenpair(λ, x), (λI − A)x = 0, x 6= 0.Q: This is only possible if ?







λI − A is singular.







λI − A is singular.

For generalλ, π(λ) ≡ det(λI − A) is calledthecharacteristic polynomial of A

π(λ) = λn − (a11 + · · · + ann)λn−1 + · · · + det(−A).

π(λ) = 0 is called thecharacteristic equation .π(λ) has exact degreen, and so hasn zerosλ1, . . . , λn, say.


Eigenvalues and eigenvectors, ctd

Spectrum of A: the setΛ(A) = {λ1, . . . , λn}.




An eigenvalueλ hasalgebraic multiplicity r if it is repeatedexactlyr times.





Q: Canreal A havecomplex eigenvalues?






Q: Let xi be an eigenvector ofA coresponding toλi.Are x1, . . . , xn linearly independent?







[

1 1

1

] [

ξ

η

]

=

[

ξ

η

]

,

It has 2 eigenvalues 1 and 1, but 1 independent eigenvector.







[

1 1

1

] [

ξ

η

]

=

[

ξ

η

]

,

It has 2 eigenvalues 1 and 1, but 1 independent eigenvector.An eigenvalueλ hasgeometric multiplicity s if hass linearlyindependent eigenvectors.


Similarity Transformations (ST)

Similarity transformations : B ≡ X−1AX.A andB are said to besimilar .




Thm. Similar matrices have the same characteristicpolynomial.





Cor. Similarity transformations preserve eigenvaluesandalgebraic multiplicities.





Cor. Similarity transformations preserve eigenvaluesandalgebraic multiplicities.

Thm. Similarity transformations preserve geometricmultiplicities.


Jordan Canonical Form (JCF)

k × k Jordan block:J (k)λ =

λ 1

· ·

λ 1

λ

It has one distinct eigenvalueλ with algebraic multiplicitykand geometric multiplicity 1.


Jordan Canonical Form (JCF)

k × k Jordan block:J (k)λ =

λ 1

· ·

λ 1

λ

It has one distinct eigenvalueλ with algebraic multiplicitykand geometric multiplicity 1.

Thm. Let A ∈ Cn×n, then there exist unique numbers

λ1, λ2, . . . , λs (complex, not necessarily distinct), and uniquepositive integersm1,m2, . . . ,ms, and nonsingularX ∈ C

n×n

giving the JCF ofA

X−1AX = J ≡ diag[J (m1)λ1

, . . . , J(ms)λs

]. 2


Jordan Canonical Form, ctd

We seeAX = XJ , X = [x1, · · · , xn].

Q: Which ofx1, . . . , xn are eigenvectors?



We seeAX = XJ , X = [x1, · · · , xn].


Q: Can we find the algebraic multiplicity and geometricmultiplicity of an eigenvalue ofA from its JCF?



We seeAX = XJ , X = [x1, · · · , xn].



Q: What is the relationship between the algebraic multiplicityand geometric multiplicity of an eigenvalue?



We seeAX = XJ , X = [x1, · · · , xn].




If all the Jordan blocks corresponding to the same eigenvaluehave dimension 1, the eigenvalue isnondefective orsemisimple , elsedefective .



We seeAX = XJ , X = [x1, · · · , xn].




If all the Jordan blocks corresponding to the same eigenvaluehave dimension 1, the eigenvalue isnondefective orsemisimple , elsedefective .

If all the eigenvalues ofA are nondefective,A is nondefectiveor semisimple .



If J is diagonal,A is diagonalizable .




Q. Is “nondefective” = “diagonalizabl” ?





Q: If a matrixA ∈ Cn×n hasn distinctive eigenvalues, isA

diagonalizable?






diagonalizable?

If two or more Jordan blocks have the same eigenvalue, theeigenvalue isderogatory , elsenonderogatory .






diagonalizable?


If a matrix has a derogatory eigenvalue, the matrix isderogatory , elsenonderogatory .






diagonalizable?



If an eigenvalue is distinct, it issimple .






diagonalizable?



If an eigenvalue is distinct, it issimple .

If all eigenvalues are distinct, the matrix issimple .



The JCF is classical theory — important in describingproperties of various linear differential equations, and so isimportant in control theory etc.




Two problems with JCF:

For a generalA, X in X−1AX = J can be veryill-conditioned. Rounding errors in the JCF computationcan be greatly magnified:

fl(X−1AX) = X−1AX+E, ‖E‖2 = O(u)κ2(X)‖A‖2.







A small perturbation inA may change the dimensions ofthe Jordan blocks completely.







A small perturbation inA may change the dimensions ofthe Jordan blocks completely.

If possible, avoid the JCF in computations.


Unitary similarity transformations

Unitary transformations preserve size.

If Q ∈ Cn×n is unitary,B ≡ QHAQ is aunitary similarity

transformation of A.






Q: What are the relations between the eigenvalues and singularvalues ofB and those ofA?






Q: What are the relations between the eigenvalues and singularvalues ofB and those ofA?

Q: What is “simplest” form of the aboveB?


Schur Decomposition

Schur’s Theorem. There exists a similarity transformation withunitaryQ such that

QHAQ = R, R upper triangular.


Schur Decomposition

Schur’s Theorem. There exists a similarity transformation withunitaryQ such that


Pf. A has at least one eigenvectorx, so suppose

Ax = λx, xHx = 1, QH1 x = e1, with Q1 unitary,

givesQ1 = [x, F ] say, then

QH1 AQ1 =

[

xHAx xHAF

FHAx FHAF

]

=

[

λ xHAF

O FHAF

]

,

and the same argument can be applied toFHAF , etc.2EIGENVALUE PROBLEMS – p. 10/45

Real Schur decomposition

If A ∈ Rn×n, then there exists real orthogonalQ, giving

QTAQ = R,

R is quasi–upper triangular, with blocks on the diagonalcorresponding to complex conjugate pairs, e.g.,

R =

× × × × ×

× × × ×

× × × ×

× ×

× ×

two complex conjugate pairs, and one real eigenvalue.EIGENVALUE PROBLEMS – p. 11/45

Real Schur decomposition, ctd

Q: If A = AH , the Schur form has real diagonalR ?




Hermitian matrices havereal eigenvalues,and a complete set ofunitary eigenvectorsQ.





Any matrix having a complete set of unitary eigenvectors iscalled anormal matrix .






Q: ShowA is normal iffAAH = AHA.






Q: ShowA is normal iffAAH = AHA.

Q: If A is real symmetrc, then it has a complete set oforthogonal eigenvectors.


Algorithms for the EVP

If A is real,A = AT ,

A = Q diag(λi)QT , QTQ = I,

the eigendecomposition ofA, is the same as the SVD (apartfrom signs).






Q: What is one way of computing this?






Q: What is one way of computing this?Jacobi’s method (1845)






Q: What is one way of computing this?Jacobi’s method (1845)

For general A ∈ Cn×n, there exists unitaryQ such that


Also ‖QHAQ‖2,F = ‖A‖2,F , numerically desirable.So we seek suchQ andR.


Algorithms for the EVP, ctd

Q: Why must methods be iterative for the EVP?



Q: Why must methods be iterative for the EVP?Finding zeros ofπ(λ) = 0 is a nonlinear problem– cannot be solved on a fixed number of steps.




The form of such iterative algorithms for computing theSchur form is:

A1 = A

Ak+1 = QHk AkQk, Qk is unitary, k = 1, 2, . . .

= QHk QH

k−1 · · ·QH1 AQ1 · · ·Qk−1Qk,

We try to design theQk so thatAk −→ upper triangular.




The form of such iterative algorithms for computing theSchur form is:

A1 = A

Ak+1 = QHk AkQk, Qk is unitary, k = 1, 2, . . .

= QHk QH

k−1 · · ·QH1 AQ1 · · ·Qk−1Qk,

We try to design theQk so thatAk −→ upper triangular.

If A is real, we should try to design orthogonalQk so thatAk −→ quasi-upper triangular.


The QR algorithm

Basic algorithm:

A1 = Afor k = 1, 2, . . . until convergence do

QR factorization of Ak:Ak = QkRk, Qk unitary,Rk upper triangular

recombine in the reverse order:Ak+1 = RkQk

end


The QR algorithm, ctd

1.Ak+1 = QH

k AkQk = QHk · · ·QH

1 AQ1 · · ·Qk

is a unitary similarity transformation ofA.

With refinements this is one of the most effective matrixcomputation algorithms for transformingA to uppertriangular form using unitary similarity transformations.


The QR algorithm, ctd

1.Ak+1 = QH

k AkQk = QHk · · ·QH

1 AQ1 · · ·Qk

is a unitary similarity transformation ofA.

With refinements this is one of the most effective matrixcomputation algorithms for transformingA to uppertriangular form using unitary similarity transformations.

2. If A is real, thenQk are orthogonal,

Ak+1 = QTk AkQk = QT

k · · ·QT1 AQ1 · · ·Qk.

With some refinements, the algorithm will transformA toquasi-upper triangular form.


Hessenberg Reduction:

Each iteration (called a QR step) costsO(n3) flops— very expensive.




Q. Can we reduceA to a matrix with special structure so thateach QR step costsO(n2) flops?




Q. Can we reduceA to a matrix with special structure so thateach QR step costsO(n2) flops?

Reduce the matrixA to

the Hessenberg form:

× × × ×

× × × ×

× × ×

× ×

,

the closest we can get to upper triangular form using a fixednumber ofunitary similarity transformations.



Q. How to do the Hessenberg reduction?




Q. What is its cost ifA is real ?10n3/3 flops.





Q. Do we lose the upper Hessenberg form in the iterations?





Q. Do we lose the upper Hessenberg form in the iterations?No if we use Givens rotations in the QR factn.

Q. What is the cost of each QR step now ifA is real ?

6n2 flops.


QR algorithm with Hessenberg reduction:

Compute Hessenberg reductionA1 := QH0 AQ0

whereQ0 := H1 . . . Hn−2

for k = 1, 2, . . . until convergence doQR factorization of Ak:

Ak = QkRk, Qk unitary,Rk upper triangularrecombine in the reverse order:

Ak+1 = RkQk

end


QR algorithm with Hessenberg reduction:

Compute Hessenberg reductionA1 := QH0 AQ0

whereQ0 := H1 . . . Hn−2

for k = 1, 2, . . . until convergence doQR factorization of Ak:

Ak = QkRk, Qk unitary,Rk upper triangularrecombine in the reverse order:

Ak+1 = RkQk

end

It is still slow.


Shifting

Let |λ1| ≥ · · · ≥ |λn|. If |λi| > |λi+1|, we can show

a(k)i+1,i → 0, as

∣

∣

∣

∣

λi+1

λi

∣

∣

∣

∣

k

→ 0

so it haslinear convergence .


Shifting, ctd

Q: What are the eigenvalues ofA − µI, and what is the new

measure of convergence fora(k)i+1,i → 0 ?


Shifting, ctd



Eigenvalues ofA − µI: λi − µ, i = 1 : n.


Shifting, ctd




Suppose we renumber the eigenvalues ofA so that

|λ1 − µ| ≥ · · · ≥ |λn − µ|

New measure of convergence:|λi+1 − µ|

|λi − µ|.


Shifting, ctd





|λ1 − µ| ≥ · · · ≥ |λn − µ|


|λi − µ|.

Q: How should we chooseµ to obtain fast convergence?


Shifting, ctd





|λ1 − µ| ≥ · · · ≥ |λn − µ|


|λi − µ|.

Q: How should we chooseµ to obtain fast convergence?

Chooseµ to be close to an eigenvalue ofA.


Shifting, ctd

Suppose|λn − µ| < |λn−1 − µ|. If we apply the QR iterations

to A1 = A1 − µI, thena(k)n,n−1 will converge to zero quickly.

Suppose afterk0 − 1 QR steps,a(k0)n−1,n is small enough,

we regard it as zero and add the shift back on:

Ak0+ µI =

× × × × ×

× × × × ×

× × × ×

× × ×

0 λ

Thenλ is an eigenvalue ofA.


Shifting, ctd

The above process can be written asA1 := A1 − µIfor k = 1 : k0 − 1

Ak = QkRk

Ak+1 = RkQk

endAk0

:= Ak0+ µI


Shifting, ctd

The above process can be written asA1 := A1 − µIfor k = 1 : k0 − 1

Ak = QkRk

Ak+1 = RkQk

endAk0

:= Ak0+ µI

It is easy to show the above process is equivalent to

for k = 1 : k0 − 1Ak − µI = QkRk

Ak+1 = RkQk + µIend


Shifting, ctd

But there is no reason to use the same shiftµ in all QR steps.


Shifting, ctd


Also we usually can only find a good approximation to aneigenvalue during the QR iterations.


Shifting, ctd



So we should use different shifts in the algorithm for differentQR steps.


Shifting, ctd



So we should use different shifts in the algorithm for differentQR steps.

Shifted QR Algorithm with Hessenberg Reduction :Compute the Hessenberg reductionA1 = QH

0 AQ0,whereQ0 := H1 . . . Hn−2

for k = 1, 2, . . . until convergence doAk − µkI = QkRk

Ak+1 = RkQk + µkIend


Shifting, ctd

Remarks:Ak+1 = QH

k (Ak − µkI)Qk + µkI = QHk AkQk.

Now Qk depends onµk.


Shifting, ctd

Remarks:Ak+1 = QH



With correct choice of shift we getquadratic convergence , andif A = AH we can get approximatelycubic convergence .


Shifting, ctd

Remarks:Ak+1 = QH



With correct choice of shift we getquadratic convergence , andif A = AH we can get approximatelycubic convergence .

If a(k)n,n−1 is small, thena(k)

nn is close to an eigenvalue.

So one possible choice forµ is a(k)nn . This shift is called the

Rayleigh quotient shift .


Example: no shifting

>> A = [8 1; -2 1];>> A1 = A;

>> [Q1,R1] = qr(A1);>> A2 = R1 * Q1

A2 = 7.8235e+00 -2.7059e+002.9412e-01 1.1765e+00

>> [Q2,R2] = qr(A2);>> A3 = R2 *Q2

A3 = 7.7236e+00 2.9520e+00-4.7985e-02 1.2764e+00

>> [Q3,R3] = qr(A3);>> A4 = R3 *Q3

A4 = 7.7053e+00 -2.9920e+00 EIGENVALUE PROBLEMS – p. 26/45

Example: shifting

>> A1 = A;>> I = eye (2);

>> [Q1,R1] = qr(A1 - A1(2,2)*I);

>> A2 = R1 * Q1 + A1(2,2)*IA2 = 7.7358e+00 -2.9245e+00

7.5472e-02 1.2642e+00

>> [Q2,R2] = qr(A2 - A2(2,2)*I);

>> A3 = R2 * Q2 + A2(2,2)*IA3 = 7.7017e+00 2.9996e+00

-3.9768e-04 1.2983e+00

>> [Q3,R3] = qr(A3 - A3(2,2)*I);EIGENVALUE PROBLEMS – p. 27/45

Deflation Technique

Whena(k)n,n−1 is small enough, wedeflate by ignoring the last

row and column.× × × × ×

× × × × ×

× × × ×

× × ×

0 ×

Note the remaining eigenvalues ofA are those ofAk(1 :n − 1, 1:n − 1).

Apply the QR algorithm toAk(1 :n − 1, 1:n − 1).


Deflation Technique, ctd

Note: During iterations, it may happen that

somea(k)i+1,i other thana(k)

n−1,n becomes very small.Then we just regard it as zero, and work with

Ak(1 : i, 1: i), Ak(i + 1:n, i + 1:n)


Special caseA = AH

Q: What does the upper Hessenberg form become?


Special caseA = AH


Tridiagonal, costs4n3/3 flops if A is real.


Special caseA = AH



Q: Is it preserved in theQR algorithm?


Special caseA = AH




Yes

Q: What is the cost perQR step ifA is real ?


Special caseA = AH




Yes


12n flops.


Special caseA = AH




Yes


12n flops.

It has cubic convergence and the eigenvectors are immediatelyavailable.QHAQ −→ D diagonal, so the eigenvectors are thecolumns ofQ.


Numerical Difficulty with Shifting

FormA − µI with e.g.µ = ann. Suppose

A =

10−6 10−4

10−4 1 102

102 1016

fl(A − 1016I) =

−1016 10−4 0

10−4 −1016 102

0 102 0

It loses information unnecessarily.


Numerical Difficulty with Shifting

After one QR step,

A2 ≈

0 10−4 −3.9 ∗ 10−34

10−4 0 0

0 10−26 1016

leading to the eignevalues−10−4, 10−4, 1016.

More accuare computed eignevluaes by MATLAB are:9.9 ∗ 10−7, 1, 1016.

We can useimplicit shifts in order to avoid this loss.


Implicitly shifted QR alg. for real unsym A

A real unsymmetricA usually has complex conjugate pairs ofeigenvelues.




A complexµ in QH(A − µI) requirescomplex arithmetic.





But for a real matrix, we would like to avoid complexarithmetic as much as possible.






When the QR algorithm converges to a complex conjugate pair,we find the(n − 1, n − 2) entry converges to 0, and eventuallywe can deflate.







Q: How do we deflate here?







Q: How do we deflate here?

Ignore the last 2 rows and columns.



When the(n − 1, n − 2) entry is small, the eigenvaluesµ1, µ2

of the bottom right hand corner2 × 2 block are goodapproximations to eigenvalues ofA.



When the(n − 1, n − 2) entry is small, the eigenvaluesµ1, µ2

of the bottom right hand corner2 × 2 block are goodapproximations to eigenvalues ofA.

If they are a complex conjugate pair, we could do one QR stepwith µ1, the next withµ2 = µ1.



SupposeA1 is real upper Hessenberg. One step of double QRwith explicit shiftsµ1 andµ2 is

A1 − µ1I = Q1R1, A2 = R1Q1 + µ1I,

A2 − µ2I = Q2R2, A3 = R2Q2 + µ2I.



SupposeA1 is real upper Hessenberg. One step of double QRwith explicit shiftsµ1 andµ2 is

A1 − µ1I = Q1R1, A2 = R1Q1 + µ1I,

A2 − µ2I = Q2R2, A3 = R2Q2 + µ2I.

Two drawbacks:

AlthoughA1 is real, complex arithmetic is involved ifµ1

andµ2 are complex.

Explicit shifting may cause numerical difficulties.



Q. Show that

1. A3 = QH2 A2Q2 = (Q1Q2)

HA1(Q1Q2).



Q. Show that

1. A3 = QH2 A2Q2 = (Q1Q2)

HA1(Q1Q2).

2. N ≡ (A1 − µ1I)(A1 − µ2I) is real,N = Q1Q2R2R1.



Q. Show that

1. A3 = QH2 A2Q2 = (Q1Q2)

HA1(Q1Q2).


3. Nij = 0 for i ≥ j + 3, i.e.,N has two nonzerosubdiagonals.



Q. Show that

1. A3 = QH2 A2Q2 = (Q1Q2)

HA1(Q1Q2).



SinceN is real, we can choose the Q-factorQ1Q2 of its QRfactorization to be real. So we can obtain the QR factorizationof N to getQ1Q2 and then use it to obtainA3 — avoidingcomplex arithmetic.



Q. Show that

1. A3 = QH2 A2Q2 = (Q1Q2)

HA1(Q1Q2).




But there are two problems:i. FormingN costsO(n3)—too expensive;



Q. Show that

1. A3 = QH2 A2Q2 = (Q1Q2)

HA1(Q1Q2).




But there are two problems:i. FormingN costsO(n3)—too expensive;ii. Essentially explicit shifting is still involved.


One step of double QR iteration

We can avoid these two problems by the following algorithm.



We can avoid these two problems by the following algorithm.Do the 1st step of Householder QR ofN : N = Q1Q2R2R1.




Computen1 = Ne1 = [τ, σ, ν, 0, · · · , 0]T .




Computen1 = Ne1 = [τ, σ, ν, 0, · · · , 0]T .Design a real Householder transformationH0 such that

HT0 n1 =

× × ×

× × ×

× × ×

1. . .

1

×

×

×

0...0

=

×

0

...

0




Computen1 = Ne1 = [τ, σ, ν, 0, · · · , 0]T .Design a real Householder transformationH0 such that

HT0 n1 =

× × ×

× × ×

× × ×

1. . .

1

×

×

×

0...0

=

×

0

...

0

NoteH0e1 = (Q1Q2)e1.


One step of double QR iteration, ctd

Apply HT0 & H0 to A1 from left and right, respectively:

HT0 A1H0 =

× × × × × ×

× × × × × ×

2 × × × × ×

2 2 × × × ×

× × ×

× ×



Apply HT0 & H0 to A1 from left and right, respectively:

HT0 A1H0 =

× × × × × ×

× × × × × ×

2 × × × × ×

2 2 × × × ×

× × ×

× ×

Then use real Householder transformationsH1, . . . , Hn−2 totransformHT

0 A1H0 back to upper Hessenberg form:

A3 = HTn−2 · · ·H

T1 HT

0 A1H0H1 · · ·Hn−2



Remember what we want isA3. But A3 is essentially justA3

due to the following result:



Remember what we want isA3. But A3 is essentially justA3

due to the following result:

The Implicit Q-Theorem .SupposeQ andP are two real orthogonal matrices such thatQTAQ andP TAP are both upper Hessenberg matrices, oneof which is unreduced (i.e., all of its subdiagonal entries arenonzero). If

Qe1 = ±Pe1,

thenQej = ±Pej, j = 2, . . . , n

and

QTAQ = D−1P TAPD, D = diag(±1, . . . ,±1)



Two Hessenberg reductions:

A3 = (Q1Q2)TA1(Q1Q2)

A3 = (H0H1 · · ·Hn−2)TA1(H0H1 · · ·Hn−2)




A3 = (Q1Q2)TA1(Q1Q2)

A3 = (H0H1 · · ·Hn−2)TA1(H0H1 · · ·Hn−2)

Q. Show(Q1Q2)e1 = (H0H1 · · ·Hn−2)e1.




A3 = (Q1Q2)TA1(Q1Q2)

A3 = (H0H1 · · ·Hn−2)TA1(H0H1 · · ·Hn−2)

Q. Show(Q1Q2)e1 = (H0H1 · · ·Hn−2)e1.

Q. SupposeA1 is unreduced Hessenberg andµ1 andµ2 are notits eigenvalues. Show thatA3 is unreduced Hessenberg.




A3 = (Q1Q2)TA1(Q1Q2)

A3 = (H0H1 · · ·Hn−2)TA1(H0H1 · · ·Hn−2)

Q. Show(Q1Q2)e1 = (H0H1 · · ·Hn−2)e1.

Q. SupposeA1 is unreduced Hessenberg andµ1 andµ2 are notits eigenvalues. Show thatA3 is unreduced Hessenberg.

Conclusion : A3 = D−1A3D.


QR Algorithm

GivenA ∈ Rn×n and a tolerancetol greater than the unit

roundoff, this algorithm computes the real SchurdecompositionQTAQ = R.

If Q andR are desired, thenR is stored inA.

If only the eigenvalues are desired, then diagonal blocks inRare stored in the corresponding positions inA.


1. Compute the Hessenberg reduction

A := QT AQ where Q = H1 · · ·Hn−2.

If the final Q is desired, form Q := H1 · · ·Hn−2.

2. until q = n

Set to zero all subdiagonal entries that satisfy:

|ai,i−1| ≤ tol(|aii| + |ai−1,i−1|).

Find the largest non-negative q and the smallest

non-negative p such that

A =

p n − p − q q

A11 A12 A13

0 A22 A23

0 0 A33

p

n − p − q

q

where A33 is upper quasi-triangular and A22 is

unreduced. (Note: either p and q may be zero.)

if q < n

Perform one step of double QR iteration on A22:

A22 := ZT A22Z

if Q is desired

Q := Qdiag(Ip, Z, Iq), A12 := A12Z , A23 := ZT A23

endend

end3. Upper triangularize all 2-by-2 diagonal blocks in A

that have real eigenvalues and accumulate

the transformations if necessary.

41-1

Inverse iteration for eigenvectors

Inverse Iteration : GivenA ∈ Cn×n.

Let 0 < |λj − µ| ≪ |λi − µ| (i 6= j).Chooseq0 with ‖q0‖2 = 1.for k = 1, 2, . . .

Solve(A − µI)zk = qk−1

qk = zk/‖zk‖2

Stop if‖(A − µI)qk‖2 ≤ cu‖A‖2.end


Inverse Iteration for eigenvectors, ctd

1. Solve(A − µI)zk = qk−1 by LU with PP, & replace any|pivot| < u‖A‖ by u‖A‖, this works even whenA − µIis singular. Ill-condition ofA − µI does not spoilcomputation.




2. When it stopsµ andqk is an exact eigenpair for a nearbyproblem: (A + Ek)qk = µqk,whereEk ≡ −rkq

Tk with rk ≡ (A − µI)qk .




2. When it stopsµ andqk is an exact eigenpair for a nearbyproblem: (A + Ek)qk = µqk,whereEk ≡ −rkq

Tk with rk ≡ (A − µI)qk .

3. When aknown eigenvalue is used asµ, usually only onestep is needed. If one step does not give desired result, startwith a new initial vector.


Convergence Analysis

AssumeA is nondefective,AX = Xdiag(λi).



AssumeA is nondefective,AX = Xdiag(λi).Let q0 = Xa =

∑ni=1 aixi, where we assumeaj 6= 0.




∑ni=1 aixi, where we assumeaj 6= 0. Thus

(A − µI)−kq0 = (A − µI)−k

n∑

i=1

aixi =n

∑

i=1

ai

1

(λi − µ)kxi

=1

(λj − µ)k

[

ajxj +∑

i6=j

ai

(λj − µ

λi − µ

)k

xi

]

.





(A − µI)−kq0 = (A − µI)−k

n∑

i=1

aixi =n

∑

i=1

ai

1

(λi − µ)kxi

=1

(λj − µ)k

[

ajxj +∑

i6=j

ai

(λj − µ

λi − µ

)k

xi

]

.

Since|λj − µ| ≪ |λi − µ|,

qk =(A − µI)−kq0

‖(A − µ)−kq0‖2→ ±

xj

‖xj‖2ask → ∞,





(A − µI)−kq0 = (A − µI)−k

n∑

i=1

aixi =n

∑

i=1

ai

1

(λi − µ)kxi

=1

(λj − µ)k

[

ajxj +∑

i6=j

ai

(λj − µ

λi − µ

)k

xi

]

.

Since|λj − µ| ≪ |λi − µ|,

qk =(A − µI)−kq0

‖(A − µ)−kq0‖2→ ±

xj

‖xj‖2ask → ∞,

i.e. qk rapidly converges to the eigenvector ofA.


Computing an eigenvector after QR alg

1. QR algorithm→ {λi}

2. Apply the inverse iteration to HessenbergA1 = QT0 AQ0,

to find an eiegnvectoryi of A1 corresponding toλi.

Inverse iteration withA1 is economical:

solving(A1 − λiI)zk = qk−1 costsO(n2) flops.

3. Letxi = Q0yi. Then

Axi = λixi, ‖xi‖2 = 1,

i.e. xi is a unit eigenvector ofA corresponding toλi.


eigenvalue problems - cs.mcgill.cachang/teaching/cs540/loginpage/lecs540/... · is...

Documents