![Page 1: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/1.jpg)
MATH 685/ CSI 700/ OR 682 Lecture Notes
Lecture 6.
Eigenvalue problems.
![Page 2: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/2.jpg)
Eigenvalue problems Eigenvalue problems occur in many areas of science
and engineering, such as structural analysis
Eigenvalues are also important in analyzing numerical methods
Theory and algorithms apply to complex matrices as well as real matrices
With complex matrices, we use conjugate transpose, AH, instead of usual transpose, AT
![Page 3: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/3.jpg)
Formulation
Matrix expands or shrinks any vector lying in direction of eigenvector by scalar factor
Expansion or contraction factor is given by corresponding eigenvalue
Eigenvalues and eigenvectors decompose complicated behavior of general linear transformation into simpler actions
![Page 4: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/4.jpg)
Examples
![Page 5: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/5.jpg)
Characteristic polynomial
![Page 6: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/6.jpg)
Example
![Page 7: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/7.jpg)
Companion matrix
![Page 8: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/8.jpg)
Characteristic polynomial
Computing eigenvalues using characteristic polynomial is not recommended because of work in computing coefficients of characteristic polynomial sensitivity of coefficients of characteristic polynomial work in solving for roots of characteristic polynomial
Characteristic polynomial is powerful theoretical tool but usually not useful computationally
![Page 9: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/9.jpg)
Example
![Page 10: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/10.jpg)
Diagonalizability
![Page 11: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/11.jpg)
Eigenspaces
![Page 12: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/12.jpg)
Some relevant definitions
![Page 13: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/13.jpg)
Examples
![Page 14: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/14.jpg)
Examples
![Page 15: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/15.jpg)
Properties of eigenvalue problems
Properties of eigenvalue problem affecting choice of algorithm and software Are all eigenvalues needed, or only a few? Are only eigenvalues needed, or are corresponding eigenvectors
also needed? Is matrix real or complex? Is matrix relatively small and dense, or large and sparse? Does matrix have any special properties, such as symmetry, or is
it general matrix? Condition of eigenvalue problem is sensitivity of
eigenvalues and eigenvectors to changes in matrix Conditioning of eigenvalue problem is not same as
conditioning of solution to linear system for same matrix Different eigenvalues and eigenvectors are not
necessarily equally sensitive to perturbations in matrix
![Page 16: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/16.jpg)
Conditioning of eigenvalues
![Page 17: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/17.jpg)
Conditioning of eigenvalues
![Page 18: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/18.jpg)
Problem transformations
![Page 19: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/19.jpg)
Similarity transformation
![Page 20: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/20.jpg)
Similarity transformation
![Page 21: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/21.jpg)
Diagonal form Eigenvalues of diagonal matrix are diagonal entries, and
eigenvectors are columns of identity matrix
Diagonal form is desirable in simplifying eigenvalue problems for general matrices by similarity transformations
But not all matrices are diagonalizable by similarity transformation
Closest one can get, in general, is Jordan form, which is nearly diagonal but may have some nonzero entries on first superdiagonal, corresponding to one or more multiple eigenvalues
![Page 22: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/22.jpg)
Triangular form
![Page 23: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/23.jpg)
Block triangular form
![Page 24: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/24.jpg)
Forms attainable by similarity
![Page 25: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/25.jpg)
Power iteration
![Page 26: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/26.jpg)
Convergence of Power iteration
![Page 27: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/27.jpg)
Example
![Page 28: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/28.jpg)
Limitations
![Page 29: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/29.jpg)
Normalized Power iteration
![Page 30: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/30.jpg)
Example
![Page 31: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/31.jpg)
Geometric interpretation
![Page 32: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/32.jpg)
Power Iteration with Shift
In earlier example, for instance, if we pick shift of σ = 1, (which is equal to other eigenvalue) then ratio becomes zero and method converges in one iteration
In general, we would not be able to make such fortuitous choice, but shifts can still be extremely useful in some contexts, as we will see later
![Page 33: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/33.jpg)
Inverse iteration
Inverse iteration converges to eigenvector corresponding to smallest eigenvalue of A. Eigenvalue obtained is dominant eigenvalue of A−1, and hence its reciprocal is smallest eigenvalue of A in modulus
![Page 34: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/34.jpg)
Example
![Page 35: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/35.jpg)
Shifted inverse iteration
![Page 36: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/36.jpg)
Rayleigh Quotient
![Page 37: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/37.jpg)
Example
![Page 38: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/38.jpg)
Rayleigh Quotient iteration
![Page 39: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/39.jpg)
Rayleigh Quotient iteration
![Page 40: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/40.jpg)
Deflation
![Page 41: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/41.jpg)
Deflation
![Page 42: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/42.jpg)
Deflation
![Page 43: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/43.jpg)
Simultaneous Iteration
![Page 44: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/44.jpg)
Orthogonal iteration
![Page 45: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/45.jpg)
QR iteration
![Page 46: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/46.jpg)
QR iteration
![Page 47: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/47.jpg)
Example
![Page 48: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/48.jpg)
QR iteration with shifts
![Page 49: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/49.jpg)
Example
![Page 50: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/50.jpg)
Preliminary reduction
![Page 51: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/51.jpg)
Preliminary reduction
![Page 52: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/52.jpg)
Preliminary reduction
![Page 53: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/53.jpg)
Cost of QR iteration
![Page 54: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/54.jpg)
Krylov subspaces methods
![Page 55: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/55.jpg)
Krylov subspaces methods
![Page 56: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/56.jpg)
Arnoldi iteration
![Page 57: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/57.jpg)
Arnoldi iteration
![Page 58: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/58.jpg)
Arnoldi iteration
![Page 59: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/59.jpg)
Lanczos Iteration
![Page 60: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/60.jpg)
Lanczos iteration
![Page 61: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/61.jpg)
Lanczos iteration
![Page 62: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/62.jpg)
Krylov subspace methods cont.
![Page 63: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/63.jpg)
Example of Lanczos iteration
![Page 64: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/64.jpg)
Jacobi method
![Page 65: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/65.jpg)
Jacobi method
![Page 66: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/66.jpg)
Plane rotation
![Page 67: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/67.jpg)
Jacobi method cont.
![Page 68: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/68.jpg)
Jacobi method cont.
![Page 69: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/69.jpg)
Jacobi method example
![Page 70: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/70.jpg)
![Page 71: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/71.jpg)
Process continues until off-diagonal entries reduced to as small as desired
Result is diagonal matrix orthogonally similar to original matrix, with the orthogonal similarity transformation given by product of plane rotations
![Page 72: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/72.jpg)
Other methods (spectrum-slicing)
![Page 73: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/73.jpg)
Sturm sequence
![Page 74: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/74.jpg)
Divide-and-conquer algorithm
![Page 75: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/75.jpg)
Relatively robust representation
![Page 76: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/76.jpg)
Generalized eigenvalue problems
![Page 77: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/77.jpg)
QZ algorithm
![Page 78: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d9c5503460f94a847c1/html5/thumbnails/78.jpg)
Computing SVD