research matters nick higham february 25, 2009 …higham/talks/ecm12...the theory of matrices...
TRANSCRIPT
![Page 1: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/1.jpg)
Research Matters
February 25, 2009
Nick HighamDirector of Research
School of Mathematics
1 / 6
The Matrix Logarithm:from Theory to Computation
Nick HighamSchool of Mathematics
The University of Manchester
[email protected]://www.ma.man.ac.uk/~higham/
6th European Congress of Mathematics, July 2012
![Page 2: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/2.jpg)
Definition Applications Theory Methods
Matrix Logarithm
DefinitionA logarithm of A ∈ Cn×n is any matrix X such that eX = A.
Implicit definition.Properties, classification?
University of Manchester Nick Higham Matrix Logarithm 2 / 40
![Page 3: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/3.jpg)
Definition Applications Theory Methods
Outline
1 Definition and Properties
2 Applications
3 Theory
4 Computing the Matrix Logarithm andits Fréchet derivative
University of Manchester Nick Higham Matrix Logarithm 3 / 40
![Page 4: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/4.jpg)
Definition Applications Theory Methods
Cayley and Sylvester
Matrix algebra developed by ArthurCayley, FRS (1821–1895) in Memoir onthe Theory of Matrices (1858).
Cayley considered matrix squareroots.
Term “matrix” coined in 1850 by JamesJoseph Sylvester, FRS (1814–1897).
Gave (1883) first definition of f (A)for general f .
University of Manchester Nick Higham Matrix Logarithm 4 / 40
![Page 5: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/5.jpg)
Definition Applications Theory Methods
Multiplicity of Definitions
There have been proposed in the literature since 1880eight distinct definitions of a matric function,
by Weyr, Sylvester and Buchheim,Giorgi, Cartan, Fantappiè, Cipolla,
Schwerdtfeger and Richter.
— R. F. Rinehart,The Equivalence of Definitions
of a Matric Function (1955)
University of Manchester Nick Higham Matrix Logarithm 5 / 40
![Page 6: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/6.jpg)
Definition Applications Theory Methods
Multiplicity of Definitions
There have been proposed in the literature since 1880eight distinct definitions of a matric function,
by Weyr, Sylvester and Buchheim,Giorgi, Cartan, Fantappiè, Cipolla,
Schwerdtfeger and Richter.
— R. F. Rinehart,The Equivalence of Definitions
of a Matric Function (1955)
University of Manchester Nick Higham Matrix Logarithm 5 / 40
![Page 7: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/7.jpg)
Definition Applications Theory Methods
Jordan Canonical Form
Z−1AZ = J = diag(J1, . . . , Jp), Jk︸︷︷︸mk×mk
=
λk 1
λk. . .. . . 1
λk
Definition
f (A) = Zf (J)Z−1 = Zdiag(f (Jk))Z−1,
f (Jk) =
f (λk) f ′(λk) . . .
f (mk−1))(λk)
(mk − 1)!
f (λk). . . .... . . f ′(λk)
f (λk)
.
University of Manchester Nick Higham Matrix Logarithm 6 / 40
![Page 8: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/8.jpg)
Definition Applications Theory Methods
Primary and Nonprimary Logarithms
A = diag(1,1,e,e).
Primary: log(A) = diag(0,0,1,1).
Nonprimary: log(A) = diag(0,2πi ,1,1).
University of Manchester Nick Higham Matrix Logarithm 7 / 40
![Page 9: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/9.jpg)
Definition Applications Theory Methods
Cauchy Integral Theorem
Definition
f (A) =1
2πi
∫Γ
f (z)(zI − A)−1 dz,
where f is analytic on and inside a closed contour Γ thatencloses λ(A).
University of Manchester Nick Higham Matrix Logarithm 8 / 40
![Page 10: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/10.jpg)
Definition Applications Theory Methods
Mercator’s Series
By integrating (1 + t)−1 = 1− t + t2 − t3 + · · · between 0and x we obtain Mercator’s series (1668),
log(1 + x) = x − x2
2+
x3
3− x4
4+ · · · , |x | < 1.
For A ∈ Cn×n,
log(I + A) = A− A2
2+
A3
3− A4
4+ · · · , ρ(A) < 1.
University of Manchester Nick Higham Matrix Logarithm 10 / 40
![Page 11: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/11.jpg)
Definition Applications Theory Methods
Composite Functions
Theoremf (t) = g(h(t)) ⇒ f (A) = g(h(A)).
Corollaryexp(log(A)) = A when log(A) is defined.
University of Manchester Nick Higham Matrix Logarithm 11 / 40
![Page 12: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/12.jpg)
Definition Applications Theory Methods
Composite Functions
Theoremf (t) = g(h(t)) ⇒ f (A) = g(h(A)).
Corollaryexp(log(A)) = A when log(A) is defined.
What about log(exp(A)) = A?
Matrix unwinding number
U(A) = A− log(exp(A))2πi
.
University of Manchester Nick Higham Matrix Logarithm 11 / 40
![Page 13: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/13.jpg)
Definition Applications Theory Methods
Outline
1 Definition and Properties
2 Applications
3 Theory
4 Computing the Matrix Logarithm andits Fréchet derivative
University of Manchester Nick Higham Matrix Logarithm 12 / 40
![Page 14: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/14.jpg)
Definition Applications Theory Methods
Toolbox of Matrix Functions
d2ydt2 + Ay = 0, y(0) = y0, y ′(0) = y ′0
has solution
y(t) = cos(√
At)y0 +(√
A)−1 sin(
√At)y ′0.
But [y ′
y
]= exp
([0 −tA
t In 0
])[y ′0y0
].
In software want to be able evaluate interesting f atmatrix args as well as scalar args.MATLAB has expm, logm, sqrtm, funm.
University of Manchester Nick Higham Matrix Logarithm 13 / 40
![Page 15: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/15.jpg)
Definition Applications Theory Methods
Toolbox of Matrix Functions
d2ydt2 + Ay = 0, y(0) = y0, y ′(0) = y ′0
has solution
y(t) = cos(√
At)y0 +(√
A)−1 sin(
√At)y ′0.
But [y ′
y
]= exp
([0 −tA
t In 0
])[y ′0y0
].
In software want to be able evaluate interesting f atmatrix args as well as scalar args.MATLAB has expm, logm, sqrtm, funm.
University of Manchester Nick Higham Matrix Logarithm 13 / 40
![Page 16: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/16.jpg)
Definition Applications Theory Methods
Toolbox of Matrix Functions
d2ydt2 + Ay = 0, y(0) = y0, y ′(0) = y ′0
has solution
y(t) = cos(√
At)y0 +(√
A)−1 sin(
√At)y ′0.
But [y ′
y
]= exp
([0 −tA
t In 0
])[y ′0y0
].
In software want to be able evaluate interesting f atmatrix args as well as scalar args.MATLAB has expm, logm, sqrtm, funm.
University of Manchester Nick Higham Matrix Logarithm 13 / 40
![Page 17: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/17.jpg)
Definition Applications Theory Methods
Application: Control Theory
Convert continuous-time system
dxdt
= Fx(t) + Gu(t),
y = Hx(t) + Ju(t),
to discrete-time state-space system
xk+1 = Axk + Buk ,
yk = Hxk + Juk .
HaveA = eFτ , B =
(∫ τ
0eFtdt
)G,
where τ is the sampling period.MATLAB Control System Toolbox: c2d and d2c.
University of Manchester Nick Higham Matrix Logarithm 14 / 40
![Page 18: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/18.jpg)
Definition Applications Theory Methods
The Average Eye
First order character of optical system characterized bytransference matrix
T =
[S δ0 1
]∈ R5×5,
where S ∈ R4×4 is symplectic:
ST JS = J =
[0 I2−I2 0
].
Average m−1∑mi=1 Ti is not a transference matrix.
Harris (2005) proposes the average exp(m−1∑mi=1 log(Ti)).
University of Manchester Nick Higham Matrix Logarithm 15 / 40
![Page 19: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/19.jpg)
Definition Applications Theory Methods
Markov Models
Time-homogeneous continuous-time Markov processwith transition probability matrix P(t) ∈ Rn×n.Transition intensity matrix Q: qij ≥ 0 (i 6= j),∑n
j=1 qij = 0, P(t) = eQt .
For discrete-time Markov processes:
Embeddability problemWhen does a given stochastic P have a real logarithm Qthat is an intensity matrix?
University of Manchester Nick Higham Matrix Logarithm 16 / 40
![Page 20: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/20.jpg)
Definition Applications Theory Methods
Markov Models (1)—Example
With x = −e−2√
3π ≈ −1.9× 10−5,
P =13
1 + 2x 1− x 1− x1− x 1 + 2x 1− x1− x 1− x 1 + 2x
.P diagonalizable, Λ(P) = {1, x , x}.Every primary log complex (can’t have complexconjugate ei’vals).Yet a generator is the non-primary log
Q = 2√
3π
−2/3 1/2 1/61/6 −2/3 1/21/2 1/6 −2/3
.University of Manchester Nick Higham Matrix Logarithm 17 / 40
![Page 21: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/21.jpg)
Definition Applications Theory Methods
Markov Models (2)
Suppose P ≡ P(1) has a generator Q = log P.Then P(t) at other times is P(t) = exp(Qt).E.g., if P transition matrix for 1 year,
P(1/12) = e1
12 log P ≡ P1/12 is matrix for 1 month.
Problem: log P, P1/k may have wrong sign patterns⇒“regularize”.
University of Manchester Nick Higham Matrix Logarithm 18 / 40
![Page 22: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/22.jpg)
Definition Applications Theory Methods
HIV to Aids Transition
Estimated 6-month transition matrix.Four AIDS-free states and 1 AIDS state.2077 observations (Charitos et al., 2008).
P =
0.8149 0.0738 0.0586 0.0407 0.01200.5622 0.1752 0.1314 0.1169 0.01430.3606 0.1860 0.1521 0.2198 0.08150.1676 0.0636 0.1444 0.4652 0.1592
0 0 0 0 1
.Want to estimate the 1-month transition matrix.
Λ(P) = {1,0.9644,0.4980,0.1493,−0.0043}.N. J. Higham and L. Lin.
On pth roots of stochastic matrices, LAA, 2011.
University of Manchester Nick Higham Matrix Logarithm 19 / 40
![Page 23: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/23.jpg)
Definition Applications Theory Methods
Outline
1 Definition and Properties
2 Applications
3 Theory
4 Computing the Matrix Logarithm andits Fréchet derivative
University of Manchester Nick Higham Matrix Logarithm 20 / 40
![Page 24: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/24.jpg)
Definition Applications Theory Methods
Logs of A = I3
B =
0 0 00 0 00 0 0
,C =
0 2π − 1 1−2π 0 0−2π 0 0
, D =
0 2π 1−2π 0 0
0 0 0
,eB = eC = eD = I3.
Λ(C) = Λ(D) = {0,2πi ,−2πi}.
University of Manchester Nick Higham Matrix Logarithm 21 / 40
![Page 25: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/25.jpg)
Definition Applications Theory Methods
All Solutions of eX = A
Theorem (Gantmacher, 1959)
A ∈ Cn×n nonsing with Jordan canonical formZ−1AZ = J = diag(J1, J2, . . . , Jp). All solutions to eX = Aare given by
X = Z U diag(L(j1)1 ,L(j2)
2 , . . . ,L(jp)p ) U
−1Z−1,
whereL(jk )
k = log(Jk(λk)) + 2 jk π i Imk ,
jk ∈ Z arbitrary, and U an arbitrary nonsing matrix thatcommutes with J.
University of Manchester Nick Higham Matrix Logarithm 22 / 40
![Page 26: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/26.jpg)
Definition Applications Theory Methods
All Solutions of eX = A: Classified
TheoremA ∈ Cn×n nonsing: p Jordan blocks, s distinct ei’vals.eX = A has a countable infinity of solutions that are primaryfunctions of A:
Xj = Zdiag(L(j1)1 ,L(j2)
2 , . . . ,L(jp)p )Z−1,
where λi = λk implies ji = jk . If s < p then eX = A hasnon-primary solutions
Xj(U) = Z U diag(L(j1)1 ,L(j2)
2 , . . . ,L(jp)p ) U
−1Z−1,
where jk ∈ Z arbitrary, U arbitrary nonsing with UJ = JU,and for each j ∃ i and k s.t. λi = λk while ji 6= jk .
University of Manchester Nick Higham Matrix Logarithm 23 / 40
![Page 27: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/27.jpg)
Definition Applications Theory Methods
Logs of A = I3 (again)
C =
0 2π − 1 1−2π 0 0−2π 0 0
, D =
0 2π 1−2π 0 0
0 0 0
,e0 = eC = eD = I3. Λ(C) = Λ(D) = {0,2πi ,−2πi}.
U =
1 α 00 1 α0 0 1
, α ∈ C,
X = U diag(2πi ,−2πi ,0)U−1 = 2π i
1 −2α 2α2
0 1 −α0 0 1
.University of Manchester Nick Higham Matrix Logarithm 24 / 40
![Page 28: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/28.jpg)
Definition Applications Theory Methods
Square Roots of Rotations
G(θ) =
[cos θ sin θ− sin θ cos θ
].
G(θ/2) is the natural square root of G(θ).
For θ = π,
G(π) =
[−1 00 −1
], G(π/2) =
[0 1−1 0
].
G(π/2) is a nonprimary square root.
University of Manchester Nick Higham Matrix Logarithm 25 / 40
![Page 29: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/29.jpg)
Definition Applications Theory Methods
Principal Logarithm and pth Root
Let A ∈ Cn×n have no eigenvalues on R− .
Principal logX = log(A) denotes unique X such that
eX = A.−π < Im
(λ(X )
)< π.
Principal pth root
For integer p > 0, X = A1/p is unique X such thatX p = A.−π/p < arg(λ(X )) < π/p.
University of Manchester Nick Higham Matrix Logarithm 26 / 40
![Page 30: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/30.jpg)
Definition Applications Theory Methods
Principal Logarithm and pth Root
Let A ∈ Cn×n have no eigenvalues on R− .
Principal logX = log(A) denotes unique X such that
eX = A.−π < Im
(λ(X )
)< π.
Principal pth root
For integer p > 0, X = A1/p is unique X such thatX p = A.−π/p < arg(λ(X )) < π/p.
University of Manchester Nick Higham Matrix Logarithm 26 / 40
![Page 31: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/31.jpg)
Definition Applications Theory Methods
Outline
1 Definition and Properties
2 Applications
3 Theory
4 Computing the Matrix Logarithm andits Fréchet derivative
University of Manchester Nick Higham Matrix Logarithm 27 / 40
![Page 32: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/32.jpg)
Definition Applications Theory Methods
Henry Briggs (1561–1630)
Arithmetica Logarithmica (1624)Logarithms to base 10 of 1–20,000 and90,000–100,000 to 14 decimal places.
Briggs must be viewed as one of thegreat figures in numerical analysis.
—Herman H. Goldstine,A History of Numerical Analysis (1977)
University of Manchester Nick Higham Matrix Logarithm 28 / 40
![Page 33: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/33.jpg)
Definition Applications Theory Methods
Henry Briggs (1561–1630)
Arithmetica Logarithmica (1624)Logarithms to base 10 of 1–20,000 and90,000–100,000 to 14 decimal places.
Briggs must be viewed as one of thegreat figures in numerical analysis.
—Herman H. Goldstine,A History of Numerical Analysis (1977)
University of Manchester Nick Higham Matrix Logarithm 28 / 40
![Page 34: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/34.jpg)
![Page 35: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/35.jpg)
![Page 36: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/36.jpg)
Definition Applications Theory Methods
Briggs’ Log Method (1617)
log(ab) = log a + log b ⇒ log a = 2 log a1/2.
Use repeatedly:log a = 2k log a1/2k
.
Write a1/2k= 1 + x and note log(1 + x) ≈ x . Briggs worked
to base 10 and used
log10 a ≈ 2k · log10 e · (a1/2k − 1).
University of Manchester Nick Higham Matrix Logarithm 31 / 40
![Page 37: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/37.jpg)
Definition Applications Theory Methods
When Does log(BC) = log(B) + log(C)?
TheoremLet B,C ∈ Cn×n commute and have no ei’vals on R−. If forevery ei’val λj of B and the corr. ei’val µj of C,|argλj + argµj | < π, then log(BC) = log(B) + log(C).
Proof. log(B) and log(C) commute, since B and C do.Therefore
elog(B)+log(C) = elog(B)elog(C) = BC.
Thus log(B) + log(C) is some logarithm of BC. Then
Im(logλj + logµj) = argλj + argµj ∈ (−π, π),
so log(B) + log(C) is the principal logarithm of BC.
University of Manchester Nick Higham Matrix Logarithm 32 / 40
![Page 38: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/38.jpg)
Definition Applications Theory Methods
When Does log(BC) = log(B) + log(C)?
TheoremLet B,C ∈ Cn×n commute and have no ei’vals on R−. If forevery ei’val λj of B and the corr. ei’val µj of C,|argλj + argµj | < π, then log(BC) = log(B) + log(C).
Proof. log(B) and log(C) commute, since B and C do.Therefore
elog(B)+log(C) = elog(B)elog(C) = BC.
Thus log(B) + log(C) is some logarithm of BC. Then
Im(logλj + logµj) = argλj + argµj ∈ (−π, π),
so log(B) + log(C) is the principal logarithm of BC.
University of Manchester Nick Higham Matrix Logarithm 32 / 40
![Page 39: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/39.jpg)
Definition Applications Theory Methods
Inverse Scaling and Squaring Method
Take B = C in previous theorem:
log A = log(A1/2 · A1/2) = 2 log(A1/2),
since argλ(A1/2) ∈ (−π/2, π/2).
Use Briggs’ idea: log A = 2k log(A1/2k)
.
Kenney & Laub’s (1989) inverse scaling and squaringmethod:
Bring A close to I by repeated square roots.Approximate log(A1/2s
) using an [m/m] Padéapproximant rm(x) ≈ log(1 + x).Rescale to find log(A).
University of Manchester Nick Higham Matrix Logarithm 33 / 40
![Page 40: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/40.jpg)
Definition Applications Theory Methods
Inverse Scaling and Squaring Method
Take B = C in previous theorem:
log A = log(A1/2 · A1/2) = 2 log(A1/2),
since argλ(A1/2) ∈ (−π/2, π/2).
Use Briggs’ idea: log A = 2k log(A1/2k)
.
Kenney & Laub’s (1989) inverse scaling and squaringmethod:
Bring A close to I by repeated square roots.Approximate log(A1/2s
) using an [m/m] Padéapproximant rm(x) ≈ log(1 + x).Rescale to find log(A).
University of Manchester Nick Higham Matrix Logarithm 33 / 40
![Page 41: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/41.jpg)
Definition Applications Theory Methods
Inverse Scaling and Squaring Method
Take B = C in previous theorem:
log A = log(A1/2 · A1/2) = 2 log(A1/2),
since argλ(A1/2) ∈ (−π/2, π/2).
Use Briggs’ idea: log A = 2k log(A1/2k)
.
Kenney & Laub’s (1989) inverse scaling and squaringmethod:
Bring A close to I by repeated square roots.Approximate log(A1/2s
) using an [m/m] Padéapproximant rm(x) ≈ log(1 + x).Rescale to find log(A).
University of Manchester Nick Higham Matrix Logarithm 33 / 40
![Page 42: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/42.jpg)
Definition Applications Theory Methods
Choice of Parameters s, m
Must have ‖I − A1/2s‖ < 1.
Larger Padé degree m means smaller s.
Let h2m+1(X ) = erm(X) − X − I.Assume ρ(rm(X )) < π, so log(erm(X)) = rm(X ). Then
rm(X ) = log(I + X + h2m+1(X )) =: log(I + X +∆X ),
where
h2m+1(X ) =∞∑
k=2m+1
ckX k .
University of Manchester Nick Higham Matrix Logarithm 34 / 40
![Page 43: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/43.jpg)
Definition Applications Theory Methods
Bounding the Backward Error
Want to bound norm of h2m+1(X ) =∑∞
k=2m+1 ckX k .
Non-normality implies ρ(A)� ‖A‖.
Note that
ρ(A) ≤ ‖Ak‖1/k ≤ ‖A‖, k = 1 : ∞.
and limk→∞ ‖Ak‖1/k = ρ(A).
Use ‖Ak‖1/k instead of ‖A‖ in the truncation bounds.
University of Manchester Nick Higham Matrix Logarithm 35 / 40
![Page 44: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/44.jpg)
A =
[0.9 5000 −0.5
].
0 5 10 15 2010
0
102
104
106
108
1010
‖A‖k2‖Ak‖2(‖A5‖1/52 )k
(‖A10‖1/102 )k
‖Ak‖1/k2
![Page 45: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/45.jpg)
Definition Applications Theory Methods
Algorithm of Al-Mohy & H (2012)
Truncation bounds use ‖Ak‖1/k rather than ‖A‖, leadingto major benefits in speed and accuracy.Matrix norms not such a blunt tool!Use estimates of ‖Ak‖ (alg of H & Tisseur (2000)).Choose s and m to achieve double precision backwarderror at minimal cost.Initial Schur decomposition: A = QTQ∗.Directly and accurately compute certain elements ofT 1/2s − I and log(T ). Use
a1/2s − 1 =a− 1∏s
i=1(1 + a1/2i ).
University of Manchester Nick Higham Matrix Logarithm 37 / 40
![Page 46: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/46.jpg)
Definition Applications Theory Methods
Frechét Derivative of Logarithm
f (A + E)− f (A)− L(A,E) = o(‖E‖).Integral formula
L(A,E) =
∫ 1
0
(t(A− I) + I
)−1E(t(A− I) + I
)−1 dt .
Method based on
f([
X E0 X
])=
[f (X ) L(X ,E)
0 f (X )
].
Kenney & Laub (1998): Kronecker–Sylvester alg,Padé of tanh(x)/x . Requires complex arithmetic.
University of Manchester Nick Higham Matrix Logarithm 38 / 40
![Page 47: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/47.jpg)
Definition Applications Theory Methods
Algorithm of Al-Mohy, H & Relton (2012)
Fréchet differentiate the ISS algorithm!
1 E0 = E2 for i = 1: s3 Compute A1/2i .4 Solve the Sylvester eqn A1/2i Ei + EiA1/2i
= Ei−1.5 end6 log(A) ≈ 2srm(A1/2s − I)7 Llog(A,E) ≈ 2sLrm(A1/2s − I,Es)
Backward Error Result
rm(X ) = log(I + X +∆X ),
Lrm(X ,E) = Llog(I + X +∆X ,E +∆E).
University of Manchester Nick Higham Matrix Logarithm 39 / 40
![Page 48: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/48.jpg)
Definition Applications Theory Methods
Conclusions & Future Directions
Log appears in a growing number of applications.Have good algorithms for log(A), Llog(A) and estimatingthe condition number.If A is real can work entirely in real arithmetic.
Conditioning of f (A).Non-primary functions.Functions ofstructured matrices.
University of Manchester Nick Higham Matrix Logarithm 40 / 40
![Page 49: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/49.jpg)
Definition Applications Theory Methods
References I
A. H. Al-Mohy and N. J. Higham.A new scaling and squaring algorithm for the matrixexponential.SIAM J. Matrix Anal. Appl., 31(3):970–989, 2009.
A. H. Al-Mohy and N. J. Higham.Improved inverse scaling and squaring algorithms forthe matrix logarithm.SIAM J. Sci. Comput., 34(4):C152–C169, 2012.
University of Manchester Nick Higham Matrix Logarithm 36 / 40
![Page 50: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/50.jpg)
Definition Applications Theory Methods
References II
A. H. Al-Mohy, N. J. Higham, and S. D. Relton.Computing the fréchet derivative of the matrix logarithmand estimating the condition number.MIMS EPrint 2012.72, Manchester Institute forMathematical Sciences, The University of Manchester,UK, July 2012.17 pp.
T. Charitos, P. R. de Waal, and L. C. van der Gaag.Computing short-interval transition matrices of adiscrete-time Markov chain from partially observeddata.Statistics in Medicine, 27:905–921, 2008.
University of Manchester Nick Higham Matrix Logarithm 37 / 40
![Page 51: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/51.jpg)
Definition Applications Theory Methods
References III
W. F. Harris.The average eye.Opthal. Physiol. Opt., 24:580–585, 2005.
N. J. Higham.The Matrix Function Toolbox.http://www.ma.man.ac.uk/~higham/mftoolbox.
N. J. Higham.Evaluating Padé approximants of the matrix logarithm.SIAM J. Matrix Anal. Appl., 22(4):1126–1135, 2001.
University of Manchester Nick Higham Matrix Logarithm 38 / 40
![Page 52: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/52.jpg)
Definition Applications Theory Methods
References IV
N. J. Higham.Functions of Matrices: Theory and Computation.Society for Industrial and Applied Mathematics,Philadelphia, PA, USA, 2008.ISBN 978-0-898716-46-7.xx+425 pp.
N. J. Higham and A. H. Al-Mohy.Computing matrix functions.Acta Numerica, 19:159–208, 2010.
N. J. Higham and L. Lin.On pth roots of stochastic matrices.Linear Algebra Appl., 435(3):448–463, 2011.
University of Manchester Nick Higham Matrix Logarithm 39 / 40
![Page 53: Research Matters Nick Higham February 25, 2009 …higham/talks/ecm12...the Theory of Matrices (1858). Cayley considered matrix square roots. Term “matrix” coined in 1850 by James](https://reader033.vdocuments.us/reader033/viewer/2022050402/5f8024f5de11c93d9c54a60c/html5/thumbnails/53.jpg)
Definition Applications Theory Methods
References V
C. S. Kenney and A. J. Laub.Condition estimates for matrix functions.SIAM J. Matrix Anal. Appl., 10(2):191–209, 1989.
C. S. Kenney and A. J. Laub.A Schur–Fréchet algorithm for computing the logarithmand exponential of a matrix.SIAM J. Matrix Anal. Appl., 19(3):640–663, 1998.
University of Manchester Nick Higham Matrix Logarithm 40 / 40