dario a. bini universit a di pisa mopnet-3 edinburgh ...mopnet.cf.ac.uk/bini.pdf · d.a. bini...

46
Overview on doubling algorithms for matrix polynomials Dario A. Bini Universit` a di Pisa MOPNET-3 Edinburgh, September, 20–22 2010 D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 1 / 44

Upload: others

Post on 17-Jul-2020

4 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

Overview on doubling algorithms for matrix polynomials

Dario A. BiniUniversita di Pisa

MOPNET-3Edinburgh, September, 20–22 2010

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 1 / 44

Page 2: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

The pictureThere are problems from applications where matrix (operator) polynomialsor matrix power series play an important role: queueing models, hyperbolicquadratic eigenvalue problems, algebraic Riccati equations, etc

A typical example from queueing models: [Neutz 89]Given m ×m nonnegative matrices A0,A1,A2, . . ., such thatA0 + A1 + A2 + · · · is stochastic, compute the minimal nonnegativesolution to the matrix equation

X = A0 + A1X + A2X2 + A3X

3 + · · ·

Compute the canonical Wiener-Hopf factorization

I −+∞∑i=−1

z iAi+1 = U(z)L(z) := (+∞∑i=0

z iUi )(I − z−1G )

where U(z) and L(z) are analytic and nonsingular inside and outside theunit disk, respectively.

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 2 / 44

Page 3: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

The picture

There exist effective algorithms based on matrix polynomialmanipulation for solving these problems

their effectiveness relies on the quadratic convergence in the genericcase and on their numerical stability

the most popular algorithms are the Strucured Doubling Algorithm(SDA) and the Cyclic Reduction (CR)

the latter is widely used in the framework of Markov chains andstochastic processes, the former is well-known in control problemsgoverned by the Riccati equations

Both of them have ancient and different origins and have been objectof many papers with adaptations and variants, but rely on the sameidea of repeated “squaring”.

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44

Page 4: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

Aim of this talk

First part:

to give an overview of this subject in the framework of matrixpolynomials

to point out the interplay of CR and SDA

Second part:

to show the richness and the nice features of CR

to present the problems that still require some work

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 4 / 44

Page 5: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

The concept of squaring

Let us recall the Graeffe-Lobachevsky-Dandelin iteration for scalarpolynomials [Ostrowski 40]:

p(z) polynomial of degree n with roots ξ1, ..., ξn such that

|ξ1| ≤ · · · ≤ |ξk | < 1 < |ξk+1| ≤ · · · ≤ |ξn|

Multiply p(z) and p(−z) and obtain

p(z)p(−z) = p1(z2), p1(z) polynomial of degree n

Remark

The roots of p1(z) are the square of the roots of p(z)

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 5 / 44

Page 6: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

The squaring property

In general, definepν+1(z2) = pν(z)pν(−z)

The roots of pν(z) are ξ2ν

i , i = 1, . . . , n so that

|ξ1|2ν ≤ · · · ≤ |ξk |2

ν︸ ︷︷ ︸ < 1 < |ξk+1|2ν ≤ · · · ≤ |ξn|2

ν︸ ︷︷ ︸↓ ↓0 ∞

In other words, for ν large enough one has p(ν)k 6= 0 and

limν

pν(z)

p(ν)k

= zk

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 6 / 44

Page 7: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

The case of matrix polynomials

Let Ai , i = 0, 1, . . . , n be m ×m matrices, define the matrix polynomial

P(z) = A0 + zA1 + · · ·+ znAn, An 6= 0

Remark

Due to lack of commutativity, P(z)P(−z) is not a matrix polynomial in z2

However, for n = 2 and P(z) = A0 + zA1 + z2A2, with det A1 6= 0, one has

P(z)A−11 P(−z) = P1(z2), P1(z) = A

(1)0 + zA

(1)1 + z2A

(1)2

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 7 / 44

Page 8: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

A

(1)0 = A0A

−11 A0

A(1)1 = −A1 + A0A

−11 A2 + A2A

−11 A0

A(1)2 = A2A

−11 A2

Remark

The roots of det P1(z) are the squares of the roots of det P(z), i.e., thesquaring property is preserved.

Define

Pν+1(z2) = Pν(z)(A

(ν)1

)−1Pν(−z)

where we assume that this sequence is well defined, i.e., det A(ν)1 6= 0

Then the roots of Pν(z) are such that

ξ(ν)i = ξ2

ν

i , i = 1, . . . ,m.

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 8 / 44

Page 9: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

If the roots of det P(z) are such that:

|ξ1| ≤ · · · ≤ |ξm| < 1 < |ξm+1| ≤ · · · < |ξ2m|

one should expect that

lim Pν(z) = zA?1, with det A?1 6= 0

that is, A(ν)0 → 0, A

(ν)2 → 0, A

(ν)1 → A?1.

Formally, the algorithm obtained this way coincides with theCyclic Reduction (CR) algorithm introduced by Gene Golub at the endof 1960’s for solving the discrete Poisson equation over a rectangle, ifapplied to a general block tridiagonal block Toeplitz system [Hockney 65]

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 9 / 44

Page 10: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

The squaring property of the roots of P(z) can be rephrased as follows:

If the m ×m matrix G solves the equation

A0 + A1X + A2X2 = 0 (1)

then the matrix G 2νsolves the equation

A(ν)0 + A

(ν)1 X + A

(ν)2 X 2 = 0

This property provides a means to compute the (semi) stable solution G ofthe quadratic equation (1) that is, such that ρ(G ) < 1 (ρ(G ) ≤ 1), orequivalently, to compute the canonical Wiener-Hopf factorization

z−1A0 + A1 + zA2 = (U0 + zU1)(I − z−1G )

U0 = A0 + A1G , U1 = A

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 10 / 44

Page 11: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

Solving the equation A0 + A1X + A2X2 = 0

{A0 + A1X + A2X

2 = 0A0X + A1X

2 + A2X3 = 0

eliminate X 2 → A0 + A(1)X + A(1)2 X 3 = 0

{A0 + A

(1)1 X + A

(1)2 X 3 = 0

A(1)0 X + A

(1)1 X 3 + A

(1)2 X 5 = 0

eliminate X 3 → A0 + A(2)X + A(2)2 X 5 = 0

At the step ν one has

A0 + A(ν)X + A(ν)2 X 2ν+1 = 0, A(ν+1) = A(ν) − A

(ν)0 (A

(ν)1 )−1A

(ν)2

Since A(ν)2 → 0, ρ(X ) ≤ 1, if A

(ν)2 has a uniformly bounded inverse then then

−(A(ν))−1A0 → X

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 11 / 44

Page 12: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

Change of the scenario: Structured Doubling Algorithms

The generalization of the Graeffe iteration enables one to generate asequence of quadratic matrix polynomials having roots which are squaredat each step and allows to solve quadratic matrix equations

Something similar can be done for linear matrix pencils [Anderson 78]

Consider the linear matrix pencil L− zU where we assume that L ∈ L,U ∈ U , and L and U are two given matrix groups. We say that the pencilis in LU–canonical form

W.l.o.g, assume that det U 6= 0 so that we may define A = U−1L.

Observe that the eigenvalue problem for A is equivalent to the generalizedeigenvalue problem for the pencil L− zU.

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 12 / 44

Page 13: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

The idea of SDA

A2 =(U−1L

) (U−1L

)= U−1

(LU−1

)L,

so that if the matrix LU−1 can be factored as

LU−1 = U−1L, L ∈ L, U ∈ U ,

thenA2 = U−1

1 L1, U1 = UU ∈ U , L1 = LL ∈ L

That is, given the pencil L− zU in canonical form one can construct thepencil L1 − zU1, still in canonical form, whose eigenvalues are the squaresof the eigenvalues of L− zU.

Problem: to compute the UL factorization of a product of type LU, or,more simply to solve the UL–LU problem where the invertibility of U is notrequired [Benner, Byers 06]:

given L ∈ L, U ∈ U , compute U ∈ U , L ∈ L such that

UL = LU

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 13 / 44

Page 14: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

Remark

The recursive application of the above formulae provides a sequence oflinear pencils Uν − zLν in canonical form such that the eigenvalues of thepencil at step ν are λ2ν

i , where λi are the eigenvalues of the original pencil.

The squaring property can be expressed in terms of deflating subspaces:

Property

If V and W are matrices of size m × k, k × k , respectively, with k < m:

=V U VW

L

i.e., if V spans a deflating subspace for the pencil, then

=

WUL

νV

ν V

that is, V still spans a deflating subspace for the pencil Lν − zUν .

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 14 / 44

Page 15: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

Different algorithms can be obtained by using different classes U , L.Since the structure of L and U is preserved, the algorithms in this class arecalled Structured Doubling Algorithms (SDA)

Example: SDA-1 [Chiang, Chu, Guo, Lin, Xu 09]

Consider the linear matrix pencil L− zU in the standard structured form,i.e.,

L =

[E 0−H I

]} n}m

, U =

[I −G0 F

]} n}m

,

Then a simple computation shows that

L1 =

[E1 0−H1 I

], U1 =

[I −G1

0 F1

]where

E1 = E (I − GH)−1E , H1 = H + F (I − HG )−1HE

G1 = G + E (I − GH)−1GF , F1 = F (I − HG )−1F

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 15 / 44

Page 16: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

Example: QR factorization

Example: SDA-2 [Chiang, Chu, Guo, Lin, Xu 09] The group conditionmay be weakened.

Consider the pencil L− zU in the 2nd standard structured form

L =

[E 0−H I

], U =

[−G IF 0

]then SDA keeps the structure and generates a sequence of pencilsLν − zUν such that

Lν =

[Eν 0−Hν I

], Uν =

[−Gν IFν 0

]

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 16 / 44

Page 17: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

Comparison of SDA and CR

SDA acts onlinear matrix pencils

CR acts onquadratic matrix polynomials

Given a matrix pencil in canonicalform

L− zU,

SDA generates a sequence of ma-trix pencils

Lν − zUν

in canonical form whose eigenval-ues have the repeated squaringproperty

Given a quadratic matrix polyno-mial

A0 + zA1 + z2A2,

CR generates a sequence ofquadratic matrix polynomials

A(ν)0 + zA

(ν)1 + z2A

(ν)2 ,

whose roots have the repeatedsquaring property

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 17 / 44

Page 18: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

Comparison of SDA and CR

SDA acts onlinear matrix pencils

CR acts onquadratic matrix polynomials

Given a matrix pencil in canonicalform

L− zU,

SDA generates a sequence of ma-trix pencils

Lν − zUν

in canonical form whose eigenval-ues have the repeated squaringproperty

Given a quadratic matrix polyno-mial

A0 + zA1 + z2A2,

CR generates a sequence ofquadratic matrix polynomials

A(ν)0 + zA

(ν)1 + z2A

(ν)2 ,

whose roots have the repeatedsquaring property

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 17 / 44

Page 19: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

Comparison of SDA and CR

If LV = UV W then

LνV = UνV W 2ν

If A0 + A1X + A2X2 = 0

A(ν)0 + A

(ν)1 X 2ν

+ A(ν)2 (X 2ν

)2 = 0

SDA can be applied if

det(I − GνHν) 6= 0

CR can be applied if

det A(ν)1 6= 0

A natural question arises: are they the same algorithm?

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 18 / 44

Page 20: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

Comparison of SDA and CR

If LV = UV W then

LνV = UνV W 2ν

If A0 + A1X + A2X2 = 0

A(ν)0 + A

(ν)1 X 2ν

+ A(ν)2 (X 2ν

)2 = 0

SDA can be applied if

det(I − GνHν) 6= 0

CR can be applied if

det A(ν)1 6= 0

A natural question arises: are they the same algorithm?

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 18 / 44

Page 21: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

Linearization and quadraticization

A matrix polynomial

P(z) = A0 + zA1 + z2A2

can be “linearized” into a matrix pencil

A(z) =

[0 IA0 A1

]− z

[I 00 −A2

]where [

0 IA0 A1

] [IX

]=

[I 00 −A2

] [IX

]X

and X is any solution of the equation A0 + A1X + A2X2 = 0

Correspondence between deflating subspace of the matrix pencil andsolutions of the quadratic matrix equation

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 19 / 44

Page 22: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

Linearization and quadraticization

A matrix pencil

A(z) =

[L11 L12

L21 L22

]− z

[U11 U12

U21 U22

]can be transformed into a quadratic matrix polynomial:

P(z) =

[I 00 zI

]A(z)

=

[L11 L12

0 0

]+ z

[U11 U12

L21 L22

]+ z2

[0 0

U21 U22

]

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 20 / 44

Page 23: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

SDA-1 is a specific CR

Consider the linear pencil associated with SDA-1

A(z) =

[E 0−H I

]− z

[I −G0 F

]construct the quadratic matrix polynomial

P(z) =

[I 00 zI

]A(z) =

[E 00 0

]+ z

[I −G−H I

]+ z2

[0 00 F

]apply CR to P(z) and get

Pk(z) =

[I 00 zI

]Ak(z)

Then Ak(z) are the linear pencils generated by SDA–1 [B., Meini, Poloni10]

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 21 / 44

Page 24: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

CR is SDA-2

GivenA0 + A1z + A2z

2

consider the following linearization [Guo 08][0 IA0 0

]+ z

[A2 0−A1 −I

]applying SDA-2 yields the sequence[

−A(ν) I

A(ν)0 0

]+ z

[A

(ν)2 0

A(ν)1 −I

]

where Pν(z) = A(ν)0 + zA

(ν)1 + z2A

(ν)2 is the polynomial sequence generated

by CR.

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 22 / 44

Page 25: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

Resuming

CR→SDA linearization

SDA→CR quadraticization

Advantages of having two different points of view: more tools for

proving applicability conditions

proving convergence conditions

proving convergence properties

analyzing critical cases

solving problems from applications

finding generalizations

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 23 / 44

Page 26: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

The world of CR is richer than that of SDACR and analytic functions

Define the Laurent polynomial ϕ(z) := z−1P(z) = A0z−1 + A1 + A2z

and the matrix function ψ(z) := ϕ(z)−1 defined for z 6= ξi

ψ(z) is analytic in the annulus A = {z ∈ C : r = |ξn| < |z | < |ξn+1| = R}

1 Rr

therefore it can be represented as a Laurent series

ψ(z) =+∞∑

i=−∞z iHi , z ∈ A

For the analyticity of ψ in A one has ∀ ε > 0 ∃ θ > 0 such that{||Hi || ≤ θ(r + ε)i , i > 0

||Hi || ≤ θ(R − ε)i , i < 0

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 24 / 44

Page 27: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

CR and analytic functions

Now, for the polynomials Pν(z) generated by CR denote

ϕν(z) := z−1Pν(z), ψν(z) := ϕν(z)−1

One has

ϕ1(z2) = ϕ(z)A−11 ϕ(−z) =

(ϕ(−z)−1 + ϕ(z)−1

2

)−1

so that ψν+1(z2) = (ψν(z) + ψν(−z))/2 and

ψν(z) =∞∑

i=−∞z iHi2ν , z ∈ A

i.e., ψν(z) converges double exponentially to the constant H0.

Under the assumption det H0 6= 0, the sequence ϕν(z) converges to H−10 .

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 25 / 44

Page 28: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

CR and analytic functions

The following equivalent conditions imply that det H0 6= 0:

there exist G and F such that A0 + A1G + A2G2 = 0,

A2 + A1F + A0F2 = 0, ρ(G ), ρ(F ) < 1

there exist G and Z such that A0 + A1G + A2G2 = 0,

A0 + ZA1 + Z 2A2 = 0, ρ(G ), ρ(Z ) < 1

there exist the canonical Wiener-Hopf factorizations ofz−1A0 + A1 + zA2 and z−1A2 + A1 + zA0

Moreover,G = H−1H

−10 , F = H1H

−10

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 26 / 44

Page 29: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

CR and Schur complements

Consider the Laurent polynomial

ϕ(z) = z−1P(z) = z−1A0 + A1 + zA2

From the identity:

ϕ1(z2) = ϕ(z)A−11 ϕ(−z) = A1 − (z−1A0 + zA2)A−1

1 (z−1A0 + zA2)

one discovers a Schur complement in functional form

Matrix translation:

associate with ϕ(z) the infinite block tridiagonal block Toeplitz matrix

T = Trid(A0,A1,A2) =

. . .

. . .. . . 0

A0 A1 A2

0. . .

. . .. . .

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 27 / 44

Page 30: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

CR and Schur complements

The Schur complement of the submatrix of T formed by the evennumbered rows and column is the block tridiagonal matrix T1

associated with the Laurent polynomial ϕ1(z).

Applicability of CR holds for diagonally dominant matrices, symmetricpositive definite matrices, M-matrices; numerical stability is undercontrol;

The νth step of CR can be performed if and only ifTrid2ν−1(A0,A1,A2) is nonsingular

If Trid2ν−1(A0,A1,A2) should be singular or ill-conditioned, thensimple formulas, based on Schur complements can be designed forskipping the νth step → possibility to implement Look-aheadstrategies

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 28 / 44

Page 31: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

Extensions: matrix power series

The Graeffe-Lobachevsky-Dandelin algorithm can be generalized to matrixLaurent power series ϕ(z) analytic and invertible for z ∈ A:

ϕ(z) =+∞∑

i=−∞Aiz

i , ψ(z) := ϕ(z)−1 =+∞∑

i=−∞z iHi

ϕ1(z2) = ϕ(z)

(ϕ(z) + ϕ(−z)

2

)−1

ϕ(−z) =

(ψ(−z) + ψ(z)

2

)−1

is analytic and invertible in A and

ψ1(z2) =ψ(z) + ψ(−z)

2=

+∞∑i=−∞

z iH2i

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 29 / 44

Page 32: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

The general convergence Theorem

Consider the sequence generated by generalized CR

ϕν+1(z2) = ϕν(z)

(ϕν(z) + ϕν(−z)

2

)−1

ϕν(−z)

One deduces that

ϕν(z)−1 =+∞∑

i=−∞z iHi2ν

The analyticity of ψ(z) implies

Theorem

If detϕ(z) 6= 0 for z ∈ A, and det H0 6= 0, where ϕ(z)−1 =∑+∞

i=−∞ z iHi ,and if CR can be carried out with no breakdown then the sequence ϕν(z)generated by CR converges double exponentially to the constant H−1

0 .

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 30 / 44

Page 33: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

The same formula can be written in terms of Schur complement as

ϕ1(z) = zϕeven − ϕodd(z)ϕeven(z)−1ϕodd(z)

where

ϕeven(z2) =ϕ(z) + ϕ(−z)

2,

ϕodd(z2) =ϕ(z)− ϕ(−z)

2

The nice properties of Schur complements still apply

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 31 / 44

Page 34: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

Similarly to the quadratic case, we are able to complement CR withsuitable relations in order to compute the Wiener-Hopf factorization ofϕ(z) and of ϕ(−z) in the following cases

ϕ(z) =+∞∑i=−1

z iAi = (+∞∑i=0

z iUi )(I − z−1G )

ϕ(z) =+∞∑

i=−k

z−iAi = (+∞∑i=0

z iUi )(k∑

i=0

z−iLi ), k > 1

that include M/G/1 and G/M/1 Markov Chains Non-Skip-Free MarkovChains (k > 1) [Neutz 89], [B., Latouche, Meini 05]

Implementations of these algorithms for problems encountered in queueingmodels are contained in the package SMCSolver

[Van Houdt], ftp://ftp.win.ua.ac.be/pub/pats/tools/[B, Meini, Steffe, Van Houdt]http://bezout.dm.unipi.it/SMCSolver/

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 32 / 44

Page 35: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

Some questionsWe are not yet able to design algorithms for computing the Wiener-Hopffactorization of a general Laurent series

ϕ(z) =+∞∑

i=−∞z iAi =

+∞∑i=0

z iUi

+∞∑i=0

z−iLi

Remark. Since ϕ(z) is analytic in A, its coefficients decay exponentially.Therefore, numerically ϕ(z) is approximated by the sequence of Laurentpolynomials

φk(z) =k∑

i=−k

z iAi

Question 1

Under which conditions there exists the W-H factorization φk(z) =Uk(z)Lk(z) and do the coefficients of Uk(z) and Lk(z) converge to thecorresponding coefficients of U(z) and L(z)?

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 33 / 44

Page 36: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

Some questions

In certain applications, quadratic polynomials are encountered where thematrix coefficients A0,A1,A2 have infinite size.

Question 2

Assuming that there exists the W-F factorization

ϕ(z) = U(z)L(z) of ϕ(z) = z−1A0 + A1 + zA2,

under which conditions there exists the W-H factorization

φk(z) = Uk(z)Lk(z)

of the function φk(z) obtained by truncating the blocks A0,A1,A2 to finitesize k, and under which assumptions the k × k coefficients of Uk(z) andLk(z) converge to the corresponding coefficients of U(z) and L(z)?

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 34 / 44

Page 37: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

Some questionsFor ϕ(z) =

∑+∞i=−1 z iAi the following sufficient condition for det H0 6= 0

holds [B, Meini, Spitkovsky]:

Theorem

If there exist solutions G and F to the equations

∞∑i=−1

AiXi+1 = 0,

∞∑i=−1

Y i+1Ai = 0,

such that ρ(G ) < 1, ρ(F ) < 1, then det H0 6= 0 and ϕν(z) convergedouble exponentially to the constant power series H0.

This result is false for general Laurent power series.

Question 3

Find conditions under which the existence of the W-H factorizations of ϕ(z)and ϕ(−z) imply the nonsingularity of H0, for a general ϕ(z).

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 35 / 44

Page 38: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

Some questions

If some roots of det(A0 + zA1 + z2A2) lie in the unit circle convergence ofCR is more critical.

Some results are available in [Guo, Higham, Tisseur] which guarantee(linear) convergence

Experiments show that, even though convergence of −A(ν)A0 to G mayfail, the non-unitary eigenvalues of −A(ν)A0 still converge. Some resultsare available from [Li, Chu, Lin, JCAM 2010] but the analysis is notcomplete

The case of ϕ(z) =∑+∞

i=−k z iAi such that detϕ(z) is zero for some z inthe unit circle is not covered yet

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 36 / 44

Page 39: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

Some references

Anderson, B. Second-order convergent algorithm for the steady-state Riccatiequation. Int. J. Control 28, 295–306 (1978)

P. Benner, R. Byers. An arithmetic for matrix pencils: theory and newalgorithms. Numer. Math., 103(4):539–573, 2006.

D.A. Bini, B. Meini. The cyclic reduction algorithm: from Poisson equationto stochastic processes and beyond. In memoriam of Gene H. Golub.Numer. Algorithms, 51(1):23–60, 2009.

D. A. Bini, G. Latouche, and B. Meini. Numerical methods for structuredMarkov chains. Oxford University Press, New York, 2005.

D.A. Bini, B. Meini, and F. Poloni. Transforming algebraic Riccati equationsinto unilateral quadratic matrix equations. Numer. Math., 2010.

Bini, D.A., Meini, B., Spitkovsky, I.M.: Shift techniques and canonicalfactorizations in the solution of M/G/1-type Markov chains. Stoch. Models21(2–3), 279–302 (2005)

B. L. Buzbee, G. H. Golub, and C. W. Nielson. On direct methods forsolving Poisson’s equations. SIAM J. Numer. Anal., 7:627–656, 1970.

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 37 / 44

Page 40: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

Chun-Yueh Chiang, Eric King-Wah Chu, Chun-Hua Guo, Tsung-MingHuang, Wen-Wei Lin, and Shu-Fang Xu. Convergence analysis of thedoubling algorithm for several nonlinear matrix equations in the critical case.SIAM J. Matrix Anal. Appl., 31(2):227– 247, 2009.

E. K.-W. Chu, H.-Y. Fan, and W.-W. Lin. A structure-preserving doublingalgorithm for continuous-time algebraic Riccati equations. Linear AlgebraAppl., 396:55–80, 2005.

C.-H. Guo. Efficient methods for solving a nonsymmetric algebraic Riccatiequation arising in stochastic fluid models. J. Comput. Appl. Math.,192(2):353–373, 2006.

C.-H. Guo, N. Hogham, F. Tisseur, Detecting and Solving HyperbolicQuadratic Eigenvalue Problems, SIAM J. Matrix Anal. Appl.,30(4):1593-1613, 2009

C.-H. Guo, B. Iannazzo, and B. Meini. On the doubling algorithm for a(shifted) nonsymmetric algebraic Riccati equation. SIAM J. Matrix Anal.Appl., 29(4):1083–1100, 2007.

Hockney, R.W.: A fast direct solution of Poisson’s equation using Fourieranalysis. J. Assoc. Comput. Mach. 12, 95–113 (1965)

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 38 / 44

Page 41: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

T.-M. Hwang, E. K.-W. Chu, and W.-W. Lin. A generalizedstructure-preserving doubling algorithm for generalized discrete-timealgebraic Riccati equations. Internat. J. Control, 78(14):1063–1075,2005.

Neuts, M.F. Structured Stochastic Matrices of M/G/1 Type andTheir Applications, Marcel Dekker, New York (1989)

Ostrowski, A.: Recherches sur la methode de Graeffe et les zeros despolynomes et des series de Laurent. Chapitres III et IV. Acta Math.72, 157–257 (1940)

Ostrowski, A.: Addition a notre memoire: “Recherches sur lamethode de Graeffe et les zeros des polynomes et des series deLaurent.” Acta Math. 75, 183–186 (1943)

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 39 / 44

Page 42: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

Further extensions: some reductions

reduction of a matrix polynomial to a quadratic polynomial

If G is a solution ofn∑

i=−1

AiXi+1 = 0

then the enlarged matrix

G =

0 . . . 0 G0 . . . 0 G 2

......

......

0 . . . 0 Gn

solves the enlarged equation

A0 +A1X +A2X 2 = 0

where

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 40 / 44

Page 43: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

A0 +A1X +A2X 2 = 0

A0 =

0 . . . 0 A−1

0 . . . 0 0...

......

...0 . . . 0 0

, A1 =

A0 . . . . . . An−1

A−1 A0. . .

.... . .

. . ....

0 A−1 A0

,

A2 =

An 0

An−1. . .

.... . .

. . .

A1 A2 . . . An

An m ×m matrix polynomial equation of degree n is reduced to an(mn)× (mn) quadratic matrix equation

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 41 / 44

Page 44: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

A different reduction: [Ramaswami]

G =

G 0 . . . 0G 2 0 . . . 0...

......

...G n 0 . . . 0

solves the enlarged equation A0 +A1X +A2X 2 = 0 where

A0 =

A−1 0 . . . 0

0 0 . . . 0...

. . .. . .

...0 0 . . . 0

, A1 =

A0 A1 . . . An−1

I. . .

I

,

A2 =

0

−I. . .. . .

. . .

−I 0

In the case of a matrix power series, one gets a quadratic matrix equation

with semi-infinite blocksD.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 42 / 44

Page 45: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

Further extensions: Sign function iteration

Define the following functions:

J(t) = (t + t−1)/2 Joukowski

C (t) = (t − 1)/(t + 1) Cayley

S(t) = t2 Square

It holdsC (J(t)) = S(C (t)), C (−S(t)) = J(C (t)),

C (−t) = 1/C (t), C (t−1) = −C (t)

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 43 / 44

Page 46: Dario A. Bini Universit a di Pisa MOPNET-3 Edinburgh ...mopnet.cf.ac.uk/bini.pdf · D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 3 / 44. Aim of this talk First part: to

This implies the following

Property

P(z)(A0 − A2)−1P(z−1) = P1(J(z))

The roots of P1(z) coincide with J(λi ), where λi are the roots of P(z).

Sign function iteration for matrix polynomials:

the roots of Pν(z) areJ ◦ · · · ◦ J︸ ︷︷ ︸(λi )

lim Pν(z) = (−1 + z2)A?1

Application: computing the solutions X+ and X− of the equationA0 + A1X + A2X

2 = 0 such that σ(X+) ⊂ C+, σ(X−) ⊂ C−.

D.A. Bini (Pisa) Doubling algorithms Mopnet-3 Edinburgh 44 / 44