a goal oriented low rank dual adi iteration for balanced ... · bt in a nut lrcf-adi existing...

MAX PLANCK INSTITUTE

FOR DYNAMICS OF COMPLEX

TECHNICAL SYSTEMS

MAGDEBURG

GAMM Workshop onApplied and Numerical Linear Algebra

with Special Emphasis on Model Reduction

September 22nd and 23rd , 2011

A Goal Oriented Low Rank Dual ADIIteration for Balanced Truncation

Jens Saakjoint work with

Peter Benner and Patrick Kurschner

Max Planck Institute for Dynamics of Complex Technical SystemsComputational Methods in Systems and Control Theory

Max Planck Institute Magdeburg Jens Saak, Dual ADI for Balanced Truncation 1/25

BT in a Nut ∗LRCF-ADI Existing Approaches Dual LR-BT-ADI Numerical Results Conclusions

Outline

1 Balanced Truncation in a Nutshell

2 Solving Large Lyapunov Equations

3 Existing Approaches “Simultaneously” Treating the Dual LyapunovEquations

4 Dual LR-BT-ADI

5 Numerical Results

6 Conclusions and Future Perspectives



Balanced Truncation in a NutshellBasic Idea

Idea:

The system Σ, in realization (A,B,C ), is called balanced, if thesolutions P,Q of the Lyapunov equations

AP + PAT + BBT = 0, ATQ + QA + CTC = 0,

satisfy: P = Q = diag(σ1, . . . , σn) where σ1 ≥ σ2 ≥ . . . ≥ σn > 0.

{σ1, . . . , σn} are the Hankel singular values (HSVs) of Σ.

A balanced realization is computed via state space transformation

T : (A,B,C) 7→ (TAT−1,TB,CT−1)

=

([A11 A12

A21 A22

],

[B1

B2

],[C1 C2

]).

Truncation reduced order model: (A, B, C ) = (A11,B1,C1).



Balanced Truncation in a NutshellImplementation

The SR Method1 Compute (Cholesky)factors of the solutions to the Lyapunov

equation,P = STS , Q = RTR.

2 Compute singular value decomposition

SRT = [U1, U2 ]

[Σ1

Σ2

] [V T1

V T2

].

3 DefineW := RTV1Σ

−1/21 , V := STU1Σ

−1/21 .

4 Then the reduced order model is (W TAV ,W TB,CV ).



Solving Large Lyapunov EquationsLRCF-ADI

Lyapunov equation

Consider

FX + XFT = −GGT

with F ∈ Rn×n Hurwitz, G ∈ Rn×m, m� n.

Observation in practice:[Penzl ’99, Ant./Sor./Zhou ’02, Grasedyck ’04]

rank(X , τ) = r � n

⇒ Compute low-rank solution factorZ ∈ Rn×r , r � n.

X ≈ Z ZT . 50 100 2000 250010−25

10−13

10−1

· · ·

u

singular values for n = 2 500 example

σj (X )



Solving Large Lyapunov EquationsLRCF-ADI e.g.,[Benner/Li/Penzl ’08]

Consider FX + XFT = −GGT F ∈ Rn×n,G ∈ Rn×p

Task Find Z ∈ Cn,nz , such that nz � n and X ≈ ZZH

Algorithm

V1 =√−2Re p1(F + p1I )

−1G , Z1 = V1

Vi =

√Re pi√

Re pi−1

[I − (pi + pi−1)(F + pi I )

−1]Vi−1 Zi = [Zi−1Vi ]

For certain shift parameters {p1, ..., pJ} ⊂ C<0.

Stop if

‖ViVHi ‖ is small, or

‖FZiZHi + ZiZ

Hi FT + GGT‖ is small.



Solving Large Lyapunov EquationsG-LRCF-ADI (E invertible) e.g.,[S. ’09]

Consider FXET + EXFT = −GGT E ,F ∈ Rn×n,G ∈ Rn×p


Algorithm

V1 =√−2Re p1(F + p1E)−1G , Z1 = V1

Vi =

√Re pi√

Re pi−1

[I − (pi + pi−1)(F + piE)−1

]EVi−1 Zi = [Zi−1Vi ]


Stop if


‖FZiZHi ET + EZiZ

Hi FT + GGT‖ is small.



Solving Large Lyapunov EquationsS-LRCF-ADI ((E ,F ) index 1) e.g.,[Rommes/Freitas/Martins ’08]

Consider FXET11 + E 11XFT = −G GT E 11, F ∈ Rn×n, G ∈ Rn×p


Algorithm[V1

∗

]=√−2Re p1

[F11 + p1E11 F12

F21 F22

]−1 [B1

B2

], Z1 = V1[

Vi

∗

]=

√Re pi√

Re pi−1

[I − (pi + pi−1)

[F11 + piE11 F12

F21 F22

]−1] [E11Vi−1

0

]Zi = [Zi−1Vi ]


Stop if


‖FZiZHi ET

11 + E 11ZiZHi FT + G GT‖ is small.



Solving Large Lyapunov EquationsS-LRCF-ADI e.g.,[Rommes/Freitas/Martins ’08]

Consider FXET11 + E 11XFT = −G GT E 11, F ∈ Rn×n, G ∈ Rn×p


Algorithm[V1

∗

]=√−2Re p1

[F11 + p1E11 F12

F21 F22

]−1 [B1

B2

], Z1 = V1[

Vi

∗

]=

√Re pi√

Re pi−1

[I − (pi + pi−1)

[F11 + piE11 F12

F21 F22

]−1] [E11Vi−1

0

]Zi = [Zi−1Vi ]

Can ensure Z ∈ Rn,nz even if {p1, ..., pJ} 6⊂ Rsee next talk by Patrick Kurschner



Solving Large Lyapunov EquationsDrawbacks of LR-ADI Based BT

Performing LR-ADI based BT is computationally expensivecompared to simple Krylov based approaches.

Main effort is solving the dual Lyapunov equations.

Central Question

Can we reduce the time for computingthe balancing transformations?



Existing Approaches “Simultaneously” Treating theDual Lyapunov EquationsCross Gramian based Methods e.g. [Fernando/Nicholson ’83; Sorensen/Antoulas ’02]

IdeaSolve AX + XA = −BC for the cross gramian X ,

Proceed with knowledge X 2 = PQ, i.e., for the HSV σi = |λj(X )|.

e.g., repeatedly solve projected equations of the form

+ = for

Requires symmetric system or symmetrizer Ψ, such that

AΨ = ΨA∗, B = ΨC∗,

Ψ may be expensive to compute

number of inputs and outputs must coincide.



Existing Approaches “Simultaneously” Treating theDual Lyapunov EquationsApproximate Implicit Subspace Iteration with Alternating Directions (AISIAD)[Zhou/Sorensen ’08]

Idea

Solve AP +PAT = −BBT and ATQ +QA = −CTC simultaneously.

Alternate the projection spaces.

alternatingly solve 2 equations of the form

+ = + =




Idea



alternatingly solve 2 equations of the form

+ = + =

After convergence use and to compute balancing transformations.




Idea



Initial projection basis required.

2 growingly expensive linear equations per step.

Reuse of solvers for the step equations?



Dual LR-BT-ADISimultaneous Solution of the Dual Lyapunov Equations

Key Target

Reuse LU-factorizations in a sparse direct solver framework.

Vi =

√Re pi√Re pi−1

[I − (pi + pi−1)(F + piE )−1

]EVi−1,

LU := (F + piE ) ⇒ UHLH = (FT + piET )

complex shifts come in conjugate pair,

they are used one after another,

reverse order of complex conjugate shifts for the second equation.



Dual LR-BT-ADISimultaneous Solution of the Dual Lyapunov Equations

Key Target

Reuse LU-factorizations in a sparse direct solver framework.

Vi =


[I − (pi + pi−1)U−1L−1

]EVi−1,

Wi =


[I − (pi + pi−1)L−HU−H

]ETWi−1.

complex shifts come in conjugate pair,

they are used one after another,

reverse order of complex conjugate shifts for the second equation.



Dual LR-BT-ADIStopping the Iteration

ProblemNumber of columns in LRCFs limits ROM dimension.

Lyapunov residuals usually totally unrelated to ROM quality.

Idea

Use goal oriented stopping criteria.

Identify and monitor the property of interest to underlying application.

In Balanced Truncation MOR:

Assume we are interested in an order k ROM.

Monitor the relative change of k leading HSVs.

Stop when leading HSVs do no longer change.



Dual LR-BT-ADIStopping the Iteration

ProblemNumber of columns in LRCFs limits ROM dimension.

Lyapunov residuals usually totally unrelated to ROM quality.

Idea

Use goal oriented stopping criteria.Identify and monitor the property of interest to underlying application.

In Balanced Truncation MOR:

Assume we are interested in an order k ROM.

Monitor the relative change of k leading HSVs.

Stop when leading HSVs do no longer change.



Numerical ResultsTest Examples and Hardware

Rail Model [Benner/S. ’03-’05]

Oberwolfach Collection Rail Coolinga

n = 1357

inputs 7, outputs 6

ahttp://portal.uni-freiburg.de/imteksimulation/downloads/benchmark/

Steel\Profiles\(38881)

BIPS07 [Freitas/Rommes/Martins ’08]

index 1 descriptor model of a power networkb

n = 13 275, nE11 = 1693

inputs 4, outputs 4

condest(A22) = 8.275 · 1026

bhttp://sites.google.com/site/rommes/software/bips07_1693.mat


http://portal.uni-freiburg.de/imteksimulation/downloads/benchmark/Steel\ Profiles\ (38881)

http://portal.uni-freiburg.de/imteksimulation/downloads/benchmark/Steel\ Profiles\ (38881)

http://sites.google.com/site/rommes/software/bips07_1693.mat


Numerical ResultsTest Examples and Hardware

Hardware

dual Intel®Xeon®W3503 @ 2.40GHz

Cache Size: 4MB

RAM: 6GB

SofwareOS: Ubuntu Linux 10.04LTS

32bit kernel 2.6.32-25-generic-pae

MATLAB® 7.11.0 (R2010b)



Numerical ResultsRail Model

20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi

Leading Hankel Singular Values ( σiσmax≥ eps)

Original

dualADI

Iteration: 28




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 29




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 30




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 31




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 32




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 33




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 34




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 35




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 36




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 37




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 38




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 39




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 40




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 41




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 42




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 43




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 44




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 45




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 46




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 47




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 48




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 49




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 50




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 51




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 52




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 53




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 54




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 55




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 56




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 57




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 58




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 59




20 40 60 80 100 120 140 16010−22

10−17

10−12

10−7

10−2

i

σi


Original

dualADI

Iteration: 60




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax

relative change of HSVs

Iteration: 29




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 30




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 31




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 32




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 33




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 34




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 35




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 36




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 37




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 38




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 39




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 40




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 41




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 42




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 43




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 44




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 45




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 46




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 47




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 48




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 49




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 50




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 51




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 52




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 53




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 54




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 55




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 56




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 57




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 58




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 59




20 40 60 80 100 120 140 16010−24

10−18

10−12

10−6

100

i

|σ−σ(old) |

σmax


Iteration: 60




ObservationLeading HSVs converge from below.

Qualitative explanation:

LRCF-ADI is a shifted rational Arnoldi process.

HSVs are thus approximated via Ritz values.

Ritz values converge to the eigenvalues from the inside of thespectrum.

Quantitative analysis: work in progress.



Numerical ResultsBIPS07

Test Setup

We take the same settings as in [Freitas/Rommes/Martins ’08], i.e.:

shift parameters

50 Penzl shiftsfrom 80 Ritz values and 80 harmonic Ritz values

reduced order model dimension 32

maximum iteration number 80

α-shifted S-LRCF-ADI with α = 0.05




5 10 15 20 25 3010−1

100

101

102

i

σi

Leading 32 Hankel Singular Values

Original

S-LRCF-ADI

dual-S-LRCF-ADI

Do not trust residuals!




10−2 10−1 100 101 102100

101

102

103

ω

σmax(G

(ω

))

Transfer functions of original and reduced systems

original system

reduced system

dual reduced




10−2 10−1 100 101 10210−1

100

101

102

ω

σmax(G

(ω

)−

Gr(ω

))

absolute model reduction error

2 S-LRCF-ADIs

dual-S-LRCF-ADI




10−2 10−1 100 101 10210−3

10−2

10−1

100

ω

σmax(G

(ω)−

Gr(ω))

σmax(G

(ω))

relative model reduction error

2 S-LRCF-ADIs

dual-S-LRCF-ADI




10 20 30 40 50 60 70 8010−8

10−7

10−6

10−5

10−4

niter

res(niter

)re

s 0Residual History for S-LRCF-ADI

Controllability

Observability




5 10 15 20 25 30

10−1

100

101

102

i

σi


Original

S-LRCF-ADI(80)

dual-S-LRCF-ADI(80)

S-LRCF-ADI(15)





5 10 15 20 25 30

10−1

100

101

102

i

σi


Original

S-LRCF-ADI(80)

dual-S-LRCF-ADI(80)

S-LRCF-ADI(15)

dual-S-LRCF-ADI(250)





maxiter 50 75 100 125 150

tdual in s 2.97 5.22 7.31 10.33 14.72tS-LRCF-ADI in s 19.87 27.82 37.03 47.65 58.97

speedup 6.68 5.33 5.06 4.61 4.01

change HSV 2.151e-1 1.123e-1 7.440e-2 6.579e-2 6.438e-2final res. B 3.237e-5 3.237e-5 3.237e-5 3.237e-5 3.237e-5final res. C 5.513e-8 6.194e-8 5.900e-8 5.778e-8 5.710e-8

175 200 225 250

20.20 29.99 39.34 52.5172.16 86.32 98.50 111.7

3.57 2.88 2.50 2.13

6.389e-2 6.282e-2 6.189e-2 6.058e-23.237e-5 3.237e-5 3.237e-5 3.237e-55.703e-8 5.706e-8 5.707e-8 5.710e-8

Column compression via RRQR can reduce the SVD cost in the dual method:

maxiter = 250, CC-frequency=20, RRQR-tolerance=eps tdual ≈ 37s



Conclusions and Future Perspectives

ConclusionsSimultaneous handling of the dual Lyapunov equations candrastically improve the performance.

Goal oriented stopping criteria enhance the reliability of the results.

Future WorkQuantify convergence results.

Investigate reliable adaptive ROM dimension control.

Adapt stopping criterion to finite arithmetic.

Thank you very muchfor listening.


a goal oriented low rank dual adi iteration for balanced ... · bt in a nut lrcf-adi existing...

Documents