emgr - Empirical Gramian Framework

Christian Himpe (christian.himpe@uni-muenster.de)Mario Ohlberger (mario.ohlberger@uni-muenster.de)

WWU MünsterInstitute for Computational and Applied Mathematics

ESCO2014(Software Afternoon)

About

General Topic:Model ReductionSystem IdentificationUncertainty Quantification

Outline:Mathematical BackgroundCapabilities and FeaturesExamples and Demos

source: clker.com

Why?

Tasks:fMRI / fNIRS, EEG / MEGConnectivity AnalysisHyperbolic NetworksDiscretized PDEs

Challenges:Large-Scale ODEsNonlinearites

source: clker.com

emgr (At A Glance)

emgr - EMpirical GRamian FrameworkVersion 2.2Open-Source (BSD 2-Clause-License)Compatible with

OctaveMatlab(FreeMat)

about 400 LoC

emgr (Areas of Application)

Model Order ReductionState ReductionParameter ReductionCombined Reduction

Parameter IdentificationSensitivity Analysis

source: clker.com

emgr Features

Implementation Details:Uniform InterfaceAdaptable Solver BackendOnly Matrix and Vector OperationsGeneralized Transpositions

Performance Details:VectorizedImplicit ParallelizationExplicitly ParallelizableProfiled & Benchmarked

source: clker.com

Control Systems

Attributes:Input(Internal) StateOutputDynamics (ODE / PDE)

General Control Systems

x(t) = f (x(t), u(t))y(t) = g(x(t), u(t))x(0) = x0

Input / Control: u ∈ RM

State: x ∈ RN

Output: y ∈ RO

Vector Field: f : RN ×RM ×R→ RN

Output Functional: g : RN ×RM → RO

Linear Control Systems1

x(t) = Ax(t) + Bu(t)y(t) = Cx(t)x(0) = x0

Input / Control: u ∈ RM

State: x ∈ RN

Output: y ∈ RO

System Matrix: A ∈ RN×N

Input Matrix: B ∈ RN×M

Output Matrix: C ∈ RO×N

1Adjoint System: ˙x = AT x + CT u; y = BT x

Large-Scale Systems

Large-Scale Control System:

x = f (x , u)y = g(x , u)

State: x � 1Input: u � xOutput: y � x

Large-Scale Parametrized Control System:

x = f (x , u, θ)y = g(x , u, θ)

Parameter: θ � 1

Controllability & Observability [Moore’81]

Controllability Operator:

C(u) :=∫ ∞

0eAtBu(t)dt

Observability Operator:

O(x0) := CeAtx0

Controllability Gramian

Controllability Gramian:

WC := CCT

=

∫ ∞0

eAtBBT eAT tdt

⇒ AWC + WCAT = −BBT WC

Observability Gramian

Observability Gramian:

WO := OTO

=

∫ ∞0

eAT tCTCeAtdt

⇒WOAT + AWO = −CTCWO

Cross Gramian [Fernando & Nicholson’83]

Cross Gramian2 :

WX := CO

=

∫ ∞0

eAtBCeAtdt

⇒ AWX + WXA = −BCWX

2The control system has to be square, better: symmetric!

Model Reduction for Linear Control Systems

Balanced Truncation:

WOCholesky

= LOLTO

WCCholesky

= LCLTC

LCLTO

SVD= UDV T

σ1 > . . . > σN

→ U =

(U1U2

), V =

(V1V2

)T

A = V1AU1

B = V1B

C = CU1

x0 = V1x0

Direct Truncation:

WXSVD= UDV T

σ1 > . . . > σN

→ U =

(U1U2

)

A = UT1 AU1

B = UT1 B

C = CU1

x0 = UT1 x0

Model Reduction for Nonlinear Control Systems

Balanced Truncation:

WOCholesky

= LOLTO

WCCholesky

= LCLTC

LCLTO

SVD= UDV T

σ1 > . . . > σN

→ U =

(U1U2

), V =

(V1V2

)T

f = V1f (U1x , u, θ)g = g(U1x , u, θ)x0 = V1x0

Direct Truncation:

WXSVD= UDV T

σ1 > . . . > σN

→ U =

(U1U2

)

f = UT1 f (U1x , u, θ)

g = g(U1x , u, θ)

x0 = UT1 x0

Empirical Gramians (About)

System Gramians:

WC =

∫ ∞0

(eAtB) (eAtB)Tdt

WO =

∫ ∞0

(eAT tCT ) (eAT tCT )Tdt

WX =

∫ ∞0

(eAtB) (eAT tCT )Tdt

Integrands are just impulse responses!

→ Compute gramians empirically.

emgr (Capabilities)

Empirical Gramians:1 Empirical Controllability Gramian ( WC )2 Empirical Observability Gramian ( WO )3 Empirical Linear Cross Gramian ( WY )4 Empirical Cross Gramians ( WX )

5 Empirical Sensitivity Gramian ( WS )6 Empirical Identifiability Gramian ( WI )7 Empirical Joint Gramian ( WJ )

emgr (Signature)

Minimal Signature:W = emgr(f, g, q, t, w);

Full Signature:W = emgr(f, g, q, t, w, pr, nf, ut, us, xs, um, xm, yd);

f - System Function Handleg - Output Function Handleq - System Dimension(Inputs, States, Outputs)t - Time(Start, Step, Stop)w - Gramian Type(’c’, ’o’, ’x’, ’y’, ’s’, ’i’, ’j’)

pr - Parametersnf - Configurationut - Inputus - Steady State Inputxs - Steady Stateum - Input Scalesxm - State Scalesyd - Data

Empirical Gramians (Perturbations)

Impulse Input Perturbations: uhij(t) = chSiejδ(t)

Eu = {ei ∈ Rm; ‖ei‖ = 1; eiej 6=i = 0; i = 1, . . . ,m},Ru = {Si ∈ Rm×m; ST

i Si = 1; i = 1, . . . , s},Qu = {ci ∈ R; ci > 0; i = 1, . . . , q},

Initial State Perturbation: xkla0 = dkTl fa

Ex = {fi ∈ Rn; ‖fi‖ = 1; fi fj 6=i = 0; i = 1, . . . , n},Rx = {Ti ∈ Rn×n;TT

i Ti = 1; i = 1, . . . , t},Qx = {di ∈ R; di > 0; i = 1, . . . , r}.

emgr (Perturbations)

Impulse Input Perturbations: uhij(t) = chSiejδ(t)

Eu = {eRm

i=1...m},Ru = {−1,1},Qu = {ci ∈ R; ci > 0; i = 1, . . . , q},

Initial State Perturbation: xkla0 = dkTl fa

Ex = {eRn

i=1...n},Rx = {−1,1},Qx = {di ∈ R; di > 0; i = 1, . . . , r}.

Example System3

Linear MIMO Control System:

u(t) = δ(t)x(t) = Ax(t) + Bu(t)y(t) = Cx(t)

x0 = 0

Dimensions:Input: u(t) ∈ R16

State: x(t) ∈ R256

Output: y(t) ∈ R16

Attributes:State-Space Symmetric:A = AT , B = CT

⇒ WC = WO = WX

3Random System via Inverse Lypunov Procedure (see [Smith’03])

Empirical Controllability Gramian [Lall et al’99]

Analytic Controllability Gramian

WC =

∫ ∞0

eAtBBT eAT tdt

Linear Empirical Controllability Gramian

W ′C =

∫ ∞0

x(t) xT (t)dt

Empirical Controllability Gramian

W ′′C =

1|Qu||Ru|

|Qu |∑h=1

|Ru |∑i=1

m∑j=1

1c2h

∫ ∞0

Ψhij (t)dt,

Ψhij (t) = (xhij (t)− x)(xhij (t)− x)T ∈ Rn×n

Discrete Empirical Controllability Gramian

W ′′′C =

1|Qu||Ru|

|Qu |∑h=1

|Ru |∑i=1

m∑j=1

∆tc2h

T∑t=0

Ψhijt ,

Ψhijt = (xhij

t − x)(xhijt − x)T ∈ Rn×n

Example: POD

System:

x = Ax + Bu

Dimensions:

Input: u(t) ∈ R16

State: x(t) ∈ R256

Relative L2-Error:

Code:f = @(x,u,p) A*x+B*u;g = 1;q = [16,256,16];t = [0,0.01,1.0];WC = emgr(f,g,q,t,’c’);[U, D, V] = svd(WC);

Empirical Observability Gramian [Lall et al’99]

Analytic Observability Gramian

WO =

∫ ∞0

eAT tCTCeAtdt

Linear Empirical Observability Gramian

W ′O =

∫ ∞0

x(t) xT (t)dt

Empirical Observability Gramian

W ′′O =

1|Qx ||Rx |

|Qx |∑k=1

|Rx |∑l=1

1d2

k

∫ ∞0

TlΨkl (t)TT

l dt,

Ψklab = (ykla(t)− y)T (yklb(t)− y) ∈ R

Discrete Empirical Observability Gramian

W ′′′O =

1|Qx ||Rx |

|Qx |∑k=1

|Rx |∑l=1

∆td2

k

T∑t=0

TlΨtklTTl ,

Ψklt,ab = (ykla

t − y)T (yklbt − y) ∈ R

Example: Balanced Truncation

System:

x = Ax + Bu

y = Cx

Dimensions:

Input: u(t) ∈ R16

State: x(t) ∈ R256

Output: y(t) ∈ R16

Relative L2-Error:

Code:f = @(x,u,p) A*x+B*u;g = @(x,u,p) C*x;q = [16,256,16];t = [0,0.01,1.0];WC = emgr(f,g,q,t,’c’);WO = emgr(f,g,q,t,’o’);[U, D, V] = balance(WC,WO);

Example: Balanced Truncation

System:

x = Ax + Bu

y = Cx

Dimensions:

Input: u(t) ∈ R16

State: x(t) ∈ R256

Output: y(t) ∈ R16

Relative L2-Error:

Code:f = @(x,u,p) A*x+B*u;F = @(x,u,p) A’*x+C’*u;q = [16,256,16];t = [0,0.01,1.0];WC = emgr(f,1,q,t,’c’);WO = emgr(F,1,q,t,’c’);[U, D, V] = balance(WC,WO);

Empirical Cross Gramian [Streif’06, H.’14]

Analytic Cross Gramian

WX =

∫ ∞0

eAtBCeAtdt

Linear Empirical Cross Gramian

W ′X =

∫ T

0x(t)xT (t)dt

Example: (Linear) Direct Truncation

System:

x = Ax + Bu

y = Cx

Dimensions:

Input: u(t) ∈ R16

State: x(t) ∈ R256

Output: y(t) ∈ R16

Relative L2-Error:

Code:f = @(x,u,p) A*x+B*u;g = @(x,u,p) A’*x+C’*u;q = [16,256,16];t = [0,0.01,1.0];WY = emgr(f,g,q,t,’y’);[U, D, V] = svd(WY);

Empirical Cross Gramian [Streif’06, H.’14]

Analytic Cross Gramian

WX =

∫ ∞0

eAtBCeAtdt

Linear Empirical Cross Gramian

W ′X =

∫ T

0x(t)xT (t)dt

Empirical Cross Gramian

W ′′X =

1|Qu||Ru|m|Qx ||Rx |

|Qu |∑h=1

|Ru |∑i=1

m∑j=1

|Qx |∑k=1

|Rx |∑l=1

1chdk

∫ ∞0

TlΨhijkl (t)TT

l dt,

Ψhijklab (t) = f T

b TTk (xhij (t)− x)eT

i STh (ykla(t)− y)

Discrete Empirical Cross Gramian

W ′′′X =

1|Qu||Ru|m|Qx ||Rx |

|Qu |∑h=1

|Ru |∑i=1

m∑j=1

|Qx |∑k=1

|Rx |∑l=1

∆tchdk

T∑t=0

TlΨhijklt TT

l ,

Ψhijklt,ab = f T

b TTk (xhij

t − x)eTi ST

h (yklat − y)

Example: Direct Truncation

System:

x = Ax + Bu

y = Cx

Dimensions:

Input: u(t) ∈ R16

State: x(t) ∈ R256

Output: y(t) ∈ R16

Relative L2-Error:

Code:f = @(x,u,p) A*x+B*u;g = @(x,u,p) C*x;q = [16,256,16];t = [0,0.01,1.0];WX = emgr(f,g,q,t,’x’);[U, D, V] = svd(WX);

Parametrized Example Systems4

Parametrized General Control System:

x(t) = f (x(t), u(t), θ)y(t) = g(x(t), u(t), θ)

Parametrized Linear Control System:

x(t) = A(θ)x(t) + B(θ)u(t)y(t) = C (θ)x(t)

4Diagonal Entries of System Matrix A are considered parameters here.

Empirical Sensitivity Gramian [Sun’06, H.’13]

Parameter Decomposed System:

x = f (x , u) +P∑

k=1

f (x , θk)

⇒WC = WC ,0 +P∑

k=1

WC ,k

Sensitivity Gramian:

WS ,ii = tr(WC ,i )

Example: Sensitivity Analysis

System:

x = A(θ)x + Bu

y = Cx

Dimensions:

Input: u(t) ∈ R16

State: x(t) ∈ R256

Output: y(t) ∈ R16

Parameter: θ ∈ R256

Parameter Sensitivity:

Code:f = @(x,u,p) (A+diag(p))*x+B*u;g = @(x,u,p) C*x;q = [16,256,16];t = [0,0.01,1.0];WS = emgr(f,g,q,t,’s’,p,0,1,1);

Empirical Identifiability Gramian [Geffen’08]

Parameter Augmented System:(xθ

)=

(A 00 0

)(xθ

)+

(B0

)u

y =(C 0

)(xθ

)(

x(0)θ(0)

)=

(x0θ0

)Observability Gramian of Parameter Augmented System:

WO =

(WO WMW T

M WP

)Identifiability Gramian:(Schur Complement of Augmented Observability Gramian)

WI := WP −WMW−1O W T

M ≈WP

Example: Parameter Reduction

System:

x = A(θ)x + Bu

y = Cx

Dimensions:

Input: u(t) ∈ R16

State: x(t) ∈ R256

Output: y(t) ∈ R16

Parameter: θ ∈ R256

Relative L2-Error:

Code:f = @(x,u,p) (A+diag(p))*x+B*u;g = @(x,u,p) C*x;q = [16,256,16];t = [0,0.01,1.0];WI = emgr(f,g,q,t,’i’,p,0,1,0,1);[P, S, Q] = svd(WI{2});

Empirical Joint Gramian [H.’14]

Parameter Augmented System:(xθ

)=

(A 00 0

)(xθ

)+

(B0

)u

y =(C 0

)(xθ

)(

x(0)θ(0)

)=

(x0θ0

)Joint Gramian:(Cross Gramian of Parameter Augmented System)

WJ =

(WX WM0 0

)Cross-Identifiability Gramian:(Schur Complement of Symmetric Part of Joint Gramian)

WI := −12WM(WX + W T

X )−1W TM

Example: Combined Reduction

System:

x = A(θ)x + Bu

y = Cx

Dimensions:

Input: u(t) ∈ R16

State: x(t) ∈ R256

Output: y(t) ∈ R16

Parameter: θ ∈ R256

Relative L2-Error:

Code:f = @(x,u,p) (A+diag(p))*x+B*u;g = @(x,u,p) C*x;q = [16,256,16];t = [0,0.01,1.0];WJ = emgr(f,g,q,t,’j’,p,0,1,0,1);[U, D, V] = svd(WJ{1});[P, S, Q] = svd(WJ{2});

More emgr Use Cases

Sensitivity Analysis via System GainNonlinearity QuantificationDecentralized ControlSolve Lyapunov Equations and Sylvester Equations

emgr (Signature)

Minimal Signature:W = emgr(f, g, q, t, w);

Full Signature:W = emgr(f, g, q, t, w, pr, nf, ut, us, xs, um, xm, yd);

f - System Function Handleg - Output Function Handleq - System Dimension(Inputs, States, Outputs)t - Time(Start, Step, Stop)w - Gramian Type(’c’, ’o’, ’x’, ’y’, ’s’, ’i’, ’j’)

pr - Parametersnf - Configurationut - Inputus - Steady State Inputxs - Steady Stateum - Input Scalesxm - State Scalesyd - Data

emgr (Configuration Options)

1 Center: Steady-State, Mean, Median, Last, Principal, Zero2 Input Scale Sequence: Linear, Log, Geometric, Single3 State Scale Sequence: Linear, Log, Geometric, Single4 Input Rotations: Unit, Reciproce, Dyadic, Single5 State Rotations: Unit, Reciproce, Dyadic, Single6 Double Run?7 Use Measured Data?8 Robust Parameters: None, Active or Passive?9 Adjust Parameter Scales?10 Schur Complement on WI ?11 Enforce Gramian Symmetry?12 Solver/Integrator: RK1,AB2,RRK2,ARK3,Leapfrog,CUSTOM

Solver signature: x = CUSTOM_ODE(f,h,T,z,u,p)

Public Service Announcement

DownloadDocumentationDemos

source: clker.com

at:

http://gramian.de

Outlook

Technical:SymmetrizersNon-Symmetric Cross GramiansParametrized Linear Cross Gramian

Implementation:Accelerator Offloading (Xeon Phi)Python Version(into pyMOR: http://pymor.org)(Julia Version)(asm.js Version)

Listings

ORMS (Oberwolfach References on Mathematical Software)

swMATH (Information Service for Mathematical Software)

MoRePas (Model Reduction for Parametrized Systems)

MORwiki (Model Order Reduction Wiki)

GitHub

tl;dl

emgr - Empirical Gramian Framework

http://gramian.de

Thanks!

Code & References

Companion Source Code: http://j.mp/esco2014

C. Himpe, M. Ohlberger; “A Unified Software Frameworkfor Empirical Gramians”; Journal of Mathematics 2013:1-6,2013

C. Himpe, M. Ohlberger; “Cross-Gramian Based CombinedState and Parameter Reduction for Large-Scale ControlSystems”; Mathematical Problems in Engineering (Accepted),2014

C. Himpe, M. Ohlberger; “Model Reduction for ComplexHyperbolic Networks” Proceedings of the ECC’14(Accepted), 2014