vikram garg pecosroystgnr/uncertainty_propagation-talk.pdfinstitute for computational engineering...

PECOSPredictive Engineering and Computational Sciences

Accelerating Forward Uncertainty Propagation

Roy H. StognerVikram Garg

Institute for Computational Engineering and SciencesThe University of Texas at Austin

October 18, 2012

Stogner & Garg Uncertainty Propagation October 18, 2012 1 / 31

Outline

1 Notation

2 Enhanced Model EvaluationGoal-Oriented Adaptivity

3 Enhanced Monte CarloEnhanced Sampling StrategiesEnhanced EstimatorsResults


Notation

Notation

Primal/Forward Problem

Parameter value(s) ξ ∈ ΞR ⊂ Ξ ≡ RNp

System solution u(x, t; ξ) ∈ UR ⊂ U

Primal weighted residual R(u(ξ),v; ξ) ≡ 0 ∀v ∈ V

Quantity of interest functional Q : UR ×ΞR → R

Quantity of interest output q = Q(u(ξ), ξ)

• Highly nonlinear forward solves

• Large Np-dimensional parameter space

• Few quantities of interest


Enhanced Model Evaluation Goal-Oriented Adaptivity

Notation

Adjoint Problem

Adjoint solution z(x, t; ξ) ∈ V

Adjoint equation Ru(u, z; ξ)(u) ≡ Qu(u; ξ)(u) ∀u ∈ U

Adjoint-based sensitivities dqdξ = Qξ(u; ξ)−Rξ(u, z; ξ)

Adjoint-based error Q(uh)−Q(u) = R(uh, z; ξ) +H.O.T.

• Linear, relatively cheap adjoint solves

• One linear solve per QoI

• Sensitivity: One dot product per parameter

• Error: One weighted residual evaluation per QoI



Adjoint Residual Error Indicator

Error Indicators• Efficiently bounding eQ via per-element terms

• From our error estimator,

R(uh, z; ξ) =∑E

RE( uh∣∣∣E, z|E ; ξ)

• zh is cheaper than higher order approximation of z;

• zh is already needed for sensitivity calculations

Ignoring higher order terms:

q − qh = −Ru(u, z − zh; ξ)(u− uh)∣∣∣q − qh∣∣∣ ≤∑E

∣∣∣∣REu ∣∣∣∣B(UE ,VE∗)

∣∣∣∣∣∣ u|E − uh∣∣∣E

∣∣∣∣∣∣UE

∣∣∣∣∣∣ z|E − zh∣∣∣E

∣∣∣∣∣∣VE



Adjoint Residual Error Indicator

ηE ≡∣∣∣∣ u|E − uh∣∣E∣∣∣∣UE

∣∣∣∣ z|E − zh∣∣E∣∣∣∣VE

(neglecting∣∣∣∣REu ∣∣∣∣)

Primal Error Adjoint Error QoI Error



Norm vs QoI Based Refinement

2 2.5 3 3.5 4 4.5−6.5

−6

−5.5

−5

−4.5

−4

−3.5

−3

−2.5

Log10

(Dofs), Degrees of Freedom

Lo

g1

0(A

bso

lute

Err

or)

Convergence of the interior QoI, ε = 10−8

Uniform Refinements

Flux Jump Error Estimator

Adjoint Error Estimator

• Global error indicatorignores QoI sensitivity,plateaus

• Rapid convergence fromadjoint-based refinement



Multiphysics and Adjoint Weighting

Where does∣∣∣∣REu ∣∣∣∣B(UE ,VE∗)

become non-negligible?

• Spatially varying constitutive parameters, nonlinearities

• Variable scaling in dimensionalized multiphysics problems

• Interaction pattern between multiphysics variables

Generalized Adjoint Residual Error Indicator

∣∣∣Q(uh)−Q(u)∣∣∣ ≤ ∣∣∣R(uh, z − zh)(u− uh)

∣∣∣≤∑E

∣∣∣∣∣∣zi − zhi∣∣∣∣∣∣iMij

∣∣∣∣∣∣uj − uhj∣∣∣∣∣∣j

Choose M , ||·||i, ||·||j to match physics



Multiphysics Adjoint Weighting Example

Stokes Flow

Q(u)−Q(uh) =

∫Ω∇(~u− ~uh) · ∇( ~u∗ − ~u∗

h) dx

−∫

Ω∇ · (~u− ~uh) (p∗ − p∗h) dx−

∫Ω∇ · ( ~u∗ − ~u∗

h) (p− ph) dx

Matrix Form

Q(u)−Q(uh) ≤[e(u1)e(u2)

]T [1 00 1

] [e(u∗1)e(u∗2)

]

+

e(u1,1)e(u2,2)e(p)

T 0 0 10 0 11 1 0

e(u∗1,1)

e(u∗2,2)

e(p∗)



Multiphysics Adjoint Weighting Example

2.5 3 3.5 4 4.5−8

−7

−6

−5

−4

−3

−2

Log10

(Dofs), Degrees of Freedom

Log

10(R

ela

tive E

rror)

UniformAdjoint Residual (naive weighting)Adjoint Residual (matrix weighting)

• Stokes flow +transport, cornersingularities

• Naive weighting“staggers”

• Matrix weightingconvergessmoothly topenalty BCprecision


Enhanced Monte Carlo Enhanced Sampling Strategies

Notation

Uncertainty Propagation Problems

Input PDF ξ ∼ pξ

Expected (mean) output q ≡ E [q] ≡∫ΞR q(ξ)dpξ

Output standard deviation σq ≡√

E[(q − q)2

]Output risk PC ≡ E [q ∈ Cq]

• Integrals in high dimensional spaces RNp• “Curse of dimensionality”

I Deterministic integration cost exponential in Np

• Monte Carlo: dimension independent



Monte Carlo Integration

Errors

• qMC − q variance: σ2e[q] = σ2

Ns

•(σMCq

)2 − σ2q variance: σ2

e[σ2] = σ4(

2Ns−1 + κ

Ns

)• PMC

C − PC variance: σ2e[PC ] =

PC−P 2C

Ns

• Sampling-based scheme errors are PDFs

• Errors based on variance σ2 of sampled entity

• Error “bound” width: σe ∝ N−1/2s

• Importance sampling improves convergence constant, not rate



Latin Hypercube Sampling

Algorithm• Quantiles in each parameter• Samples permuted to bins

I Reducing correlations?

• Random sample placementwithin bins

Uses• Reduces variance from

additive components

• Higher-order convergencefor separable functions



Hierarchical Latin Hypercube Sequences

• LHS improves convergence but not naturally incremental

• HLHS based methods offer more flexible incrementation

(a) Hierarchical Latin Hypercubes

X5,0

X4,0

X3,3

X2,7

X3,2

X4,1

X2,6

i = 5, 25 samples each




(b) Tree Structure



Numerical Experiments

• Correlation reduction used for both ILHS and HLHS

• HILHS points show finer grained increments

100

101

102

103

104

10−3

10−2

10−1

100

log10

(Ns)

log

10(e

µ)

Mean error in the mean

HILHS

ILHS

SRS

(c) Q(ξ) = round(∑Np=16i=1 ξi)

100

101

102

103

104

10−2

10−1

100

101

log10

(Ns)

log

10(e

µ)

Mean error in the mean

HILHS

ILHS

(d) Q(ξ) = e

Np=16∑i=1

ξi


Enhanced Monte Carlo Enhanced Estimators

Control VariateKnown Surrogate• Quantity of interest q has correlated surrogate statistic qs• “Known” mean qs ≡ E [qs]

cqqs ≡E [(q − q)(qs − qs)]

σqσqs

Unbiased, Variance-reduced estimator

E [q] = E [q − αqs] + E [αqs] = E [s]

s ≡ q − αqs + αqs

σ2s = σ2

q − 2αcqqsσqσqs + α2σ2qs

Integrating s via MC sampling, α ≡ 1, qs → q gives σe[q] → 0.



Sensitivity Derivative EnhancementSDEMC Surrogate• Justification: Adjoints are cheap• One linearization at input mean

I Adds one forward, one sensitivity solve

• qs(ξ) ≡ q(ξ) + qξ(ξ)(ξ − ξ)



Local Sensitivity Derivative Enhancement

LSDEMC Surrogate• Take any input sample set• Evaluate forward and adjoint at each

I Adds one sensitivity solve per sample

• Linearize around each nearby sample



Bias

Problem• Use every sample to construct surrogate?

• E[∑Ns

i=1 q(ξi)− qs(ξi)]

= 0

• E[qL]

= qLs 6= q

• May have systemic bias for any problem

• Huge error for high-dimensional benchmark problems

Solution• Subdivide sample set (e.g. 2, 4, 8 subsets; HLHS applicable)

• Use one subset to construct surrogate, remainder to integrate bias

• Repeat for all subsets; average.



Bias

Unbiased LSDEMC Estimator

1

NR

NR∑r=1

1

Nss

Nss∑l=1

Qr,1(ξl) +1

Ns − NsNR

Ns− NsNR∑

c=1

(Q(ξc)−Qr,1(ξc))



Convergence

Proofs• Holding Ns/NR constant

• Assuming sufficiently smooth Q

• LSDEMC converges asymptotically faster than Monte Carlo

Issues• How many subsets is optimal?

I More subsets == larger N integrating bias termI Fewer subsets == smaller σ in bias termI Numerical results show dimension dependence

• What convergence rate to expect?


Enhanced Monte Carlo Results

Visualizing 250 Thousand Trials

Methodology• Dependent variable: mean(abs(error)) from 256 trials per case

• 32, 128, 512 true sample evaluations per trial

• 32, 128, 512 surrogate samples per true sample

• Statistics: mean, standard deviation

• Sampling: SRS, HLHS

• Methods: MC, SDEMC, LSDEMC2/4/8

Graph Key• Purple: MC, Red: SDEMC

• Blue/Cyan/Green: LSDEMC2/4/8

• X: SRS, O: LHS1e-06

1e-05

0.0001

0.001

0.01

0.1

10 100 1000 10000

Err

or

in A

ppro

xim

ate

d O

utp

ut

Number of Forward Evaluations

Forward UQ Convergence: Mean

MC+SRSSDEMC+SRS

LSDMC2+SRSLSDMC4+SRSLSDMC8+SRS

MC+LHSSDEMC+LHS

LSDMC2+LHSLSDMC4+LHSLSDMC8+LHS



Benchmark Problem: Smooth

Normal→ Lognormal Distribution

ξ ≡(ξ1, ...ξNp

)q(ξ) ≡ e

∑i ξi

ξi ∼ N

(µ

Np,

σ√Np

)q ∼ Log-N (µ, σ)

• µ ≡ 1, σ ≡ 1

• Arbitrary dimensionality Np

• All parameters equally significant

• Simple analytic exact solution moments

• Variance, MC error independent of Np



1 Parameter

0.01

0.1

1

10 100 1000

Err

or

in A

pp

roxim

ate

d O

utp

ut


Forward UQ Convergence: Exp Benchmark, Mean

MC+SRSSDEMC+SRS

LSDEMC2+SRSLSDEMC4+SRSLSDEMC8+SRS

MC+LHSSDEMC+LHS

LSDEMC2+LHSLSDEMC4+LHSLSDEMC8+LHS



4 Parameters

0.01

0.1

1

10 100 1000

Err

or

in A

pp

roxim

ate

d O

utp

ut



MC+SRSSDEMC+SRS


MC+LHSSDEMC+LHS




16 Parameters

0.01

0.1

1

10 100 1000

Err

or

in A

pp

roxim

ate

d O

utp

ut



MC+SRSSDEMC+SRS


MC+LHSSDEMC+LHS




64 Parameters1

10 100 1000

Err

or

in A

pp

roxim

ate

d O

utp

ut



MC+SRSSDEMC+SRS


MC+LHSSDEMC+LHS




Benchmark Problem: C0 abs

Normal→ Folded Normal Distribution

ξ ≡(ξ1, ...ξNp

)q(ξ) ≡

∣∣∣∣∣∑i

ξi

∣∣∣∣∣ξi ∼ N

(µ

Np,

σ√(Np

)q ∼ F −N (µ, σ)

• µ ≡ 1, σ ≡ 1

• Same benefits as previous benchmark

• Response function now piecewise linear

• Differentiable except on one hyperplane

• Derivative defined as ~0 there



4 Parameters

0.001

0.01

0.1

1

10 100 1000

Err

or

in A

ppro

xim

ate

d O

utp

ut


Forward UQ Convergence: Abs Benchmark, Mean

MC+SRSSDEMC+SRS


MC+LHSSDEMC+LHS




64 Parameters

0.01

0.1

1

10 100 1000

Err

or

in A

pp

roxim

ate

d O

utp

ut


Forward UQ Convergence: Abs Benchmark, Mean

MC+SRSSDEMC+SRS


MC+LHSSDEMC+LHS




Future Improvements

Adaptivity• Regularization of “peak value” QoIs

HILHS• Correlation Reduction improvements

LSDEMC• Stochastic Voronoi moments lemma

• Anisotropic Voronoi metric

• Smooth (multi-sample-based) surrogates

Questions?


vikram garg pecosroystgnr/uncertainty_propagation-talk.pdfinstitute for computational engineering...

Documents