Uncertainty Quantification in Bayesian Inversion

Andrew Stuart

Department of MathematicsUniversity of Warwick

EPSRC, ERC and ONR.

SIAM UQApril 1st, 2014

Savannah, Georgia, USA

(Warwick University) UQ in Bayesian Inversion Andrew Stuart 1 / 25

The Context

Outline

1 The Context

2 Examples

3 Mathematical Foundations

4 Algorithms

5 Conclusions

(Warwick University) UQ in Bayesian Inversion Andrew Stuart 2 / 25

The Context

Vanilla Uncertainty Quantification (UQ)

(Warwick University) UQ in Bayesian Inversion Andrew Stuart 3 / 25

The Context

Bayesian Inverse Problem (BIP)

(Warwick University) UQ in Bayesian Inversion Andrew Stuart 4 / 25

The Context

BIP+UQ (“Inverse UQ”)

(Warwick University) UQ in Bayesian Inversion Andrew Stuart 5 / 25

The Context

Example: BIP in Subsurface (Permeability)

200 400 600 800 1000 1200 1400 1600

200

400

600

800

1000

1200

1400

1600

X (meters)

Y (

me

ters

)

−31.5

−31

−30.5

−30

−29.5

−29

−28.5

−28

−27.5

−27

−26.5

20 40 60 80 100 120 140 160

20

40

60

80

100

120

140

160

X (meters)

Y (

me

ters

)

−31

−30

−29

−28

−27

−26

20 40 60 80 100 120 140 160

20

40

60

80

100

120

140

160

X (meters)

Y (

me

ters

)

−32

−31

−30

−29

−28

−27

−26

20 40 60 80 100 120 140 160

20

40

60

80

100

120

140

160

X (meters)

Y (

me

ters

)

−30.5

−30

−29.5

−29

−28.5

−28

−27.5

−27

−26.5

−26

−25.5

200 400 600 800 1000 1200 1400 1600

200

400

600

800

1000

1200

1400

1600

X (meters)

Y (

me

ters

)

−30

−29.5

−29

−28.5

−28

−27.5

−27

−26.5

−26

200 400 600 800 1000 1200 1400 1600

200

400

600

800

1000

1200

1400

1600

X (meters)

Y (

me

ters

)

−30.5

−30

−29.5

−29

−28.5

−28

−27.5

−27

−26.5

200 400 600 800 1000 1200 1400 1600

200

400

600

800

1000

1200

1400

1600

X (meters)

Y (

me

ters

)

−30

−29.5

−29

−28.5

−28

−27.5

−27

−26.5

200 400 600 800 1000 1200 1400 1600

200

400

600

800

1000

1200

1400

1600

X (meters)

Y (

me

ters

)

−30.5

−30

−29.5

−29

−28.5

−28

−27.5

−27

−26.5

−26

Figure: Prior samples (top row) versus posterior samples (bottom row)

(Warwick University) UQ in Bayesian Inversion Andrew Stuart 6 / 25

The Context

Example: BIP+UQ in Subsurface (QoI=Oil Production)

Vanilla UQ on permeability⇒ larger uncertainty in QoI prediction.BIP+UQ on permeability⇒ smaller uncertainty in QoI prediction.

0 1 2 3 4 5 6 70

0.1

0.2

0.3

0.4

0.5

0.6

0.7

oil

re

co

ve

ry f

ac

tor

time (years)0 1 2 3 4 5 6 7

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

time (years)

oil

re

co

ve

ry f

ac

tor

200 400 600 800 1000 1200 1400 1600

200

400

600

800

1000

1200

1400

1600

X (meters)

Y (

me

ters

)

−31.5

−31

−30.5

−30

−29.5

−29

−28.5

−28

−27.5

−27

−26.5

200 400 600 800 1000 1200 1400 1600

200

400

600

800

1000

1200

1400

1600

X (meters)

Y (

me

ters

)

−30.5

−30

−29.5

−29

−28.5

−28

−27.5

−27

−26.5

−26

(Warwick University) UQ in Bayesian Inversion Andrew Stuart 7 / 25

The Context

Approximation of Posterior µ on Banach Space X by µN on RN

(Warwick University) UQ in Bayesian Inversion Andrew Stuart 8 / 25

The Context

History

Vanilla UQ: Ghanem and Spanos 91, . . . , Schwab et al 00s.Classical Inversion: Keller 66, . . . , Engl et al 96 – no uncertainty.Bayesian Inversion: Franklin 70 – heat equation: N =∞.Bayesian Inversion: Kaipio/Somersalo 05 – methodology: N <∞.Bayesian Inversion: Lasanen 02, 12 – rigorous N =∞.Bayesian Inversion: Lassas/Siltanen 04, 09 – discretization inv.BIP Applications (Subsurface): Tarantola 90s, Oliver 00s. N <∞.BIP Applications (Fluids): Bennett, Lorenc 90s N <∞.MCMC for BIP: Efendiev et al 05 —> N <∞.Approx. for BIP: Marzouk/Xui 09, Mallick/Efendiev 10. N <∞.

(Warwick University) UQ in Bayesian Inversion Andrew Stuart 9 / 25

Examples

Outline

1 The Context

2 Examples

3 Mathematical Foundations

4 Algorithms

5 Conclusions

(Warwick University) UQ in Bayesian Inversion Andrew Stuart 10 / 25

Examples

Linear Inverse Problem

Consider the bounded linear map:

K : X → Y .

Find u ∈ X .Given noisy observations

y = Ku + η.

Abstractly: for G : X 7→ Y find u given

y = G(u) + η, noise.

(Warwick University) UQ in Bayesian Inversion Andrew Stuart 11 / 25

Examples

Groundwater Flow

Consider the Darcy Flow, with log permeability u ∈ X = L∞(D) :

−∇ ·(exp(u)∇p

)= 0, x ∈ D,

u = g, x ∈ ∂D.

Find u ∈ X .Given noisy observations

yj = p(xj) + ηj .

Abstractly: for G : X 7→ Y = RJ find u given

y = G(u) + η, noise.

(Warwick University) UQ in Bayesian Inversion Andrew Stuart 12 / 25

Examples

Data Assimilation in Fluid Mechanics

Consider the Navier-Stokes equation:

dvdt

+ νAv + B(v , v) = f , v(0) = u ∈ X := L2div(T2)

Find u ∈ X .Given noisy Eulerian observations

yj,k = v(xj , tk ) + ηj,k

Abstractly: for G : X 7→ Y = RJK find u given

y = G(u) + η, noise.

(Warwick University) UQ in Bayesian Inversion Andrew Stuart 13 / 25

Mathematical Foundations

Outline

1 The Context

2 Examples

3 Mathematical Foundations

4 Algorithms

5 Conclusions

(Warwick University) UQ in Bayesian Inversion Andrew Stuart 14 / 25

Mathematical Foundations

Priors: Random Functions

{ϕj}∞j=1 an infinite sequence in the Banach space X , ‖ϕj‖ = 1.

γ = {γj}∞j=1 ∈ `pw (R).

ξ = {ξj}∞j=1 i.i.d centred sequence in R, Eξ21 = 1.

uj = γjξj .

u = m +∞∑

j=1

ujϕj .

Gaussian (Karhunen-Loeve 47), Besov (Lassas et al 09), CompactlySupported (Schwab, 00s); Kahane (Random functions 96). All N =∞.

(Warwick University) UQ in Bayesian Inversion Andrew Stuart 15 / 25

Mathematical Foundations

Priors: Hierarchical

Write Gaussian prior as u ∼ N(m,C) where

Cϕj = γ2j ϕj .

(example) Hierarchical Prior:

u|δ ∼ N(0, δ−1C).

δ ∼ Ga(α, β).

Bardsley 12. N <∞.Agapiou, Bardsley, Papaspiliopoulis, S 13. N =∞.

(Warwick University) UQ in Bayesian Inversion Andrew Stuart 16 / 25

Mathematical Foundations

Priors: Geometric

a1b1

a2b2

a3

b3......

anbn ,

a1b1

a2

b2a3

b3......

anbn,

Carter 04, Landa and Horne 97, Xie/Efendiev/Datta-Gupta 11 N <∞.Iglesias, Lin, S 13. N =∞.

(Warwick University) UQ in Bayesian Inversion Andrew Stuart 17 / 25

Mathematical Foundations

Well-Posed Posterior (for above Priors/Examples)

Prior u ∼ µ0,

Likelihood y |u ∼ N(G(u), Γ).

TheoremAssume G ∈ C(X ,Y ), µ0(X ) = 1. Posterior µy on u|y :

µy (du) ∝ exp(−Φ(u; y)

)µ0(du), Φ(u; y) :=

12∥∥Γ−

12(y − G(u)

)∥∥2;

and y 7→ µy is Hölder in the Hellinger metric.

Cotter, Dashti, Robinson S 09; Dashti, Harris, S 12; Iglesias, Lin, S 14.

(Warwick University) UQ in Bayesian Inversion Andrew Stuart 18 / 25

Mathematical Foundations

Recovery of Truth

Consider data y given from truth u† by

y = G(u†) + ε η0, η0 ∼ N(0, Γ0).

How close is posterior µy to the truth u†? For many of the precedingproblems we have (refinements of) results of the type:

Theorem

For any δ > 0, Pµy (|u − u†| > δ

)→ 0 as ε→ 0.

Linear: Van Der Vaart et al 12, Agapiou, Larsson, S 13; Ray 13Groundwater: Vollmer 13; Navier-Stokes: Sanz and S 14.

(Warwick University) UQ in Bayesian Inversion Andrew Stuart 19 / 25

Algorithms

Outline

1 The Context

2 Examples

3 Mathematical Foundations

4 Algorithms

5 Conclusions

(Warwick University) UQ in Bayesian Inversion Andrew Stuart 20 / 25

Algorithms

Forward Error = Inverse Error

Approximate forward map G by a numerical method to obtain GN

satisfying, for u in X ,

|G(u)− GN(u)| ≤ ψ(N)→ 0

as N →∞. Leads to approximate posterior measure µy ,N .

TheoremFor appropriate class of test functions f : X → S:

‖Eµyf (u)− Eµ

y,Nf (u)‖S ≤ Cψ(N).

Cotter, Dashti S 10.

(Warwick University) UQ in Bayesian Inversion Andrew Stuart 21 / 25

Algorithms

Faster MCMC

TheoremFor a wide range of priors/model problems, usingdiffusion limits to SPDEs andspectral gaps in X we have:

Standard MCMC algorithms mix in O(Na) steps, a > 0.

New MCMC algorithms mix in O(1) steps.

Mattingly, Pillai, S, Thiery 12–14, TheoryHairer, S, Vollmer 14, TheoryAgapiou, Bardsley, Papaspiliopoulis, S 13, Theory (hierarchical)Ghattas/Bui-Thanh 10s, ApplicationsCliffe/Ernst/Sprungk 14, Cui/Law/Marzouk 14, Improved Algorithms

(Warwick University) UQ in Bayesian Inversion Andrew Stuart 22 / 25

Algorithms

Other Directions

Sparse determinsitic approximation of posterior expectations.Schwab and S 12, Schwab and Schillings 14.Multilevel MCMC.Ha Hoang, Schwab, S 13, Ketelsen, Scheichl, Teckentrup 13.Benchmarking ad hoc algorithmsLaw and S 12 (Fluids), Iglesias, Law and S 13 (Subsurface).Best Dirac approximation ν = δu? (MAP).Dahsti, Law, S, Voss 13, Dunlop, S 14.Best Gaussian approximation ν = N(m,C) wrt relative entropy.Pinski, Simpson, S and Weber 13.

(Warwick University) UQ in Bayesian Inversion Andrew Stuart 23 / 25

Conclusions

Outline

1 The Context

2 Examples

3 Mathematical Foundations

4 Algorithms

5 Conclusions

(Warwick University) UQ in Bayesian Inversion Andrew Stuart 24 / 25

Conclusions

Conclusions

UQ+BIP: formidable mathematical and computational challenges.

Balance: numerical analysis and statistical approximations.

Think N =∞, implement N <∞.

This is the beginning not the end.

(Warwick University) UQ in Bayesian Inversion Andrew Stuart 25 / 25

Conclusions

Conclusions

UQ+BIP: formidable mathematical and computational challenges.

Balance: numerical analysis and statistical approximations.

Think N =∞, implement N <∞.

This is the beginning not the end.

(Warwick University) UQ in Bayesian Inversion Andrew Stuart 25 / 25