![Page 1: Uncertainty Quantification in Bayesian Inversionhomepages.warwick.ac.uk/~masdr/TALKS/SIAM_UQ.pdf · 2014. 4. 1. · New MCMC algorithms mix in O(1) steps. Mattingly, Pillai, S, Thiery](https://reader035.vdocuments.us/reader035/viewer/2022071420/6119d6357dc49d2eb021420c/html5/thumbnails/1.jpg)
Uncertainty Quantification in Bayesian Inversion
Andrew Stuart
Department of MathematicsUniversity of Warwick
EPSRC, ERC and ONR.
SIAM UQApril 1st, 2014
Savannah, Georgia, USA
(Warwick University) UQ in Bayesian Inversion Andrew Stuart 1 / 25
![Page 2: Uncertainty Quantification in Bayesian Inversionhomepages.warwick.ac.uk/~masdr/TALKS/SIAM_UQ.pdf · 2014. 4. 1. · New MCMC algorithms mix in O(1) steps. Mattingly, Pillai, S, Thiery](https://reader035.vdocuments.us/reader035/viewer/2022071420/6119d6357dc49d2eb021420c/html5/thumbnails/2.jpg)
The Context
Outline
1 The Context
2 Examples
3 Mathematical Foundations
4 Algorithms
5 Conclusions
(Warwick University) UQ in Bayesian Inversion Andrew Stuart 2 / 25
![Page 3: Uncertainty Quantification in Bayesian Inversionhomepages.warwick.ac.uk/~masdr/TALKS/SIAM_UQ.pdf · 2014. 4. 1. · New MCMC algorithms mix in O(1) steps. Mattingly, Pillai, S, Thiery](https://reader035.vdocuments.us/reader035/viewer/2022071420/6119d6357dc49d2eb021420c/html5/thumbnails/3.jpg)
The Context
Vanilla Uncertainty Quantification (UQ)
(Warwick University) UQ in Bayesian Inversion Andrew Stuart 3 / 25
![Page 4: Uncertainty Quantification in Bayesian Inversionhomepages.warwick.ac.uk/~masdr/TALKS/SIAM_UQ.pdf · 2014. 4. 1. · New MCMC algorithms mix in O(1) steps. Mattingly, Pillai, S, Thiery](https://reader035.vdocuments.us/reader035/viewer/2022071420/6119d6357dc49d2eb021420c/html5/thumbnails/4.jpg)
The Context
Bayesian Inverse Problem (BIP)
(Warwick University) UQ in Bayesian Inversion Andrew Stuart 4 / 25
![Page 5: Uncertainty Quantification in Bayesian Inversionhomepages.warwick.ac.uk/~masdr/TALKS/SIAM_UQ.pdf · 2014. 4. 1. · New MCMC algorithms mix in O(1) steps. Mattingly, Pillai, S, Thiery](https://reader035.vdocuments.us/reader035/viewer/2022071420/6119d6357dc49d2eb021420c/html5/thumbnails/5.jpg)
The Context
BIP+UQ (“Inverse UQ”)
(Warwick University) UQ in Bayesian Inversion Andrew Stuart 5 / 25
![Page 6: Uncertainty Quantification in Bayesian Inversionhomepages.warwick.ac.uk/~masdr/TALKS/SIAM_UQ.pdf · 2014. 4. 1. · New MCMC algorithms mix in O(1) steps. Mattingly, Pillai, S, Thiery](https://reader035.vdocuments.us/reader035/viewer/2022071420/6119d6357dc49d2eb021420c/html5/thumbnails/6.jpg)
The Context
Example: BIP in Subsurface (Permeability)
200 400 600 800 1000 1200 1400 1600
200
400
600
800
1000
1200
1400
1600
X (meters)
Y (
me
ters
)
−31.5
−31
−30.5
−30
−29.5
−29
−28.5
−28
−27.5
−27
−26.5
20 40 60 80 100 120 140 160
20
40
60
80
100
120
140
160
X (meters)
Y (
me
ters
)
−31
−30
−29
−28
−27
−26
20 40 60 80 100 120 140 160
20
40
60
80
100
120
140
160
X (meters)
Y (
me
ters
)
−32
−31
−30
−29
−28
−27
−26
20 40 60 80 100 120 140 160
20
40
60
80
100
120
140
160
X (meters)
Y (
me
ters
)
−30.5
−30
−29.5
−29
−28.5
−28
−27.5
−27
−26.5
−26
−25.5
200 400 600 800 1000 1200 1400 1600
200
400
600
800
1000
1200
1400
1600
X (meters)
Y (
me
ters
)
−30
−29.5
−29
−28.5
−28
−27.5
−27
−26.5
−26
200 400 600 800 1000 1200 1400 1600
200
400
600
800
1000
1200
1400
1600
X (meters)
Y (
me
ters
)
−30.5
−30
−29.5
−29
−28.5
−28
−27.5
−27
−26.5
200 400 600 800 1000 1200 1400 1600
200
400
600
800
1000
1200
1400
1600
X (meters)
Y (
me
ters
)
−30
−29.5
−29
−28.5
−28
−27.5
−27
−26.5
200 400 600 800 1000 1200 1400 1600
200
400
600
800
1000
1200
1400
1600
X (meters)
Y (
me
ters
)
−30.5
−30
−29.5
−29
−28.5
−28
−27.5
−27
−26.5
−26
Figure: Prior samples (top row) versus posterior samples (bottom row)
(Warwick University) UQ in Bayesian Inversion Andrew Stuart 6 / 25
![Page 7: Uncertainty Quantification in Bayesian Inversionhomepages.warwick.ac.uk/~masdr/TALKS/SIAM_UQ.pdf · 2014. 4. 1. · New MCMC algorithms mix in O(1) steps. Mattingly, Pillai, S, Thiery](https://reader035.vdocuments.us/reader035/viewer/2022071420/6119d6357dc49d2eb021420c/html5/thumbnails/7.jpg)
The Context
Example: BIP+UQ in Subsurface (QoI=Oil Production)
Vanilla UQ on permeability⇒ larger uncertainty in QoI prediction.BIP+UQ on permeability⇒ smaller uncertainty in QoI prediction.
0 1 2 3 4 5 6 70
0.1
0.2
0.3
0.4
0.5
0.6
0.7
oil
re
co
ve
ry f
ac
tor
time (years)0 1 2 3 4 5 6 7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
time (years)
oil
re
co
ve
ry f
ac
tor
200 400 600 800 1000 1200 1400 1600
200
400
600
800
1000
1200
1400
1600
X (meters)
Y (
me
ters
)
−31.5
−31
−30.5
−30
−29.5
−29
−28.5
−28
−27.5
−27
−26.5
200 400 600 800 1000 1200 1400 1600
200
400
600
800
1000
1200
1400
1600
X (meters)
Y (
me
ters
)
−30.5
−30
−29.5
−29
−28.5
−28
−27.5
−27
−26.5
−26
(Warwick University) UQ in Bayesian Inversion Andrew Stuart 7 / 25
![Page 8: Uncertainty Quantification in Bayesian Inversionhomepages.warwick.ac.uk/~masdr/TALKS/SIAM_UQ.pdf · 2014. 4. 1. · New MCMC algorithms mix in O(1) steps. Mattingly, Pillai, S, Thiery](https://reader035.vdocuments.us/reader035/viewer/2022071420/6119d6357dc49d2eb021420c/html5/thumbnails/8.jpg)
The Context
Approximation of Posterior µ on Banach Space X by µN on RN
(Warwick University) UQ in Bayesian Inversion Andrew Stuart 8 / 25
![Page 9: Uncertainty Quantification in Bayesian Inversionhomepages.warwick.ac.uk/~masdr/TALKS/SIAM_UQ.pdf · 2014. 4. 1. · New MCMC algorithms mix in O(1) steps. Mattingly, Pillai, S, Thiery](https://reader035.vdocuments.us/reader035/viewer/2022071420/6119d6357dc49d2eb021420c/html5/thumbnails/9.jpg)
The Context
History
Vanilla UQ: Ghanem and Spanos 91, . . . , Schwab et al 00s.Classical Inversion: Keller 66, . . . , Engl et al 96 – no uncertainty.Bayesian Inversion: Franklin 70 – heat equation: N =∞.Bayesian Inversion: Kaipio/Somersalo 05 – methodology: N <∞.Bayesian Inversion: Lasanen 02, 12 – rigorous N =∞.Bayesian Inversion: Lassas/Siltanen 04, 09 – discretization inv.BIP Applications (Subsurface): Tarantola 90s, Oliver 00s. N <∞.BIP Applications (Fluids): Bennett, Lorenc 90s N <∞.MCMC for BIP: Efendiev et al 05 —> N <∞.Approx. for BIP: Marzouk/Xui 09, Mallick/Efendiev 10. N <∞.
(Warwick University) UQ in Bayesian Inversion Andrew Stuart 9 / 25
![Page 10: Uncertainty Quantification in Bayesian Inversionhomepages.warwick.ac.uk/~masdr/TALKS/SIAM_UQ.pdf · 2014. 4. 1. · New MCMC algorithms mix in O(1) steps. Mattingly, Pillai, S, Thiery](https://reader035.vdocuments.us/reader035/viewer/2022071420/6119d6357dc49d2eb021420c/html5/thumbnails/10.jpg)
Examples
Outline
1 The Context
2 Examples
3 Mathematical Foundations
4 Algorithms
5 Conclusions
(Warwick University) UQ in Bayesian Inversion Andrew Stuart 10 / 25
![Page 11: Uncertainty Quantification in Bayesian Inversionhomepages.warwick.ac.uk/~masdr/TALKS/SIAM_UQ.pdf · 2014. 4. 1. · New MCMC algorithms mix in O(1) steps. Mattingly, Pillai, S, Thiery](https://reader035.vdocuments.us/reader035/viewer/2022071420/6119d6357dc49d2eb021420c/html5/thumbnails/11.jpg)
Examples
Linear Inverse Problem
Consider the bounded linear map:
K : X → Y .
Find u ∈ X .Given noisy observations
y = Ku + η.
Abstractly: for G : X 7→ Y find u given
y = G(u) + η, noise.
(Warwick University) UQ in Bayesian Inversion Andrew Stuart 11 / 25
![Page 12: Uncertainty Quantification in Bayesian Inversionhomepages.warwick.ac.uk/~masdr/TALKS/SIAM_UQ.pdf · 2014. 4. 1. · New MCMC algorithms mix in O(1) steps. Mattingly, Pillai, S, Thiery](https://reader035.vdocuments.us/reader035/viewer/2022071420/6119d6357dc49d2eb021420c/html5/thumbnails/12.jpg)
Examples
Groundwater Flow
Consider the Darcy Flow, with log permeability u ∈ X = L∞(D) :
−∇ ·(exp(u)∇p
)= 0, x ∈ D,
u = g, x ∈ ∂D.
Find u ∈ X .Given noisy observations
yj = p(xj) + ηj .
Abstractly: for G : X 7→ Y = RJ find u given
y = G(u) + η, noise.
(Warwick University) UQ in Bayesian Inversion Andrew Stuart 12 / 25
![Page 13: Uncertainty Quantification in Bayesian Inversionhomepages.warwick.ac.uk/~masdr/TALKS/SIAM_UQ.pdf · 2014. 4. 1. · New MCMC algorithms mix in O(1) steps. Mattingly, Pillai, S, Thiery](https://reader035.vdocuments.us/reader035/viewer/2022071420/6119d6357dc49d2eb021420c/html5/thumbnails/13.jpg)
Examples
Data Assimilation in Fluid Mechanics
Consider the Navier-Stokes equation:
dvdt
+ νAv + B(v , v) = f , v(0) = u ∈ X := L2div(T2)
Find u ∈ X .Given noisy Eulerian observations
yj,k = v(xj , tk ) + ηj,k
Abstractly: for G : X 7→ Y = RJK find u given
y = G(u) + η, noise.
(Warwick University) UQ in Bayesian Inversion Andrew Stuart 13 / 25
![Page 14: Uncertainty Quantification in Bayesian Inversionhomepages.warwick.ac.uk/~masdr/TALKS/SIAM_UQ.pdf · 2014. 4. 1. · New MCMC algorithms mix in O(1) steps. Mattingly, Pillai, S, Thiery](https://reader035.vdocuments.us/reader035/viewer/2022071420/6119d6357dc49d2eb021420c/html5/thumbnails/14.jpg)
Mathematical Foundations
Outline
1 The Context
2 Examples
3 Mathematical Foundations
4 Algorithms
5 Conclusions
(Warwick University) UQ in Bayesian Inversion Andrew Stuart 14 / 25
![Page 15: Uncertainty Quantification in Bayesian Inversionhomepages.warwick.ac.uk/~masdr/TALKS/SIAM_UQ.pdf · 2014. 4. 1. · New MCMC algorithms mix in O(1) steps. Mattingly, Pillai, S, Thiery](https://reader035.vdocuments.us/reader035/viewer/2022071420/6119d6357dc49d2eb021420c/html5/thumbnails/15.jpg)
Mathematical Foundations
Priors: Random Functions
{ϕj}∞j=1 an infinite sequence in the Banach space X , ‖ϕj‖ = 1.
γ = {γj}∞j=1 ∈ `pw (R).
ξ = {ξj}∞j=1 i.i.d centred sequence in R, Eξ21 = 1.
uj = γjξj .
u = m +∞∑
j=1
ujϕj .
Gaussian (Karhunen-Loeve 47), Besov (Lassas et al 09), CompactlySupported (Schwab, 00s); Kahane (Random functions 96). All N =∞.
(Warwick University) UQ in Bayesian Inversion Andrew Stuart 15 / 25
![Page 16: Uncertainty Quantification in Bayesian Inversionhomepages.warwick.ac.uk/~masdr/TALKS/SIAM_UQ.pdf · 2014. 4. 1. · New MCMC algorithms mix in O(1) steps. Mattingly, Pillai, S, Thiery](https://reader035.vdocuments.us/reader035/viewer/2022071420/6119d6357dc49d2eb021420c/html5/thumbnails/16.jpg)
Mathematical Foundations
Priors: Hierarchical
Write Gaussian prior as u ∼ N(m,C) where
Cϕj = γ2j ϕj .
(example) Hierarchical Prior:
u|δ ∼ N(0, δ−1C).
δ ∼ Ga(α, β).
Bardsley 12. N <∞.Agapiou, Bardsley, Papaspiliopoulis, S 13. N =∞.
(Warwick University) UQ in Bayesian Inversion Andrew Stuart 16 / 25
![Page 17: Uncertainty Quantification in Bayesian Inversionhomepages.warwick.ac.uk/~masdr/TALKS/SIAM_UQ.pdf · 2014. 4. 1. · New MCMC algorithms mix in O(1) steps. Mattingly, Pillai, S, Thiery](https://reader035.vdocuments.us/reader035/viewer/2022071420/6119d6357dc49d2eb021420c/html5/thumbnails/17.jpg)
Mathematical Foundations
Priors: Geometric
a1b1
a2b2
a3
b3......
anbn ,
a1b1
a2
b2a3
b3......
anbn,
Carter 04, Landa and Horne 97, Xie/Efendiev/Datta-Gupta 11 N <∞.Iglesias, Lin, S 13. N =∞.
(Warwick University) UQ in Bayesian Inversion Andrew Stuart 17 / 25
![Page 18: Uncertainty Quantification in Bayesian Inversionhomepages.warwick.ac.uk/~masdr/TALKS/SIAM_UQ.pdf · 2014. 4. 1. · New MCMC algorithms mix in O(1) steps. Mattingly, Pillai, S, Thiery](https://reader035.vdocuments.us/reader035/viewer/2022071420/6119d6357dc49d2eb021420c/html5/thumbnails/18.jpg)
Mathematical Foundations
Well-Posed Posterior (for above Priors/Examples)
Prior u ∼ µ0,
Likelihood y |u ∼ N(G(u), Γ).
TheoremAssume G ∈ C(X ,Y ), µ0(X ) = 1. Posterior µy on u|y :
µy (du) ∝ exp(−Φ(u; y)
)µ0(du), Φ(u; y) :=
12∥∥Γ−
12(y − G(u)
)∥∥2;
and y 7→ µy is Hölder in the Hellinger metric.
Cotter, Dashti, Robinson S 09; Dashti, Harris, S 12; Iglesias, Lin, S 14.
(Warwick University) UQ in Bayesian Inversion Andrew Stuart 18 / 25
![Page 19: Uncertainty Quantification in Bayesian Inversionhomepages.warwick.ac.uk/~masdr/TALKS/SIAM_UQ.pdf · 2014. 4. 1. · New MCMC algorithms mix in O(1) steps. Mattingly, Pillai, S, Thiery](https://reader035.vdocuments.us/reader035/viewer/2022071420/6119d6357dc49d2eb021420c/html5/thumbnails/19.jpg)
Mathematical Foundations
Recovery of Truth
Consider data y given from truth u† by
y = G(u†) + ε η0, η0 ∼ N(0, Γ0).
How close is posterior µy to the truth u†? For many of the precedingproblems we have (refinements of) results of the type:
Theorem
For any δ > 0, Pµy (|u − u†| > δ
)→ 0 as ε→ 0.
Linear: Van Der Vaart et al 12, Agapiou, Larsson, S 13; Ray 13Groundwater: Vollmer 13; Navier-Stokes: Sanz and S 14.
(Warwick University) UQ in Bayesian Inversion Andrew Stuart 19 / 25
![Page 20: Uncertainty Quantification in Bayesian Inversionhomepages.warwick.ac.uk/~masdr/TALKS/SIAM_UQ.pdf · 2014. 4. 1. · New MCMC algorithms mix in O(1) steps. Mattingly, Pillai, S, Thiery](https://reader035.vdocuments.us/reader035/viewer/2022071420/6119d6357dc49d2eb021420c/html5/thumbnails/20.jpg)
Algorithms
Outline
1 The Context
2 Examples
3 Mathematical Foundations
4 Algorithms
5 Conclusions
(Warwick University) UQ in Bayesian Inversion Andrew Stuart 20 / 25
![Page 21: Uncertainty Quantification in Bayesian Inversionhomepages.warwick.ac.uk/~masdr/TALKS/SIAM_UQ.pdf · 2014. 4. 1. · New MCMC algorithms mix in O(1) steps. Mattingly, Pillai, S, Thiery](https://reader035.vdocuments.us/reader035/viewer/2022071420/6119d6357dc49d2eb021420c/html5/thumbnails/21.jpg)
Algorithms
Forward Error = Inverse Error
Approximate forward map G by a numerical method to obtain GN
satisfying, for u in X ,
|G(u)− GN(u)| ≤ ψ(N)→ 0
as N →∞. Leads to approximate posterior measure µy ,N .
TheoremFor appropriate class of test functions f : X → S:
‖Eµyf (u)− Eµ
y,Nf (u)‖S ≤ Cψ(N).
Cotter, Dashti S 10.
(Warwick University) UQ in Bayesian Inversion Andrew Stuart 21 / 25
![Page 22: Uncertainty Quantification in Bayesian Inversionhomepages.warwick.ac.uk/~masdr/TALKS/SIAM_UQ.pdf · 2014. 4. 1. · New MCMC algorithms mix in O(1) steps. Mattingly, Pillai, S, Thiery](https://reader035.vdocuments.us/reader035/viewer/2022071420/6119d6357dc49d2eb021420c/html5/thumbnails/22.jpg)
Algorithms
Faster MCMC
TheoremFor a wide range of priors/model problems, usingdiffusion limits to SPDEs andspectral gaps in X we have:
Standard MCMC algorithms mix in O(Na) steps, a > 0.
New MCMC algorithms mix in O(1) steps.
Mattingly, Pillai, S, Thiery 12–14, TheoryHairer, S, Vollmer 14, TheoryAgapiou, Bardsley, Papaspiliopoulis, S 13, Theory (hierarchical)Ghattas/Bui-Thanh 10s, ApplicationsCliffe/Ernst/Sprungk 14, Cui/Law/Marzouk 14, Improved Algorithms
(Warwick University) UQ in Bayesian Inversion Andrew Stuart 22 / 25
![Page 23: Uncertainty Quantification in Bayesian Inversionhomepages.warwick.ac.uk/~masdr/TALKS/SIAM_UQ.pdf · 2014. 4. 1. · New MCMC algorithms mix in O(1) steps. Mattingly, Pillai, S, Thiery](https://reader035.vdocuments.us/reader035/viewer/2022071420/6119d6357dc49d2eb021420c/html5/thumbnails/23.jpg)
Algorithms
Other Directions
Sparse determinsitic approximation of posterior expectations.Schwab and S 12, Schwab and Schillings 14.Multilevel MCMC.Ha Hoang, Schwab, S 13, Ketelsen, Scheichl, Teckentrup 13.Benchmarking ad hoc algorithmsLaw and S 12 (Fluids), Iglesias, Law and S 13 (Subsurface).Best Dirac approximation ν = δu? (MAP).Dahsti, Law, S, Voss 13, Dunlop, S 14.Best Gaussian approximation ν = N(m,C) wrt relative entropy.Pinski, Simpson, S and Weber 13.
(Warwick University) UQ in Bayesian Inversion Andrew Stuart 23 / 25
![Page 24: Uncertainty Quantification in Bayesian Inversionhomepages.warwick.ac.uk/~masdr/TALKS/SIAM_UQ.pdf · 2014. 4. 1. · New MCMC algorithms mix in O(1) steps. Mattingly, Pillai, S, Thiery](https://reader035.vdocuments.us/reader035/viewer/2022071420/6119d6357dc49d2eb021420c/html5/thumbnails/24.jpg)
Conclusions
Outline
1 The Context
2 Examples
3 Mathematical Foundations
4 Algorithms
5 Conclusions
(Warwick University) UQ in Bayesian Inversion Andrew Stuart 24 / 25
![Page 25: Uncertainty Quantification in Bayesian Inversionhomepages.warwick.ac.uk/~masdr/TALKS/SIAM_UQ.pdf · 2014. 4. 1. · New MCMC algorithms mix in O(1) steps. Mattingly, Pillai, S, Thiery](https://reader035.vdocuments.us/reader035/viewer/2022071420/6119d6357dc49d2eb021420c/html5/thumbnails/25.jpg)
Conclusions
Conclusions
UQ+BIP: formidable mathematical and computational challenges.
Balance: numerical analysis and statistical approximations.
Think N =∞, implement N <∞.
This is the beginning not the end.
(Warwick University) UQ in Bayesian Inversion Andrew Stuart 25 / 25
![Page 26: Uncertainty Quantification in Bayesian Inversionhomepages.warwick.ac.uk/~masdr/TALKS/SIAM_UQ.pdf · 2014. 4. 1. · New MCMC algorithms mix in O(1) steps. Mattingly, Pillai, S, Thiery](https://reader035.vdocuments.us/reader035/viewer/2022071420/6119d6357dc49d2eb021420c/html5/thumbnails/26.jpg)
Conclusions
Conclusions
UQ+BIP: formidable mathematical and computational challenges.
Balance: numerical analysis and statistical approximations.
Think N =∞, implement N <∞.
This is the beginning not the end.
(Warwick University) UQ in Bayesian Inversion Andrew Stuart 25 / 25