bayesian static parameter estimation using multilevel...

Introduction MLMC BIP Coupling PMCMC PMC(ML)MC Numerical simulations Summary

Bayesian static parameter estimation usingMultilevel Monte Carlo

Kody Law

joint with A. Jasra (NUS), K. Kamatani (Osaka), & Y. Zhou

MCQMC 2018Rennes, FR

July 4, 2018


Outline

1 Introduction

2 Multilevel Monte Carlo sampling

3 Bayesian inference problem

4 Approximate coupling

5 Particle Markov chain Monte Carlo

6 Particle Markov chain Multilevel Monte Carlo

7 Numerical simulations

8 Summary


Orientation

Aim: Approximate posterior expectations of the state path andstatic parameters associated to a discretely observed diffusion,which must be finitely approximated.Solution: Apply an approximate coupling strategy so that themultilevel Monte Carlo (MLMC) method can be used within aparticle marginal Metropolis-Hastings (PMMH) algorithm [B02,AR08, ADH10].

MLMC methods reduce cost to error= O(ε) [H00, G08];Recently this methodology has been applied to inference,mostly in cases where target can be evaluated up to anormalizing constant [HSS13, DKST15, HTL16, BJLTZ17].Here it cannot, but using PMMH we are able to sampleconsistently from an approximate coupling of successivetargets [JKLZ18].


Outline

1 Introduction







8 Summary


Example: expectation for SDE [G08]

Estimation of expectation of solution of intractable stochasticdifferential equation (SDE).

dX = f (X )dt + σ(X )dW , X0 = x0 .

Aim: estimate E(g(XT )).We need to(1) Approximate, e.g. by Euler-Maruyama method with

resolution h:

Xn+1 = Xn + hf (Xn) +√

hσ(Xn)ξn, ξn ∼ N(0,1).

(2) Sample {X (i)NT}Ni=1, NT = T/h.


Multilevel Monte Carlo [H00, G08, CGST11]

Assume hl = 2−l , Xl approximates XT with stepsize hl , andthere are α, β, ζ > 0 such that

(i) weak error |E[g(Xl)− g(X )]| = O(hαl ).

(ii) strong error E|g(Xl)− g(X )|2 = O(hβl )⇒ Vl = O(hβl ),(iii) computational cost for a realization of g(Xl)− g(Xl−1),

Cl ∝ h−ζl .Then, it is possible to choose L and {Nl}Ll=0 so as to achieve amean squared error of O(ε2) at a cost

COST ≤ C

ε−2, if β > γ,

ε−2| log(ε)|2, if β = γ,

ε−(2+ γ−βα ), if β < γ.

Compared with O(ε−2−γ/α) for single level.


Outline

1 Introduction







8 Summary


Parameter inference

Estimate the posterior expectation of a function ϕ of the jointpath X1:T and parameters θ, of an intractable SDE

dX = fθ(X )dt + σθ(X )dW , X0 ∼ µθ ,

given noisy partial observations

Yn ∼ gθ(Xn, ·) , n = 1, . . . ,T .

Aim: estimate E[ϕ(θ,X0:T )|y1:T ], where y1:T := {y1, . . . , yT}.

The hidden process {Xn} is a Markov chain.

Discretize with resolution h and denote the transition kernelFθ,h

(xp−1, dxp

)– this can be simulated from, but its density

cannot be evaluated.


Return to ML

The joint measure is

Πh(dθ, dx0:n) ∝ Π(dθ)µθ(dx0)n∏

p=1

gθ(xp, yp)Fθ,h(xp−1, dxp) ,

For +∞ > h0 > · · · > hL > 0, we would like to compute

EΠhL[ϕ(θ,X0:n)] =

L∑l=0

{EΠhl

[ϕ(θ,X0:n)]− EΠhl−1[ϕ(θ,X0:n)]

}where EΠh−1

[·] := 0.


Outline

1 Introduction







8 Summary


Approximate coupling

Consider a single pair EΠh [ϕ(θ,X0:n)]− EΠh′ [ϕ(θ,X0:n)], h < h′.

Let z = (x , x ′) and let Qθ,h,h′(z, dz) be a coupling of(Fθ,h(x , dx),Fθ,h′(x ′, dx ′)).

Let Gp,θ(z) = max{gθ(x , yp),gθ(x ′, yp)}.

We will sample from the joint coarse/fine filter

Πh,h′(dθ, dz0:n) ∝ Π(dθ)νθ(dz0)n∏

p=1

Gp,θ(zp)Qθ,h,h′(zp−1, dzp),

where νθ is the initial coupling

νθ(d(x , x ′)) = µθ(dx)δx (dx ′).


Change of measure

We haveEΠh [ϕ(θ,X0:n)]− EΠh′ [ϕ(θ,X ′0:n)] =

EΠh,h′ [ϕ(θ,X0:n)H1,θ(θ,Z0:n)]

EΠh,h′ [H1,θ(θ,Z0:n)]−

EΠh,h′ [ϕ(θ,X ′0:n)H2,θ(θ,Z0:n)]

EΠh,h′ [H2,θ(θ,Z0:n)]

where

H1,θ(θ, z0:n) =n∏

p=1

gθ(xp, yp)

Gp,θ(zp)

H2,θ(θ, z0:n) =n∏

p=1

gθ(x ′p, yp)

Gp,θ(zp).


Outline

1 Introduction







8 Summary


Particle filter, for fixed θ

Let M ≥ 1 and θ be fixed, and introduce a0:n−1 ∈ {1, . . . ,M}n.The bootstrap particle filter [Del04] approximates

Πh,h′(dz0:n|θ) ∝ νθ(dz0)n∏

p=1

Gp,θ(zp)Qθ,h,h′(zp−1, dzp)

by sampling from

P(a1:M0:n−1, dz1:M

0:n |θ) =( M∏

i=1

νθ(dz i0)) n∏

p=1

M∏i=1

( Gp−1,θ(zai

p−1p−1 )∑M

j=1 Gp−1,θ(z jp−1)

Qθ,h,h′(zai

p−1p−1 , dz i

p)),

where G0,θ := 1,

i.e. zai

p−1p−1 is resampled with the probability

Gp−1,θ(zaip−1

p−1 )∑Mj=1 Gp−1,θ(z j

p−1).


Unbiased estimator, for fixed θ

Draw J with probability proportional to Gn,θ(z in) for i = 1, . . . ,M.

Let zn = z jn, and trace its ancestral lineage

zn−1 = zaj

n−1n−1 , zn−2 = z

aajn−1

n−2n−2 , and so on .

Define pMh,h′(y0:n|θ) =

∏np=1

(1M∑M

j=1 Gp,θ(z jp))

, let E denote

expectation w.r.t. (A1:M0:n−1,Z

1:M0:n , J), and note that [Del04]

E[ϕ(z0:n)pM

h,h′(y0:n|θ)]

=

∫Zn+1

ϕ(z0:n)νθ(dz0)n∏

p=1

Gp,θ(zp)Qθ,h,h′(zp−1, dzp) .

Therefore, one can run a MH chain {θi , z0:n(θi)} targeting∝ pM

h,h′(y0:n|θ)Π(dθ), and it is consistent with respect toΠh,h′(dz0:n, dθ) [B03, AR08, ADH10].


Particle marginal MH (PMMH) [ADH10]

1 Sample θ0 ∼ π(dθ) and (a1:M0:n−1, z

1:M0:n ) from particle filter

P(da1:M0:n−1, dz1:M

0:n |θ0), and store pMh,h′(y0:n|θ0).

2 Select a path z00:n: draw z j

n with probability proportional to Gn,θ0 (z jn), let

z0n = z j

n, and trace back its ancestral lineage

z0n−1 = z

ajn−1

n−1 , z0n−2 = z

aajn−1

n−2n−2 , and so on ; Set i = 1 .

3 Sample θ∗|θi−1 according to R(dθ∗|θi−1) = r(θ∗|θi−1)dθ∗, then samplefrom particle filter P(da1:M

0:n−1, dz1:M0:n |θ∗). Select one path z∗0:n as above.

4 Set θi = θ∗, z i0:n = z∗0:n with probability:

min

{1,

pMh,h′(y0:n|θ∗)

pMh,h′(y0:n|θi−1)

π(θ∗)r(θi−1|θ∗)π(θi−1)r(θ∗|θi−1)

}

otherwise θi = θi−1, z i0:n = z i−1

0:n .5 Set i = i + 1 and return to the start of 3.


PMMH increment estimator

1N∑N

i=1 ϕ(θi , x i0:n)H1,θ(θi , z i

0:n)1N∑N

i=1 H1,θ(θi , z i0:n)

−1N∑N

i=1 ϕ(θi , x ′i0:n)H2,θ(θi , z i0:n)

1N∑N

i=1 H2,θ(θi , z i0:n)

.

−→N→∞

EΠh,h′ [ϕ(θ,X0:n)H1,θ(θ,Z0:n)]

EΠh,h′ [H1,θ(θ,Z0:n)]−

EΠh,h′ [ϕ(θ,X ′0:n)H2,θ(θ,Z0:n)]

EΠh,h′ [H2,θ(θ,Z0:n)]

=

EΠh [ϕ(θ,X0:n)]− EΠh′ [ϕ(θ,X ′0:n)]


Outline

1 Introduction







8 Summary


Multilevel estimatorConsider

L∑l=0

ENll (ϕ) , ENl

l (ϕ) = ENll (ϕ)− El(ϕ) , (1)

where

ENll (ϕ) =

1Nl

∑Nli=1 ϕ(θi , , x i

0:n)H1,θ(θi , z i0:n)

1Nl

∑Nli=1 H1,θ(θi , z i

0:n)−

1Nl

∑Nli=1 ϕ(θi , x ′i0:n)H2,θ(θi , z i

0:n)

1Nl

∑Nli=1 H2,θ(θi , z i

0:n)

is a consistent estimator of

El(ϕ) := EΠhl[ϕ(θ,X0:n)]− EΠhl−1

[ϕ(θ,X0:n)] .

⇒ (1) is a consistent estimator of EΠhL[ϕ(θ,X0:n)].

One must bound

E[(L∑

l=0

ENll (ϕ))2] =

L∑l=0

E[ENll (ϕ)2] .


Assumptions

(A1) ∀ y ∈ T, ∃ C > 0 such that ∀ x ∈ S, θ ∈ Θ,

C ≤ gθ(x , y) ≤ C−1.

And ∀ y ∈ T, gθ(x , y) is globally Lipschitz on S×Θ.

(A2) ∀ 0 ≤ k ≤ n, ∃ β > 0 such that ∀ϕ ∈ Bb(Θ× Sk+1) ∩ Lip(Θ× Sk+1) ∃ C > 0∫

Θ×S2k+2|ϕ(θ, x0:k )−ϕ(θ, x ′0:k )|2Π(dθ)νθ(dz0)

k∏p=1

Qθ,h,h′(zp−1, dzp) ≤ C(h′)β .

(A3) Suppose that ∀ n > 0, ∃ ξ ∈ (0,1) and ν ∈ P(W) such thatfor each w ∈W, ϕ ∈ Bb(W) ∩ Lip(W), h,h′:∫

Wϕ(w ′)K (w ,dw ′) ≥ ξ

∫Wϕ(v)ν(dv).

K is η-reversible, where η is the joint on the extended space.


Main result

Theorem (JKLZ18)Assume (A1-3). Then ∀ n > 0, ∃ β > 0 such that ∀ϕ ∈ Bb(Θ× Sn+1) ∩ Lip(Θ× Sn+1) ∃ C > 0 such that

E[ENll (ϕ)2] ≤

ChβlNl

,

and β is from (A2).


(A4) ∃ γ, α,C > 0 such that the cost to simulate ENll is controlled by

C(ENll ) ≤ CNlh

−γl , and the bias is controlled by

|EΠhL(ϕ(θ,X0:n))− EΠ0 (ϕ(θ,X0:n))| ≤ ChαL .

Corollary

Assume (A1-4). ∀ n > 0 and ϕ ∈ Bb(Θ× Sn+1) ∩ Lip(Θ× Sn+1) ∃C > 0 such that ∀ ε > 0 one can choose (L, {Nl}L

l=0) so

E

[|

L∑l=0

ENll (ϕ)− EΠ0 (ϕ(θ,X0:n))|2

]≤ Cε2 ,

with a total cost (per time step)

COST ≤ C

ε−2, if β > γ,

ε−2| log(ε)|2, if β = γ,

ε−(2+ γ−βα ), if β < γ.


Outline

1 Introduction







8 Summary


Ornstein-Uhlenbeck process

dXt = θ(µ− Xt )dt + σdWt , X0 = x0 ,

Yk |Xδk ∼ N (Xδk , τ2) ,

θ ∼ G(1,1) ,

σ ∼ G(1,0.5) .

N (m, τ2) denotes the Normal with mean m and varianceτ2.G(a,b) denotes the Gamma with shape a and scale b.x0 = 0, µ = 0, δ = 0.5, and τ2 = 0.2.100 observations simulated with θ = 1 and σ = 0.5.


Langevin SDE

dXt =12∇ log π(Xt )dt + σdWt , X0 = x0

Yk |Xk ∼ N (0, τ2 exp Xk ) ,

θ ∼ G(1,1) ,

σ ∼ G(1,0.5) .

π(x) denotes the probability density function of a Student’st-distribution with θ degrees of freedom.x0 = 0.1,000 observations simulated with θ = 10, σ = 1, andτ2 = 1.


Langevin SDE Ornstein-Uhlenbech process

0 10 20 30 0 10 20 30

0.00

0.25

0.50

0.75

1.00

Lag

ACF

Parameter 𝜎 𝜃

Figure: Autocorrelation of a typical PMCMC chain.


Lagenvin SDE

𝜎

Lagenvin SDE

𝜃

Ornstein-Uhlenbech process

𝜎

Ornstein-Uhlenbech process

𝜃

2−8 2−7 2−6 2−5 2−4 2−3 2−2 2−5 2−4 2−3 2−2 2−1 20

2−8 2−7 2−6 2−5 2−4 2−3 2−6 2−5 2−4 2−3 2−2 2−122242628210

22242628210

22242628210

22242628210

Error

CostAlgorithm ML-PMCMC PMCMC

Figure: Cost vs. MSE for the 2 parameters for each of the 2 SDEs.


Model Parameter ML-PMCMC PMCMCOU θ −1.022 −1.463

σ −1.065 −1.522Langevin θ −1.060 −1.508

σ −1.023 −1.481

Table: Estimated rates of convergence of MSE with respect to cost forvarious parameters, fitted to the curves.


Outline

1 Introduction







8 Summary


Summary

New approximate coupling strategy can be used to applyMLMC to PMCMC for static parameter estimation[JKLZ18.i].Same strategy can be employed for multi-index MCMC[JKLZ18.ii].In progress: MISMC2 [JLX18.s]PhD and postdocs wanted – please inquire !


References[G08]: Giles. "Multilevel Monte Carlo path simulation." Op.Res., 56, 607-617 (2008).[H00] Heinrich. "Multilevel Monte Carlo methods." LSSCproceedings (2001).[CGST11] Cliffe, Giles, Scheichl, & Teckentrup. "MLMCand applications to elliptic PDEs." Computing andVisualization in Science, 14(1), 3 (2011).[D04]: Del Moral. "Feynman-Kac Formulae." Springer:New York (2004).[B02] Beaumont. "Estimation of population growth..."Genetics 164(3) 1139-1160 (2003).[AR08] Andrieu & Roberts. "The pseudo-marginalapproach." Annals of Stat. 37(2) 697-725 (2009).[ADH10] Andrieu, Doucet, and Holenstein. "ParticleMCMC methods." JRSSB 72(3) 269-342 (2010).


References[JKLZ18.i]: Jasra, Kamatani, Law, Zhou. "MLMC for staticBayesian parameter estimation." SISC 40, A887-A902(2018).[JKLZ18.ii]: Jasra, Kamatani, Law, Zhou. "A Multi-IndexMarkov Chain Monte Carlo Method." IJUQ 8(1), 61-73(2018).[JLX18.s]: Jasra, Law, Xu. "Multi-index SMC2." Submitted.[BJLTZ15]: Beskos, Jasra, Law, Tempone, Zhou."Multilevel Sequential Monte Carlo samplers." SPA 127:5,1417–1440 (2017).[HSS13]: Hoang, Schwab, Stuart. Inverse Prob., 29,085010 (2013).[DKST13]: Dodwell, Ketelsen, Scheichl, Teckentrup. "Ahierarchical MLMCMC algorithm." SIAM/ASA JUQ 3(1)1075-1108 (2015).


References[CHLNT17] Chernov, Hoel, Law, Nobile, Tempone."Multilevel ensemble Kalman filtering for spatio-temporalprocesses." arXiv:1608.08558 (2017).[JLS17] Jasra, Law, Suciu. "Advanced Multilevel MonteCarlo Methods." arXiv:1704.07272 (2017).[DJLZ16]: Del Moral, Jasra, Law, Zhou. "MultilevelSequential Monte Carlo samplers for normalizingconstants." ToMACS 27(3), 20 (2017).[JKLZ15]: Jasra, Kamatani, Law, Zhou. "Multilevel particlefilter." SINUM 55(6), 3068–3096 (2017).[JLZ16]: Jasra, Law, and Zhou. "Forward and Inverse UQMLMC Algorithms for an Elliptic Nonlocal Equation." IJUQ6(6), 501–514 (2016).[HLT15]: Hoel, Law, Tempone. "Multilevel ensembleKalman filter." SINUM 54(3), 1813–1839 (2016).


Thank you

bayesian static parameter estimation using multilevel...

Documents