Background Independent SMC Examples Summary
Sequential Monte Carlo for static Bayesianmodels with independent MCMC proposals
Dr Christopher DrovandiQueensland University of Technology and
Australian Centre of Excellence for Mathematical andStatistical Frontiers
Collaborators: Leah South, Tony Pettitt, Adam ClementsAcknowledgments: ARC funding, EcoSta organisers
17th June, 2017
Chris Drovandi
EcoSta 2017 1 / 29
Background Independent SMC Examples Summary
Outline
1 BackgroundBayesian Posterior SamplingMarkov Chain Monte CarloSequential Monte Carlo
2 Independent SMCIndependent MCMC Proposals within SMCRecycling
3 ExamplesBEGE ModelChallenging ODE Example
4 Summary
Chris Drovandi
EcoSta 2017 2 / 29
Background Independent SMC Examples Summary
Bayesian Posterior Sampling
Interest is in sampling from the posterior π(θ|y), where
π(θ|y) =f (y |θ)π(θ)
Z
and Z =∫
Θ f (y |θ)π(θ)dθ.
Z is referred to as the evidence and is useful for model choice.
Chris Drovandi
EcoSta 2017 3 / 29
Background Independent SMC Examples Summary
Markov Chain Monte Carlo
MCMC methods construct en ergodic Markov chain with theposterior as its limiting distrubution.
A common MCMC algorithm is Metropolis Hastings (MH-)MCMC, where proposals θ∗ are accepted with probability
min
(1,
f (y|θ∗)π(θ∗)q(θ|θ∗)
f (y|θ)π(θ)q(θ∗|θ)
),
where q(·) is the proposal density.
Chris Drovandi
EcoSta 2017 4 / 29
Background Independent SMC Examples Summary
Markov Chain Monte Carlo
Some Limitations:
Difficult to automate and adapt the method
Need to tune the proposal distribution for good performance.
Convergence can be difficult to assess
Can have difficulty exploring irregular posteriors (egmulti-modality)
Standard MCMC is a serial algorithm
Chris Drovandi
EcoSta 2017 5 / 29
Background Independent SMC Examples Summary
Sequential Monte Carlo (Chopin (2002) and Del Moralet al. (2006))
SMC methods can be a useful alternative to MCMC in someapplications as they are...
Naturally adaptive
Easily parallelisable
More capable of dealing with multimodal or complexdistributions
Able to produce an estimate of the unknown normalisingconstant
Chris Drovandi
EcoSta 2017 6 / 29
Background Independent SMC Examples Summary
Sequential Monte Carlo
Basic idea:
Moving a population of N particles through a sequence ofdistributions (starting with one easy to sample from andfinishing at the target posterior)
Can introduce the effect of either the data (data annealing) orthe likelihood (likelihood annealing) sequentially
Chris Drovandi
EcoSta 2017 7 / 29
Background Independent SMC Examples Summary
Sequential Monte Carlo
In likelihood annealing, the power posteriors are defined by
πt(θ|y) ∝ f (y |θ)γtπ(θ),
where 0 = γ0 < γt < γT = 1 and 0 < t < 1.
At each iteration, the following steps are applied
reweighting
resampling
moving to avoid particle degeneration, for example by severalruns of an MCMC kernel (largest impact on estimates, highestcost). We can make use of population of particles. ‘Standard’approach is multivariate normal random walk.
Chris Drovandi
EcoSta 2017 8 / 29
Background Independent SMC Examples Summary
Sequential Monte Carlo
Denote the collection of particles representing target t asθi
t ,Wit Ni=1. Re-weight step:
w it+1 = W i
t f (y |θit)γt+1−γt
Effective sample size (ESS) can be estimated asESS = 1/
∑i (W
it+1)2. The sequence of γt can be selected
adaptively to maintain a particular ESS, e.g. N/2.
Boost ESS back up to N via resampling. Multinomial Re-sampling.
(Diversify Particles) Apply MCMC kernel with πt+1-invariantdistribution Rt+1 times. Determine Rt+1 adaptively by performing1 iteration on each particle and inspecting acceptance rate.
Chris Drovandi
EcoSta 2017 9 / 29
Background Independent SMC Examples Summary
Sequential Monte Carlo - Estimating the Evidence
SMC provides convenient estimate of evidence, Z
We can write Z as:
Z =ZT
Z0=
T−1∏t=0
Zt+1
Zt,
with Z0 = 1. It is easy to show that
Zt+1/Zt ≈N∑i=1
w it+1.
Call this the “standard” SMC estimator.
Chris Drovandi
EcoSta 2017 10 / 29
Background Independent SMC Examples Summary
Independent Proposals
We form efficient independent proposals in SMC
Independent proposals achieve uniform ergodicity rather thanstandard geometric ergodicity
Generally difficult to construct efficient independent proposalsin MCMC
In SMC, we can use population of particles
Obtain estimates of evidence with less bias and lower variancecompared with standard SMC estimator
Chris Drovandi
EcoSta 2017 11 / 29
Background Independent SMC Examples Summary
Independent Proposals
Benefits of independent proposals in SMC
Takes advantage of the population of particles to formefficient independent proposals
Take advantage of the parallelisable nature of SMC
Re-use all information generated in the SMC process (betterestimates of posterior quantities and evidence compared withstandard SMC)
Chris Drovandi
EcoSta 2017 12 / 29
Background Independent SMC Examples Summary
Our Independent Proposal
Our efficient independent proposals are based on a mixture(Pearson, 1894; Dempster et al., 1977) of Gaussian copulas
Copulas (Sklar, 1959) model dependence between parameterswhile also modelling their marginals separately
Requires choice of marginal distributions
Basic concept
Marginals are transformed to U [0, 1] using the marginal CDFsand then to N (0, 1).
Dependence is modelled with a multivariate Gaussian mixturemodel.
We can simulate from the resulting density and also evaluate it.
Chris Drovandi
EcoSta 2017 13 / 29
Background Independent SMC Examples Summary
Recycling all ProposalsIn importance sampling (IS, Neal (2001)), the followingunnormalised weights are used for samples θiNi=1 drawn fromimportance distribution qφ(·)
ωi =f (y|θi )π(θi )
qφ(θi )for i = 1, . . . ,N.
Normalised weights are denoted ΩiNi=1.
ESS of the weighted set is ESS = 1/∑N
i=1(Ωi )2
Better importance distribution → higher ESS.The evidence can be obtained by averaging the unnormalisedweights
Z =1
N
N∑i=1
ωi .
Chris Drovandi
EcoSta 2017 14 / 29
Background Independent SMC Examples Summary
Recycling all Proposals
We can use independent proposals qφt (·) for t = 0, . . . ,T asimportance distributions.
T+1 separate estimators of the evidence
Z =1
Nt
Nt∑i=1
ωit , where ωi
t =f (y|θi
t)π(θit)
qφt (θit)
A single evidence estimator
Z =1∑T
k=0 Nk
T∑t=0
Nt∑i=1
ωit
This estimator doesn’t take into account the different ESS’s fromdifferent temperatures.
Chris Drovandi
EcoSta 2017 15 / 29
Background Independent SMC Examples Summary
Recycling all Proposals
A novel importance sampling method (similar to Nguyen et al.(2014))
Define λt = ESSt∑Tt=0 ESSt
Weights are scaled by λt
The evidence estimator is
Z =T∑t=0
λtRtN
RtN∑m=1
ωmt .
We refer to this method as CIS.
Chris Drovandi
EcoSta 2017 16 / 29
Background Independent SMC Examples Summary
“bad environment-good environment” (BEGE) ExampleBEGE model (Bekaert et al 2015):
rt+1 = µt + ut+1,
ut+1 = σpωp,t+1 − σnωn,t+1,
ωp,t+1 ∼ Γ(pt , 1),
ωn,t+1 ∼ Γ(nt , 1),
pt = p0 + ρppt−1 +φ+p
2σ2p
u2t Iut≥0 +
φ−p2σ2
p
u2t (1− Iut≥0),
nt = n0 + ρnnt−1 +φ+n
2σ2n
u2t Iut≥0 +
φ−n2σ2
n
u2t (1− Iut≥0),
θ = (p0, n0, ρp, ρn, φ+p , φ
+n , φ
−p , φ
−n , σp, σn) with µt = 0.
Likelihood is computable but involves lots of numerical integrations(cdf), and finite differencing (pdf), quite slow for large datasets.
Chris Drovandi
EcoSta 2017 17 / 29
Background Independent SMC Examples Summary
BEGE Example - Data
0 1000 2000 3000 4000 5000 6000 7000
Re
turn
s
-0.1
-0.05
0
0.05
0.1
0.15
S&P 500 daily returns from January 1990 to July 2016.
Chris Drovandi
EcoSta 2017 18 / 29
Background Independent SMC Examples Summary
BEGE Results
50 independent runs with N = 1000 particles for both randomwalk (RW) and independent (IND) proposals.
RW requires 3.7 times more log-likelihood calculations thanIND - IND much more effective at diversifying particles.
ESS for usual SMC is N = 1000.
After ‘power posterior’ (Gramacy et al 2010, Nguyen et al2014) recycling ESS is 2670 on average.
After recycling all independent proposals ESS is 3450 onaverage.
Chris Drovandi
EcoSta 2017 19 / 29
Background Independent SMC Examples Summary
BEGE Example - Evidence Results
Method
RW IND IND CISIP
Log E
vid
ence E
stim
ate
×104
2.2097
2.2098
2.2098
2.2099
2.2099
2.2100
2.21
Chris Drovandi
EcoSta 2017 20 / 29
Background Independent SMC Examples Summary
BEGE Example - Evidence Results
Method
RW IND IND CISIP
log(VAR·TLL)
5
6
7
8
9
10
Chris Drovandi
EcoSta 2017 21 / 29
Background Independent SMC Examples Summary
Challenging ODE Example
This challenging ODE example pushes the limits of what cancurrently be achieved with our independent proposal.
Nonlinear ODE system for modelling biochemical pathways
Used for investigating the enzymatic activation of protein Rinto its active form Rpp by enzyme S .
Chris Drovandi
EcoSta 2017 22 / 29
Background Independent SMC Examples Summary
Challenging ODE ExampleThis four-dimensional system of coupled ODEs has been describedin Geyer (1991) and investigated from a Bayesian context inGirolami (2008):
dSdt = −k1S
dDdt = k1S
dRdt = − V1RS
Km1+R + V2RppKm2+Rpp
dRppdt = V1RS
Km1+R −V2Rpp
Km2+Rpp .
Rpp is observed for 20 time points based on parameterconfiguration by Girolami (2008)Denote observations by y and assume y(t) ∼ N (Rpp(t), σ2).We set σ = 0.02.Nine parameters (very complex posterior)
Chris Drovandi
EcoSta 2017 23 / 29
Background Independent SMC Examples Summary
Failure of Independent Proposal - Challenging ODEExample
In this example we found that the independent proposal failed toprovide proper tail coverage.Posterior estimates:
-6 -4 -20
0.5
1
1.5
(a) log k1
-4 -2 0 20
0.5
1
(b) logV1
-10 -5 0 50
0.2
0.4
(c) logKm1
-6 -4 -2 00
0.5
1
(d) logKm2
-4 -2 00
0.5
1
1.5
(e) logV2
MCMCRWIND
Chris Drovandi
EcoSta 2017 24 / 29
Background Independent SMC Examples Summary
ODE Example - Evidence Results
Method
RW IND IND CISIP
Lo
g E
vid
en
ce
Estim
ate
21
21.5
22
Chris Drovandi
EcoSta 2017 25 / 29
Background Independent SMC Examples Summary
ODE Example - Evidence Results
Method
RW IND IND CISIP
log(VAR·TLL)
8
9
10
11
12
Chris Drovandi
EcoSta 2017 26 / 29
Background Independent SMC Examples Summary
Independent SMC Summary
Benefits:
Efficient MCMC proposal within SMC
Reuse all proposals in estimating the posterior and evidence
Significant improvement in posterior inference compared to norecycling or accepted particle recycling, if independentproposals cover the tails of the targetPrecise estimates of the evidence via IS identities
Challenges:
Difficulty achieving tail coverage with current independentproposals
Chris Drovandi
EcoSta 2017 27 / 29
Background Independent SMC Examples Summary
Future Work
Need for efficient independent proposals with improved tailcoverage
Defensive mixture distributionsMore flexible copula models
Further variance reduction through randomised quasi-MonteCarlo sample from independent proposal.
Chris Drovandi
EcoSta 2017 28 / 29
Background Independent SMC Examples Summary
Key References
Chopin, N. (2002). A sequential particle filter method for static models.Biometrika, 89(3):539-552.
Nguyen, T. L. T., Septier, F., Peters, G. W., and Delignon, Y. (2014).Improving SMC sampler estimate by recycling all past simulated particles. In2014 IEEE Workshop. In Statistical Signal Processing (SSP), pages p117-120.
Del Moral, P., Doucet, A., and Jasra, A. (2006). Sequential Monte Carlosamplers. Journal of the Royal Statistical Society: Series B (StatisticalMethodology), 68:411-436.
Bekaert, G., Engstrom, E., and Ermolov, A. (2015). Bad environments, goodenvironments: A non-Gaussian asymmetric volatility model. Journal ofEconometrics, 186:258-275.
South, L. F., Drovandi, C. C., and Pettitt, A. N. (2017). SequentialMonte Carlo for static models with independent MCMC proposals.Revision Submitted to Statistics and Computing.https://eprints.qut.edu.au/view/person/Drovandi,_Christopher.htm
Chris Drovandi
EcoSta 2017 29 / 29