tutorial on particle filterstutorial on particle filters 4 pf suppl 1... · 2015-11-24 · tutorial...
TRANSCRIPT
Tutorial on Particle filtersTutorial on Particle filters
Keith CopseyPattern and Information
Processing GroupProcessing GroupDERA Malvern
NCAF January Meeting, Aston University, Birmingham.
Outline
Introduction to particle filters
– Recursive Bayesian estimationBayesian Importance sampling
– Sequential Importance sampling (SIS)Sequential Importance sampling (SIS)
– Sampling Importance resampling (SIR)Improvements to SIR
– On-line Markov chain Monte CarloBasic Particle Filter algorithmExamplesExamplesConclusionsDemonstration
NCAF January Meeting, Aston University, Birmingham.
Particle Filters
Sequential Monte Carlo methods for on-line learning within a Bayesian frameworka Bayesian framework.
Known as
– Particle filters
– Sequential sampling-importance resampling (SIR)
– Bootstrap filters
– Condensation trackersCo de sat o t ac e s
– Interacting particle approximations
Survival of the fittest– Survival of the fittest
NCAF January Meeting, Aston University, Birmingham.
Recursive Bayesian estimation (I)y ( )
Recursive filter:
– System model:
)|( ),( 11 −− ↔= kkkkkk xxpxfx ω
– Measurement model:
)|( ),( kkkkkk xypxhy ↔= υ
– Information available:
)( 1 kk yyD = ),,( 1 kk yyD …=
)( 0xp
NCAF January Meeting, Aston University, Birmingham.
Recursive Bayesian estimation (II)
Seek: )|( kik Dxp +
y ( )
– i = 0: filtering.
– i > 0: prediction.
– i<0: smoothing.
P di tiPrediction:
∫ −−−−− = 11111 )|()|()|( kkkkkkk dxDxpxxpDxp– since:
∫ −−−− = 1111 )|,()|( kkkkkk dxDxxpDxp ∫
NCAF January Meeting, Aston University, Birmingham.
Recursive Bayesian estimation (III)
Update:
y ( )
)|()|()|()|(
1
1
−
−=kk
kkkkkk Dyp
DxpxypDxp
where:
∫ dDD )|()|()|(
– since:
∫ −− = kkkkkkk dxDxpxypDyp )|()|()|( 11
– since:
∫ −− = kkkkkk dxDxypDyp )|,()|( 11
NCAF January Meeting, Aston University, Birmingham.
Classical approximations
Analytical methods:
– Extended Kalman filter,
– Gaussian sums… (Alspach et al. 1971)
• Perform poorly in numerous cases of interest
Numerical methods:
– point masses approximations,
– splines. (Bucy 1971, de Figueiro 1974…)
• Very complex to implement, not flexible.
NCAF January Meeting, Aston University, Birmingham.
Perfect Monte Carlo simulation (I)( )
Introduce the notation
Represent posterior distribution using a set of samples or
),,( 0:0 kk xxx …=
particles.
∑=N
kxkk dxN
Dxp ik
:0:0 )(1)|(0
δ
Random samples are drawn from the posterior distribution
=ixN k1 :0
ikx :0
distribution.
NCAF January Meeting, Aston University, Birmingham.
Perfect Monte Carlo simulation (II)
Easy to approximate expectations of the form:
( )
∫= kkkkk dxDxpxgxgE :0:0:0:0 )|()())((
– by:
∑N
i1 ∑=
=i
ikk xg
NxgE
1:0:0 )(1))((
NCAF January Meeting, Aston University, Birmingham.
Random samples and the pdf (I)( )
Take p(x)=Gamma(4,1)Generate some random samplesGenerate some random samplesPlot histogram and basic approximation to pdf
0.412
0 25
0.3
0.35
8
10
0.15
0.2
0.25
4
6
0 2 4 6 8 10 12 14 16 18 200
0.05
0.1
0 20 40 60 80 100 120 140 160 180 2000
2
0 20 40 60 80 100 120 140 160 180 200
200 samples
NCAF January Meeting, Aston University, Birmingham.
Random samples and the pdf (II)( )
0 45 0.35
0 3
0.35
0.4
0.45
0.25
0.3
0.35
0.15
0.2
0.25
0.3
0 1
0.15
0.2
0 2 4 6 8 10 12 14 16 18 200
0.05
0.1
0 2 4 6 8 10 12 14 16 18 200
0.05
0.1
0 2 4 6 8 10 12 14 16 18 20
500 samples 1000 samples
NCAF January Meeting, Aston University, Birmingham.
Random samples and the pdf (III)( )
0.25 0.25
0.15
0.2
0 15
0.2
0.1
0.15
0.1
0.15
0 5 10 15 20 250
0.05
0 5 10 15 20 250
0.05
200000 samples5000 samples
NCAF January Meeting, Aston University, Birmingham.
Bayesian Importance Sampling (I)Unfortunately it is often not possible to sample directly from the posterior distribution.
y g ( )
Circumvent by drawing from a known easy to sample proposal distribution giving:)|( :0 kk Dxq
∫= kkkkk
kkkk dxDxq
DxqDxpxgxgE :0:0
:0
:0:0:0 )|(
)|()|()())((
∫= kkkkkk
kkkk dxDxq
DxqDpxpxDpxg :0:0
:0
:0:0:0
)(
)|()|()()()|()(
∫= kkkk
kkk dxDxq
Dpxwxg :0:0
:0:0 )|(
)()()(
NCAF January Meeting, Aston University, Birmingham.
Bayesian Importance Sampling (II)y g ( )
where are unnormalised importance weights:)( :0 kk xw
)|()()|()(
0
:0:0:0
kk
kkkkk Dxq
xpxDpxw =
Now:
)|( :0 kk Dxq
∫ dDD )()(
∫∫
=
=
kkkkkk
kkkk
dxD
DxqxpxDpdxxDpDp
:0:0:0:0
:0:0
)|(
)|()()|( ),()(
∫
∫= kkkkk
kkk
dxDxqxwDxq
:0:0:0
:0:0
)|()()|(
NCAF January Meeting, Aston University, Birmingham.
Bayesian Importance Sampling (III)y g ( )
Giving:
∫ dxDxqxwxg )|())()((
∫∫=
kkkkk
kkkkkkk dxDxqxw
dxDxqxwxgxgE
:0:0:0
:0:0:0:0:0 )|()(
)|())()(())((
so that:
∑N
ii xwxg )()(1
∑∑
∑
=
= ==N
i
ikk
ikN
i
ikkk
k xwxgxw
xwxgN
xgE1
:0:01
:0:0
:0 )(~)()(1
)()())((
where are normalised importance weights
∑=
i
ikk xw
N1
1:0 )(
)(~~0i
kkik xww =where are normalised importance weights
– and are independent random samples from
)( :0 kkk xww =i
kx :0 )|( :0 kk Dxq
NCAF January Meeting, Aston University, Birmingham.
Sequential Importance Sampling (I)g ( )
Factorising the proposal distribution:k∏=
−=k
jjjjkk DxxqxqDxq
11:00:0 ),|()()|(
and remembering that the state evolution is modelled as a Markov process
obtain a recursive estimate of the importance weights:
)|()|( 1kkkk xxpxyp),|(
)|()|(:0
11
kkk
kkkkkk Dxxq
xxpxypww −−=
NCAF January Meeting, Aston University, Birmingham.
Derivation of SIR weightsg
Since:k k∏=
−=k
jjjk xxpxpxp
110:0 )|()()( and ∏
==
k
jjjkk xypxDp
1:0 )|()|(
We have:
)()|( :0:0 kkk xpxDpw
1)()|()|(),|(
)()|(
:0:011:01:0
:0:0
kkkkkkkk
kkkk
xpxDpw
DxqDxxqppw
−−−
=
=
)|()|(),|()()|(
11
1:01:01:011
kkkkk
kkkkkkk
xxpxypw
DxxqxpxDpw
−−−−−
−
=
=
),|( 1:01
kkkk Dxxq
w−
−=
NCAF January Meeting, Aston University, Birmingham.
Sequential Importance Sampling (II)g ( )
Choice of the proposal distribution:
Choose proposal function to minimise variance of
),|( 1:0 kkk Dxxq −
kw(Doucet et al. 1999):
),|(),|( 1:01:0 kkkkkk DxxpDxxq −− =
Although Common choice is the prior distribution:
1:01:0 kkkkkk
)|(),|( 11:0 −− = kkkkk xxpDxxq
NCAF January Meeting, Aston University, Birmingham.
Sequential Importance Sampling (III)
Illustration of SIS:
g ( )
w
Time 1
Time 10
w
Time 19
ww
Degeneracy problems:
– variance of importance ratios i t h ti ll ti (K t l 1994 D t
)|(/)|( :0:0 kkkk DxqDxpincreases stochastically over time (Kong et al. 1994; Doucet et al. 1999).
NCAF January Meeting, Aston University, Birmingham.
SIS - why variance increase is bady
Suppose we want to sample from the posterior
– choose a proposal density to be very close to the posterior density
Th• Then
d
1)|()|(
:0
:0 =⎟⎟⎠
⎞⎜⎜⎝
⎛
kk
kkq Dxq
DxpE
• and01
)|()|(
)|()|(var
2
:0
:0
:0
:0 =⎟⎟
⎠
⎞
⎜⎜
⎝
⎛⎟⎟⎠
⎞⎜⎜⎝
⎛−=⎟⎟
⎠
⎞⎜⎜⎝
⎛
kk
kkq
kk
kkq Dxq
DxpEDxqDxp
So we expect the variance to be close to 0 to obtain reasonable estimates
)|()|( :0:0 ⎠⎝ ⎠⎝⎠⎝ kkkk qq
– thus a variance increase has a harmful effect on accuracy
NCAF January Meeting, Aston University, Birmingham.
Sequential Importance Sampling (IV)g ( )
Illustration of degeneracy: Time 1
w
Time 10
w
Time 19
w
NCAF January Meeting, Aston University, Birmingham.
Sampling-Importance Resamplingg g
SIS suffers from degeneracy problems so we don’t want to d th t!do that!Introduce a selection (resampling) step to eliminate samples with low importance ratios and multiply samples
ith hi h i t tiwith high importance ratios. Resampling maps the weighted random measure on to the equally weighted random measure
)}(~,{ :0:0i
kki
k xwx}{ 1-
:0 Nx jk
– by sampling uniformly with replacement from with probabilities
}{ :0 k},,1;{ :0 Nixi
k …=},,1;~{ Niwi
k …=
Scheme generates children such that and satisfies:
NNN
ii =∑
=1iN
i 1iki wNNE ~)( =
)~1(~)var( ik
iki wwNN −=
NCAF January Meeting, Aston University, Birmingham.
kki
Improvements to SIR (I)( )
Variety of resampling schemes with varying performance in terms of the variance of the particles :)var( iNterms of the variance of the particles :
– Residual sampling (Liu & Chen, 1998).
Systematic sampling (Carpenter et al 1999)
)var( iN
– Systematic sampling (Carpenter et al., 1999).
– Mixture of SIS and SIR, only resample when necessary (Liu & Chen 1995; Doucet et al 1999)Chen, 1995; Doucet et al., 1999).
Degeneracy may still be a problem:
– During resampling a sample with high importance weight may be duplicated many times.
– Samples may eventually collapse to a single point.
NCAF January Meeting, Aston University, Birmingham.
Improvements to SIR (II)( )
To alleviate numerical degeneracy problems, sample smoothing methods may be adoptedsmoothing methods may be adopted.
– Roughening (Gordon et al., 1993).
• Adds an independent jitter to the resampled• Adds an independent jitter to the resampled particles
– Prior boosting (Gordon et al 1993)– Prior boosting (Gordon et al., 1993).
• Increase the number of samples from the proposal distribution to M>N,proposal distribution to M N,
• but in the resampling stage only draw N particles.p
NCAF January Meeting, Aston University, Birmingham.
Improvements to SIR (III)( )
Local Monte Carlo methods for alleviating degeneracy:
– Local linearisation - using an EKF (Doucet, 1999; Pitt & Shephard, 1999) or UKF (Doucet et al, 2000) to estimate the importance distributionimportance distribution.
– Rejection methods (Müller, 1991; Doucet, 1999; Pitt & Shephard, 1999).)
– Auxiliary particle filters (Pitt & Shephard, 1999)
– Kernel smoothing (Gordon 1994; Hürzeler & Künsch 1998; Liu &Kernel smoothing (Gordon, 1994; Hürzeler & Künsch, 1998; Liu & West, 2000; Musso et al., 2000).
– MCMC methods (Müller, 1992; Gordon & Whitby, 1995; Berzuini et ( , ; y, ;al., 1997; Gilks & Berzuini, 1998; Andrieu et al., 1999).
NCAF January Meeting, Aston University, Birmingham.
Improvements to SIR (IV)( )
Illustration of SIR with sample smoothing:
w
Time 1
Time 10
Time 19
ww
NCAF January Meeting, Aston University, Birmingham.
MCMC move stepImprove results by introducing MCMC steps with invariant distribution .)|( :0 kk Dxp
– By applying a Markov transition kernel, the total variation of the current distribution w.r.t. the invariant distribution can only decreasedecrease.
Introduces possibility ofIntroduces possibility of variable dimension state space through the use of reversible jump MCMC (de Freitas et al., j p ( ,1999; Gilks & Berzuini, 2001)
NCAF January Meeting, Aston University, Birmingham.
Ingredients for SMCg
Importance sampling functioni– Gordon et al →
– Optimal →
)|( 1ikk xxp −
),|( 1:0 ki
kk Dxxp −– UKF → pdf from UKF at
Redistribution scheme
ikx 1−
– Gordon et al → SIR
– Liu & Chen → Residual
– Carpenter et al → Systematic
– Liu & Chen, Doucet et al → Resample when necessary
Careful initialisation procedure (for efficiency)
NCAF January Meeting, Aston University, Birmingham.
Basic Particle Filter - SchematicInitialisation
0k t0=k
1+→ kk
measurement
ky
I tR li
},{ 1:0
−Nxik
Importancesampling step
Resamplingstep
)}(~,{ :0:0i
kki
k xwx
Extract estimate, kx :0ˆ
NCAF January Meeting, Aston University, Birmingham.
Basic Particle Filter algorithm (I)g ( )
Initialisation
–
– For sample
0=kNi ,,1…= )(~ 00 xpxi
– and set
I ti t id h i t t k t l f
1=k
In practice, to avoid having to take too many samples, for the first step we may want to ensure that we have a reasonable number of particles in the region of high likelihoodlikelihood
– perhaps use MCMC techniques
NCAF January Meeting, Aston University, Birmingham.
Basic Particle Filter algorithm (II)g ( )
Importance Sampling step
– For sample Ni ,,1…= )|(~~1
ikk
ik xxpx −
),(~1:0:0
ik
ik
ik xxx −=and set
– For evaluate the importance weightsNi ,,1…=
)~|( ikk
ik xypw =
– Normalise the importance weights, ∑=N
jk
ik
ik www /~
)|( kkk yp
∑=j
kkk1
NCAF January Meeting, Aston University, Birmingham.
Basic Particle Filter algorithm (III)g ( )
Resampling step
– Resample with replacement particles:N),,1;( :0 Nixi
k …=– from the set:
),,1;~( :0 Nix ik …=
– according to the normalised importance weights, ikw~
Set
– proceed to the Importance Sampling step, as the next
1+→ kk
measurement arrives.
NCAF January Meeting, Aston University, Birmingham.
Example
On-line Data Fusion (Marrs, 2000).
NCAF January Meeting, Aston University, Birmingham.
Example - Sensor Deploymenty
Aim to reduce target sd below some thresholdthreshold...
… and keep it there
… by placing the minimum number of sensors possible
Sensor positions chosen according to particle distribution.
NCAF January Meeting, Aston University, Birmingham.
E l I it it i f i i d tExample - In-situ monitoring of growing semiconductor crystal composition
Si1-xGex
substrate
NCAF January Meeting, Aston University, Birmingham.
Book Advert (or put this in or your fired)( y )
Sequential Monte Carlo methods in practice, Editors: Doucet de Freitas Gordon Springer-Verlag (2001)Doucet, de Freitas, Gordon, Springer-Verlag (2001).
– Theorectical foundations - plus convergence proofsp g p
– Efficiency measures
– Applications:– Applications:• Target tracking; missile guidance; image tracking; terrain
referenced navigation; exchange rate prediction; portfolio allocation; ellipsometry; electricity load forecasting;allocation; ellipsometry; electricity load forecasting; pollution monitoring; population biology; communications and audio engineering.
ISBN=0-387-95146-6, Price=$79.95.
NCAF January Meeting, Aston University, Birmingham.
Conclusions
On-line Bayesian learning a realistic proposition for many applicationsapplications.Appropriate for complex non-linear/non-Gaussian models
– don’t bother if KF based solution adequate.qRepresentation of full posterior pdf leading to
– estimation of moments.
– estimation of HPD regions.
– multi-modality easy to deal with.u t oda ty easy to dea tModel order can be included in unknowns.Can mix SMC and KF based solutions
NCAF January Meeting, Aston University, Birmingham.
Tracking Demog
Illustrate a running particle filter
– compare with Kalman Filter
Running as we watch - not pre-recordedRunning as we watch - not pre-recorded
Pre-defined scenarios, or design your own
– available to play with at coffee and lunch breaks.
Tracking Demo
NCAF January Meeting, Aston University, Birmingham.
2nd Book Advert
Statistical Pattern RecognitionAndrew Webb DERAAndrew Webb, DERAISBN 0340741643, Paperback: 1999: £29.99Butterworth Heinemann
Contents:
– Introduction to SPR, Estimation, Density estimation, Linear discriminant analysis, Nonlinear discriminant analysis - neural networks, Nonlinear discriminant analysis - statistical methods, Classification trees, Feature selction and extraction, Clustering, Additional topics, Measures of dissimilarity, Parameter estimation, Linear algebra, Data, Probability theory.
NCAF January Meeting, Aston University, Birmingham.