unit 11 sampling methods and particle filter

Institute of Communications Engineering, EE, NCTU 1

Unit 11 Sampling Methods and

Particle Filter

Sampling Methods

Unit 10 : Sampling Methods and Particle Filter Sau-Hsuan Wu


For most probabilistic models of practical interest, exact

inference is intractable

E.x. we wish to estimate

the expectation of

E{ f }= f (z) p(z) dz

The general idea behind

sampling is to obtain a

set of samples z(i), i = 1,…, N drawn independently from p(z)

We approximate (estimate) E{ f } with

And

( )

1

1ˆ ˆ( ) whose E{ } = E{ }N

i

i

f f f fN

z

2 22 21ˆ ˆ ˆ ˆvar( ) { } = { }- { } ( - { }) f E f E f E f E f E f E f

N



We note that the accuracy of the estimator does not

depend on the dimensionality of z

High accuracy may be achieved with a relatively small

number of i.i.d. samples z(i).

In practice, N 10 ~ 20, as long as z(i) are i.i.d.

The problem, however, is that the samples {z(i)} might not

be independent effective sample size « N

If f (z) is small in regions where p(z) is large, and, vice

versa, then is dominated by regions of small

probability if sample size are not large enough to

represent the distribution of z

relatively large sample sizes will be required

f̂



Basic sampling algorithms

We shall assume that an algorithm has been provided to

generate pseudo-random numbers distributed uniformly

over (0,1)

Transformation technique for standard distributions

Suppose z ~ U(0,1)

Let y = f (z) p(y) = p(z)|dz / dy| where, in this case, p(z) = 1

Goal: choose f (z) such that y have

some desired distribution p(y)

Let

If z = h(y) F(z)=z=h(y), dz/dy=p(y)

Then y = h -1(z) with z ~ U(0,1)

ˆ ˆ( ) ( ) ( )

y

h y F y p y dy

z



For example, p(y) = exp (- y) for 0 y

h(y) = 1- exp (- y)

y = - -1 ln (1 – z) with y ~ p(y) and z ~ U(0,1)

Box-Muller method for generating Gaussian distributed

random numbers

Generate z1 and z2 with U(-1,1), which

can be done with z = 2y – 1, y ~ U(0,1)

Discard (z1, z2) unless r2 = z12 + z2

2 1

p(z1, z2) = 1/



Consequently, we have

HW1: Prove the Box-Muller method

For generating random numbers y ~ N ( , )

Let = LLT , making use of the Cholesky decomposition

Then y = + Lz

The transformation technique depends on the ability to

calculate and invert the indefinite integral of the desired

distribution



Rejection sampling

Suppose we want to

sample from a p(z)

which is not a simple

standard distribution

Nevertheless, we are

able to evaluate p(z) z up to some normalizing constant Zp ,

namely

where Zp is unknown

We can readily draw samples from some simpler distribution

q(z), which is sometimes called a proposal distribution

We next introduce a constant k such that ( ) ( ), kq p z z z



The rejection sampling method is as follows

First, we first generate a number z0 from q(z)

Next, we generate a number u0 from U[0, kq(z0)]

Finally, if , then the sample is rejected

The corresponding z values are distributed according to p(z)

The probability that a sample will be accepted is given by

Thus, the constant k should be as small as possible

0 0( )u p z



Example:

We want to sample from the gamma distribution

A suitable proposal distribution

q(z) is Cauchy

because it is bell-shaped too

q(z) can be generated with

the transformation method

by having

z = tan (y) with y ~ U(0,)

2

1 1( ) =

1+zq z



To make sure , we now have z = b tan (y) + c

with y ~ U(0,1)

This leads to

The minimum rejection ratio is obtained by setting c=a-1

and b2 = 2a-1

HW2: Show the above formulas for this example

2 2

k( ) =

1+(z-c) /q z

b

( ) ( )kq pz z



Importance sampling

The probability of interest will often have much of their

mass confined to relatively small regions of z space

The objective of importance sampling is aimed to sample

the distribution in the region of importance

We also use a proposal distribution q(z) from which it is

easy to draw samples

Then approximate E{f (z)} by

( )( ) ( )

( )1 1

( )E{ }= ( ) ( )d ( ) ( )d

( )

1 ( ) 1( ) ( )

( )

iN Ni i

iii i

pf f p f q

q

pf w f

N q N

zz z z z z z

z

zz z

z



wi = p(z(i))/q(z(i)) are known as importance weights

When p(z) can only be evaluated up to a normalizing

constant Zp , i.e.

Similarly, we may wish to use for sampling

Then,

where

Where

Hence,

( ) = ( ) / pp p Zz z

( )

1

( ) 1E{ }= ( ) ( )d ( ) ( )d ( )

( )

Nq q i

i

ip p

Z Zpf f p f q w f

Z q Z N

z

z z z z z z zz

( ) = ( ) / qq q Zz z

1

1 ( ) 1= ( )d ( )d

( )

Np

i

iq q

Z pp q w

Z Z q N

z

z z z zz

( )

1

1

E{ }= ( ) with N

i iNi i

ii

i

wf w f w

w

z

( )

( )( )

( )

i

ii

pw

qz

z



Sampling-importance-resampling (SIR)

There are two stages of the scheme

Draw N, z(1),…, z(N) samples from q(z)

Compute weights wi, i=1,…, N

Draw a second set of N samples from (z(1),…, z(N)) with

probabilities given by the weights (w1,…, wN)

Multinomial resampling

Draw a u ~ U(0,1)

Compute si

Find si s.t. si-1 u si

The particle z(i) is chosen

Suppose each z(i) is chosen

Ni times E{Ni } = N wi

Var{Ni } = N wi (1- wi) z(i)

wi

si=j=1i wj



The resultant N samples are only approximately distributed

according to p(z), but becomes correct when N

To see this, consider the CDF of the resampled values

where I(.) is the indicator function

When N



The resampling occurs between important sampling steps

Resampling can be taken at every steps or only when needed

Resampling is critical in importance sampling

If (w1,…, wN)

are uneven

distributed

If w1 are skewed,

resampling can

provide chances

for selecting

important

samples



Markov Chain Monte Carlo

Rejection and importance sampling methods suffer from

severe limitations in spaces of high dimensionality

As with rejection and importance sampling, we again

sample from a proposal distribution

Maintain a record of the current state z()

The proposal distribution q(z | z()) depends on z()

If we write , we assume can readily be

evaluated, although Zp may be unknown

The proposal distribution q(z | z()) should be sufficiently

simple and straightforward to draw samples from it directly

At each cycle, we generate a candidate sample z*, and accept

the sample according to an appropriate criterion

( ) = ( ) / pp p Zz z ( )p z



Metropolis algorithm

Assume the proposal distribution is symmetric, i.e.

q(zA | zB) = q(zB | zA) , zA and zB

1) Draw a candidate sample z* from q(z | z())

2) The candidate sample z* is accepted with probability

This can be achieved by choosing a random number u with

U(0,1) and then accept the sample if A(z*, z()) > u

Note: if the step from z() to z* cause an increase in the value

of p(z), then the candidate z* is certain to be kept

3) If the candidate z* is accepted, then z(+1) = z*, otherwise

z* is discarded, and set z(+1) = z() and go back to 1).

** ( )

( )

( )( , ) min 1,

( )

pA

p

zz z

z



The sequence of samples z(1), z(2),… forms a Markov chain

In rejection sampling, rejected samples are simply discarded.

In the metropolis method,

when a candidate is rejected,

the previous sample is

included instead in the

final list of samples.

z(1) (z(2)= z(3)) z(4) …

If q(zA | zB) > 0, then the

distribution of z p(z)

if

Why?



A natural question is under what conditions will a

Markov chain converge to the desired distribution

Consider a first order Markov chain of z(1), z(2),…, z(M) with

p(z(m+1) | z(1), z(2),…, z(M)) = p(z(m+1) | z(1))

The Markov chain can be specified with p(z(0)) and a

transition probability Tm(z(m), z(m+1)) p(z(m+1) | z(m))

A Markov chain is called homogeneous if Tm(z(m), z(m+1)) are

the same for all m.

For a homogeneous Markov chain with T(z’, z),

p*(z) is invariant if p*(z) = z T(z’, z) p*(z’)

A sufficient condition for p*(z) to be invariant is to choose a

T(z’, z) that satisfies p*(z) T(z, z’) = p*(z’) T(z’, z) since

p*(z) = z’ p(z’ | z) p*(z) = z’ T(z, z’) p*(z) = z’ T(z’, z) p*(z’)



p*(z) T(z, z’) = p*(z’) T(z’, z) is referred to as the property of

detailed balance for p*(z)

A Markov chain that respects detailed balance is said to be

reversible

If p(z(m+1) ) converges to the required invariant p*(z) for m

irrespective of the choice of initial distribution p(z(0)),

this property is called ergodicity. p*(z)

The invariant p*(z) is then called the equilibrium

distribution

It can be shown that a homogeneous Markov chain will be

ergodic subject to weak restrictions on p*(z) and T(z’, z)

An ergodic Markov chain can have only one equilibrium

distribution The goal is to find T(z’, z) that leads to p*(z)



The Metropolis-Hasting (MH) algorithm

In contrast to the Metropolis algorithm, the proposal distribution q(zA | zB) is no longer to be symmetric

Draw a candidate sample z* from q(z | z())

The candidate sample z* is accepted with probability

For a symmetric q(z | z()), the MH criterion reduces to the Metropolis criterion

The probability that Markov process stays at z() is given by

* ( ) ** ( )

( ) * ( )

( ) ( | )( , ) min 1,

( ) ( | )

p qA

p q

z z zz z

z z z

( ) ( )1 ( , ) ( | )dA q z

z z z z z



The purpose of introducing A(z*, z()) is to make q(z | z())

satisfy the property of detailed balance, namely

p(z’) q(z | z’) A(z, z’) = p(z) q(z’ | z) A( z’, z)

To see this, it follows that

p(z’) q(z | z’) A(z, z’) = min(p(z’) q(z | z’), p(z) q(z’ | z))

= min(p(z) q(z’ | z), p(z’) q(z | z’))= p(z) q(z’ | z) A(z’, z)

We note that, in general, we do not know how fast the

Markov chain will converge to an equilibrium distribution

Only samples that are drawn after the Markov chain

approaches the equilibrium are regarded as the

representative draws

The time to reach the equilibrium is called the burn-in time



In summary, a generic Metropolis-Hastings algorithm

proceeds as follows

1) For i = 1,…, N, set = 0, draw z () from a p(z())

2) Generate u from U(0,1) and draw z* from q(z | z())

3) If u < A(z*, z()) z(+1) = z*, otherwise z(+1) = z()

4) = + 1, repeat 2) and 3) until T, the burn-in time

5) Store x(i) = z(T)

6) i = i + 1, repeat 1) to 6) until N samples are drawn

7) Return samples {x(1) ,…, x(N)} which are ~ p(x)

A common choice of proposal distribution is a Gaussian

centered on z(), i.e. N(z(), 2)

where the scale should be of the same order as min


Unit 11 Sampling Methods and

Particle Filter

Sequential Monte Carlo Estimation

(Particle Filter)



Sequential Monte Carlo Estimation

Given Yn ={y1,…, yn}, our goal is to approximate the

posterior probability: p(xn | Yn ) of xn with

where Np is the number of particles and x(i) are assumed to

be i.i.d. drawn from p(xn | Yn )

By this approximation, we can estimate the posterior mean

of f (xn)

n n

pN

( i )

n n n n

i 1P

1ˆp( | ) ( ) p( | )

N

x x x xY Y



In practice, it is common and much easier to sample from a

proposal distribution q(xn | Yn ), hence

where the weighting function is given by



Rewrite

By i.i.d. samples from q(xn | Yn ),



Now, suppose q(x0:n | Yn ) has a factorized form of

In addition, the joint posterior PDF

Thus, the importance weight can be updated recursively



In practice, however, we are more interested in p(xn | Yn )

Suppose q(xn | x0:n-1, Yn ) = q(xn | x0:n-1, yn ), the weighting

function can be simplified as



The two most frequently used importance functions are

the prior and the optimal importance function

The prior importance is p(xn | xn-1(i)), consequently

Wn(i) Wn-1

(i) p(yn | xn(i))

The optimal importance functions minimizes the variance of

the importance weights conditioned on the x0:n-1(i) and Yn

and is given by p(xn | x0:n-1(i), Yn)

The corresponding weight vector is given by

( )( ) 0:

( ) ( ) ( )

0: 1 0: 1 1

( ) ( ) ( ) ( )( ) ( )0: 1 0: 1 1 0: 1 1

1 1( ) ( ) ( )

0: 1 0: 1 1

( | )

( | , ) ( | )

( | , ) ( | , ) ( | )( | )

( | , ) ( | )

ii n n

n i i i

n n n n n

i i i ii in n n n n n n n

n n ni i i

n n n n n

p

q q

p p pp

p q

xW

x x x

x x y x xW y x

x x x

Y

Y Y

Y Y Y

Y Y



Sequential importance sampling (SIS) filter

The problem of the SIS filter is that the distribution of the

sampling weights become more and more skewed

To monitor how bad is the weight degeneration, a suggested

method is the so-called effective sample size Neff

The second equality follows that Eq(.)



The SIS algorithm



Bootstrap/ Sampling Importance Resampling filter (SIR)

SIR uses transition prior p(xn | xn-1) as q(xn | x0:n-1, Yn )

Consequently, Wn = p(yn | xn )



Remarks:

Both SIS and SIR filters use importance sampling scheme

The difference between SIS and SIR is that in SIR, resampling

is always performed; whereas in SIS, importance weights are

calculated sequentially. Resampling is only used when needed

Resampling step is suggested to be done after filtering,

because resampling brings extra random variation to the

current samples. Normally, the posterior estimate should be

calculated before resampling



MCMC Particle filter

The performance of particle filters depends to a large extent

on the choices of proposal distribution

To tackle more general and complex probability distribution,

MCMC methods are needed

In particle filter framework, MCMC is used for drawing

samples either in sampling or resampling step

The desired distribution is

p*(x0:n | Yn) p(yn | xn) p(xn | xn-1) p(x0:n-1 | Yn-1)

Now perform MCMC at the sampling step, i.e. to replace xn

with a particle xn* with an acceptance probability A(xn

*, xn),

where xn* is generated with a proposal distribution q(xn

*| xn)



In the framework of particle filtering, the acceptance probability is given by

Provided that we use the prior, q(xn*| xn) = p(xn

* | xn-1), as proposal, we have

Or, we may have a proposal with q(xn* | xn) = q(xn | xn

*), e.g. N(xn

* | xn) = N(xn | xn*)

* * ( ) ( ) ** 1

( ) ( ) ( ) * ( )

1

( | ) ( | ) ( | )( , ) min 1,

( | ) ( | ) ( | )

i i

n n n n n nn n i i i i

n n n n n n

p p qA

p p q

y x x x x xx x

y x x x x x

**

( )

( | )( , ) min 1,

( | )

n nn n i

n n

pA

p

y xx x

y x

* * ( )* 1

( ) ( ) ( )

1

( | ) ( | )( , ) min 1,

( | ) ( | )

i

n n n nn n i i i

n n n n

p pA

p p

y x x xx x

y x x x



Applications of Particle filters on blind equalization

Detection of transmitted data without using training data

Suppose stT = [st ,…, st-L+1] and ht

T = [ht,0 ,…, ht,L-1],

yt = htTst + vt with vt ~ N(0,v

2 )

The objective is to obtain p(s0:t | y0:t)

Time-Invariant channels (ht = h)

Let st {-1,1} and

The posterior PDF of h is

1 1~ ( , ) h hN( )( ) ( )

0: 0:( | , ) ( , )mm m

tt t tp s y h hN



The likelihood function, p(yt |s0:t(m), y0:t), of a trajectory s0:t

(m)

can be obtain by marginalizing out h w.r.t.

yielding

( )

0: 1 0: 1( | , )m

t tp s y h



The optimal importance function p(st |s0:t-1(m), y0:t) is

proportional to p(yt | st , s0:t-1(m), y0:t-1), and is thus equal to

which is in fact a Bernoulli distribution

Draw a number st(m) from p(st |s0:t-1

(m), y0:t) to form s0:t(m)

And the updating of the particle weights is carried out by

Wt(m) Wt-1

(m) p(yt | s0:t-1(m), y0:t-1) which is equal to



Note that the

complexity does

not increase

with time

The number of

trajectories is

fixed



Time-Variant channels

Still we have stT = [st ,…, st-L+1] and ht

T = [ht,0 ,…, ht,L-1],

yt = htTst + vt with vt ~ N(0,v

2 )

ht, l = - k=1ra ak ht-k, l + ut, l

Define xtT = [ht

T ,…, ht-ra+1T]

The state space system model is given by

a

T

t t t 1 t L 1r L

s ,s ...,s ,0,0,...

s



Again we consider optimal importance function

p(st |s0:t-1(m), y0:t) p(yt | st , s0:t-1

(m), y0:t-1) which is equal to

Given st , s0:t-1(m), p(xt | st , s0:t-1

(m), y0:t-1) is Gaussian

distributed, denoted by N( t(m), t

(m) ), whose mean t(m)

and variance t(m) can be obtained with Kalman filtering

Therefore, p(st |s0:t-1(m), y0:t)

a

T( m ) ( m ) ( m )

t t t 1 t L 1r L

s ,s ...,s ,0,0,...

s



We note that t(m) and t

(m) are the predictive mean and

variance of xt of the mth trajectory

Similarly, the corresponding weights is given by

where

We note that to compute Wt(m) for each particle, we evaluate

the means and covariances of two Kalman filters, each

corresponding to the symbols st = 1 and st = -1

Then draw st(m) and update the corresponding t

(m) and t(m)

a

T( m ) ( m ) ( m )

t t t 1 t L 1r L

s ,s ...,s ,0,0,...

s



Joint estimation and detection in flat fading channels

The system model is given by

Method I: use prior distribution as the important function

=

The weighting function is given by



Method II: use optimal importance function given by

A sample st(m) is first obtained from

Followed by a sample ht(m) from

p(ht | ht-1(m), st

(m), yt) = N( t(m), t

(m) )

with



The weight update equation is given by

where

unit 11 sampling methods and particle filter

Documents