EM Algorithm in HMM and Linear EM Algorithm in HMM and Linear Dynamical SystemsDynamical Systems

by Yang Jinsan

2

ReferencesReferences

An Introduction to the Kalman Filter (1999), Technical Report, by G. Welch and G. Bishop

Parameter Estimation for Linear Dynamical Systems (1999), Technical Report, by Z. Gharhamani and G.E. Hinton

From Hidden Markov Models to Linear Dynamical Systems (1999), Technical Report, by T.P. Minka

Gaussian Process (1999), Technical Report, by D. Mackay A comparison between the EM and subspace identification algorithms

for time-invariant linear dynamic systems (2000), Technical Report, by

GA. Smith and AJ Robinson.

3

Outline Outline HMM

Forward Algorithm Backward Algorithm Baum-Welch Algorithm EM Algorithm

Linear Dynamic System Kalman Filter EM Algorithm

HMM and Linear Dynamic System

4

HMM-Forward AlgorithmHMM-Forward Algorithm

Question: What is the probability of having ‘Sunny’ weather today if the seaweed was dry, damp , soggy during last three days ?

Answer: P (‘Sunny’ today) = Sum of all the path for P(‘Sunny’; a path to‘Sunny’)

# summation = (# state)(#state)…(#state)

5

Belief Propagation in Markov Chain:

})1()1|2({

)2|3())1(|(

)1()1|2())1(|(

),)1(,,2,1(

)(

1

)1(2

1 2 )1(

1 2 )1(

day

Tdayday

day day Tday

day day Tday

weatherPweatherweatherP

weatherweatherPTweathersunnyP

weatherPweatherweatherPTweathersunnyP

sunnyTweatherweatherweatherP

SunnyP

6

)()|()|()|()|(

)()|()|()|,()(

111111,,,

1111)1(~11

1 pppppkkkkiii

ki kkkkkkk

iiipixpiipixp

iiipixpSixpi

pkk

k

(Probability of arriving to state ik+1 with x1 ~ xk+1 : Forward probability from i

1)

(Where ik+1 represents the state of hidden variable in time step k+1)

(Probability of starting from state ik with xk+1 ~ xN : Backward probability from iN )

)()|()|()|()|(

)()|()|(),|()(

,,, 1111

1111~)1(

21

1

qiii qqqqkkkk

ki kkkkkNkk

iiipixpiipixp

iiipixpSixpi

qkk

k

7

Baum-Welch re-estimation:Baum-Welch re-estimation:

The probability of passing through state ik with all the observations:

The probability of passing through state ik and ik+1 with all the observations:

)|,,(

)()(),,,|()(

)()(),|,,()|,,,(

)|,,,(),,,(

11

11

11

Sxxp

iiSxxipi

iiSixxpSxxip

Sxxipxxi

N

kkNkk

kkkNkkk

NkNk

)|,,(

)()|()|()(

)|,,(

)|,,;,(),,,|,(),(

)|,,;,(),,;,(

1

1111

1

11111

1111

Sxxp

iixpiipi

Sxxp

SxxiipSxxiipii

Sxxiipxxii

N

kkkkkk

N

NkkNkkkk

NkkNkk

8

The estimate for the number of occurrence of state i during N stages given the model S and N observations :

The estimate for the number of transitions (state i state j) during N stages given the model S and N observations :

Re-estimation formulas for the unknown model parameters:

k k ii )(

1

11 ),(

N

kkk jiii

)()(ˆ

)(

)(

)|(ˆ)(ˆ,)(

),()|(ˆˆ

11

1

,1

1

1

1

11

iiiip

ii

ii

irxprbii

jiiiijpa

i

N

kk

N

rxkk

iiN

kk

N

kkk

ijk

9

Calculation of Likelihood:

Re-estimation Algorithm (A special case of the EM algorithm)

1. Estimate indicator ( , ) for the transition and emission process. (E-Step) from the given HMM parameters.

2. Update S = (A, B, ) using indicator from step 1.

stateallstateall

NN jiiiSxxp )()()|,,( 11

10

Linear Dynamic SystemLinear Dynamic System

conditioninitialQNs

unknownandfixedareRQBA

NkRNvvBsx

QNwwAss

kkkk

kkkk

:),(~

.),,,(

,,2,1),0(~,

),0(~,

111

1

S1 S2 S3 . . . . . SN

X1 X2 X3 . . . . . XN

11

Posterior estimation by Kalman gain K.

( Vk :posterior covariance of sk | xk , : prior covariance of )

For smaller R, x is trusted more

For smaller prior covariance of estimate for s,

predicted measurement(estimate) using prior x is trusted more

Forecast Correct Hidden State

(Predict: Forward) (Update)

1

0lim

kk

RBK

k

):ˆ:ˆ()ˆ(ˆˆ priorsposteriorssBxKss kkkkkkkk

0lim0

k

VK

k

),())ˆ)(ˆ(,ˆ(~)|(:ˆ kkT

kkkkkkkk VsNssssEsENxsps

kV

kk xs |

12

13

Kalman FilterKalman Filter

.),(

,2)]([

,)]([

):)(

)(

)0)),((()()(

}))]ˆ)(()ˆ))][(ˆ)(()ˆ{[(

}))]ˆ(ˆ())][(ˆ(ˆ({[(

)ˆ)(ˆ(

21

1211

symmetricbemustCsquarebemustAB

da

dsda

ds

da

ds

dA

dsAC

dA

ACAtrdB

dA

ABtrd

svectorstateinelementstheallofMSEofsumVtrace

KRBVBKKBVVBKV

vssCovwhereKRKBKIVBKI

sBvsBKsssBvsBKssE

sBxKsssBxKssE

ssssEV

TT

k

Tkk

Tkkkk

Tk

Tkkkkkk

kkkT

kkkT

kkkkk

Tkkkkkkkkkkkkkkkk

Tkkkkkkkkkkkk

Tkkkkk

14

kkk

Tkk

Tkkkkk

kkkTkkk

Tkkkk

kTkkk

Tkkk

kTkkkk

Tkk

k

k

VBKI

KRBVBKV

VBRBVBBVVV

gainKalmanRBVBBVK

RHVHKVHdK

Vtrd

)(

)(

)(*

)()(

0)(2)(2)(

1

1

15

Step 1 (E): Given the model, estimate hidden states using Kalman filter

Step 2 (M): Update parameters: A, B, R, Q, Q1using the estimations of

hidden state from step 1 and the log-likelihood :

kkkkkkkkkk

kTkkk

Tkkk

kTkkkkkkk

VHKIVsBxKss

RBVBBVKUpdating

QAVAVsAsgForecastin

)(,)ˆ(ˆˆ

,)()(

,ˆˆ)(1

11

},,|),,;,,({log 111'. NNNsswrt xxxxsspEt

2log2

12log

22log

2)1(||log

2||log

2

1

||log2

1||log

2)()(

2

1

)()(2

1)()(

2

1),,;,,(log

1

11

11

21

1111

111111

NN

NNR

NQ

QN

RN

BxxRBxx

AssQAsssQsxxssp

N

ttttt

N

tttttNN

16

11111111111111

111

1111

,11

,1

11,1

11,,11

11

1

1

1

1

2 12 1,12

1,1

~111,~1~1

ˆˆ),ˆˆ(2

1

2

1

ˆ,0)ˆ(

)(1

1,0)(

2

1

2

1

,0)(2

1

2

1

)ˆ(1

,0)ˆ2

1

2

1(

2

))(ˆ(,0)ˆ(2

))((,0)(2

2)(

,)()(

*

)|(),|(),|(ˆ

ssPQssPQQ

sQs

PAPN

QPAPQN

APAAPSinceAAPAPAPPQN

Q

xsBxxN

RxsBBBPxxRN

R

PsxBBPsxRB

PPAAPPQA

Axx

Axxl

x

xl

x

lx

xssEPxssEPxsEs

new

new

ttnew

tnew

N

ttt

newt

tnew

tt

N

ttttttt

ttnew

ttnew

ttt

N

ttt

tttnew

t

N

ttt

tttnew

t

N

ttt

NttttNtttNtt

17

))((

)(

)(

)ˆˆ(ˆˆ

)(

:ˆˆ

ˆˆ,)|(ˆ

11,

211,1212,1

1111

1111

11111

11,1,

~1

NNN

NN

ttNttttt

Ntt

ttN

tttN

t

tNttt

Nt

tt

ttt

Nt

Nt

Ntttt

Nt

Nt

NttNt

Nt

AVBKIVbydinitialize

JAVVJJVV

JVVJVV

sAsJss

VAVJ

ssVP

andssVPxsEsofnComputatioT

T

(From Shumway and Stoffer (1982). An approach to time series smoothing and forecasting using the EM algorithm. J. Time Series Analysis, 3(4):253-264. )

18

Methods by forward and backward propaMethods by forward and backward propagation (Minka (1999))gation (Minka (1999))

19

Calculation of data likelihoodCalculation of data likelihood

:

:.2

).(

.,|

)|()(),(:.1

?),,(

)|()|()|()|()|()(),,;,,(

1

2211112111

aboveas

foundbecouldxeachforsolutionexactAncasedynamicaFor

BishopseefoundbecouldsolutionexactAn

xofondistributithefindsxandsofonsdistributitheGiven

sxpspxspcasestaticaFor

xxp

sxpsxpsxpsspsspspxxssp

t

N

NNNNNN

sshiddenofNEXPECTATIOan

takingbylikelihooddatacompleteaisQlikelihoodThe

t '

log*

20