![Page 1: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/1.jpg)
11
Hidden Markov ModelHidden Markov Model
Observation : O1,O2, . . . Observation : O1,O2, . . .
States in time : q1, q2, . . .States in time : q1, q2, . . .
All states : s1, s2, . . .All states : s1, s2, . . .
tOOOO ,,,, 321
tqqqq ,,,, 321
Si Sjjiaija
![Page 2: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/2.jpg)
22
Hidden Markov Model (Cont’d)Hidden Markov Model (Cont’d)
Discrete Markov ModelDiscrete Markov Model
)|(
),,,|(
1
121
itjt
zktitjt
sqsqP
sqsqsqsqP
Degree 1 Markov Model
![Page 3: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/3.jpg)
33
Hidden Markov Model (Cont’d)Hidden Markov Model (Cont’d)
)|( 1, itjtji sqsqPa
ija : Transition Probability from Si to Sj ,
Nji ,1
![Page 4: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/4.jpg)
44
Hidden Markov Model Hidden Markov Model ExampleExample
S1 : The weather is rainyS2 : The weather is cloudyS3 : The weather is sunny
8.01.01.0
2.06.02.0
3.03.04.0
}{ ijaA
rainy cloudy sunnyrainy
cloudy
sunny
![Page 5: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/5.jpg)
55
Hidden Markov Model Example Hidden Markov Model Example (Cont’d)(Cont’d)
Question 1:How much is this probability:Sunny-Sunny-Sunny-Rainy-Rainy-Sunny-Cloudy-Cloudy
22311333 ssssssss
22321311313333 aaaaaaa
87654321 qqqqqqqq410536.1
![Page 6: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/6.jpg)
66
Hidden Markov Model Example Hidden Markov Model Example (Cont’d)(Cont’d)
Question 2:The probability of staying in a state for d days if we are in state Si?
NisqP ii 1),( 1The probability of being in state i in time t=1
)()1()( 1 dPaassssP iiidiiijiii
d Days
![Page 7: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/7.jpg)
77
HMM ComponentsHMM Components
N : Number Of StatesN : Number Of States
M : Number Of OutputsM : Number Of Outputs
A : State Transition Probability MatrixA : State Transition Probability Matrix
B : Output Occurrence Probability in B : Output Occurrence Probability in each stateeach state
: Primary Occurrence Probability: Primary Occurrence Probability),,( BA : Set of HMM Parameters
![Page 8: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/8.jpg)
88
Three Basic HMM ProblemsThree Basic HMM Problems
Given an HMM Given an HMM and a sequence of and a sequence of observations observations O,O,what is the probability what is the probability ? ?
Given a model and a sequence of Given a model and a sequence of observations observations OO, what is the most likely , what is the most likely state sequence in the model that produced state sequence in the model that produced the observations?the observations?
Given a model Given a model and a sequence of and a sequence of observationsobservations O, O, how should we adjust how should we adjust model parameters in order to maximize model parameters in order to maximize ? ?
)|( OP
)|( OP
![Page 9: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/9.jpg)
99
First Problem SolutionFirst Problem Solution
)(),|(),|(11
tq
T
ttt
T
tobqoPqoP
t
TT qqqqqqq aaaqP132211
)|(
)()|(),( yPyxPyxP
)|(),|()|,( zyPzyxPzyxP We Know That:
And
![Page 10: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/10.jpg)
1010
First Problem Solution (Cont’d)First Problem Solution (Cont’d)
)|(),|()|,( qPqoPqoP
)()()(
)|,(
122111 21 Tqqqqqqqq obaobaob
qoP
TTT
T
TTTqqq
Tqqqqqqqq
q
obaobaob
qoPoP
21
122111)()()(
)|,()|(
21
Account Order : )2( TTNO
![Page 11: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/11.jpg)
1111
Forward Backward ApproachForward Backward Approach
)|,,,,()( 21 iqoooPi ttt
Niobi ii 1),()( 11
Computing )(it
1) Initialization
![Page 12: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/12.jpg)
1212
Forward Backward Approach Forward Backward Approach (Cont’d)(Cont’d)
NjTt
obaij tjij
N
itt
1,11
)(])([)( 11
1 2) Induction :
3) Termination :
N
iT ioP
1
)()|(
Account Order : )( 2TNO
![Page 13: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/13.jpg)
1313
Backward Variable ApproachBackward Variable Approach
),|,,,()( 21 iqoooPi tTttt
NiiT 1,1)(1) Initialization
2)Induction
NjAndTTt
jobaiN
jttjijt
11,,2,1
)()()(1
11
![Page 14: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/14.jpg)
1414
Second Problem SolutionSecond Problem Solution
Finding the most likely state sequenceFinding the most likely state sequence
N
itt
ttN
it
t
ttt
ii
ii
iqoP
iqoP
oP
iqoPoiqPi
11
)()(
)()(
)|,(
)|,(
)|(
)|,(),|()(
Individually most likely state :
NntTtiq tt 1,1)],(max[arg*
![Page 15: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/15.jpg)
1515
Viterbi AlgorithmViterbi Algorithm
Define : Define :
Ni
qqq
oooiqqqqP
i
t
ttt
t
1
,,,
]|,,,,,,,,[max
)(
121
21121
P is the most likely state sequence with this conditions : state i , time t and observation o
![Page 16: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/16.jpg)
1616
Viterbi Algorithm (Cont’d)Viterbi Algorithm (Cont’d)
)(].)(max[)( 11 tjijti
t obaij
1) Initialization
0)(
1),()(
1
11
i
Niobi ii
)(it Is the most likely state before state i at time t-1
![Page 17: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/17.jpg)
1717
Viterbi Algorithm (Cont’d)Viterbi Algorithm (Cont’d)
NjTt
aij
obaij
ijtNi
t
tjijtNi
t
1,2
])([maxarg)(
)(])([max)(
11
11
2) Recursion
![Page 18: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/18.jpg)
1818
Viterbi Algorithm (Cont’d)Viterbi Algorithm (Cont’d)
)]([maxarg
)]([max
1
*
1
*
iq
ip
TNi
T
TNi
3) Termination:
4)Backtracking:
1,,2,1),( *11
* TTtqq ttt
![Page 19: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/19.jpg)
1919
Third Problem SolutionThird Problem Solution
Parameters Estimation using Baum-Parameters Estimation using Baum-Welch Or Expectation Maximization Welch Or Expectation Maximization (EM) Approach(EM) Approach
Define:
N
i
N
jttjijt
ttjijt
tt
ttt
jobai
jobai
oP
jqiqoP
ojqiqPji
1 111
11
1
1
)()()(
)()()(
)|(
)|,,(
),|,(),(
![Page 20: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/20.jpg)
2020
Third Problem Solution Third Problem Solution (Cont’d)(Cont’d)
N
jtt jii
1
),()(
1
1
)(T
tt i
T
tt ji
1
),(
: Expectation value of the number of jumps from state i
: Expectation value of the number of jumps from state i to state j
![Page 21: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/21.jpg)
2121
Third Problem Solution Third Problem Solution (Cont’d)(Cont’d)
)(1 ii
T
tt
T
tt
ij
i
jia
1
1
)(
),(
T
tt
Vo
T
tt
j
j
j
kb kt
1
1
)(
)(
)(
![Page 22: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/22.jpg)
2222
Baum Auxiliary FunctionBaum Auxiliary Function
q
qoPqoPQ )|,(log)'|,()|( '
)|()|(
)',(),(: ''
oPoP
QQif
By this approach we will reach to a local optimum
![Page 23: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/23.jpg)
2323
Restrictions Of Restrictions Of Reestimation FormulasReestimation Formulas
11
N
ii
NiaN
jij
1,11
NjkbM
kj
1,1)(1
![Page 24: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/24.jpg)
2424
Continuous Observation Continuous Observation DensityDensity
We have amounts of a PDF instead of We have amounts of a PDF instead of
We haveWe have
)|()( jqVoPkb tktj
1)(,),,()(1
dooboCob j
M
kjkjkjkj
Mixture Coefficients
Average Variance
![Page 25: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/25.jpg)
2525
Continuous Observation Continuous Observation DensityDensity
Mixture in HMMMixture in HMM
),,()( jkjkjkk
j oCMaxob
M2|1M1|1
M4|1M3|1
M2|3M1|3
M4|3M3|3
M2|2M1|2
M4|2M3|2
S1 S2 S3Dominant Mixture:
![Page 26: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/26.jpg)
2626
Continuous Observation Continuous Observation Density (Cont’d)Density (Cont’d)
Model Parameters:Model Parameters:
),,,,( CA
N×N N×M×K×KN×M×KN×M1×N
N : Number Of StatesM : Number Of Mixtures In Each StateK : Dimension Of Observation Vector
![Page 27: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/27.jpg)
2727
Continuous Observation Continuous Observation Density (Cont’d)Density (Cont’d)
T
t
M
kt
T
tt
jk
kj
kjC
1 1
1
),(
),(
T
tt
t
T
tt
jk
kj
okj
1
1
),(
),(
![Page 28: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/28.jpg)
2828
Continuous Observation Continuous Observation Density (Cont’d)Density (Cont’d)
T
tt
jktjkt
T
tt
jk
kj
ookj
1
1
),(
)()(),(
),( kjt Probability of event j’th state and k’th mixture at time t
![Page 29: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/29.jpg)
2929
State Duration ModelingState Duration Modeling
Si Sj
Probability of staying d times in state i :
)1()( 1ii
diii aadP
jia
ija
![Page 30: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/30.jpg)
3030
State Duration Modeling State Duration Modeling (Cont’d)(Cont’d)
Si Sjjia
……. …….
HMM With clear duration
ija )(dPj)(dPi
![Page 31: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/31.jpg)
3131
State Duration Modeling State Duration Modeling (Cont’d)(Cont’d)
HMM consideration with State Duration :HMM consideration with State Duration :– Selecting using ‘sSelecting using ‘s– Selecting usingSelecting using– Selecting Observation Sequence Selecting Observation Sequence
using using in practice we assume the following in practice we assume the following
independence:independence:
– Selecting next state using transition probabilities Selecting next state using transition probabilities . We also have an additional constraint: . We also have an additional constraint:
),(),,,(1
1
11 121 tq
d
tdq OtbOOOb
iiq 1
dOOO ,,, 21 )(
1dPq1d
21qqa
),,,(11 21 dq OOOb
jq 2
011qqa
![Page 32: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/32.jpg)
3232
Training In HMMTraining In HMM
Maximum Likelihood (ML)Maximum Likelihood (ML)
Maximum Mutual Information (MMI)Maximum Mutual Information (MMI)
Minimum Discrimination Information (MDI)Minimum Discrimination Information (MDI)
![Page 33: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/33.jpg)
3333
Training In HMMTraining In HMM
Maximum Likelihood (ML)Maximum Likelihood (ML)
)|( 1oP
)|( 2oP)|( 3oP
)|( noP
.
.
.
)]|([*V
rOPMaximumP
ObservationSequence
![Page 34: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/34.jpg)
3434
Training In HMM (Cont’d)Training In HMM (Cont’d)
Maximum Mutual Information (MMI)Maximum Mutual Information (MMI)
)()(
)|,(log),(
POP
OPOI
v
ww
v
wPwOP
OPOI
1
)(),|(log
)|(log),(
Mutual Information
}{, v
![Page 35: 1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj](https://reader035.vdocuments.us/reader035/viewer/2022062809/5697c0091a28abf838cc6fa5/html5/thumbnails/35.jpg)
3535
Training In HMM (Cont’d)Training In HMM (Cont’d)
Minimum Discrimination Information Minimum Discrimination Information (MDI)(MDI)
dooP
oqoqPQI )|(
)(log)():(
),,,( 21 TOOOO
),,,( 21 tRRRR Observation :
Auto correlation :
):(inf),( PQIPR )(RQ