hidden markov models - cs department
TRANSCRIPT
![Page 1: Hidden Markov Models - CS Department](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14d607961420d3e1a7cf9/html5/thumbnails/1.jpg)
Hidden Markov ModelsCAP5610: Machine Learning
Instructor: Guo-Jun Qi
![Page 2: Hidden Markov Models - CS Department](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14d607961420d3e1a7cf9/html5/thumbnails/2.jpg)
Dynamic Model VS. Static Model
β’ So far, we have learned a bunch of static machine learning modelsβ’ Given a input feature vector X, we learn a static model to predict its label
from the training set.β’ In the static model, given a sequence {ππππ|ππ = 1,2, β¦ ,ππ} of feature vectors,
the predictions on them are made independently.β’ A decision on a previous feature vector will not affect how we predict on the
next feature vector.β’ A static Bayes model β class conditional model
ππ ππππ ππππ = οΏ½ππ=1
ππ
ππ(ππππ|ππππ)
![Page 3: Hidden Markov Models - CS Department](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14d607961420d3e1a7cf9/html5/thumbnails/3.jpg)
Dynamic Model VS. Static Modelβ’ In real world, many decisions/predictions are made sequentially
which are often dependent on the context (e.g., the previous and/or next decisions/predictions)
β’ Speech recognition, recognizing a word is not independent of the previous word recognized.
Please tell the people falling asleep what have discussed in class.
β’ Exploring the dependency in the context should be very informative and important to many machine learning tasks.
![Page 4: Hidden Markov Models - CS Department](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14d607961420d3e1a7cf9/html5/thumbnails/4.jpg)
Dynamic Model
β’ A joint decisions will be made given a dynamic sequence of input feature vector
ππ ππππ ππππ = οΏ½ππ=1
ππ
ππ(ππππ|ππππ)ππ(ππππ|ππcontext ππ )
Where context(ππ) denotes the context in which decision ππππ is made.
![Page 5: Hidden Markov Models - CS Department](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14d607961420d3e1a7cf9/html5/thumbnails/5.jpg)
A graphical representation of Static and dynamic modelsβ’ Static model
β’ All decision is made only dependent on the current feature vector (Observation)
Xi
Yi
Xn
Yn
X1
Y1
X2
Y2
ππ ππππ ππππ = οΏ½ππ=1
ππ
ππ(ππππ|ππππ)
![Page 6: Hidden Markov Models - CS Department](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14d607961420d3e1a7cf9/html5/thumbnails/6.jpg)
A graphical representation of Static and dynamic modelsβ’ Dynamic model
where context(i) includes {i-2,i-1}.
Xi
Yi
Xn
Yn
X1
Y1
X2
Y2
Xi-1
Yi-1
Xi+1
Yi+1
Xi-2
Yi-2β¦
β¦ β¦
β¦
ππ ππππ ππππ = οΏ½ππ=1
ππ
ππ(ππππ|ππππ)ππ(ππππ|ππcontext ππ )
![Page 7: Hidden Markov Models - CS Department](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14d607961420d3e1a7cf9/html5/thumbnails/7.jpg)
Markov model
β’ Joint distribution of n decision variables, by Bayes rule
Here ππ1:ππβ1 is a short for {ππππ , ππ = 1,2, β¦ , ππ β 1}.
β’ Markov assumption which simplifies the dependence:
ππ ππππ ππ1:ππβ1 = ππ(ππππ|ππππβ1)
where A decision is only dependent on its immediate predecessor.
ππ ππππ = ππ ππ1 ππ ππ2 ππ1 ππ ππ3 ππ1,ππ2 β¦ππ ππππ ππ1:ππβ1 β¦ππ(ππππ|ππ1:ππβ1)
![Page 8: Hidden Markov Models - CS Department](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14d607961420d3e1a7cf9/html5/thumbnails/8.jpg)
Markov Assumption
β’ Dynamic model
where context(i) includes {i-1}.
Xi
Yi
Xn
Yn
X1
Y1
X2
Y2
Xi-1
Yi-1
Xi+1
Yi+1
Xi-2
Yi-2β¦
β¦ β¦
β¦
ππ ππππ ππππ = οΏ½ππ=1
ππ
ππ(ππππ|ππππ)ππ(ππππ|ππcontext ππ )
![Page 9: Hidden Markov Models - CS Department](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14d607961420d3e1a7cf9/html5/thumbnails/9.jpg)
Hidden Markov Model
β’ Decision variables are hidden variables to be inferred.β’ Markov dependence is imposed on the hidden variables.
β’ The sequence of feature vectors constitute the observed variables.β’ The dependence between observed variables are made implicitly via the
hidden variables, where they are not independent of each other.
Xi
Yi
Xn
Yn
X1
Y1
X2
Y2
Xi-1
Yi-1
Xi+1
Yi+1
Xi-2
Yi-2β¦
β¦ β¦
β¦
![Page 10: Hidden Markov Models - CS Department](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14d607961420d3e1a7cf9/html5/thumbnails/10.jpg)
2nd order Markov Assumption
β’ Each decision depends on two immediate predecessors
β’ where context(i) includes {i-1,i-2}.
Xi
Yi
Xn
Yn
X1
Y1
X2
Y2
Xi-1
Yi-1
Xi+1
Yi+1
Xi-2
Yi-2β¦
β¦ β¦
β¦
ππ ππππ ππππ = οΏ½ππ=1
ππ
ππ ππππ|ππππ ππ(ππππ|ππππβ1,ππππβ2)
![Page 11: Hidden Markov Models - CS Department](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14d607961420d3e1a7cf9/html5/thumbnails/11.jpg)
m order Markov Assumption
β’ Probabilistic model
ππ ππππ ππππ = οΏ½ππ=1
ππ
ππ ππππ|ππππ ππ(ππππ|ππππβ1,ππππβ2, β¦ ,ππππβππ)
β’ Each decision depends on the previous m decisions.
![Page 12: Hidden Markov Models - CS Department](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14d607961420d3e1a7cf9/html5/thumbnails/12.jpg)
Parameterization β Hidden Markov Modal
β’ HMM:
β’ State space ππ = {1,2, β¦ πΌπΌ}, from which ππππ samples its value.β’ Representing a particular label that denotes the class of Xiβ’ A decision that is made based on the observations through i
β’ Initial state distribution ππ ππ1 = π π = πππ π β’ Transition distribution ππ ππππ = ππ|ππππβ1 = ππ = ππππππ
β’ Stationary assumption: the transition distribution does not change over time, i.e., independent of i, the transition distribution stays the same.
ππ ππππ ππππ = ππ ππ1 ππ ππ1 ππ1 οΏ½ππ=2
ππ
ππ ππππ|ππππ ππ(ππππ|ππππβ1)
![Page 13: Hidden Markov Models - CS Department](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14d607961420d3e1a7cf9/html5/thumbnails/13.jpg)
Parameterization β Hidden Markov Modalβ’ HMM:
β’ Emission distribution ππ(ππππ|ππππ) β given the current state ππππ, the distribution that generates ππππ
β’ Stationary assumption: the emission distribution does not change over time, only depending on the current state
β’ The observation has a discrete value from O = {1,2, β¦ , K}ππ ππππ = ππ|ππππ = π π = πππ π ππ
β’ The observation is a feature vector in a high dimensional feature spaceππ ππππ = ππ|ππππ = π π = ππ(πππ π , Ξ£π π )
ππ ππππ ππππ = ππ ππ1 ππ ππ1 ππ1 οΏ½ππ=2
ππ
ππ ππππ|ππππ ππ(ππππ|ππππβ1)
![Page 14: Hidden Markov Models - CS Department](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14d607961420d3e1a7cf9/html5/thumbnails/14.jpg)
An example
β’ The dishonest casino (Poczos & Singh)β’ A casino has two dices
β’ A fair dice denoted by Fππ ππ = π₯π₯ ππ = πΉπΉ = 1
6, for x=1,2,3,4,5,6
β’ A loaded dice denoted by Lππ ππ = π₯π₯ ππ = πΏπΏ = 1
10, for x=1,2,3,4,5
ππ ππ = 6 ππ = πΉπΉ =12
β’ Casino player switches back and forth between fair and loaded dices with 5% probability
β’ Transition distribution β’ ππ ππππ = πΉπΉ ππππβ1 = πΏπΏ = ππ ππππ = πΏπΏ ππππβ1 = πΉπΉ = 0.05β’ ππ ππππ = πΉπΉ ππππβ1 = πΉπΉ = ππ ππππ = πΏπΏ ππππβ1 = πΏπΏ = 0.95
![Page 15: Hidden Markov Models - CS Department](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14d607961420d3e1a7cf9/html5/thumbnails/15.jpg)
HMM Problems
β’ A sequence of rolls by a casino player
β’ We wonderβ’ Which rolls are generated by the fair/loaded die?
β’ Decoding problem: inferring the states associated with each observation.β’ How likely is this sequence generated by the model of this casino player?
β’ Evaluation problem: computing the likelihood given the model parameters (initial, transition, emission distributions)
β’ How βloadedβ is the loaded die? How fair is the fair die? How often does the casino player change between two dies?
β’ Learning problem: learning the parameters of the casino playerβs model.
![Page 16: Hidden Markov Models - CS Department](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14d607961420d3e1a7cf9/html5/thumbnails/16.jpg)
HMM examples
![Page 17: Hidden Markov Models - CS Department](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14d607961420d3e1a7cf9/html5/thumbnails/17.jpg)
HMM parameters
β’ Transition probability between two dies
β’ Model parameters
![Page 18: Hidden Markov Models - CS Department](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14d607961420d3e1a7cf9/html5/thumbnails/18.jpg)
Formalizing three HMM problems
β’ Evaluation: Given HMM parameters ππ & observation sequences ππ ={πππ‘π‘|π‘π‘ = 1,2, β¦ππ}, compute the probability of observing ππ, i.e.,
ππ πππ‘π‘ π‘π‘=1ππ ππ = οΏ½
ππ
ππ πππ‘π‘ π‘π‘=1ππ , πππ‘π‘ π‘π‘=1
ππ ππ
β’ Decoding: Given HMM parameters ππ & observation sequences ππ ={πππ‘π‘|π‘π‘ = 1,2, β¦ππ}, find the most likely sequence of states, i.e.,
ππ = argmaxππππ πππ‘π‘ π‘π‘=1ππ πππ‘π‘ π‘π‘=1
ππ ,ππ
![Page 19: Hidden Markov Models - CS Department](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14d607961420d3e1a7cf9/html5/thumbnails/19.jpg)
Formalizing three HMM problems
β’ Learning: Given a sequence of observations ππ = {πππ‘π‘|π‘π‘ = 1,2, β¦ππ}, find the HMM parameters that maximize the likelihood of observing ππ
ππβ = argmaxππππ πππ‘π‘ π‘π‘=1ππ ππ
β’ We will apply EM algorithm to solve learning problemβ’ E-step: estimate the ππ πππ‘π‘ π‘π‘=1
ππ πππ‘π‘ π‘π‘=1ππ ,ππ(ππ) (similar to decoding)
β’ M-step: maximize the expected log likelihood of complete data sequence
ππ ππ,ππ ππ = οΏ½ππ
ππ πππ‘π‘ π‘π‘=1ππ πππ‘π‘ π‘π‘=1
ππ ,ππ(ππ) logππ πππ‘π‘ π‘π‘=1ππ , πππ‘π‘ π‘π‘=1
ππ ππ
![Page 20: Hidden Markov Models - CS Department](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14d607961420d3e1a7cf9/html5/thumbnails/20.jpg)
Evaluation Problem
β’ Given HMM parameters ππ & observation sequences ππ = {πππ‘π‘|π‘π‘ = 1,2, β¦ππ}, β’ compute the probability of observing ππ, i.e.,
ππ πππ‘π‘ π‘π‘=1ππ ππ = οΏ½
ππ
ππ πππ‘π‘ π‘π‘=1ππ , πππ‘π‘ π‘π‘=1
ππ ππ
= βππ ππ πππ‘π‘ π‘π‘=1ππ πππ‘π‘ π‘π‘=1
ππ ,ππ ππ( πππ‘π‘ π‘π‘=1ππ |ππ) by Bayes rule
= βππΞ π‘π‘=1ππ ππ πππ‘π‘ πππ‘π‘ ππ( πππ‘π‘ π‘π‘=1ππ |ππ) by independence of observations given states
= βππΞ π‘π‘=1ππ ππ πππ‘π‘ πππ‘π‘ ππ ππ1 ππ Ξ t=2T ππ(πππ‘π‘|πππ‘π‘β1) by Markov assumption
It is exponentially in enumerating all possible assignment of S, leading to computational complexity O(KT).
![Page 21: Hidden Markov Models - CS Department](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14d607961420d3e1a7cf9/html5/thumbnails/21.jpg)
Forward Probabilityβ’ A dynamic programming algorithmβ’ Let πΌπΌππππ = ππ( πππ‘π‘ π‘π‘=1
ππ , ππππ = ππ), then ππ πππ‘π‘ π‘π‘=1ππ = βππ=1πΎπΎ πΌπΌππππ
β’ Recursively compute πΌπΌππππ in forward fashionπΌπΌππππ = ππ πππ‘π‘ π‘π‘=1
ππ , ππππ = ππ= βππβ² ππ πππ‘π‘ π‘π‘=1
ππβ1,ππππ , ππππβ1 = ππβ², ππππ = ππ margalizing ππππβ1= βππβ² ππ ππππ , ππππ = ππ| πππ‘π‘ π‘π‘=1
ππβ1, ππππβ1 = ππβ² ππ( πππ‘π‘ π‘π‘=1ππβ1, ππππβ1 = ππβ²) by Bayes rule
= βππβ² ππ ππππ = ππ|ππππβ1 = ππβ² ππ ππππ|ππππ = ππ ππ( πππ‘π‘ π‘π‘=1ππβ1, ππππβ1 = ππβ²) by Markov assumption
= βππβ² ππ ππππ = ππ|ππππβ1 = ππβ² ππ ππππ|ππππ = ππ πΌπΌππβ1ππβ² by recursive definition.
Each recursive
![Page 22: Hidden Markov Models - CS Department](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14d607961420d3e1a7cf9/html5/thumbnails/22.jpg)
Computational complexity
β’ πΌπΌππππ = βππβ² ππ ππππ = ππ|ππππβ1 = ππβ² ππ ππππ|ππππ = ππ πΌπΌππβ1ππβ²
β’ Each recursive definition incurs O(K) of summations over the state space.
β’ We have KT of such πΌπΌππππ, so the computational complexity is about O(K2T) β a polynomial computational cost.
![Page 23: Hidden Markov Models - CS Department](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14d607961420d3e1a7cf9/html5/thumbnails/23.jpg)
Forward Algorithm
β’ Intialize πΌπΌ1ππ = ππ ππ1, ππ1 = ππ = ππ ππ1 = ππ ππ(ππ1|ππ1 = ππ)β’ Recursive steps: for t =2,β¦,T
πΌπΌπ‘π‘ππ = ππ ππππ|ππππ = ππ οΏ½ππβ²ππ πππ‘π‘ = ππ|πππ‘π‘β1 = ππβ² πΌπΌπ‘π‘β1ππβ²
β’ Termination:
P πππ‘π‘ π‘π‘=1ππ |ππ = οΏ½
ππ
πΌπΌππππ
![Page 24: Hidden Markov Models - CS Department](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14d607961420d3e1a7cf9/html5/thumbnails/24.jpg)
Summary
β’ Define Hidden Markov Model to capture a sequence of observationsβ’ Hidden states: Markov assumptionβ’ Observations: each one only depends on its current state
β’ Three main problemsβ’ Evaluation, decoding, learning
β’ Forward algorithm: evaluationβ’ Next lecture: decoding and learning