Download - 2014 CN - Material 8 - MARKOV Chain
-
8/18/2019 2014 CN - Material 8 - MARKOV Chain
1/17
.
Markov Chains
Tutorial #5
© Ydo Wexler & Dan Geiger
-
8/18/2019 2014 CN - Material 8 - MARKOV Chain
2/17
.
Statistical Parameter EstimationReminder
• The basic paradigm:
• MLE / bayesian approach
• Input data: series of observations X 1, X 2 … X t
-We assumed observations ere i.i.d !independent identica" distributed#
$ata set
Mode"
%arameters: Θ
Heads - P(H) Tails - 1-P(H)
-
8/18/2019 2014 CN - Material 8 - MARKOV Chain
3/17
3
Markov Process
• Mar&ov %roperty: The state of the system at time t +1 depends on"yon the state of the system at time t
X 1 X 2 X 3 X 4 X 5
[ ] [ ] x| X x X x x X | X x X t t t t t t t t ===== ++++ 111111 Pr Pr
• 'tationary (ssumption: Transition probabi"ities are independent oftime !t #
[ ]1Pr t t ab X b | X a p+ = = =
)ounded memory transition mode"
-
8/18/2019 2014 CN - Material 8 - MARKOV Chain
4/17
4
Weather:
• raining today 40% rain tomorrow
60% no rain tomorrow
• not raining today 20% rain tomorrow
80% no rain tomorrow
Markov ProcessSimple Example
rain no rain
0.60.4 0.8
0.2
'tochastic *'M:
-
8/18/2019 2014 CN - Material 8 - MARKOV Chain
5/17
5
Weather:
• raining today 40% rain tomorrow
60% no rain tomorrow
• not raining today 20% rain tomorrow
80% no rain tomorrow
Markov ProcessSimple Example
=8.02.0
6.04.0 P
• 'tochastic matri+:,os sum up to
• $oub"e stochastic matri+:,os and co"umns sum up to
The transition matri+:
-
8/18/2019 2014 CN - Material 8 - MARKOV Chain
6/17
6
amb"er starts ith 01
- (t each p"ay e have one of the fo""oing:
• amb"er ins 0 ith probabi"ity p
• amb"er "ooses 0 ith probabi"ity 1- p
ame ends hen gamb"er goes bro&e2 or gains a fortune of 011
!)oth 1 and 11 are absorbing states#
0 1 2 99 100
p p p p
1-p 1-p 1-p 1-p
Start
(10$)
Markov ProcessGambler’s Example
-
8/18/2019 2014 CN - Material 8 - MARKOV Chain
7/177
• Markov proe!! - described by a stochastic *'M• Markov hain - a random a"& on this graph
!distribution over paths#
• Edge-eights give us• We can as& more comp"e+ 3uestions2 "i&e
Markov Process
[ ]1
Pr t t ab
X b | X a p+
= = =
[ ] ?Pr 2
===+ ba | X X t t
0 1 2 99 100
p p p p
1-p 1-p 1-p 1-p
Start
(10$)
-
8/18/2019 2014 CN - Material 8 - MARKOV Chain
8/178
• iven that a person4s "ast co"a purchase as 5o&e2there is a 617 chance that his ne+t co"a purchase i""a"so be 5o&e.
• If a person4s "ast co"a purchase as %epsi2 there is
an 817 chance that his ne+t co"a purchase i"" a"so be%epsi.
coke pepsi
0.10.9 0.8
0.2
Markov ProcessCoke vs. Pepsi Example
=
8.02.0
1.09.0 P
transition matri+:
-
8/18/2019 2014 CN - Material 8 - MARKOV Chain
9/179
iven that a person is current"y a %epsi purchaser2hat is the probabi"ity that he i"" purchase 5o&e topurchases from no9
Pr %epsi95o&e ; =
Pr %epsi5o&e5o&e ; + Pr %epsi %epsi 5o&e ; =
0.2 * 0.9 + 0.8 * 0.2 = 0.34
== 66.034.017.083.0
8.02.0
1.09.0
8.02.0
1.09.02 P
Markov ProcessCoke vs. Pepsi Example (cont)
%epsi 9 9 5o&e
= 8.02.0
1.09.0
P
-
8/18/2019 2014 CN - Material 8 - MARKOV Chain
10/1710
iven that a person is current"y a 5o&e purchaser2hat is the probabi"ity that he i"" purchase %epsi three purchases from no9
Markov ProcessCoke vs. Pepsi Example (cont)
=
=
562.0438.0
219.0781.0
66.034.0
17.083.0
8.02.0
1.09.03 P
-
8/18/2019 2014 CN - Material 8 - MARKOV Chain
11/1711
•(ssume each person ma&es one co"a purchase per ee&
•'uppose
-
8/18/2019 2014 CN - Material 8 - MARKOV Chain
12/1712
'imu"ation:
Markov ProcessCoke vs. Pepsi Example (cont)
#eek - i
P r X
i =
! o k e "
23
[ ] [ ]313231328.02.0
1.09.0=
stationary distribution
coke pepsi
0.10.9 0.8
0.2
-
8/18/2019 2014 CN - Material 8 - MARKOV Chain
13/1713
Hidden Markov Models HMM
" #
" 2
" $%#
" $ " i
>idden states
?bserveddata
& #
& 2
& $%#
& $ & i
-
8/18/2019 2014 CN - Material 8 - MARKOV Chain
14/1714
0'(
)air loaded
H HT T
0'(0'
0'
*2 *4+*4*2
Hidden Markov Models HMMCoin!ossin" Example
%&ir'o&e
e&&i
X 1
X 2
X L-1
X L X i
& #
& 2
& $%#
& $ & i
transition probabi"ities
emission probabi"ities
-
8/18/2019 2014 CN - Material 8 - MARKOV Chain
15/1715
Hidden Markov Models HMMCG #slands Example
Regular
DNA
C-G island
,- i!land!: enome regions hich are very rich in 5 and
A
C
G
T
change
A
C
G
T
./*4
/*6
1*4
1*4
1*4
1*4 /
/
1
1
11/
/
.1*6
.1*+
p*+
p*+
p*6
-
8/18/2019 2014 CN - Material 8 - MARKOV Chain
16/17
-
8/18/2019 2014 CN - Material 8 - MARKOV Chain
17/17
Mid !est $%&'
1. Observe the queing mehanism in !ubli !lae"!lease de#ine b$ $oursel#%
. 'laborate the !arameters o# suh mehanism(
i.e. )ervie time( ost o# *aiting( disi!line( +endall
,otation( et
. ro!ose the ne* queing mehanism
17