2014 cn - material 8 - markov chain

8/18/2019 2014 CN - Material 8 - MARKOV Chain

1/17

.

Markov Chains

Tutorial #5

© Ydo Wexler & Dan Geiger


2/17

.

Statistical Parameter EstimationReminder

• The basic paradigm:

• MLE / bayesian approach

• Input data: series of observations X 1, X 2 … X t

-We assumed observations ere i.i.d !independent identica" distributed#

$ata set

Mode"

%arameters: Θ

Heads - P(H) Tails - 1-P(H)


3/17

3

Markov Process

• Mar&ov %roperty: The state of the system at time t +1 depends on"yon the state of the system at time t

X 1 X 2 X 3 X 4 X 5

[ ] [ ] x| X x X x x X | X x X t t t t t t t t ===== ++++ 111111 Pr Pr

• 'tationary (ssumption: Transition probabi"ities are independent oftime !t #

[ ]1Pr t t ab X b | X a p+ = = =

)ounded memory transition mode"


4/17

4

Weather:

• raining today 40% rain tomorrow

60% no rain tomorrow

• not raining today 20% rain tomorrow


Markov ProcessSimple Example

rain no rain

0.60.4 0.8

0.2

'tochastic *'M:


5/17

5

Weather:

• raining today 40% rain tomorrow


• not raining today 20% rain tomorrow


Markov ProcessSimple Example

=8.02.0

6.04.0 P

• 'tochastic matri+:,os sum up to

• $oub"e stochastic matri+:,os and co"umns sum up to

The transition matri+:


6/17

6

amb"er starts ith 01

- (t each p"ay e have one of the fo""oing:

• amb"er ins 0 ith probabi"ity p

• amb"er "ooses 0 ith probabi"ity 1- p

ame ends hen gamb"er goes bro&e2 or gains a fortune of 011

!)oth 1 and 11 are absorbing states#

0 1 2 99 100

p p p p

1-p 1-p 1-p 1-p

Start

(10$)

Markov ProcessGambler’s Example


7/177

• Markov proe!! - described by a stochastic *'M• Markov hain - a random a"& on this graph

!distribution over paths#

• Edge-eights give us• We can as& more comp"e+ 3uestions2 "i&e

Markov Process

[ ]1

Pr t t ab

X b | X a p+

= = =

[ ] ?Pr 2

===+ ba | X X t t

0 1 2 99 100

p p p p

1-p 1-p 1-p 1-p

Start

(10$)


8/178

• iven that a person4s "ast co"a purchase as 5o&e2there is a 617 chance that his ne+t co"a purchase i""a"so be 5o&e.

• If a person4s "ast co"a purchase as %epsi2 there is

an 817 chance that his ne+t co"a purchase i"" a"so be%epsi.

coke pepsi

0.10.9 0.8

0.2

Markov ProcessCoke vs. Pepsi Example

=

8.02.0

1.09.0 P

transition matri+:


9/179

iven that a person is current"y a %epsi purchaser2hat is the probabi"ity that he i"" purchase 5o&e topurchases from no9

Pr %epsi95o&e ; =

Pr %epsi5o&e5o&e ; + Pr %epsi %epsi 5o&e ; =

0.2 * 0.9 + 0.8 * 0.2 = 0.34

== 66.034.017.083.0

8.02.0

1.09.0

8.02.0

1.09.02 P

Markov ProcessCoke vs. Pepsi Example (cont)

%epsi 9 9 5o&e

= 8.02.0

1.09.0

P


10/1710

iven that a person is current"y a 5o&e purchaser2hat is the probabi"ity that he i"" purchase %epsi three purchases from no9


=

=

562.0438.0

219.0781.0

66.034.0

17.083.0

8.02.0

1.09.03 P


11/1711

•(ssume each person ma&es one co"a purchase per ee&

•'uppose


12/1712

'imu"ation:


#eek - i

P r X

i =

! o k e "

23

[ ] [ ]313231328.02.0

1.09.0=

stationary distribution

coke pepsi

0.10.9 0.8

0.2


13/1713

Hidden Markov Models HMM

" #

" 2

" $%#

" $ " i

>idden states

?bserveddata

& #

& 2

& $%#

& $ & i


14/1714

0'(

)air loaded

H HT T

0'(0'

0'

*2 *4+*4*2

Hidden Markov Models HMMCoin!ossin" Example

%&ir'o&e

e&&i

X 1

X 2

X L-1

X L X i

& #

& 2

& $%#

& $ & i

transition probabi"ities

emission probabi"ities


15/1715

Hidden Markov Models HMMCG #slands Example

Regular

DNA

C-G island

,- i!land!: enome regions hich are very rich in 5 and

A

C

G

T

change

A

C

G

T

./*4

/*6

1*4

1*4

1*4

1*4 /

/

1

1

11/

/

.1*6

.1*+

p*+

p*+

p*6


16/17


17/17

Mid !est $%&'

1. Observe the queing mehanism in !ubli !lae"!lease de#ine b$ $oursel#%

. 'laborate the !arameters o# suh mehanism(

i.e. )ervie time( ost o# *aiting( disi!line( +endall

,otation( et

. ro!ose the ne* queing mehanism

17

2014 cn - material 8 - markov chain

Documents