CH – 11CH – 11

Markov analysisMarkov analysis

Learning objectivesLearning objectives::

After completing this chapter , you should be able toAfter completing this chapter , you should be able to::

11 . . Give examples of systems that may lend themselves Give examples of systems that may lend themselves to description by a markov modelto description by a markov model..

22 . . Explain the meaning of transition probabilitiesExplain the meaning of transition probabilities..

33 . . Use a tree diagram to analyze system behaviorUse a tree diagram to analyze system behavior..

44 . . Use matrix multiplication to analyze system behaviorUse matrix multiplication to analyze system behavior..

55 . . Use algebraic method to solve for steady state Use algebraic method to solve for steady state valuesvalues..

SummarySummary

Markov analysis can be useful for describing the behavior of a Markov analysis can be useful for describing the behavior of a certain class of system that change from state to state on a certain class of system that change from state to state on a period – by – period basis according to know transition period – by – period basis according to know transition probabilitiesprobabilities..

Customers patterns , market share , and equipment breakdowns Customers patterns , market share , and equipment breakdowns sometimes lend themselves to description in markov termssometimes lend themselves to description in markov terms..

GlossaryGlossaryMarkov processMarkov process::

Steady stateSteady state::

Transition matrixTransition matrix::

A closed system that change A closed system that change from state to state according from state to state according to stable transition to stable transition probabilitiesprobabilities..

The long- term tendencies of The long- term tendencies of amarkov system to be in its amarkov system to be in its various statesvarious states..

A matrix that shows the A matrix that shows the probabilities of markov probabilities of markov system changing from is system changing from is current state to each current state to each possible state in the next possible state in the next periodperiod..

CH – 11CH – 11

Markov analysisMarkov analysis

A markov system has these characteristicsA markov system has these characteristics

11 . . It will operate or exist for a number of periodsIt will operate or exist for a number of periods..

22 . . In each period , the system can assume one of a number of In each period , the system can assume one of a number of states or conditionsstates or conditions..

33 . . System changes between states from period to period can be System changes between states from period to period can be described by described by transition probabilitiestransition probabilities , which remain constant , which remain constant..

Example of system that may be described as markovExample of system that may be described as markov

Brand switching

Proportion of customersWho buy brand A

Brand BBrand C etc

ProbabilityThat a customerWill switch from

Brand A to brand B , etc

Transition probabilitiesTransition probabilities

Which indicate the tendencies of the system to change from one Which indicate the tendencies of the system to change from one period to the nextperiod to the next

ExampleExample: :

A car rent agency that has to offices at each of a city’s two A car rent agency that has to offices at each of a city’s two airportsairports..

Customers are allowed to return a rented car to either airport , Customers are allowed to return a rented car to either airport , regardless of which airport they rented fromregardless of which airport they rented from..

Suppose that the manager of the rented agency has made a study Suppose that the manager of the rented agency has made a study of return behavior and has found the following Infoof return behavior and has found the following Info: :

70%70% of cars rented from airport A tend to be returned to that of cars rented from airport A tend to be returned to that airport , and 30% of A tend to returned to Bairport , and 30% of A tend to returned to B..

10%10% of cars rented from airport B are returned to airport A. and of cars rented from airport B are returned to airport A. and 90% returned to B90% returned to B..

Transition probabilities for car agencyTransition probabilities for car agency

Returned toReturned to

A BA B

A .70 .30 = 1.00A .70 .30 = 1.00

rentedrented

fromfrom

B .10 .90 = 1.00B .10 .90 = 1.00

The manager has several questions concerning the systemThe manager has several questions concerning the system: :

11 . . What proportion of cars will be returned to each airport at the What proportion of cars will be returned to each airport at the short-run , over the next several daysshort-run , over the next several days..

22 . . What proportion of cars will be to each location over the long – What proportion of cars will be to each location over the long – runrun. .

Methods of analysisMethods of analysis

For a short – runFor a short – run

11 . .Tree diagramTree diagram

22 . .Matrix multiplicationMatrix multiplication

For a long – runFor a long – run

33 . .Algebraic methodAlgebraic method..

Tree diagramTree diagram

For one periodFor one period

00 11

AA

A A At this point At this point

B A = .70B A = .70

B = .30B = .30

00 11

AA

B B B = .90 B = .90

B A = .10B A = .10

Stratingfrom

Stratingfrom

.70

.30

.10

.90

What we are doing forWhat we are doing for

Several periodsSeveral periods? ?

ExampleExample-: -:

Use the Info. In the previous example , and prepare a tree Use the Info. In the previous example , and prepare a tree diagram for two period. Then compute joint probabilities and diagram for two period. Then compute joint probabilities and use them to determine how many cars will be at location A if A use them to determine how many cars will be at location A if A

originally has 100 cars and location B has 80 carsoriginally has 100 cars and location B has 80 cars. .

Joint probabilitiesJoint probabilities

Period A BPeriod A B

00 11 22. . 70x.70=.49 .70x.30=.2170x.70=.49 .70x.30=.21

A .30x.10=.03 .30x.90=.27A .30x.10=.03 .30x.90=.27

AA

B .52 .48B .52 .48

A AA A

AA

BB

.70

.70

.30

.30

.10

.90

Joint probabilitiesJoint probabilities

Period A BPeriod A B

00 11 22. . 10x.70=.07 .10x.30=.0310x.70=.07 .10x.30=.03

A .90x.10=.09 .90x.90=.81A .90x.10=.09 .90x.90=.81

AA

B .16 .84B .16 .84

A AA A

AA

BB

At zero point A=100 cars , B = 80 carsAt zero point A=100 cars , B = 80 cars

A BA B

100100.(.(5252 + ) + )8080.(.(1616 = ) = )6464 cars in A at the second periodcars in A at the second period..

.70

.10

.90

.30

.10

.90

→ →matrix multiplicationmatrix multiplication

We need two matrix to use I solutionWe need two matrix to use I solution: :

11 . . Shares matrixShares matrix

22 . . Transition probabilities matrixTransition probabilities matrix..

Share’s matrix can be obtained from the calculations period to period . But for Share’s matrix can be obtained from the calculations period to period . But for the first period ) zero period ( we may assume the sharesthe first period ) zero period ( we may assume the shares..

for our examplefor our example: :

shares for A , B would be this mean that all cars are in Ashares for A , B would be this mean that all cars are in A..

for period onefor period one

A BA B

= = 11.(.(7070 ) )11.(.(3030))

+ + 00.(.(1010+ )+ )00.(.(9090))

This share willThis share will

Use in the next periodUse in the next period..

1A 0B

1 0

90.

30.

10.

70.

70. 30.

→ →for the 2 periodfor the 2 period

A BA B

. = . = 7070.(.(7070. = ). = )4949. . 7070.(.(3030. = ). = )2121

. . 3030.(.(1010. = ). = )0303. . 3030.(.(9090. = ). = )2727

. . 5252. . 4848

This share willThis share will

Use in the 3 periodUse in the 3 period

→ →algebraic Solutionalgebraic Solution

The basic Equation isThe basic Equation is

A+B = 1A+B = 1

70. 30.

10.

70.

9.

30.

10.

70.

90.

30.

B

A

A

B

1

1

From the transition probability tape we can develop the equation , From the transition probability tape we can develop the equation , for A and for Bfor A and for B

A = .70A + .10BA = .70A + .10B

B = .30A + .90BB = .30A + .90B

A = .70A + .10B B = 1-AA = .70A + .10B B = 1-A

A = .70A + .10)1-A( B = 1-.25A = .70A + .10)1-A( B = 1-.25

A = .70A + .10 - .10A B = .75A = .70A + .10 - .10A B = .75

A = .60A + .10A = .60A + .10

A - .60A = .10 .40A = .10 A = = .25A - .60A = .10 .40A = .10 A = = .25 40.

10.

→ →Analysis for 3x3 matrixAnalysis for 3x3 matrix

ExampleExample-: -:

X Y ZX Y Z

X .70 .20 .10X .70 .20 .10

Y .40 .50 .10Y .40 .50 .10

Z .30 .10 .60Z .30 .10 .60

Tree diagramTree diagram

solve for 2 periodsolve for 2 period

stating from Xstating from X..

PeriodPeriod00 11 22

xx x yx y

zz

xx x y yx y y

zz

xx z yz y

zz

Joint probabilitiesJoint probabilities

x sharex share. . 7070.(.(7070. = ). = )4949. . 2020.(.(4040. = ). = )0808. . 1010.(.(3030. = ). = )0303

. . 6060 y sharey share

. . 7070.(.(2020. = ). = )1414

. . 2020.(.(5050. = ). = )1010

. . 1010.(.(1010. = ). = )0101. . 2525

z sharez share. . 7070.(.(1010. = ). = )0707. . 2020.(.(1010. = ). = )0202. . 1010.(.(6060. = ). = )0606

. . 1515

70.

70.

20.

20.

10.

10.10.

10.

40.

50.

30.

60.

The same if you want to solve starting Y or Z?

Matrix multiplicationMatrix multiplication SolutionSolution

Period : starting from xPeriod : starting from x

x y zx y z

11.(.(7070 ) )11.(.(2020 ) )11.(.(1010))

00 00.(.(4040 ) )00.(.(5050 ) )00.(.(1010.).)

00.(.(3030 ) )00.(.(1010 ) )00.(.(6060))

. . 7070. . 2020. . 1010

22. . 7070. . 2020. . 1010. . 7070.(.(7070. ). )7070.(.(2020. ). )7070.(.(1010))

. . 2020.(.(4040. ). )2020.(.(5050. ). )2020.(.(1010))

. . 1010.(.(3030. ). )1010.(.(1010. ). )1010.(.(6060))

. . 6060. . 2525. . 1515

For the 3ed period , and so onFor the 3ed period , and so on. .

100

100

30.

40.

70.

10.

50.

20.

60.

10.

10.

30.

40.

70.

10.

50.

20.

60.

10.

10.

Algebraic solutionAlgebraic solution

equationsequations

11 . .X = .70x + .40y + .30zX = .70x + .40y + .30z

Y = .20x + .50y + .10zY = .20x + .50y + .10z

Z = .10x + .10y + .60zZ = .10x + .10y + .60z

11 = = x + y +z basic equationx + y +z basic equation

22 . .Eliminate one the equations , but not the basic equation. Suppose Eliminate one the equations , but not the basic equation. Suppose the third equation is eliminatedthe third equation is eliminated. .

Z = 1 – X - YZ = 1 – X - Y

33 . .Substitute for Z in the first and second equationsSubstitute for Z in the first and second equations

X = .70x + .40y + .30)1 – x – y(X = .70x + .40y + .30)1 – x – y(

Y = .20x + .50y + .10)1 – x – y(Y = .20x + .50y + .10)1 – x – y(

X = .70x + .40y + .30 - .30x - .30yX = .70x + .40y + .30 - .30x - .30y

Y = .40x + .30 + .10yY = .40x + .30 + .10y

X - .40x - .10y = .30X - .40x - .10y = .30

. . 60x - .10y = .360x - .10y = .3

the save for Ythe save for Y

. - . - 10x + .60y = .1010x + .60y = .10

. . 60x - .10y = .3060x - .10y = .30

.- .- 10x + .6y = .1010x + .6y = .10

. . 60x - .10y = .3060x - .10y = .30

66.-( .-( 10x + .60y = .1010x + .60y = .10) )

. . 60x - .10y = .3060x - .10y = .30

. - . - 60x + 3.6y = .6060x + 3.6y = .60

3.5y = .903.5y = .90

Y= = .257 Y= = .257

..60x - .10).257( = .3060x - .10).257( = .30

..60x - .0257 = .3060x - .0257 = .30

..60x = .30 + .025760x = .30 + .0257

..60x = .3257 X = = .543 60x = .3257 X = = .543

5.3

90.

60.

3257.

Z = 1 - .542 - .257 = .200 Z = 1 - .542 - .257 = .200

if the are 900 unit in the systemif the are 900 unit in the system

X = 900).543( = 488.7X = 900).543( = 488.7

Y = 900).257( = 231.3Y = 900).257( = 231.3

Z = 900).200( = 180.0Z = 900).200( = 180.0

900900