7/30/2019 ex5s

1/2

MTH6141 Random Processes, Spring 2012

Solutions to Exercise Sheet 5

1. If we start in state 1 we can return in 2 steps (1 2 1) or 3 steps(1 2 3 1). These are the only possibilities, so So P(R1 = 2) =

23

,P(R1 = 3) =

13

and E(R1) =23

2 + 13

3 = 73

.

If we start in state 2 then possible paths are similar (2 1 2 and2 3 1 2) and the calculation is identical, yielding P(R2 = 2) =

23

,P(R2 = 3) =

13

and E(R2) =23

2 + 13

3 = 73

.

If we start in state 3 then we may return at any odd number t = 2k + 1 ofsteps (k 1), via a path of the form 3 1 2 1 2 3. ThusP(R3 = 2k + 1) =

13

(23

)k1 for k 1 (and all other values for R3 have zeroprobability). I.e., 1

2

(R3 1) is distributed geometrically with parameter 1/3.Hence E(1

2(R3 1)) =

12E(R3)

12

= 3, and E(R3) = 7. (Or sum the seriesby hand.)

By Theorem 1.14, the equilibrium distribution is given by wk = 1/E(Rk), sois (3

7, 37

, 17

). This agrees with the answer obtained by solving wP = w (c.f.Exercise Sheet 3).

2. Suppose that a b. Then there exist r and s such that p(r)ab > 0 and p

(s)ba > 0.

Hencep(r+t+s)aa p

(r)ab p

(t)bb p

(s)ba , for all t N.

Now set u = r + s and = p(r)ab p

(s)ba , to obtain the inequality we were required

to prove.

Now suppose that a is null recurrent. Recurrence is a class property (Theo-rem 1.12) and so state b is also recurrent. Now, by Theorem 1.13, since a is

null recurrent, p(t)aa 0 as t . Now p

(t)bb

1p(t+u)aa , and the right-hand

side converges to 0 as t . Hence the left-hand side also converges to 0and, by Theorem 1.13, state b is null recurrent.

3. By considering the transition diagram we see that

f(t)00 = p0,1p1,2p2,3 . . . pt2,t1pt1,0 =

122

33

4

. . .t 1

t 1

t + 1

=1

t(t + 1) .

Now

f00 =t=1

f(t)00 =

t=1

1

t(t + 1)=

t=1

1

t

1

t + 1

= 1

and so the chain is recurrent.

1

7/30/2019 ex5s

2/2

Also,

E(R0) =

t=1

tf(t)00 =

t=1

1

t + 1=

and so the chain is null recurrent.

The transition diagram for this process has the same shape as that for thesuccess runs chain. Only the numerical values of the positive probabililtieshave changed. You could think of this process as being like the success runschain except that the longer the current run of successes is the more likelyit is to be extended. For instance a football team may be more confidentduring a long run of wins and hence more likely to win the next game.

4. As in the lecture, we argue that, in order to be at state 0 at time 2 n, we musthave made exactly n increasing transitions and n decreasing. There are2nn

possible such sequences. Each sequence occurs with probability pn(1 p)n.Thus the probability of making a transition from state 0 to state 0 in 2n stepsis2nn

pn(1 p)n. It follows that

p(2n)

00 =

2n

n

pn(1 p)n 22n[p(1 p)]n = [4p(1 p)]n,

where we have used the fact that the binomial coefficient2nn

is certainly less

than the sum 2nk=02nk = 22n of all coefficients. The function r = r(p) =

4p(1 p) achieves a maximum of 1 at the point p = 12

and is everywhere elsestrictly less than 1 (and non-negative). So when p = 1

2, r < 1 and

t=1

p(t)00 =

n=1

p(2n)

00 n=1

rn 1

1 r< ,

where we have used the fact that p(t)00 = 0 when t is odd. Hence, by

Lemma 1.12, state 0 is transient. By symmetry (or by Theorem 1.13) allstates are transient.

5. Hint: Consider the symmetric random walk (Yt) and the related stochasticprocess (Xt) given by Xt = |Yt|. Is Xt a Markov chain? If so, what are itstransition probabilities?

2