midterm.pdf

7/30/2019 midterm.pdf

1/9

MS&E 221 Midterm Examination

Ramesh Johari February 14, 2007

Instructions

1. Take alternate seating.

2. Answer all questions in the spaces provided on these sheets. If needed, additional paper will

be available at the front of the room. Answers given on any other paper will not be counted.

3. The examination begins at 1:20 pm, and ends at 2:30 pm.

4. Show your work! Partial credit will be given for correct reasoning.

Honor Code

In taking this examination, I acknowledge and accept the Stanford University Honor Code.

NAME (signed)

NAME (printed)

1


2/9

Problem 1 (40 points).

Answer each of the following short answer questions. (10 points per question)

(a) True or false: In a chain on a finite state space that has two communicating classes and a

unique invariant distribution, at least one of the two classes is transient. (Justify your answer.)

(b) Suppose the irreducible, aperiodic transition matrix P has a greater second largest eigen-

value modulus (SLEM) than the irreducible, aperiodic transition matrix Q (both on the same

finite state space). Which Markov chain converges faster to its equilibrium distribution, andwhy?

2


3/9

(c) Let X0, X1, X2, . . . be a Markov chain on the state space {0, 1, 2, 3}, and suppose a newprocess Y0, Y1, Y2, . . . is defined according to:

Yn =

0, ifXn is even;1, ifXn is odd.

Under exactly what conditions is the process Yn also a Markov chain? Explain your answer.

(d) Suppose a Markov chain is irreducible and has invariant distribution . What is the mean

number of visits to i before the first return to j?

3


4/9


Consider a Markov chain with the following transition matrix.

P =

0 1 0 0 0 0 0 0

0 0 1 0 0 0 0 01 0 0 0 0 0 0 01/6 0 1/3 0 1/2 0 0 0

0 0 0 1/6 0 0 1/3 1/20 0 0 0 0 1/2 1/2 00 0 0 0 0 0 0 10 0 0 0 0 1/3 2/3 0

.

(a) (6 points) What are the communicating classes of this chain, and which are closed?

(b) (6 points) Which communicating classes are positive recurrent, null recurrent, and/or tran-

sient?

4


5/9

(c) (6 points) Find all invariant distributions of this chain.

(d) (6 points) For what values ofi does the following limit exist? Give the limit for those valuesofi where it exists.

limn

P(Xn = i|X0 = 4).

5


6/9

(e) (6 points) Now suppose the seventh row of the matrix is changed to:

0 1/2 0 0 0 0 0 1/2

.

Does this change your answer to (d)? Why or why not?

6


7/9


A salesman travels to four different cities that are located at the vertices of the unit square (i.e.,

the square with vertices (0, 0), (0, 1), (1, 0), (0, 1)). At each time step, the salesman jumps to oneof the two adjacent vertices.

He jumps vertically with probability p, and horizontally with probability q. In all cities except(0, 0), with probability r the salesman stays in the same city in the next time step. In (0, 0) (hishome office), with probability r the salesman takes a vacation in the next time step. (Here p + q+r = 1.)

Once on vacation, the salesman stays on vacation each period with probability a, and otherwisereturns to work at his home office.

(a) (6 points) Describe the movement of the salesman as a Markov chain.

7


8/9

(b) (8 points) Assuming r = 0, find the long run fraction of time the salesman spends in eachcity.

(c) (8 points) Suppose the salesman is currently on vacation. Find the probability that, once the

salesman returns to work, he goes on vacation again without ever visiting (1, 1).

8


9/9

(d) (8 points) Suppose the salesman is currently in (1, 1). Suppose also that he earns a reward ofone dollar for each horizontal sales trip, and two dollars for each vertical sales trip. Give a

set of equations that can be used to compute the expected reward he will earn once he leaves

(1, 1), until he returns to (1, 1). You do not need to explicitly compute the answer.

9

midterm.pdf

Documents