markov process and markov chains(unit 3)

8/3/2019 Markov Process and Markov Chains(Unit 3)

1/5

UNIT III MARKOV PROCESS AND MARKOV CHAINS

PART A (QUESTION AND ANSWERS)

1. Consider the random process ( )+= ttX cos)( where is a random variable

with density function ( )22

,1


2/5

So, X(t) is not WSS.

3. Define a Random Process.

Soln:

A random process or Stochastic process X(s,t) is a function that maps each element of asample space into a time function called sample function.

4. Define First order stationary and Second order stationary process.

Soln:A random process is said to be stationary to order one if the first order density functions

defined for all the random variables of the process are same.

A random process is said to be stationary to order two if for all andtt 21 , its second

order density functions satisfy the condition ( ) ).,;,(,;, 21212121 ++= ttxxfttxxf xx5. Define Markov Process

Soln:

A random process or Stochastic process X(t) is said to be a Markov process if given the

value of X(t), the value of X(v) for v > t does not depend on values of X(u) for u < t. Inother words, the future behavior of the process depends only on the present value and not on

the past value.

6. Define Markov Chain

Soln:

A Markov process is called Markov chain if the states { }iX is discrete no matter whethert is discret or continuous.

7. Define Chapman-Kolmogrov EquationSoln:

The Chapman-Kolmogrov equation provides a method to compute the n-step transition

probabilities. The equation can be represented as

=

+=

0

0,k

m

kj

n

ik

mn

ij mnppP .

8. When do you say that a Markov chain is irreducible?

Soln:

The Markov chain is irreducible if all states communicate with each other at some time.

9. When do you say the Markov chain is regular?Soln:

A regular Markov chain is defined as a chain having a transition matrix P such that for

some power of P, it has only non-zero positive probability values.10. When do you say that state i is periodic and aperiodic?

Soln:

Let A be the set of all positive integers such that 0)(>

n

iip and d be the Greatest Common

Divisor(G.C.D.) of the set A. We say state i is periodic if d>1 and aperiodic if d = 1.11. Define Poisson process.

Soln:Poisson process is a counting point process representing the number of occurrences of

certain event in a finite collection of non overlapping statistically independent time arrivals.

12. What are the properties of Poisson process.

Soln:


3/5

(a) The poisson process is not a stationary process. It is vivid from the expressions of

moments of poisson process that they are time dependent.

(b) The poisson process is a Markov process.

13. Determine whether the given matrix is irreducible or not.

=

8.02.00

5.04.01.0

07.03.0

P

Soln:

=

8.02.00

5.04.01.0

07.03.0

P and

=

74.024.002.0

60.033.007.0

35.049.016.02P

Here jiPn

ij ,,0)( > . So, P is irreducible.

14. When do you say the Markov chain is homogeneous?Soln:

If the one-step transition probability does not depend on the step i.e.),1(),1( mmpnnp ijij =

then the Markov chain is called a homogeneous Markov chain.15. What are the different types of Random process.Soln:

Continuous Random process, Discrete Random process, Continuous Randomsequence and Discrete Random sequence.16. Define Birth and Death process.

Soln:If X(t) represents the number of individuals present at time t in a population inwhich two types of events occur one representing birth which contributes toits increase and the other representing death which contributes to its decrease,then the discrete random process {X(t)} is called the birth and death process.17. A housewife buys 3 kinds of cereals A, B and C. She never buy the same

cereal in successive weeks. If she buys cereal A, the next week she buys B.However if she buys B or C the next week she is 3 times as likely to buy A asthe other cereal. Construct the Transition Probability Matrix.

Soln:

=

410

43

410

43

010

C

B

A

P

CBA

18. Let

= 2/12/1

10

A be a Stochastic matrix. Check whether it is regular.

Soln:

=

2/12/1

10A and

=

4/34/1

2/12/12A .

Since all entries in 2A are positive, A is regular.


4/5

19.The number of particles emitted by a radioactive source is Poisson distributed.

The source emits particles at the rate of 6 per minute. Each emitted particle has a

probability of 0.7 of being counted. Find the probability that 11 particles are

counted in 4 minutes.

Soln:

The number of particles N(t) emitted is poisson with parameter p = 6(0.7) =

4.2

( )!

2.4))((

2.4

m

temtNP

mt

==

( ).038.0

!11

)4(2.4)11)4((

11)4(2.4

===

eNP

20.Prove that the sum of two independent Poisson process is a Poisson process.

Soln:

Let )(1 tX and )(2 tX be the Poisson process with mean 21 and

respectively.

)1(

)(1

1)(

iwet

tX ew

= and )1()(

2

2)(

iwet

tX ew=

Since1

X and 2X are independent, =+ )()()( 21 wtXtX )()(1 wtX . )()(2 wtX

= )1(1 iwete )1(2iwete

= )1()( 21iwet

e+

Since1

X + 2X follows Poisson Distribution with mean .21 +

PART B

01. Define Random process. Specify the four different types of Random Process andgive an example to each type.

02. Prove that the difference of two independent Poisson Process is not a Poisson

Process.

03. The transition probability matrix of a Markov chain{ }nX , n = 0,1,2,3 having 3

states 1, 2 and 3 is

0.1 0.5 0.4

p 0.6 0.2 0.2

0.3 0.4 0.3

=

and the initial distribution is ( )p(0) 0.7,0.2,0.1= .

Find { }2P X 3= and { }3 2 1 0P X 2,X 3,X 3,X 2= = = = .


5/5

04. A random process { }X(t), t T has the probability distribution

{ }

( )

( )

n 1

n 1

at, n 1, 2,3,

1 atP X(t) n

at, n 01 at

+

=

+= =

= +

L

. Show that the process is not stationary.

05. A man either drives a car or catches a train to go to office each day. He never goes two

days in a row by train but if he drives one day, then the next day he is just as likely to

drive again or he is travel by train. Now suppose that on the first day of the week, the

man tossed a fair die and drove to work if and only if 6 appeared. Find the

probability that he takes a train on the third day. Also find the probability

that h drives to work in the long run.

markov process and markov chains(unit 3)

Documents