7/28/2019 Introduction to Stochastic Processes 30July2011

1/14

Introduction to Stochastic Processes

U Dinesh Kumar

7/28/2019 Introduction to Stochastic Processes 30July2011

2/14

Lecture Outline

Introduction and definition of SP.

Characteristics of SP.

Types of SP.

Strength and Weakness of SP models.

7/28/2019 Introduction to Stochastic Processes 30July2011

3/14

In mathematics, you don't understand things. You just get

used to them

- J von Neumann

7/28/2019 Introduction to Stochastic Processes 30July2011

4/14

Stochastic Processes - Definition

A stochastic process is a model that evolves in timeor space subject to probabilistic laws.

A stochastic process {X(t), t T} is a collection ofrandom variables indexed by t. The index t is usuallyassumed to be time and the random variable X(t) is thestate of system at time t.

If the index set T is countable set, then the process X(t)is a discrete-time stochastic process. If T is continuous,then X(t) is called continuous time process.

7/28/2019 Introduction to Stochastic Processes 30July2011

5/14

Importance of Stochastic Process

Theory of stochastic processes aims to model theinteraction of chance and time.

It is all most impossible to find a problem in business

which does not change with time and has no uncertainty.Often, SP is the only tool which can model realistic

problems in business and management.

7/28/2019 Introduction to Stochastic Processes 30July2011

6/14

Path of SP X(t)

The values taken by SP X(t) for various values of t iscalled the path of the SP X(t)

time

X(t)a sample path

a random variable for each fixed t

t

7/28/2019 Introduction to Stochastic Processes 30July2011

7/14

State Space

The range of possible values of SP, X{t}, is called the statespace.

The state space can be either discrete or continuous.

Discrete State Space

Number of customers

Credit Rating

Continuous State Space

Market Share

Stock Price

7/28/2019 Introduction to Stochastic Processes 30July2011

8/14

Examples of SP

Cash flow Market Share

Stock Price

Customer Retention

Customer Satisfaction

Inventory Level

Commodity Price

Product and service demand Availability of engineering systems

7/28/2019 Introduction to Stochastic Processes 30July2011

9/14

Stochastic Processes: Independent Increment

A continuous time SP {X(t)} is said to have independentincrements if for all t0 < t1 < t2 < < tn: the random

variables:

X(t1)X(t0), X(t2)X(t1), X(t3)X(t2), , X(tn)X(tn-

1)

are independent. In an independent increment process,

the change in the non-overlapping time intervals areindependent.

7/28/2019 Introduction to Stochastic Processes 30July2011

10/14

Stationary Processes

A SP is said to have stationary increments if X(t+s)X(t) has same distributions for all t.

In a stationary process, the distribution of change

between two points depends only on the distancebetween the two points.

7/28/2019 Introduction to Stochastic Processes 30July2011

11/14

Types of Stochastic Processes

Poisson Process Renewal Process

Markov Process

Martingales

Random Walks

Brownian Motion Process

7/28/2019 Introduction to Stochastic Processes 30July2011

12/14

Strength of SP

The element of randomness exists in any process andthus you need a tool that can model changing

phenomena.

All most all problems in finance, marketing and operationshave random element and the distribution of randomness

changes over a period of time.

7/28/2019 Introduction to Stochastic Processes 30July2011

13/14

Weakness of SP

The SP models in many cases can be intractable.

Many SP models are still unsolved.

7/28/2019 Introduction to Stochastic Processes 30July2011

14/14

Recommended Readings

Ross, S M, Introduction to Probability Models, Academic

Press, 2006.

E Cinlar, Introduction to Stochastic Processes, PrenticeHall, 1975

H C Tijms, Stochastic Models An Algorithmic Approach,

John Wiley, 1994.