Homework 4 due Wednesday, May 7.

Karlin and Taylor Ch. 5•Resnick Ch. 3•

Reading:

"marked Poisson point process": also generate some random variables with each Poisson point.

•

clumped Poisson point process: Each Poisson point actually corresponds to a random number of agents

•

The Poisson point process that we developed last time has two important extensions for more advanced applications:

A renewal process is a point process in time (in one dimension) which has the property that the length of the intervals between successive incidents are independently distributed.

A delayed renewal process is a slight extension of a renewal process that allows to have a different probability distribution

than

.

•

The one-dimensional Poisson point process is a special case of a renewal process where the common probability distribution of the is

exponential.

Renewal ProcessesFriday, April 25, 20141:59 PM

Stoch14 Page 1

than

.

The independence of the interincident times therefore means

that every time an incident happens, the distribution of future incidents is independent of the distribution of past incidents, given that we know an incident has just occurred. Note this is not quite Markov because this decoupling of future and past only occurs at the times of an incident, not at arbitrary times t. In particular, it is not in general true that the amount of time one expects to wait until the next incident happens is independent of the amount of time one has waited since the last incident. (If this were the case, then we'd have a 1-D Poisson point process because exponential distribution is the only memoryless distribution.)

•

The reason this point process is called a renewal process is that we can think of it as follows: Follow time continuously forward, and every time an incident occurs, the stochastic process starts afresh.

Equipment replacement•The times of action potentials (spikes) produced by a neuron (interspike interval distributions)

•

Example applications of renewal processes:

Generic way to define a renewal process associated to a Markov process by triggering an incident whenever the Markov chain visits a specified state or group of states.

•

i.e., biophysical chemistry, neuronal networks•

Spatial renewal processes for i.e., traffic following•

One can also use the idea of renewal processes:Poisson point processes::semi-Markov processes::continuous-time Markov chains. That is a semi-Markov process is a discrete-state stochastic process that generalizes a CTMC only in that the probability distribution for the amount of time spent in a state ( can have arbitrary probability distribution (but is still independent of each

other).

Stoch14 Page 2

Queueing theory with nonexponential times for processing requests•Branching processes with age considerations so that the times of branching are not exponentially distributed

•

Epidemiological models where the amount of time spent in the infective stage is not exponentially distributed

•

other).

The theory of semi-Markov processes is however, somewhat more complicated and we won't treat it here.

Mathematical Specification of a Renewal Process

The key defining object is the cumulative distribution function (CDF) for the interincidenttime:

.

When the interincident time intervals have a continuous distribution, then we can

equivalently refer to the PDF for the interincident times, and the CDF and PDF are related as follows:

The reason why renewal process works with CDFs is that they generalize to interincident times that might not be purely continuously distributed, i.e., an equipment replacement schedule where equipment is replaced when it breaks or when a fixed time interval has elapsed, whatever happens first. This technically doesn't have a PDF (but you can write the PDF in terms of delta-functions for the discrete component), but the CDF is perfectly well-behaved.

For the Poisson point process, the interincident PDF would be:

Stoch14 Page 3

The interincident CDF describes the "dynamics" of a renewal process. Initial condition? Pure renewal process treats in the same way as if an incident happened at t=0. Otherwise, for a delayed renewal process, the initial condition is given by a separate CDF for .

Key Random Variables Associated to Renewal Process

: counts the number of incidents that

have occurred before time t.

○

Associated counting process•

Current life : amount of time between t and the previous incident

•

Residual life amount of time between t and the next incident•Total life : amount of time between the previous and next incident whose interval covers the time t.

•

One of the goals of the theory of renewal processes is to derive the probability distributions for these random variables from the given interincident CDF. There is a machinery for doing this, but to warm up with the concepts, we'll just use direct calculation to compute these probability distributions for Poisson point process. These will reveal the important Poisson paradox.

The counting process N(t) associated to the Poisson point process is the Poisson counting process, and we know from our previous calculations in CTMC theory, that probability distribution for is given by a

Stoch14 Page 4

in CTMC theory, that probability distribution for is given by a Poisson distribution with mean

The residual life is also easy to compute. Because of memoryless property of the exponential distribution, which characterizes the interincident time intervals of a Poisson point process, the residual life has the same probability distribution as the interincident time probability distribution. We continue to refer to CDFs for proper generality.

Current life : Because the Markov property for the associated Poisson counting process is past/future symmetric, the probability distribution of the current life would be the same as for the residual life, except for the introduction of the initialization of the system at time 0. This only modifies the probability distribution as follows: If no incident has happened in the interval [0,t], then we set the current life Therefore, we obtain:

The total life has a probability distribution that can be computed from the probability distributions of and because these random variables are independent for Poisson point process (future and past conditionally independent of present.) The result can

Stoch14 Page 5

Poisson point process (future and past conditionally independent of present.) The result can be computed by convolution of the probability distributions; get something that is like. But let's focus on a more interesting characteristic of this total life that we can approach without having to do this calculation.

To do this, we'll introduce a technical device of Stieltjes integration that is often used in renewal process theory.

Given a piecewise continuous function and a function of bounded variation we define the Riemann-Stieltjes integral:

is some partition of the real line.where

Any bounded monotone function has bounded variation and that is the only kind of F we will be dealing with.

Once we allow ourselves to use Stieltjes integrals, we have a general way for computing means of random variables that need not be continuous or discrete:

where is the CDF of Y.

This gives the formulas for discrete and continuous random variables as special cases. (To see that the continuous random variable works, note

that in this case

when the derivative

exists).

Means of nonnegative random variables have a simpler expression in terms of the CDF:

Stoch14 Page 6

terms of the CDF:

If ,

(an ordinary Riemann integral).

Proof:

Stoch14 Page 7