lecture 4b: continuous-time birth and death...
TRANSCRIPT
![Page 1: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/1.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
Lecture 4b: Continuous-Time Birth and DeathProcesses
Continuous-time Markov chains are stochastic processes whose time is continuous, t ∈ [0,∞),but the random variables are discrete. Prominent examples of continuous-time Markov processesare Poisson and death and birth processes. These processes play a fundamental role in the theoryand applications that embrace queueing and inventory models, population growth, engineeringsystems, etc [3]. Here we discuss the birth and death process with its invariants.
1 Preliminaries
1.1 Probability Generating Functions
The probability generating function is a concept used for distributions with non-negative integers.
Consider a discrete random variable X taking nonnegative values. Let f denote the probabilitymass function (p.m.f.) of X, defined as
fk = Prob{X = k} = Pk , k = 0, 1, 2, · · ·
where∞∑
k=0Pk = 1.
The probability generating function (p.g.f.) of X, denoted �X, is defined as
�X(z) = E[zX] =
∞∑k=0
Pkzk , (1)
for some z ∈ R, or, the pmf,
Pk(t) =1k!
∂k�X(z)∂zk
∣∣∣∣∣∣z=0
. (2)
1
![Page 2: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/2.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
Note that
• The series converges absolutely for |z| ≤ 1:
�X(1) =
∞∑k=0
Pk(1)k
=
∞∑k=0
Pk
= 1 .
Thus, �X(z) is well defined for |z| ≤ 1.
• Also, the p.g.f. generates the probabilities associated with the distribution
�X(0) = P0 , �′
X(0) = P1 , �′′
X(0) = 2!P2 ,
which are resulted from, by taking
�X(z) = P0 + P1z + P2z2 + P3z3 + · · ·
whence at z = 0, then P0 = �X(0). And
�X(z)∂z
= P1 + 2P2z + 3P3z2 + · · ·
whence at z = 0, then P1 = �′
X(0). And so forth.
• In general, the nth derivative of the p.g.f. of X is
�(n)X (0) = n!Pn .
The p.g.f. can be used to calculate the mean and variance of a random variable X:
(1) The first derivative of �X is
�′
X(z) =d�X(z)
dz=
∞∑k=1
kPkzk−1 , for − 1 < z < 1 .
By letting z approach one from the left, z→ 1−, we obtain the mean of X
�′
X(1) =
∞∑k=1
kPk = E[X] = µX .
2
![Page 3: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/3.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
(2) The second derivative of �X is
�′′
X(z) =d2�X(z)
dz2 =
∞∑k=1
k(k − 1)Pkzk−2 ,
so that as z→ 1−,
�′′
X(1) =
∞∑k=1
k(k − 1)Pk = E[X(X − 1)] = E[X2 − X] .
If the mean is finite, then the variance of X is
�′′
X(1) + �′
X(1) −(�′
X(1))2
= E[X2 − X] + E[X] − (E[X])2
= E[X2] − E[X] + E[X] − (E[X])2
= E[X2] − (E[X])2
= Var(X) .
2 Birth and Death Processes
Consider a large class of infinite state space, continuous-time Markov chain whose states are{0, 1, 2, · · · }. Transitions of state may go only from n to n + 1 or from n to n − 1. We can viewthe state of the system as the size of a population that can increase by a “birth” (n → n + 1) ordecrease by a “death” (n → n − 1). Suppose that whenever there are n people in the system,then: (i) new individuals enter the system at an exponential rate λn, and (ii) individuals leave thesystem at an exponential rate µn. That is, whenever there are n people in the system, the time untilthe next arrival is exponentially distributed with mean 1/λn and is independent of the time untilthe next departure which is also exponentially distributed but with mean 1/µn. The parameters{λn}
∞n=0 and {µn}
∞n=1 are called the arrival (or birth) and departure (or death) rates, respectively. If
the population is 0, there are no deaths and µ0 = 0 [5, 7].
2.1 General Birth and Death Process
The continuous-time birth and death Markov chain {X(t) : t ∈ [0,∞)} may have either a finite{0, 1, 2, · · · ,N} or infinite {0, 1, 2, · · · } state space. Assume that its transition probabilities Pi j(t)are stationary, i.e.,
Pi j(t) = Prob{X(t + s) = j | X(s) = i} , for all ∆t ≥ 0 .
3
![Page 4: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/4.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
In addition, assume the infinitesimal transition probabilities for this process are [1]
Pi,i+ j(∆t) = Prob{X(t + ∆t) − X(t) = j | X(t) = i}
=
λi∆t + o(∆t) , j = 1
µi∆t + o(∆t) , j = −1
1 − (λi + µi)∆t + o(∆t) , j = 0
o(∆t) , j , −1, 0, 1 .
(3)
for ∆t sufficiently small, µ0 = 0, λ0 > 0, and λi > 0, µi > 0 for i = 1, 2, · · · . It is often the casethat λ0 = 0, except when there is immigration. The initial conditions are
Pi j(0) = δi j =
1 , i = j
0 , i , j .
In a small time interval ∆t, at most one change in state can occur, either a birth, i → i + 1 or adeath, i→ i − 1 [1].
In the same way as for the Poisson process, we define Pi(t) = Prob{X(t) = i} and assume X(0) =
0. If ∆t > 0, i ≥ 1, by invoking the law of total probability and the Markov property, we obtainfrom the Chapman-Kolmogorov equations that,
Pi j(t + ∆t) =
∞∑k=0
Pik(t) Pk j(∆t)
=
∞∑k=0
Pik(t) · Prob{X(t + ∆t) = j | X(t) = k}
=
∞∑k=0
Pik(t) · Prob{X(t + ∆t) − X(t) = j − k | X(t) = k}
=
k= j∑k=0
Pik(t) · Prob{X(t + ∆t) − X(t) = j − k | X(t) = k} .
Now, for k = j, for the right hand side we have
= Pi j(t) · Prob{X(t + ∆t) − X(t) = j − j | X(t) = j}= Pi j(t) · Prob{X(t + ∆t) − X(t) = 0 | X(t) = j}= Pi j(t) ·
[1 − (λ j + µ j)∆t + o(∆t)
],
4
![Page 5: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/5.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
for k = j − 1,
= Pi, j−1(t) · Prob{X(t + ∆t) − X(t) = j − ( j − 1) | X(t) = j − 1}= Pi, j−1(t) · Prob{X(t + ∆t) − X(t) = 1 | X(t) = j − 1}= Pi, j−1(t) ·
[λ j−1∆t + o(∆t)
],
for k = j + 1,
= Pi, j+1(t) · Prob{X(t + ∆t) − X(t) = j − ( j + 1) | X(t) = j + 1}= Pi, j+1(t) · Prob{X(t + ∆t) − X(t) = −1 | X(t) = j + 1}= Pi, j+1(t) ·
[µ j+1∆t + o(∆t)
],
whereas for k , j, j − 1, j + 1, that is for k ≤ j − 2,
= Pik(t) · Prob{X(t + ∆t) − X(t) ≥ 2 | X(t) = k}= Pik(t) · o(∆t) ,
and k ≥ j + 2,
= Pik(t) · Prob{X(t + ∆t) − X(t) ≤ −2 | X(t) = k}= Pik(t) · o(∆t) .
Collecting all together,
Pi j(t + ∆t) = Pi j(t) ·[1 − (λ j + µ j)∆t + o(∆t)
]+ Pi, j−1(t) ·
[λ j−1∆t + o(∆t)
]+
Pi, j+1(t) ·[µ j+1∆t + o(∆t)
]+ Pik(t) · o(∆t)
= Pi, j−1(t)λ j−1∆t + Pi, j+1(t)µ j+1∆t + Pi j(t)[1 − (λ j + µ j)∆t
]+ o(∆t) ,
which holds for all i and j in the state space with the exception of j = 0 and j = N (if thepopulation size is finite).
If j = 0, then (and due to µ0 = 0)
Pi0(t + ∆t) = Pi1(t)µ1∆t + Pi0(t)[1 − λ0∆t
]+ o(∆t) .
If j = N, which is the maximum population size, then
PiN(t + ∆t) = Pi,N−1(t)λN−1∆t + PiN(t)[1 − µN)∆t
]+ o(∆t) ,
5
![Page 6: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/6.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
where λN = 0 and PkN = 0 for k > N. Subtracting Pi j(t), Pi0(t), and PiN(t) from the precedingthree equations, respectively, dividing by ∆t, and taking the limit as ∆t → 0, yields the forwardKolmogorov differential equations for the general birth and death process,
dPi j(t)dt
= λ j−1Pi, j−1(t) − (λ j + µ j)Pi j(t) + µ j+1Pi, j+1(t)
dPi0(t)dt
= −λ0Pi0(t) + µ1Pi1(t) , for j = 0
dPiN(t)dt
= λN−1Pi,N−1(t) − µN PiN(t) , for j = N .
(4)
The forward Kolmogorov differential equations can be written in matrix notation,
dP(t)dt
= P(t)Q
dPi0/dt
dPi1/dt
dPi2/dt
dPi3/dt
...
=
[Pi0 Pi1 Pi2 Pi3 · · ·
]
−λ0 λ0 0 0 · · ·
µ1 −(λ1 + µ1) λ1 0 · · ·
0 µ2 −(λ2 + µ2) λ2 · · ·
0 0 µ3 −(λ3 + µ3) · · ·
......
......
,
where the generator matrix Q for the infinite state space
Q =
−λ0 λ0 0 0 · · ·
µ1 −(λ1 + µ1) λ1 0 · · ·
0 µ2 −(λ2 + µ2) λ2 · · ·
0 0 µ3 −(λ3 + µ3) · · ·
......
......
, (5)
6
![Page 7: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/7.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
and for the finite state space
Q =
−λ0 λ0 0 0 · · ·
µ1 −(λ1 + µ1) λ1 0 · · ·
......
... · · ·...
0 0 µN−1 −(λN−1 + µN−1) λN−1
0 0 0 µN −µN
. (6)
Similarly, to obtain the backward Kolmogorov differential equations, we start from the Chapman-Kolmogorov equations,
Pi j(∆t + t) =
∞∑k=0
Pik(∆t) Pk j(t)
= Pi,i−1(∆t)Pi−1, j(t)︸ ︷︷ ︸k = i−1
+ Pii(∆t)Pi j(t)︸ ︷︷ ︸k = i
+ Pi,i+1(∆t)Pi+1, j(t)︸ ︷︷ ︸k = i+1
+∑
k
Pik(∆t)Pk j(t) ,
where the last summation is over all k , i − 1, i + 1, i.
For k = i,Pii(∆t)Pi j(t) = Pi j(t) ·
[1 − (λi + µi)∆t + o(∆t)
],
for k = i − 1,
Pi,i−1(∆t)Pi−1, j(t) = Pi−1, j(t) ·[µi∆t + o(∆t)
],
for k = i + 1,
Pi,i+1(∆t)Pi+1, j(t) = Pi+1, j(t) ·[λi∆t + o(∆t)
],
and for k , j, j − 1, j + 1, we have∑k
Pik(∆t)Pk j(t) = Pk j(t) · o(∆t) .
Collecting all together,
Pi j(t + ∆t) =[1 − (λi + µi)∆t
]Pi j(t) + µi∆tPi−1, j(t) + λi∆tPi+1, j(t) + o(∆t) ,
7
![Page 8: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/8.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
or, rearranging,
Pi j(t + ∆t) − Pi j(t) = µi∆tPi−1, j(t) − (λi + µi)∆tPi j(t) + λi∆tPi+1, j(t) + o(∆t) .
Dividing by ∆t and taking the limit as ∆t → 0, we obtain the backward Kolmogorov differentialequations
dPi j(t)dt
= µiPi−1, j(t) − (λi + µi)Pi j(t) + λiPi+1, j(t) ,
dP0 j(t)dt
= −λ0P0 j(t) + λ0P1 j(t) , for i = 0
dPN j(t)dt
= µN PN−1, j(t) − µN PN j(t) , for i = N .
(7)
where µ0 = 0 and λN = 0, and j ≥ 0, i ≥ 0.
The backward Kolmogorov differential equations can be written in matrix notation,
dP(t)dt
= QP(t)
dP0 j/dt
dP1 j/dt
dP2 j/dt
...
dPN j/dt
=
−λ0 λ0 0 0 · · ·
µ1 −(λ1 + µ1) λ1 0 · · ·
0 µ2 −(λ2 + µ2) λ2 · · ·
......
......
0 0 0 µN −µN
P0 j
P1 j
P2 j
...
PN j
.
2.2 Stationary Probability Distribution (The Limiting Behavior of Birthand Death Processes)
For a general birth and death process that has no absorbing states, the limits
limt→∞
Pi j(t) = π j ≥ 0 (8)
exist and are independent of the initial state i. The limits form the limiting distribution (orlimiting probability) of the process, or also known as a stationary probability distribution. For
8
![Page 9: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/9.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
a more detailed discussion on this topic, see subsection 1.7 of Lecture 4a (Continuous-TimeMarkov Chain Models). To determine if a limiting distribution exists and what its values are, werewrite the forward Kolmogorov differential equations in Eq. (4),
dPi0(t)dt
= −λ0Pi0(t) + µ1Pi1(t) ,
dPi j(t)dt
= λ j−1Pi, j−1(t) − (λ j + µ j)Pi j(t) + µ j+1Pi, j+1(t) , j ≥ 1 ,
with initial conditions Pi j(0) = δi j. If we take the limit as t → ∞ to these equations, the limit ofthe right hand sides exists according to Eq. (8). The limit on the left hand side, the derivativesP′i j(t), also exists. Since the probabilities are converging to a constant, the limit of these deriva-tives must be zero [9]. These equations can be solved recursively. For the first equation, or whenj = 0:
limt→∞
dPi0(t)dt
= −λ0 limt→∞
Pi0(t) + µ1 limt→∞
Pi1(t)
0 = −λ0π0 + µ1π1 ,
orπ1 =
λ0
µ1π0 .
From the second equation,
limt→∞
dPi j(t)dt
= λ j−1 limt→∞
Pi, j−1(t) − (λ j + µ j) limt→∞
Pi j(t) + µ j+1 limt→∞
Pi, j+1(t)
0 = λ j−1π j−1 − (λ j + µ j)π j + µ j+1π j+1 ,
when j = 1:
π2 =λ1
µ2π1 =
λ0λ1
µ1µ2π0 ,
when j = 2:
π3 =λ2
µ3π2 =
λ0λ1λ2
µ1µ2µ3π0 ,
and so on. Thus,
πi+1 =λi
µi+1πi . (9)
9
![Page 10: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/10.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
Or, applying the induction hypothesis, the stationary probability distribution equals
πi =λ0λ1 · · · λi−1
µ1µ2 · · · µiπ0 , i = 1, 2, 3, · · · (10)
Then
µi+1πi+1 = (λi + µi)πi − λi−1πi−1
=
(λ0λ1 · · · λi−1(λi + µi)
µ1µ2 · · · µi−λ0λ1 · · · λi−1
µ1µ2 · · · µi−1
)π0
=λ0λ1 · · · λi−1
µ1µ2 · · · µi−1
(λ1 + µi
µi− 1
)π0 ,
or,
πi+1 =λ0λ1 · · · λi
µ1µ2 · · · µi+1π0 . (11)
Thus, if the state space is infinite, {0, 1, 2, · · · }, a unique positive stationary probability distribu-tion π for a general birth and death process exists, with
µi > 0 and λi−1 > 0 for i = 1, 2, 3, · · ·
Since∞∑
i=0πi = 1, which we expand
∞∑i=0
πi = 1
π0 +
∞∑i=1
πi = 1
π0
1 +
∞∑i=1
πi
π0
= 1 ,
from which we solve for π0 by invoking Eq. (10),
π0 =1
1 +∞∑
i=1
λ0λ1 · · · λi−1
µ1µ2 · · · µi
. (12)
10
![Page 11: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/11.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
If the state space is finite, {0, 1, 2, · · · ,N}, then a unique positive stationary probability distribu-tion π exists if and only if
µi > 0 and λi−1 > 0 for i = 1, 2, · · · ,N .
The stationary probability distribution is given by Eqs. (10) and (12), where the index and thesummation extend from i = 1, 2, · · · ,N.
2.3 Pure Birth
In a pure birth process, the chance of an event occurring at a given instant of time dependsupon the number of events that have already occurred. An example of this phenomenon is thereproduction of living organisms, hence the name of the process. This process considers only thebirth or arrival of new members, permitted with certain conditions, such as, sufficient food, nomortality, and no migration out of the system [9].
Let X(t) represent the number of births in the time interval (0, t]. We define a pure birth processas a Markov process satisfying the following:
Pi,i+ j(∆t) = Prob{X(t + ∆t) − X(t) = j | X(t) = i}
=
λi∆t + o(∆t) , j = 1
1 − λi∆t + o(∆t) , j = 0
0 , j < 0 .
(13)
By analyzing the possibilities at time t just prior to time t + ∆t, for ∆t is sufficiently small, wederive a system of Kolmogorov’s forward differential equations for a pure birth process. Theprobabilities Pn(t) = Prob{X(t) = n} are solutions of the differential equations
dP0(t)dt
= −λ0P0(t) ,
dPn(t)dt
= −λnPn(t) + λn−1Pn−1(t) ,(14)
with initial conditionsP0(0) = 1 , Pn(0) = 0 , n > 0 .
The first equation of Eq. (14) can be solved immediately. With P0(0) = 1, then
P0(t) = e−λ0t , for t > 0 .
For the second equation of Eq. (14), there are several ways to compute the solutions:
11
![Page 12: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/12.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
(1) To solve the differential equations recursively, introduce
Rn(t) = eλntPn(t) , for n = 0, 1, 2, · · ·
and rearrange the second equation of Eq. (14) to get
λn−1Pn−1(t) =dPn(t)
dt+ λnPn(t) ,
which is used for the following. Differentiating Rn with respect to t yields
dRn(t)dt
= λneλntPn(t) + eλnt dPn(t)dt
= eλnt[λnPn(t) +
Pn(t)dt
]= eλntλn−1Pn−1(t) .
Integrating both sides of these equations and using the initial conditionRn(0) = 0 gives
Rn(t) =
∫ t
0eλn xλn−1Pn−1(x)dx , for n ≥ 1 .
Substituting Rn(t) on the left hand side,
eλntPn(t) =
∫ t
0eλn xλn−1Pn−1(x)dx ,
or
Pn(t) = e−λnt∫ t
0eλn xλn−1Pn−1(x)dx
= λn−1e−λnt∫ t
0eλn xPn−1(x)dx , n = 1, 2, · · · (15)
It can be seen that all Pk(t) ≥ 0, but there is still a possibility that [9]
∞∑n=0
Pn(t) < 1 .
To ensure that∑∞
n=0 Pn(t) = 1 for all t, we must restrict the λk according to the following[9]
∞∑n=0
Pn(t) = 1 if and only if∞∑
n=0
1λn
= ∞ .
12
![Page 13: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/13.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
The integral equation (15) is solved as follows
n = 1 : P1(t) = λ0e−λ1t
t∫0
eλ1 xP0(x)dx
= λ0
(e−λ0t
λ1 − λ0+
e−λ1t
λ0 − λ1
),
n = 2 : P2(t) = λ1e−λ2t
t∫0
eλ2 xP1(x)dx
= λ0λ1
(e−λ0t
(λ1 − λ0)(λ2 − λ0)+
e−λ1t
(λ0 − λ1)(λ2 − λ1)+
e−λ2t
(λ0 − λ2)(λ1 − λ2)
).
Continuing the computations, we obtain that for n > 1 and initial conditionPn(0) = 0,
Pn(t) = Prob{X(t) = n | X(0) = 0}
= λ0 · · · λn−1
(B0,ne−λ0t + · · · + Bn,ne−λnt
),
(16)
whereB0,n =
1(λ1 − λ0)(λ2 − λ0) · · · (λn − λ0)
Bk,n =1
(λ0 − λk) · · · (λk−1 − λk)(λk+1 − λk) · · · (λn − λk)
Bn,n =1
(λ0 − λn)(λ1 − λn) · · · (λn−1 − λn).
(17)
(2) Alternatively, we can also explicitly determine Pi j(t) in the case of a pure birth process, byletting Xk denote the time the process spends in state k before making a transition into statek + 1 for k ≥ 1. Suppose that the process is presently in state i, and let state j > i. ThenXi is the time it spends in state i before moving to state i + 1, and Xi+1 is the time it thenspends in state i + 1 before moving to state i + 2, and so on. It follows that
∑ j−1k=i Xk is the
time it takes until the process enters state j. If the process has not yet entered state j bytime t, then its state at time t is smaller than j, and vice versa [7], that is
X(t) < j if and only if Xi + · · · + X j−1 > t .
Therefore, for i < j, we have for a pure birth process
Pi j(t) = Prob{X(t) < j | X(0) = i} = Prob
j−1∑k=i
Xk > t
.
13
![Page 14: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/14.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
Since the random variables are hyperexponentially distributed (see hyperexponential dis-tribution and convolutions of distribution in Lecture 4a on properties of exponential distri-bution), we obtain the tail distribution function of
∑ j−1k=i Xk, that [7]
Prob{X(t) < j | X(0) = i} =
j−1∑k=i
e−λktj−1∏
r,k, r=i
λr
λr − λk.
Replacing j by j + 1 in the preceding gives
Prob{X(t) < j + 1 | X(0) = i} =
j∑k=i
e−λktj∏
r,k, r=i
λr
λr − λk.
Since
Prob{X(t) = j | X(0) = i} = Prob{X(t) < j + 1 | X(0) = i} − Prob{X(t) < j | X(0) = i} ,
and since Pii(t) = Prob{Xi > t} = e−λit, we have for a pure birth process having λi , λ j
when i , j [7]
Pi j(t) =
j∑k=i
e−λktj∏
r,k, r=i
λr
λr − λk−
j−1∑k=i
e−λktj−1∏
r,k, r=i
λr
λr − λk, i < j
Pii(t) = e−λit .
(18)
2.4 The Yule Process
The Yule process is an example of a pure birth process that commonly arises in physics andbiology. Let X(t) represent the population size at time t and the initial population is X(0) = N,so that Pi(0) = δiN . The population size can only increase in size because there are only births.The transition probabilities for sufficiently small ∆t are
Pi,i+ j(∆t) = Prob{X(t + ∆t) − X(t) = j | X(t) = i}
=
1 − iλ∆t + o(∆t) j = 0
iλ∆t + o(∆t) j = 1
o(∆t) j ≥ 2
0 j < 0 .
(19)
14
![Page 15: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/15.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
Converting these transition probabilities into differential equationsfor Pi(t) = Prob{X(t) = i} gves
Pi(t + ∆t) = Prob{X(t + ∆t) = i}
= Prob{X(t) = i, X(t + ∆t) − X(t) = 0}+ Prob{X(t) = i − 1, X(t + ∆t) − X(t) = 1}+ Prob{X(t) = i − 2, X(t + ∆t) − X(t) ≥ 2}
= Prob{X(t) = i} · Prob{X(t + ∆t) − X(t) = 0}+ Prob{X(t) = i − 1} · Prob{X(t + ∆t) − X(t) = 1}+ Prob{X(t) = i − 2} · Prob{X(t + ∆t) − X(t) ≥ 2}
= Pi(t) ·[1 − iλ∆t + o(∆t)
]+ Pi−1(t) ·
[(i − 1)λ∆t + o(∆t)
]+ Pi−2(t) · o(∆t) .
Rearranging and equating terms with o(∆t) yield
Pi(t + ∆t) − Pi(t) = (i − 1)λ∆tPi−1(t) − iλ∆tPi(t) + o(∆t) .
Dividing both sides by ∆t and taking the limit as ∆ → 0 lead to the system of forward Kol-mogorov equations
dPi(t)dt
= (i − 1)λPi−1(t) − iλPi(t) , i = N,N + 1,N + 2, · · ·
dPi(t)dt
= 0 , i = 0, 1, 2, · · · ,N − 1
(20)
Its solution is given byPi(t) = e−λt(1 − e−λt)i−1 , i ≥ 1 . (21)
Several ways of solving the differential-difference equation of the Yule process Eq. (20) are asfollows:
(1) Using the solution of the recursive method of Eq. (16):
The general solution is analogous to Eq. (16), but for pure birth processes starting fromX(0) = 1, [9]
Pi(t) = λ1 · · · λi−1
(B1,ie−λ1t + · · · + Bi,ie−λit
), i > 1 . (22)
15
![Page 16: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/16.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
When λi = iλ,
B1,i =1
(λ2 − λ1)(λ3 − λ1)(λ4 − λ1) · · · (λi − λ1)
=1
(2λ − λ)(3λ − λ)(4λ − λ) · · · (iλ − λ)
=1
λ(1)λ(2)λ(3) · · · λ(i − 1)
=1
λi−1(1)(2)(3) · · · (i − 1)
=1
λi−1(i − 1)!,
and
B2,i =1
(λ1 − λ2)(λ3 − λ2) · · · (λi − λ2)
=1
λi−1(−1)(1)(2) · · · (i − 2)
=−1
λi−1(i − 2)!,
so, in general,
Bk,i =1
(λ1 − λk) · · · (λk−1 − λk)(λk+1 − λk) · · · (λi − λk)
=(−1)k−1
λi−1(k − 1)!(i − k)!.
Thus the solution is
Pi(t) = λi−1(i − 1)!(B1,ie−λt + B2,ie−2λt + · · · + Bi,ie−iλt
)=
i∑k=1
(i − 1)!(k − 1)!(i − k)!
(−1)k−1e−kλt
= e−λtn−1∑j=0
(i − 1)!j!(i − 1 − j)!
(e−λt) j
= e−λt(1 − e−λt
)i−1, i ≥ 1 . (23)
16
![Page 17: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/17.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
(2) Alternatively, using Eq. (18), for the case i = 1:
P1 j(t) = Prob{X(t) ≤ j | X(0) = 1}
=
j∑k=1
e−kλtj∏
r,k, r=1
rr − k
−
j−1∑k=1
e−kλtj−1∏
r,k, r=1
rr − k
= e− jλtj−1∏r=1
rr − j
+
j−1∑k=1
e−kλt
j∏r,k, r=1
rr − k
−
j−1∏r,k, r=1
rr − k
= e− jλt(−1) j−1 +
j−1∑k=1
e−kλt
(j
j − k− 1
) j−1∏r,k, r=1
rr − k
.
Now,
kj − k
j−1∏r,k, r=1
rr − k
=( j − 1)!
(1 − k)(2 − k) · · · (k − 1 − k)( j − k)!
= (−1)k−1(
j − 1k − 1
).
Hence,
P1 j(t) =
j∑k=1
(j − 1k − 1
)e−kλt(−1)k−1
= e−λtj−1∑i=0
(j − 1
i
)e−iλt(−1)i
= e−λt(1 − e−λt) j−1 , j ≥ 1 , (24)
which is exactly the same as the solution in Eq. (23).
The solution in Eq. (24) states that starting with one individual, the population size at time thas a geometric distribution with mean eλt . If the population starts with i individuals, thenwe can regard each of these individuals as starting their own independent Yule process,and so the population at time t will be the sum of i independent and identically distributedgeometric random variables with parameter e−λt. But this means that the conditional dis-tribution of X(t), given that X(0) = i, is the same as the distribution of the number of timesthat a coin that lands heads on each flip with probability e−λt must be flipped to amass atotal of i heads. Hence, the population size at time t has a negative binomial distributionwith parameters i and e−λt, thus
Pi j(t) =
(j − 1i − 1
)e−iλt (1 − e−λt) j−i , j ≥ i ≥ 1 . (25)
17
![Page 18: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/18.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
(3) Using the probability generating function (p.g.f.):
Observe that the state space for this process is {1, 2, 3, · · · }. All of the states are transientand the transition probabilities Pn(t) all depend on time t or non-stationary probabilitydistribution.
Recall that the p.g.f., denoted �(t), is given by
�(z) =
∞∑i=0
Pi zi .
For the sake of generality, here we alter the initial conditions where now PN(0) = 1 andPi(0) = 0. We derive the partial differential equation for the p.g.f. by multiplying thedifferential equations by zi and sum over i,
∂�(z, t)∂t
=
∞∑i=N+1
(N + 1 − 1)λPN+1−1 zN+1 −
∞∑i=N
NλPN zN
=
∞∑i=N+1
NλPN zN−1z2 −
∞∑i=N
NλPN zN−1z
= λz2∞∑
i=N
iPi zi−1 − λz∞∑
i=N
iPi zi−1
= λz(z − 1)∞∑
i=N
iPi zi−1 .
Recall that the first derivative of � with respect to z is
∂�
∂z=
∞∑i=1
iPi zi−1 ,
then∂�(z, t)∂t
= λz(z − 1)∂�
∂z.
The initial condition is �(z, 0) = zN .
The partial differential equation for the moment generating function (m.g.f.) can be derivedby a change of variable. Let z = eθ, where dθ/dz = 1/eθ = 1/z. There exists a closerelationship between the probability generating function �X(·) and the moment generatingfunction�X(·), that is
�(θ) = E[eθX] = �(eθ) .
Then �(eθ, t) =�(θ, t). Following the chain rule,
∂�
∂z=∂�
∂θ
dθdz
=1z∂�
∂θ.
18
![Page 19: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/19.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
The m.g.f. is a solution of the following partial differential equation
∂�
∂t= λ(eθ − 1)
∂�
∂θ,
with initial condition�(θ, 0) = eNθ.
The method of characteristics is applied to find the solution of�(θ, t). Rewrite the differ-ential equation for�(θ, t)
∂�
∂t+ λ(1 − eθ)
∂�
∂θ= 0 .
The solution is found by introducing the characteristic equations
dθdτ
= λ(1 − eθ) ,dtdτ
= 1 , anddvdτ
= 0 ,
with initial conditions
θ(s, 0) = s , t(s, 0) = 0 , and v(s, 0) = eNs .
For the first characteristic equation, separation of variables and simplification lead to
dθ1 − eθ
= λdτ ande−θdθ
e−θ − 1= λdτ .
By using a substitution u = e−θ − 1, from which du = −e−θdθ, we integrate both sides of
−duu
= λdτ orduu
= −λdτ
which results in a general solution
ln(u) = −λτ + c1 or ln(e−θ − 1) = −λτ + c1 .
The general solution can also be written
e−θ − 1 = e−λτ+c1 = C0 e−λτ .
Applying the initial condition θ(s, 0) = s solves for the constant C0 = e−s − 1. Thus theparticular solution of the first characteristic equation is
e−θ − 1 = (e−s − 1)e−λτ .
The second characteristic equation can be immediately integrated
t = τ + c2 ,
19
![Page 20: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/20.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
and using the initial condition t(s, 0) = 0, the solution is simply
t = τ .
And the solution of the third characteristic equation
v(s, τ) = eNs .
Letting v(s, τ) =�(s, τ), we can write for the m.g.f.
�(s, τ) = eNs .
The solution� must be expressed in terms of the original variables θ and t. We do this byexpressing e−s in terms of θ and t, that is
e−s = 1 − eλτ(1 − e−θ) .
Since eNs = (e−s)−N , the m.g.f. for the Yule process is
�(θ, t) =(1 − eλτ(1 − e−θ)
)−N.
The p.g.f. can be found directly from the m.g.f. by replacing e−θ = z−1 since eθ = z,
�(z, t) =(1 − eλτ(1 − z−1)
)−N
= zNe−λNt(ze−λt − (z − 1)
)−N
=zNe−λNt
(1 − z(1 − e−λt))N .
Letting p = e−λt and q = 1 − e−λt, the p.g.f. is
�(z, t) =(pz)N
(1 − qz)N . (26)
Taking N = 1, Eq. (26) can be solved trivially, that is, with
�(z, t) =pz
1 − qz,
20
![Page 21: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/21.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
then∂�(z)∂z
=p
(1 − qz)2
∣∣∣∣∣z=0
= p
∂2�(z)∂z2 =
2pq(1 − qz)3
∣∣∣∣∣z=0
= 2pq
∂3�(z)∂z3 =
6pq2
(1 − qz)4
∣∣∣∣∣∣z=0
= 6pq2
∂4�(z)∂z4 =
24pq3
(1 − qz)5
∣∣∣∣∣∣z=0
= 24pq3
...
and from Eq. (2),
Pi(t) =11!
p +12!
2!pq2−1 +13!
3!pq3−1 +14!
4!pq4−1 ,
or
Pi(t) =
(i − 1i − N
)pNqi−N , for N = 1 and i = 1, 2, 3, · · ·
In general, for N ≥ 1, the p.g.f. and m.g.f. for the Yule process correspond to a negativebinomial distribution. From the binomial coefficient, i must be greater than or equal to N,or i ≥ N. Then taking i = i + N and replacing p by e−λt and q by 1 − e−λt, the probabilitiesPi(t) can also be written as,
Pi+N(t) =
(N + i − 1
i
)e−λNt (1 − e−λt)i , i = 0, 1, 2, · · · (27)
By letting k = i + N, another way to write the probabilities is
Pk(t) =
(k − 1k − N
)e−λNt(1 − e−λt)k−N , k = N,N + 1, · · ·
When N = 1 or assuming that the population starts with one individual, then k = i + 1 andEq. (27) becomes
Pi+1(t) = e−λt(1 − e−λt)i or Pi(t) = e−λt(1 − e−λt)i−1 . (28)
The mean and variance for the Yule process are
m(t) =Np
= N eλt and σ2(t) =Nqp2 = N e2λt(1 − e−λt) .
The probabilities with four different values of t using Eq. (27) and three sample paths of theYule process are shown in Fig. 1. Python code for these figures are provided in the PythonImplementation section 3.1.
21
![Page 22: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/22.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
Figure 1: Plots of probability distributions Pi+N(t) for the Yule process using Eq. (27) with N = 5(left) and population size over time with three sample paths (right), both when λ = 0.5 andX(0) = 5. The mean and variance are m(t) = 5e0.5t and σt(t) = 5et(1 − e−0.5t).
2.5 Pure Death
Complementing the increasing pure birth process is the pure death process that is decreasing.Since the only event in the pure death process is a death, the process decreases successivelythrough states N,N − 1, · · · , 2, 1 and ultimately is absorbed in state 0 (extinction). The puredeath process is specified by the death parameters µk > 0 for k = 1, 2, · · · ,N, where the so-journ/interevent time in state k is exponentially distributed with parameter µk.
The infinitesimal transition probabilities of the pure death process whose state space is 0, 1, 2, · · · ,Nare
Pi,i+ j(∆t) = Prob{X(t + ∆t) − X(t) = j | X(t) = i}
=
µi∆t + o(∆t) , j = −1
1 − µi∆t + o(∆t) , j = 0
0 , j > 0
(29)
for i = 1, 2, · · · ,N for the first and second equations and i = 0, 1, · · · ,N for the third equation.As we have previously discussed in the general birth and death process, it is common to assignµ0 = 0.
If the death parameters µ1, µ2, · · · , µN are distinct, that is µi , µ j if i , j, taking from Eq. (4), theforward Kolmogorov equation for the pure death process at i = N is given by
dPN(t)dt
= −µN PN(t) .
If the initial condition is X(0) = N, or PN(0) = Prob{X(0) = N} = 1, thus the solution, or theexplicit transition probabilities are
PN(t) = e−µN t .
22
![Page 23: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/23.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
For i < N,dPi(t)
dt= µi+1Pi+1(t) − µiPi(t) ,
and the solutions are given by
Pi(t) = Prob{X(t) = i | X(0) = N}
= µi+1µi+2 · · · µN(Ai,ie−µit + · · · + AN,ie−µN t) (30)
whereAk,i =
1(µN − µk) · · · (µk+1 − µk)(µk−1 − µk) · · · (µi − µk)
.
2.6 Linear Death
As the pure death process complements the pure birth process, the linear death process comple-ments the Yule process. In the linear death process, or, also called the simple death process, thedeath rates are proportional to population size, µi = iµ. The infinitesimal transition probabilitiesof the linear death process are
Pi,i+ j(∆t) = Prob{X(t + ∆t) − X(t) = j | X(t) = i}
=
iµ∆t + o(∆t) , j = −1
1 − iµ∆t + o(∆t) , j = 0
o(∆t) , j ≤ −2
0 , j > 0
(31)
Since the process begins in state N, the state space is {0, 1, 2, · · · ,N}.
Converting these transition probabilities into differential equations
23
![Page 24: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/24.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
for Pi(t) = Prob{X(t) = i} gives
Pi(t + ∆t) = Prob{X(t + ∆t) = i}
= Prob{X(t) = i, X(t + ∆t) − X(t) = 0}+ Prob{X(t) = i + 1, X(t + ∆t) − X(t) = −1}+ Prob{X(t) = i + 2, X(t + ∆t) − X(t) ≤ −2}
= Prob{X(t) = i} · Prob{X(t + ∆t) − X(t) = 0}+ Prob{X(t) = i + 1} · Prob{X(t + ∆t) − X(t) = −1}+ Prob{X(t) = i + 2} · Prob{X(t + ∆t) − X(t) ≤ −2}
= Pi(t) ·[1 − iµ∆t + o(∆t)
]+ Pi+1(t) ·
[(i + 1)µ∆t + o(∆t)
]+ Pi+2(t) · o(∆t) .
Rearranging and equating terms with o(∆t) yield
Pi(t + ∆t) − Pi(t) = (i + 1)µ∆tPi+1(t) − iµ∆tPi(t) + o(∆t) .
Dividing both sides by ∆t and taking the limit as ∆ → 0 lead to the system of forward Kol-mogorov equations
dPi(t)dt
= (i + 1)µPi+1(t) − iµPi(t)
dPN(t)dt
= −NµPN(t) ,
for i = 0, 1, 2, · · · ,N − 1 and with initial conditions PN(0) = 1 and Pi(0) = 0 for i , N.
As in the Yule process, the probability distribution may be found by using the probability generat-ing function and method of characteristics. First, we multiply the forward Kolmogorov equations
24
![Page 25: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/25.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
by zi and sum over i,
∂�(z, t)∂t
=
∞∑i=N−1
(i + 1)µPi+1(t)zi −
∞∑i=N
iµPi(t)zi
=
∞∑i=N−1
(N − 1 + 1)µPN−1+1(t)zN−1 −
∞∑i=N
NµPN(t)zN
=
∞∑i=N
NµPN(t)zN−1 −
∞∑i=N
NµPN(t)zN−1z
= µ
∞∑i=N
iPi(t)zi−1 − µz∞∑
i=N
iPi(t)zi−1
= µ(1 − z)∞∑
i=N
iPi(t)zi−1
to get the partial differential equation for the p.g.f.,
∂�(z, t)∂t
= µ(1 − z)∂�
∂z, �(z, 0) = zN .
Substituting z = eθ, the m.g.f. is
∂�
∂t= µ(e−θ − 1)
∂�
∂θ, �(θ, 0) = eNθ .
After applying the method of characteristics, we obtain the solutions
�(z, t) =(1 − e−µt(1 − z)
)N=
(1 − e−µt + e−µtz
)N
and�(θ, t) =
(1 − e−µt(1 − eθ)
)N.
By letting p = e−µt and q = 1 − p = 1 − e−µt, the p.g.f. is
�(z, t) = (q + pz)N ,
which corresponds to a binomial distribution b(N, p). Since
Pi(t) =1i!
∂i�
∂zi
∣∣∣∣∣∣z=0
,
25
![Page 26: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/26.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
from which we show that,
k = 0 → P0(t) = qN
k = 1 → P1(t) =11!
∂�
∂z
∣∣∣∣∣z=0
=11!
NqN−1 p
k = 2 → P2(t) =12!
∂2�
∂z2
∣∣∣∣∣∣z=0
=12!
N(N − 1) qN−2 p2
k = 3 → P3(t) =13!
∂3�
∂z3
∣∣∣∣∣∣z=0
=13!
N(N − 1)(N − 2) qN−3 p3
...
thus, the probabilities are
Pi(t) =N!
i!(N − i)!piqN−i =
(Ni
)e−iµt (1 − e−µt)N−i
, (32)
for i = 0, 1, 2, · · · ,N. The mean is obtained by first differentiating the p.g.f. with respect to z,that is
d�dz
= N(q + pz)N−1 p ,
and the mean, now denoted by m(t), is
m(t) = �′(1) = N p ,
or, by p = e−µt
m(t) = Ne−µt ,
which corresponds to exponential decay. Whereas, the variance is
σ2(t) = �′′
(1) + �′
(1) −(�′
(1))2
= Ne−µt (1 − e−µt) .Example 4b.1: Linear Death Process
Taken from Example 5.13 in [1]:
In a linear death process, the only event is a death, that is, state i → i − 1. For ∆t sufficiently
26
![Page 27: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/27.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
small, the transition probabilities are
Pi,i+ j(∆t) = Prob{X(t + ∆t) − X(t) = j | X(t) = i}
=
di∆t + o(∆t) j = −1
1 − di∆t + o(∆t) j = 0
o(∆t) j ≤ − 2
0 j > 0 .
Given X(Wi) = n, then α(n) = dn. Therefore, the interevent time Ti is
Ti = −ln(U)
dn.
The next event is a death n→ n − 1, X(Wi+1) = n − 1. The deterministic analogue of this simpledeath process is the differential equation dn/dt = −dn, with initial condition n(0) = N, and thesolution
n(t) = N e−dt . �
The probability distributions and population size over time with three sample paths of the lineardeath process are shown in Fig. 2. The Python code to produce these results are given in thePython Implementation section 3.2. The code are also available on the course website.
Figure 2: Plots of probability distributions Pk(t) for the linear death process using Eq. (32) (left)and population size over time with three sample paths (right), both when µ = 0.5 and X(0) = 20.The mean and variance are m(t) = 20e−0.5t and σ2(t) = 20e−0.5t(1 − e−0.5t).
2.7 Simple Birth and Death
In a simple birth and death process, X(t) is allowed to increase and decrease. If at time t theprocess is in state n, after a random sojourn time, it may move to either of the neighboring states
27
![Page 28: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/28.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
n + 1 or n − 1. The birth and death process is irreducible if all the states communicate, and thishappens if and only if all the λn (n ≥ 0) and all the µn (n ≥ 1) are positive.
If we let X(t) denote the state of the chain at time t, then
Pi,i+ j(∆t) = Prob{X(t + ∆t) − X(t) = j | X(t) = i}
=
1 − i(µ + λ)∆t + o(∆t) j = 0
iλ∆t + o(∆t) j = 1
iµ∆t + o(∆t) j = −1
o(∆t) j , − 1, 0, 1 .
(33)
Simple birth and death processes are also known as birth and death processes with absorbingstates. For these processes, the zero state is an absorbing state, where when the population sizebecomes zero, it remains zero thereafter.
As for the formulations in the Poisson process, we can convert the transition probabilities inEq. (35) into differential equations for Pi(t) = Prob{X(t) = i}, hence
Pi(t + ∆t) = Prob{X(t + ∆t) = i}
= Prob{X(t) = i, X(t + ∆t) − X(t) = 0}+ Prob{X(t) = i − 1, X(t + ∆t) − X(t) = 1}+ Prob{X(t) = i + 1, X(t + ∆t) − X(t) = −1}
= Prob{X(t) = i} · Prob{X(t + ∆t) = X(t)}+ Prob{X(t) = i − 1} · Prob{X(t + ∆t) − X(t) = 1}+ Prob{X(t) = i + 1} · Prob{X(t + ∆t) − X(t) = −1}
= Pi(t) ·[1 − i(µ + λ)∆t + o(∆t)
]+ Pi−1(t) ·
[(i − 1)λ∆t + o(∆t)
]+ Pi+1(t) ·
[(i + 1)µ∆t + o(∆t)
].
Rearranging and equating terms with o(∆t) give us
Pi(t + ∆t) − Pi(t) = (i + 1)µ∆tPi+1(t) + (i − 1)λ∆tPi−1(t) − i(µ + λ)∆tPi(t) + o(∆t) .
Dividing both sides by ∆t and taking the limit as ∆t → 0 lead to the following system of
28
![Page 29: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/29.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
differential-difference equations
dPi(t)dt
= (i − 1)λPi−1(t) + (i + 1)µPi+1(t) − i(µ + λ)Pi(t) ,
dP0(t)dt
= µP1(t)
(34)
for i = 1, 2, · · · with initial conditions Pi(0) = δiN . Multiplying the first equation by zi andsumming over i we obtain
∂�(z, t)∂t
=
∞∑i=N+1
(N + 1 − 1)λPN+1−1(t)zN+1 +
∞∑i=N−1
(N − 1 + 1)µPN−1+1(t)zN−1
−
∞∑i=N
N(µ + λ)PN(t)zN
=
∞∑i=N+1
NλPN(t)zN−1z2 +
∞∑i=N−1
NµPN(t)zN−1 −
∞∑i=N
N(µ + λ)PN(t)zN−1z
= λz2∞∑
i=N
iPi(t)zi−1 + µ
∞∑i=N
iPi(t)zi−1 − z(µ + λ)∞∑
i=N
iPi(t)zi−1
=[λz2 − (µ + λ)z + µ
] ∞∑i=N
iPi(t)zi−1 .
The p.g.f. is a solution of
∂�(z, t)∂t
=[µ(1 − z) + λz(z − 1)
]∂�∂z
, �(z, 0) = zN
and the m.g.f. is a solution is
∂�
∂t=
[µ(e−θ − 1) + λ(eθ − 1)
]∂�∂θ
, �(θ, 0) = eθN .
The characteristic equations are
dtdτ
= 1 ,
dθdτ
= −[µ(e−θ − 1) + λ(eθ − 1)
],
d�dτ
= 0 ,
29
![Page 30: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/30.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
after immediately letting v(s, τ) =�(s, τ). The initial conditions are
t(s, 0) = 0 , θ(s, 0) = s , and �(s, 0) = eNs .
The solutions satisfyt = τ and �(s, τ) = eNs .
We solve for τ in the following ways
• If µ = λ:By a change of variable x = eθ, the second characteristic equation becomes
dθλ(e−θ − 1) + λ(eθ − 1)
= −dτ
dθλ(eθ + e−θ) − 2λ
= −dτ
dxx[λ(x + x−1) − 2λ
] = −dτ
dxλ(x2 − 2x + 1)
= −dτ
dxλ(x − 1)2 = −dτ
Using a substitution u = x − 1, we integrate both sides
1λ
∫duu2 = −
∫dτ
−1λ u
= −τ + c1 ,
orτ =
1λ u
+ c1 =1
λ(eθ − 1)+ c1 ,
where c1 is the arbitrary constant of integration. Applying the initial condition θ(s, 0) = s,we obtain
0 =1
λ(es − 1)+ c1 ,
orc1 = −
1λ(es − 1)
.
Hence,
τ =1
λ(eθ − 1)−
1λ(es − 1)
.
30
![Page 31: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/31.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
• If µ , λ:Similar way as the above,
dθµ(e−θ − 1) + λ(eθ − 1)
= −dτ
dxx[µ(x−1 − 1) + λ(x − 1)
] = −dτ
dxµ(1 − x) + λ(x2 − x)
= −dτ
dxλx2 − (λ + µ)x + µ
= −dτ .
The left hand side equation can be resolved into partial fractions as follows
1λx2 − (λ + µ)x + µ
≡1
(λx − µ)(x − 1)
≡A
λx − µ+
Bx − 1
,
where A and B are constants to be determined by algebraic addition, i.e.,
1(λx − µ)(x − 1)
≡A(x − 1) + B(λx − µ)
(λx − µ)(x − 1).
Since the denominators are the same on each side of the identity, then the numerators areequal to each other. Thus,
1 ≡ A(x − 1) + B(λx − µ) .
The constants A and B are determined by choosing the values of x to make the term in Aor B equal to zero. That is, when x = 1, then
1 ≡ A(0) + B(λ − µ)
B =1
λ − µ,
and when x = µ/λ, then
1 ≡ A(µ
λ− 1) + B(0)
A =λ
µ − λ.
The characteristic equation becomes(λ
µ − λ·
1λx − µ
+1
λ − µ·
1x − 1
)dx = −dτ .
31
![Page 32: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/32.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
Integrating both sides and using substitutions
λ
µ − λ
∫dx
λx − µ+
1λ − µ
∫dx
x − 1= −
∫dτ
1µ − λ
ln u +1
λ − µln v = −τ + c2
−1
µ − λln
(vu
)= −τ + c2
where u = λx − µ and v = x − 1. Thus,
τ =1
µ − λln
(eθ − 1λeθ − µ
)+ c2 .
Applying the initial condition θ(s, 0) = s gives
c2 = −1
µ − λln
(es − 1λes − µ
),
and the particular solution of the characteristic equation is
τ =1
µ − λln
(eθ − 1λeθ − µ
)−
1µ − λ
ln(
es − 1λes − µ
)
=1
µ − λ
[ln
(eθ − 1)(λes − µ)(λeθ − µ)(es − 1)
].
Because�(s, τ) = (es)N , these relations are solved for es,
es =
eτ(µ−λ)(λeθ − µ) − µ(eθ − 1)eτ(µ−λ)(λeθ − µ) − λ(eθ − 1)
, if µ , λ
1 − (λτ − 1)(eθ − 1)1 − λτ(eθ − 1)
, if µ = λ .
The m.g.f. now can be expressed in terms of θ and t, that is
�(θ, t) =
(et(µ−λ)(λeθ − µ) − µ(eθ − 1)et(µ−λ)(λeθ − µ) − λ(eθ − 1)
)N
, if µ , λ
(1 − (λt − 1)(eθ − 1)
1 − λt(eθ − 1)
)N
, if µ = λ .
32
![Page 33: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/33.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
Making the change of variable θ = ln(z), we obtain the p.g.f.,
�(z, t) =
(et(µ−λ)(λz − µ) − µ(z − 1)et(µ−λ)(λz − µ) − λ(z − 1)
)N
, if µ , λ
(1 − (λt − 1)(z − 1)
1 − λt(z − 1)
)N
, if µ = λ .
Obtaining the probabilities Pi(t) is not as straightforward as it was for the Yule and linear deathprocesses. We follow calculation results in [1]. The first term in the expansion of �(z, t) isP0(t) = �(0, t):
P0(t) =
(µ − µe(µ−λ)t
λ − µe(µ−λ)t
)N
, if µ , λ
(λt
1 + λt
)N
, if µ = λ .
The probability of extinction, P0(t), has a simple expression when t → ∞,
P0(∞) = limt→∞
P0(t) =
1 , if λ ≤ µ(µ
λ
)Nif λ > µ .
If the probability of death is greater than or equal to the probability of birth, or λ ≤ µ, then in thelong run as t → ∞ the probability of losing all of the initial population N approaches one. Butif the probability of birth is greater than the probability of death, or λ > µ, then as t → ∞ theprobability of losing all of the initial population is (µ/λ)N . The result when λ > µ as t → ∞ isreminiscent of a semi-infinite random walk with an absorbing barrier at x = 0, or the gambler’sruin problem, where the probability of losing a game is µ and the probability of winning a gameis λ [1].
The mean m(t) and variance σ2(t) of the simple birth and death process are:
• For µ , λ
m(t) = Ne(λ−µ)t and σ2(t) = N(λ + µ)(λ − µ)
e(λ−µ)t(e(λ−µ)t − 1) ,
where the mean corresponds to exponential growth when λ > µ and exponential decaywhen λ < µ.
• For µ = λm(t) = N and σ2(t) = 2Nλt .
33
![Page 34: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/34.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
The population over time plots of the simple birth and death process are shown in Fig. 3. ThePython code to produce these results are given in the Python Implementation section 3.3. Thecode are also available on the course website.
Figure 3: Plots of three sample paths for the simple birth and death process when µ = 1.0,X(0) = 50, and λ = 0.1 (left) and λ = 1.0 (right). In the case λ << µ (left figure), the populationsize becomes zero and it remains in zero forever.
2.8 Simple Birth and Death Process with Immigration
A birth and death process is called a linear growth process if λi = iλ + β and µi = iµ withλ > 0, µ > 0, and β > 0. If the state i describes the current population size, then the averageinstantaneous rate of growth is iλ + β, where the factor iλ represents the natural growth of thepopulation owing to its current size and the factor β may be interpreted as the infinitesimal rateof increase of the population due to an external source, such as immigration. Note that theimmigration rate does not depend linearly on i, which looks like the Poisson process. Similarly,the probability of the state of the process decreasing by one after the elapse of a small duration oftime ∆t is iµ∆t + o(∆t), where the factor iµ gives the mean infinitesimal death rate of the presentpopulation.
For the general case of a simple birth and death process with immigration and emigration wherethe rates are λi = iλ + β and µi = iµ + ρ, state transitions are a combination of:
• birth: i −→ i + 1 at rate iλ
• death: i −→ i − 1 at rate iµ
• immigration: i −→ i + 1 at rate β
• emigration: i −→ i − 1 at rate ρ
34
![Page 35: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/35.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
The infinitesimal transition probabilities for the simple birth and death process with immigrationsare
Pi,i+ j(∆t) = Prob{X(t + ∆t) − X(t) = j | X(t) = i}
=
1 −[β + i(µ + λ)
]∆t + o(∆t) j = 0
(β + iλ)∆t + o(∆t) j = 1
iµ∆t + o(∆t) j = −1
o(∆t) j , − 1, 0, 1 .
(35)
Because of the immigration term, the value of λ0 = β > 0.
The forward Kolmogorov differential equations for the process are
• For i = 0, where λ0 = β and µ1 = µ,
dP0(t)dt
= −βP0(t) + µP1(t)
• For i ≥ 1, where λi−1 = (i − 1)λ + β, λi = iλ + β, µi+1 = (i + 1)µ, and µi = iµ,
dPi(t)dt
=[λ(i − 1) + β
]Pi−1(t) −
[(λ + µ)i + β
]Pi(t) + µ(i + 1)Pi+1(t) .
Letting X(0) = N, for the second equation with i , 1, we apply the generating function techniqueby multiplying the equation by zi and summing over i,
∂�
∂t=
∞∑i=N+1
[λ(i − 1) + β
]Pi−1(t)zi −
∞∑i=N
[iλ + iµ + β
]Pi(t)zi +
∞∑i=N−1
µ(i + 1)Pi+1(t)zi
= λz2∞∑
i=N
iPi(t)zi−1 + βz∞∑
i=N
Pi(t)zi − (λ + µ)z∞∑
i=N
iPi(t)zi−1 − β
∞∑i=N
Pi(t)zi
+ µ
∞∑i=N
iPi(t)zi−1
=[λz(z − 1) + µ(1 − z)
] ∞∑i=N
iPi(t)zi−1 + β(z − 1)∞∑
i=N
Pi(t)zi ,
we obtain the p.g.f.,∂�
∂t=
[λz(z − 1) + µ(1 − z)
]∂�∂z
+ β(z − 1)� ,
35
![Page 36: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/36.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
and subsequently, the m.g.f. [1],
∂�
∂t=
[λ(eθ − 1) + µ(e−θ − 1)
]∂�∂θ
+ β(eθ − 1)� ,
with initial condition�(θ, 0) = eNθ. The solution of the m.g.f [1] is
�(θ, t) =(λ − µ)β/λ
[µ(ρ − 1) − eθ(µρ − λ)
]N[(λρ − µ) − λeθ(ρ − 1)
]N+β/λ,
and the p.g.f.,
�(z, t) =(λ − µ)β/λ
[µ(ρ − 1) − z(µρ − λ)
]N[λρ − µ − λz(ρ − 1)
]N+β/λ,
where ρ = e(λ−µ)t, X(0) = N, and λ , µ.
The mean can be calculated by differentiating�(θ, t) with respect to θ and evaluating at θ = 0,
m(t) =
ρ(Nµ − Nλ − β) + β
µ − λ, if λ , µ
βt + N , if λ = µ .
(36)
For the cases λ > µ and λ = µ, the process is explosive, where when λ > µ, the mean increasesexponentially in time and when λ = µ, the mean increases linearly in time. However, the processis nonexplosive in the case λ < µ, where the mean approaches a constant,
m(∞) =β
µ − λ, λ < µ .
Also, for the case λ < µ, the process is irreducible and has a unique positive stationary distri-bution. These conditions of being nonexplosive, irreducible, and having a positive stationarydistribution imply that the process is positive recurrent [1]. Then, Theorem 8 in Lecture 4aimplies the limiting distribution exists and equals the (unique positive) stationary distribution.
Example 4b.1
Taken from Example 6.1 on pp. 246–247 in [1]:
A continuous-time birth and death Markov chain has a birth rate λi = b and a death rate µi = idfor i = 0, 1, 2, · · · . Find the stationary probability distribution.
36
![Page 37: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/37.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
Solution:
To find the stationary probability distribution π, we use formulations derived in subsection 2.2.
For:
λi = b ,→ then λ0λ1 · · · λi−1 = b · b · · · b = bi ,
µi = id ,→ then µ1µ2µ3 · · · µi = 1d · 2d · 3d · · · id = dii! ,
from which,λ0λ1 · · · λi−1
µ1µ2µ3 · · · µi=
bi
dii!=
(b/d)i
i!,
then, the general equation of the stationary probability distribution is given by
πi =(b/d)i
i!π0 . (37)
Since, according to the power series of an exponential function,
1 +
∞∑i=1
λ0λ1 · · · λi−1
µ1µ2µ3 · · · µi= 1 +
∞∑i=1
(b/d)i
i!
= 1 +(b/d)
1!+
(b/d)2
2!+
(b/d)3
3!+ · · ·
= eb/d ,
from Eq. (12),
π0 =1
1 +∞∑
i=1
λ0λ1 · · · λi−1
µ1µ2µ3 · · · µi
=1
eb/d
= e−b/d ,
and from Eq. (10), the unique stationary probability distribution is
πi =λ0λ1 · · · λi−1
µ1µ2µ3 · · · µiπ0
=(b/d)i
i!e−b/d ,
which is a Poisson distribution with parameter b/d, for i = 0, 1, 2, 3, · · · . The Markov chain ispositive recurrent for all λ and µ.
37
![Page 38: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/38.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
Example 4b.2
Taken from Example 6.3 on pp. 256 in [1]:
Consider a simple birth and death process with immigration. Let λ = 0.5 and µ = 1. Find thestationary probability distribution when β = 0.5, β = 1.0, β = 1.5, and β = 5.0.
Solution:
The death rate is µi = iµ = i, andthe average instantaneous rate of growth is λi = iλ + β = 0.5i + β,with various values of β as above.
Using the same steps as in Example 4b.1, that is
(1) When β = 0.5
∗ λi = 0.5(i + 1), then
λ0λ1 · · · λi−1 = [0.5(0 + 1)] · [0.5(1 + 1)] · [0.5(2 + 1)] · · · [0.5(i − 1 + 1)]= [0.5(1)] · [0.5(2)] · [0.5(3)] · · · [0.5(i)]= 0.5i i!
∗ µi = iµ, thenµ1µ2µ3 · · · µi = 1 · 2 · 3 · · · i = i! .
Combining the above,
πi =λ0λ1 · · · λi−1
µ1µ2 · · · µiπ0
=
(0.5(1)
1
) (0.5(2)
2
) (0.5(3)
3
)· · ·
(0.5(i)
i
)π0
=0.5i (i + 0)!
0! i!π0
=0.5i i!
i!π0
= (0.5)i π0 .
(2) When β = 1
∗ λi = 0.5i + 1,
λ0λ1 · · · λi−1 = [0.5(0) + 1] · [0.5(1) + 1] · [0.5(2) + 1] · · · [0.5(i − 1) + 1]= [0.5(2)] · [0.5(3)] · [0.5(4)] · · · [0.5(i + 1)]
38
![Page 39: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/39.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
∗ µi = iµ,µ1µ2µ3 · · · µi = 1 · 2 · 3 · · · i = i! .
Then
πi =λ0λ1 · · · λi−1
µ1µ2 · · · µiπ0
=
(0.5(2)
1
) (0.5(3)
2
) (0.5(4)
3
)· · ·
(0.5(i + 1)
i
)π0
=0.5i (i + 1)!
1! i!π0
=0.5i (i + 1)!
i!π0 .
(3) When β = 1.5
∗ λi = 0.5i + 1.5,
λ0λ1 · · · λi−1 = [0.5(0) + 1.5] · [0.5(1) + 1.5] · [0.5(2) + 1.5] · · · [0.5(i − 1) + 1.5]= [0.5(3)] · [0.5(4)] · [0.5(5)] · · · [0.5(i + 2)]
∗ µi = iµ,µ1µ2µ3 · · · µi = 1 · 2 · 3 · · · i = i! .
Then
πi =λ0λ1 · · · λi−1
µ1µ2 · · · µiπ0
=
(0.5(3)
1
) (0.5(4)
2
) (0.5(5)
3
)· · ·
(0.5(i + 2)
i
)π0
=0.5i (i + 2)!
2! i!π0 .
(4) When β = 5
∗ λi = 0.5i + 5,
λ0λ1 · · · λi−1 = [0.5(0) + 5] · [0.5(1) + 5] · [0.5(2) + 5] · · · [0.5(i − 1) + 5]= [0.5(10)] · [0.5(11)] · [0.5(12)] · · · [0.5(i + 9)]
∗ µi = iµ,µ1µ2µ3 · · · µi = 1 · 2 · 3 · · · i = i! .
39
![Page 40: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/40.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
Then
πi =λ0λ1 · · · λi−1
µ1µ2 · · · µiπ0
=
(0.5(10)
1
) (0.5(11)
2
) (0.5(12)
3
)· · ·
(0.5(i + 9)
i
)π0
=0.5i (i + 9)!
9! i!π0 .
We can deduce that, the general equation of the stationary probability distribution πi, with birthrate λ, migration rate β, and death rate µ, is given by
πi =λi (i + β/λ − 1)!(β/λ − 1)! µi i!
π0 ,
orπi = (λ/µ)i ·
(i + β/λ − 1)!i! (β/λ − 1)!
π0 . (38)
Taking β/λ = n, the term(i + n − 1)!i! (n − 1)!
=
(i + n − 1
i
),
is a binomial coefficient. Hence, we can write (38) as
πi = (λ/µ)i
(i + n − 1
i
)π0 . (39)
To calculate π0, we may use the following
π0 =1
1 +∞∑
i=1
λ0λ1 · · · λi−1
µ1µ2 · · · µi
,
(1) when β = 0.5, and n = 0.5/0.5 = 1,
1 +
∞∑i=1
λ0λ1 · · · λi−1
µ1µ2 · · · µi= 1 +
∞∑i=1
(0.5)i
= 1 + 0.5 + (0.5)2 + (0.5)3 + · · · (geometric series)
=1
1 − 0.5
= 2 ,
40
![Page 41: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/41.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
from which,
π0 =12.
The stationary distribution is a geometric distribution,
πi = (0.5)i+1 , i = 0, 1, 2, · · ·
(2) when β = 1, and n = 1/0.5 = 2,
1 +
∞∑i=1
λ0λ1 · · · λi−1
µ1µ2 · · · µi= 1 +
∞∑i=1
(0.5)i (i + 1)!i!
= 1 +(0.5) 2!
1!+
(0.5)2 3!2!
+(0.5)3 4!
3!+ · · · (binomial series)
=
∞∑i=0
(0.5)i
(2 + i − 1
i
)
=1
(1 − 0.5)2
= 4 ,
from which,
π0 =14.
The unique stationary distribution is
πi =14·
(0.5)i (i + 1)!i!
=(0.5)i+2 (i + 1)!
i!, i = 0, 1, 2, · · ·
(3) when β = 1.5, and n = 1.5/0.5 = 3,
1 +
∞∑i=1
λ0λ1 · · · λi−1
µ1µ2 · · · µi= 1 +
∞∑i=1
(0.5)i (i + 2)!2! i!
= 1 +(0.5) 3!
2! 1!+
(0.5)2 4!2! 2!
+(0.5)3 5!
2! 3!+ · · · (binomial series)
=
∞∑i=0
(0.5)i
(3 + i − 1
i
)
=1
(1 − 0.5)3
= 8 ,
41
![Page 42: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/42.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
from which,
π0 =18.
The unique stationary distribution is
πi =18·
(0.5)i (i + 2)!2! i!
=(0.5)i+3 (i + 2)!
2! i!, i = 0, 1, 2, · · ·
(4) when β = 5, and n = 5/0.5 = 10,
1 +
∞∑i=1
λ0λ1 · · · λi−1
µ1µ2 · · · µi= 1 +
∞∑i=1
(0.5)i (i + 9)!9! i!
= 1 +(0.5) 10!
9! 1!+
(0.5)2 11!9! 2!
+(0.5)3 12!
9! 3!+ · · · (binomial series)
=
∞∑i=0
(0.5)i
(10 + i − 1
i
)
=1
(1 − 0.5)10
= 210 ,
from which,
π0 =
(12
)10
.
The unique stationary distribution is
πi =
(12
)10
·(0.5)i (i + 9)!
9! i!=
(0.5)i+10 (i + 9)!9! i!
, i = 0, 1, 2, · · ·
Hence, we can deduce thatπ0 = (0.5)n .
However, for general parameter values, π0 is actually given by
π0 =
(1 −
λ
µ
)n
.
Since, in this problem, β/λ = n is an integer, then the m.g.f. corresponds to a negative binomialdistribution with parameter p = 1 − λ/µ, if and only if λ < µ and β > 0. The general formula forthe stationary probability distribution is then given by
42
![Page 43: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/43.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
πi =
(i + n − 1
i
)pn (1 − p)i , i = 0, 1, 2, · · · (40)
In the case n = 1, the negative binomial distribution is the same as the geometric distribution, aswe have seen with β = 0.5 and λ = 0.5. The graphs of the stationary probability distributions forfour different sets of the immigration rate β is shown in Fig. 4.
Figure 4: Stationary probability distributions for the simple birth, death, and immigration processwith λ = 0.5, µ = 1, and β = 0.5 (blue), 1.0 (magenta), 1.5 (green), and 5 (orange).
Using
m =β
µ − λand σ2 =
βµ
(µ − λ)2
for the mean and variance, respectively, then the mean values for the stationary probability distri-butions corresponding to each parameter set are m = 1, 2, 3, and 10, and the variance σ2 = 2, 4, 6,and 20.
The plots of population size over time with four sample paths of the birth and death process withmigration are shown in Fig. 5. The Python code to produce these results are given in the PythonImplementation section 3.4. The code are also available on the course website.
43
![Page 44: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/44.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
Figure 5: Four sample paths corresponding to the birth, death, and immigration process whenX(0) = 20, λ = 0.5, µ = 1, and β = 0.5, 1.0, 1.5, 5.
3 Python Implementation
All simulations were performed using Python v 3.6. All code presented here are available on thecourse website.
3.1 Yule Process Simulation
To simulate the Yule process, it is necessary to know the random variable for the time betweenbirths or interevent/sojourn time. It is shown that the random variable for the interevent/sojourntime is exponentially distributed; if the population is of size N, then the time T to the next event(or interevent/sojourn time) has a distribution equal to [1]
Prob{T ≥ h} = exp(−bNs) .
To simulate a value h ∈ T , a uniformly distributed random number Y from the uniform distribu-
44
![Page 45: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/45.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
tion U(0, 1) is selected in the range 0 ≤ Y ≤ 1 [1]. Then
Y = exp(−bNh) ,
and solving for h yields
h = −ln(Y)bN
.
Note that in the simple birth process, as N increases, h = − ln(Y)/bN decreases, or, the intereven-t/sojourn time decreases as the population size increases [1].
In Python, to generate a random sample Y from the uniform distribution, we use:
import numpy as np
Y = np.random.rand()
which is used to get h:
h = -np.log(Y)/(b*N)
or:
h = -np.log(np.random.rand())/(b*N)
What we are calculating here is the interevent/sojourn times as the population size increases oneby one, by using:
for j in range(numberofpaths):for i in range(N-1):
h = - np.log(np.random.rand())/(b*X[i])
s[j,i+1] = s[j,i] + h
and after which, the population is increased:
X[j,i+1] = X[j,i] + 1
The whole Python code for the Yule process can be written as follows:
import numpy as npimport matplotlib.pyplot as plt
np.random.seed(100)
def simplebirth(N, samplepaths , b, N0):s = np.zeros((samplepaths , N))
X = np.zeros((samplepaths , N))
X[:,0] = N0
45
![Page 46: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/46.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
for j in range(samplepaths):for i in range(N-1):
h = - np.log(np.random.rand())/(b*X[j,i])
s[j,i+1] = s[j,i] + h
X[j,i+1] = X[j,i] + 1
return [s, X]
def deterministic(s, N, b, N0):tmax = max(s[:, -1])
t = np.linspace(0, tmax, 10*N)
yd = N0*np.exp(b*t)
return[t, yd]
def solve_and_plot(N, paths, b, X0):fig, ax = plt.subplots()
[sojourn, population] = simplebirth(N, paths, b, X0)
[time, detsol] = deterministic(sojourn, N, b, X0)
## Sets axis ranges for plotting
xmax = max([max(sojourn[k,:]) for k in range(paths)])ymin = min([min(population[k,:]) for k in range(paths)])ymax = max([max(population[k,:]) for k in range(paths)])
## Generates plots
for r in range(walkers):plt.step(sojourn[r,:], population[r,:], where=’post’, label="Path %s
" % str(r+1))
plt.plot(time, detsol, ’k--’, label="Deterministic")
plt.axis([-0.2, xmax+0.2, ymin-2, ymax+2])
ax.set_xlabel(’Time’, fontsize=14)
ax.set_ylabel(’Population Size’, fontsize=14)
plt.xticks(fontsize=14)
plt.yticks(fontsize=14)
plt.tight_layout()
plt.legend(loc=2)
plt.show()
maxPop = 100
walkers = 3
birthrate = 0.5
initPop = 5
solve_and_plot(maxPop, walkers, birthrate , initPop)
46
![Page 47: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/47.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
The code for calculating probability distributions of the Yule process given in Eq. (27) is
import numpy as npimport matplotlib.pyplot as pltimport scipy.miscimport scipy.specialfrom matplotlib.lines import Line2Dfrom cycler import cycler
def probdist(N, b):t = np.arange(1, N)
k = np.arange(0, 51) # population vector
p = np.zeros((len(t), len(k)))
for i in range(len(t)):for j in range(len(k)):
p[i][j] = scipy.special.binom(N+k[j]-1, k[j])\
*np.exp(-b*N*t[i])\
*(1-np.exp(-b*t[i]))**k[j]
return [p, t, k]
def solve_and_plot(N, b):fig, ax = plt.subplots()
mycolors = [’cornflowerblue’, ’orchid’, ’olive’, ’gold’]
colors = ax.set_prop_cycle(cycler(’color’, mycolors))
marker_style = dict(color=colors, linestyle=’:’,
marker=’o’, markersize=6)
[p, t, k] = probdist(N, b)
for y, fill_style in enumerate(Line2D.fillStyles):for r in range(len(t)):
ax.plot(k, p[r][:], fillstyle=fill_style , **marker_style)
ax.set_xlabel(’Population Size $k$’, fontsize=14)
ax.set_ylabel(’Probability $P_k(t)$’, fontsize=14)
plt.legend([’$t=%s$’ % str(i+1) for i in range(len(t))], fontsize=14,loc=1)
plt.xticks(fontsize=14)
plt.yticks(fontsize=14)
plt.text(0.5, 0.9, ’$\lambda = %s$’ % b, fontsize=14,horizontalalignment=’center’,
verticalalignment=’center’,
transform=ax.transAxes)
plt.tight_layout()
plt.show()
N = 5
47
![Page 48: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/48.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
birth = 0.5
solve_and_plot(N, birth)
3.2 Linear Death Process Simulation
import numpy as npimport matplotlib.pyplot as plt
np.random.seed(100)
def simpledeath(N, samplepaths , d, N0):s = np.zeros((samplepaths , N))
X = np.zeros((samplepaths , N))
X[:,0] = N0
for j in range(samplepaths):for i in range(N-1):
h = - np.log(np.random.rand())/(d*X[j,i])
s[j,i+1] = s[j,i] + h
X[j,i+1] = X[j,i] - 1
return [s, X]
def deterministic(s, N, d, N0):tmax = max(s[:,-1])
t = np.linspace(0, tmax, 10*N)
yd = N0*np.exp(-d*t)
return [t, yd]
def solve_and_plot(N, paths, d, X0):fig, ax = plt.subplots()
[sojourn, population] = simpledeath(N, paths, d, X0)
[time, detsol] = deterministic(sojourn, N, d, X0)
## Sets axis ranges for plotting
xmax = max([max(sojourn[k,:]) for k in range(paths)])ymax = max([max(population[k,:]) for k in range(paths)])
## Generates plots
for r in range(walkers):plt.step(sojourn[r,:], population[r,:], where=’pre’,
label="Path %s" % str(r+1))
plt.plot(time, detsol, ’k--’, label="Deterministic")
48
![Page 49: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/49.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
plt.axis([-0.1, xmax+0.2, 0, ymax+(ymax*0.01)])
ax.set_xlabel(’Time’, fontsize=14)
ax.set_ylabel(’Population Size’, fontsize=14)
plt.xticks(fontsize=14)
plt.yticks(fontsize=14)
plt.tight_layout()
plt.legend(loc=1)
plt.show()
maxPop = 20
walkers = 3
deathrate = 0.5
initPop = maxPop
solve_and_plot(maxPop, walkers, deathrate , initPop)
Note that the code for the Yule and linear death processes are almost identical, except for thefollowing:
• Initial population in the linear death process must be equal to its maximal population size,whereas in the Yule process the initial population must be smaller than the maximal popu-lation size,
• In the Yule process the population increases, X[j,i+1]=X[j,i]+1, and in the linear deathprocess the population decreases, X[j,i+1]=X[j,i]-1.
The code for probability distributions of linear death process using Eq. (32) is as follows:
import numpy as npimport matplotlib.pyplot as pltimport scipy.miscimport scipy.specialfrom matplotlib.lines import Line2Dfrom cycler import cycler
def probdist(N, mu):t = np.arange(1, 5)
k = np.arange(0, N+1)
p = np.zeros((len(t), len(k)))
for i in range(len(t)):for j in range(len(k)):
p[i][j] = scipy.special.binom(N, j)\
*np.exp(-j*mu*t[i])\
*(1-np.exp(-mu*t[i]))**(N-j)
return [p, t, k]
49
![Page 50: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/50.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
def solve_and_plot(N, mu):fig, ax = plt.subplots()
mycolors = [’cornflowerblue’, ’orchid’, ’olive’, ’gold’]
colors = ax.set_prop_cycle(cycler(’color’, mycolors))
marker_style = dict(color=colors, linestyle=’:’,
marker=’o’, markersize=6)
[p, t, k] = probdist(N, mu)
for y, fill_style in enumerate(Line2D.fillStyles):for r in range(len(t)):
ax.plot(k, p[r][:], fillstyle=fill_style , **marker_style)
ax.set_xlabel(’Population Size $k$’, fontsize=14)
ax.set_ylabel(’Probability $P_k(t)$’, fontsize=14)
plt.legend([’$t=%s$’ % str(i+1) for i in range(len(t))],fontsize=14, loc=1)
plt.xticks(fontsize=14)
plt.yticks(fontsize=14)
plt.text(0.5, 0.9, ’$\mu = %s$’ % mu, fontsize=14,horizontalalignment=’center’,
verticalalignment=’center’,
transform=ax.transAxes)
plt.tight_layout()
plt.show()
N = 20
death = 0.5
solve_and_plot(N, death)
3.3 Birth and Death Process Simulation
For birth and death processes, we first look at the transition probabilities
Pi,i+ j(∆t) = Prob{X(t + ∆t) − X(t) = j | X(t) = i}
=
1 − (µi + λi)∆t + o(∆t) j = 0
λi∆t + o(∆t) j = 1
µi∆t + o(∆t) j = −1
o(∆t) j , − 1, 0, 1 .
(41)
50
![Page 51: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/51.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
According to Eq. (41), during a small time duration of length ∆t, a transition occurs from state ito state i+1 with probability λi∆t+o(∆t) and from state i to state i−1 with probability µi∆t+o(∆t).Given that a transition occurs at time t, the probability of:
• moving to state i + 1 isλi
λi + µi
• moving to state i − 1 isµi
λi + µi
Recall that the random variable for the interevent/sojourn time is continuous and nonnegative,that is Ti = Wi+1 − Wi ≥ 0, where Wi is the time of the ith jump. In the linear birth and deathprocess, if the events consist of births with rate λi = iλ and deaths with rate µi = iµ, the intereventtime Ti is given by
Ti = −ln(U1)
(λ + µ)i,
where U1 is a generated random number.
Since there are two events, to determine whether there is a birth or a death, the unit interval [0, 1]is divided into two subintervals:
• one subinterval has probability
λi
λi + µi=
iλiλ + iµ
=λ
λ + µ,
in which a birth occurs, and
• the other has probability probability
µi
λi + µi=
iµiλ + iµ
=µ
λ + µ
in which a death occurs.
We generate another random number, say U2, and if U2 < λ/(λ + µ), then this random numberlies in the first subinterval and a birth occurs. Otherwise, if U2 ≥ λ/(λ + µ), the random numberlies in the second subinterval and a death occurs.
The deterministic analogue of this simple birth and death process is the differential equationdn/dt = (λ − µ)n, with initial condition n(0) = N, and the solution is given by
n(t) = N e(λ−µ)t .
51
![Page 52: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/52.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
import numpy as npimport matplotlib.pyplot as plt
np.random.seed(100)
def simplebirthdeath(N, samplepaths , b, d):X0 = 50 # initial population size
s = np.zeros((samplepaths , N))
X = np.zeros((samplepaths , N))
X[:,0] = X0
s[:,0] = 0.001
for j in range(samplepaths):i = 0
while X[j,i] > 0 and i < (N-1):U1 = np.random.rand()
U2 = np.random.rand()
h = - np.log(U1)/((b+d)*X[j,i])
s[j,i+1] = s[j,i] + h
if U2 < b/(b+d):X[j,i+1] = X[j,i] + 1 # birth occurs
else:X[j,i+1] = X[j,i] - 1 # death occurs
i += 1
return [s, X]
def solve_and_plot(N, paths, b, d):fig, ax = plt.subplots()
[sojourn, population] = simplebirthdeath(N, paths, b, d)
## Sets axis ranges for plotting
xmax = max([max(sojourn[k,:]) for k in range(paths)])ymax = max([max(population[k,:]) for k in range(paths)])
sojourn[sojourn==0] = np.nan
## Generates plots
for r in range(paths):plt.step(sojourn[r,:], population[r,:], where=’pre’, label="Path %s"
% str(r+1))
plt.axis([-0.1, xmax+0.2, 0, ymax+2])
ax.set_xlabel(’Time’, fontsize=14)
ax.set_ylabel(’Population Size’, fontsize=14)
plt.xticks(fontsize=14)
52
![Page 53: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/53.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
plt.yticks(fontsize=14)
plt.tight_layout()
plt.legend(loc=1)
plt.show()
maxPop = 2000
walkers = 3
birth = 1.0
death = 1.0
solve_and_plot(maxPop, walkers, birth, death)
3.4 Birth and Death Process with Migration Simulation
A birth and death process with (im)migration is called a linear growth process if the increase ofa population is determined by λi = iλ + β and the decrease of population by µi = iµ, with λ > 0,β > 0, and µ > 0. Here β represents the rate of immigration into the population and λ representsthe individual birth rate.
For this process, there are three events and to determine whether population increases or de-creases, the unit interval [0, 1] is divided into three subintervals:
• the first subinterval has probability
iλiλ + β + iµ
in which a birth occurs,
• the first subinterval has probability probability
iµiλ + β + iµ
in which a death occurs, and
• the third subinterval has probability
β
iλ + β + iµ
in which an immigration occurs.
The endpoints of the unit interval are:
0,iλ
i(λ + µ) + β,
i(λ + µ)i(λ + µ) + β
, 1.
53
![Page 54: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/54.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
The unit interval [0, 1] having three subintervals with each probability and four end points isdepicted in the following figure:
If a generated random number, say U2, is less than iλ/(i(λ+µ) +β), then the random number liesin the first subinterval and a birth occurs. If U2 ≥ iλ/(i(λ+µ)+β) and U2 < i(λ+µ)/(i(λ+µ)+β),the random number lies in the second subinterval and a death occurs. Otherwise, if U2 ≥ i(λ +
µ)/(i(λ + µ) + β), the random number lies in the third subinterval and an immigration occurs.
In general, if there are k events with rates αk(n), k = 1, 2, 3, · · · , where n is the current state, theunit interval must be divided into k subintervals (and k + 1 endpoints). The probabilities of theseevents are
αk(n)α(n)
, α(n) =∑
k
αk(n) , k = 1, 2, 3, · · ·
The python code for the simple birth and death process with immigration is as follows
import numpy as npimport matplotlib.pyplot as plt
np.random.seed(10)
def birthdeathmigration(b, d, c, paths):X0 = 20 # initial population size
N = 200 # maximal population size
# Pre-allocation and initialization
s = np.zeros((paths, N))
X = np.zeros((paths, N))
X[:,0] = X0
s[:,0] = 0.00
for j in range(paths):i = 0
while X[j,i] >= 0 and i < (N-1):U1 = np.random.rand()
U2 = np.random.rand()
h = - np.log(U1)/((b+d)*X[j,i]+c)
s[j,i+1] = s[j,i] + h
54
![Page 55: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/55.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
if U2 < b*X[j,i]/((b+d)*X[j,i]+c):X[j,i+1] = X[j,i] + 1 # a birth occurs
elif U2 >= b*X[j,i]/((b+d)*X[j,i]+c)\and U2 < (b+d)*X[j,i]/((b+d)*X[j,i]+c):
X[j,i+1] = X[j,i] - 1 # a death occurs
elif U2 >= (b+d)*X[j,i]/((b+d)*X[j,i]+c):X[j,i+1] = X[j,i] + 1 # immigration occurs
i += 1
return [X, s]
def solve_and_plot(b, d, c, paths):fig, ax = plt.subplots()
[population , sojourn] = birthdeathmigration(b, d, c, paths)
## Sets axis ranges for plotting
xmax = max([max(sojourn[k,:]) for k in range(paths)])
## Generates plots
for r in range(paths):plt.step(sojourn[r,:], population[r,:],
where=’pre’, label="Path %s" % str(r+1))
plt.axis([-0.2, xmax+0.2, -2, 32]) #ymax+2])
ax.set_xlabel(’Time’, fontsize=14)
ax.set_ylabel(’Population Size’, fontsize=14)
plt.text(0.5, 0.9, ’$\\beta = %s$’ % c, fontsize=14,horizontalalignment=’center’,
verticalalignment=’center’,
transform=ax.transAxes)
plt.xticks(fontsize=14)
plt.yticks(fontsize=14)
plt.tight_layout()
plt.legend(loc=1)
plt.grid(True)
plt.show()
## Uncomment to save the figure
#fig.savefig(’Figures/bd_mig_1p0.png’, dpi=600)
b = 0.5 # birth rate
d = 1.0 # death rate
c = 1.0 # immigration rate
walkers = 4
solve_and_plot(b, d, c, walkers)
55
![Page 56: Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General](https://reader034.vdocuments.us/reader034/viewer/2022050420/5f8f871b1cf428558d4f915a/html5/thumbnails/56.jpg)
Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html
References
[1] Linda J. S. Allen. An Introduction to Stochastic Processes with Applications to Biology2nd Edition. CRC Press, 2010.
[2] Paul G. Hoel, Sidney C. Port, Charles J. Stone. Introduction to Stochastic Processes.Houghton Mifflin Company, 1972.
[3] Samuel Karlin, Howard M. Taylor. A First Course in Stochastic Processes. AcademicPress, 1975.
[4] Samuel Karlin, Howard M. Taylor. A Second Course in Stochastic Processes. AcademicPress, 1981.
[5] Gregory F. Lawler. Introduction to Stochastic Processes 2nd Edition. Chapman & Hall,2006.
[6] Mario Lefebvre. Applied Stochastic Processes. Springer, 2007.
[7] Sheldon M. Ross. Introduction to Probability Models 9th Edition. Elsevier, 2007.
[8] Sheldon M. Ross. Stochastic Processes 2nd Edition. John Wiley & Sons Inc., 1996.
[9] Howard M. Taylor, Samuel Karlin. An Introduction to Stochastic Modeling 3rd Edition.Academic Press, 1998.
[10] Henry C. Tuckwell. Elementary Applications of Probability Theory 2nd Edition. Springer-Science+Business Media, BV, 1995.
[11] Law of Total Probability,https://en.wikipedia.org/wiki/Law of total probability
[12] Point Process,https://en.wikipedia.org/wiki/Point process
[13] Counting Processeshttps://www.probabilitycourse.com/chapter11/11 1 1 counting processes.php
56