1 transient dynamics of reduced order models of genetic regulatory networks · 2011-12-20 · 1...
TRANSCRIPT
1
Transient dynamics of reduced order
models of genetic regulatory networksRanadip Pal, Member, IEEE, Sonal Bhattacharya,
Abstract
In systems biology, a number of detailed genetic regulatory networks models have been
proposed that are capable of modeling the fine-scale dynamics of gene expression. However,
limitations on the type and sampling frequency of experimental data often prevent the parameter
estimation of the detailed models. Furthermore, the high computational complexity involved in the
simulation of a detailed model restricts its use. In such a scenario, reduced order models capturing
the coarse scale behavior of the network are frequently applied. In this paper, we analyze the
dynamics of a reduced order Markov Chain model approximating a detailed Stochastic Master
Equation model. Utilizing a reduction mapping that maintains the aggregated steady state probability
distribution of stochastic master equation models, we provide bounds on the deviation of the Markov
Chain transient distribution from the transient aggregated distributions of the stochastic master
equation model.
Index Terms
Genetic Regulatory Network Modeling Robustness, Transient Analysis, Markov Chains
✦
1 INTRODUCTION
Conceptualization of biological regulation as detailed mathematical system models dates back several
decades [1]. The recent technological advances in the measurement of the regulome (genes, proteins
and metabolites involved in gene regulation) have provideda significant impetus to modeling in systems
biology. However, we are still not at the stage where the enormous complexity of a biological system can
be represented by a single detailed composite model explaining all the interactions between RNAs, Protein
• R. Pal and S. Bhattacharya is with the Department of Electrical and Computer Engineering, Texas Tech University, Lubbock,
TX, 79409, USA.
E-mail: [email protected], [email protected]
• This research was supported by NSF Grant CCF0953366.
December 20, 2011 DRAFT
2
and Metabolites. Furthermore, a conceived model capturingall the interactions will be extremely hard to
validate due to existing limitations on expression measurements. Usually only parts of the regulome are
made explicit in a mathematical model. The complexity of regulation in a single level (transcriptome,
proteome or metabolome) can in itself be enormous and it is often not feasible to reliable estimate the
parameters of a detailed model of a single level from experimental data.
Stochastic Master Equation (SME) models are a form of detailed stochastic genetic regulatory net-
work (GRN) models that are presumed to provide a fine scale description of the underlying genomic
regulation process [2], [3], [4]. The inference of the parameters of an SME model preferably require
cell specific measurements whereas most of the available datasets on gene expression measurements are
based on microarray type approaches that provide population averaged data. Availability of cell specific
measurements based on green fluorescent protein type approaches is still extremely limited due to higher
cost and complexity as compared to existing approaches. Theinference of SME model parameters also
lacks preciseness due to low sampling rates in biological experiments and the extrinsic noise in data
extraction. Furthermore, the use of SME type models for representing GRNs are also limited by the
enormous complexity involved in its simulation and reliance on monte-carlo type approaches for design
of any intervention approaches based on such models. To provide an estimate of the complexity of an
SME model, let us consider that we are trying to model the interactions between 10 mRNAs using
an SME model and the number of mRNA molecules that can be created for each mRNA is between
0 and 99. The SME model can be considered as a continuous time Markov Chain with any possible
combination of the number of RNA molecules being a state. Thus, we will have a10010 = 1020 state
continuous time Markov chain with enormous complexity involved in its simulation. In such a scenario,
reduced order models are often used. One of the ways to reducethe complexity of a SME model is
to consider a model that captures moments (expectations or variances) of the distribution. Deterministic
differential equation models are often used to model the average behavior of a stochastic system [5],
[6], [7], [8], [9]. On the other hand, we can also consider a coarse-scale stochastic model to capture
our dynamic behavior. For our earlier example, if we binarize the expression levels of the mRNAs, then
we have to work with only210 = 1024 state Markov Chain. But the natural question arises: whether
such a coarse model is useful? The answer is in-fact yes as fora number of systems medicine scenarios
where we are trying to predict the onset of a genetic disease or effect of application of a drug or gene
silencing, we are often able to predict the phenotypes basedon binarized or ternarized expressions of
molecular markers [10], [11], [12]. In this article, we willconsider a coarse-scale Markov Chain (CSM)
to represent the reduced-order model and an SME model to represent the detailed model. We will use a
mapping that was recently introduced [13] to map the detailed model to the coarse-model. The mapping
was shown to maintain the aggregated steady state probability distribution of the detailed model and
the transition probabilities of the reduced order model canbe independently estimated from coarse-scale
December 20, 2011 DRAFT
3
steady state data [13]. However, the bounds on the deviationof the transient dynamics (whether the
transient response of the reduced order models is comparable to the coarse-scale transient response of
the finer models) was not explored. The transient dynamics can be highly significant if our purpose is
intervention for systems medicine and we want to maintain the probability of a state within specific
limits. If our design is based on the approximate model, we are aware of the state probabilities of the
approximate model but we would be interested in bounding thedeviation of the state probabilities of
the coarse-model from the aggregated state probabilities of the detailed SME model.
In this paper, we analyze and provide bounds for the difference in the transient behavior of the CSM
and SME models. We also consider the bound on the difference in state probabilities for the CSM and
aggregated SME models when a stationary control policy designed on the coarse-model is applied to the
CSM and SME models. We also derive stochastic bounds for the deviations in the transient dynamics.
The stochastic bounds are computationally inexpensive to compute and they require partial distribution
information of the entries of the transition probability matrix for their calculation.
The paper is organized as follows. The background to the problem including a review of SME, CSM
models and the mapping from SME to CSM is presented in Section2. Section 3 characterizes the
deviations for single and multiple time steps and provides the corresponding upper bounds. Section 4
provides stochastic bounds for the maximum deviation. Transient analysis under the action of stationary
control is provided in Section 5. Simulation results are included in Section 6 and conclusions are provided
in Section 7.
2 BACKGROUND
2.1 SME Models
To explain a SME model, we will consider a system withn molecular species andm different reaction
channels where the state of the system is defined byθ = [χ1, ..., χn] , θ ∈ Nn is a vector of integers
representing a specific population of each of then molecular species. For such a system, given the
probability density vectorp(θ, t) at time t, we can derive the differential equation [4]
p(θ; t) = −p(θ; t)m∑
j=1
aj(θ) +m∑
j=1
p(θ − vj ; t)aj(θ − vj) (1)
whereaj(θ)dt denotes the probability that thejth reaction will happen in a time step of lengthdt and
vj is the stoichiometric transition vector.
By considering all the reactions beginning or ending at state θ, the time derivative of the prob-
ability density of stateθ can be written in the form [14]:P(Θ; t) = P(Θ; t)A whereP(Θ; t) =
[p(θ1, t), p(θ2, t), · · · ] is the complete probability density state vector at timet andA is the state reaction
matrix. For the case of finite number of reachable states, theexact solution to the SME can be computed
December 20, 2011 DRAFT
4
asP(Θ; t) = P(Θ, 0)eAt [14]. This kind of situation can arise in biological systemswhere the number
of mRNA/Protein molecules that can be generated are bounded. If for systems, the dimension ofΘ is
infinite, we can apply the Finite State Projection approach [14] to arrive at a finite truncation of the state
space.
2.2 Coarse-scale Markov Chain Models
Coarse-scale deterministic models with synchronous timing have been used since the late 1960s in the
form of the Boolean network model [15]. In this model, gene expression is quantized to two levels:
ON and OFF denoted by 1 and 0, respectively. The expression level (state) of a gene is functionally
related via a logical rule to the expression states of other genes. Specifically, a Boolean network (BN)
is composed of a setV = {ν1, ν2, · · · , νn} consisting ofn binary variables, each denoting a gene
expression, and a setF = {f1, f2, · · · , fn} of regulatory functions, such that for discrete time,t =
0, 1, 2, · · · , νk(t+1) = fk(ν1(t), · · · , νn(t)). At any time point, the state of the network is given by an
expression vector(ν1(t), · · · , νn(t)), called the gene activity profile (GAP). However the assumption of
a single transition rule for each gene can be problematic dueto inherent stochasticity of gene expression,
presence of latent variables and noise in the experimental data used for inference. Thus, a probabilistic
Boolean network (PBN) [16] was proposed. The probabilisticstructure of the PBN can be modeled as
a Markov chain. Relative to their Markovian structure, PBNsare related to Bayesian networks [17];
specifically, the transition probability structure of a PBNcan be represented as a dynamic Bayesian
network and every dynamic Bayesian network can be represented as a PBN (actually, a class of PBNs)
[18]. As the reduced order model in this paper, we will consider coarse scale Markov Chain (CSM) which
includes the class of PBNs. The CSM model provides the transition probabilities between states in the
next discrete time duration. The primary difference from the SME model is the significant small number
of states considered and the discrete time dynamics. The state aggregations are based on thresholds
used for binarization. As an example, if we consider an SME model of 2 genes and each producing a
maximum of39 mRNA molecules, then the SME model will have40× 40 = 1600 states. For the CSM
model, if we use a binarization threshold of20, then a state like [30 11] in the SME model will be
mapped to state [1 0] in the CSM model. The CSM model will have atotal of 2× 2 = 4 states.
2.3 SME and its CSM approximation
Based on the state transitions of a SME model, it can be considered as a continuous time Markov Chain
with a huge number of states. To explore mappings from fine-scale SME models to CSM models, let
us considerM andN to denote the number of states of the SME and CSM model respectively. We
consider a sequence 0 =a0, a1, ..., aN = M such thataj−aj−1 for j ∈ {1, 2, ..., 2n} denote the number
of states in the SME model that map to statej in the CSM model. Note that the number of states in
December 20, 2011 DRAFT
5
the SME model that map to statej in the CSM model can vary withj depending on the thresholds
used for binarization. For instance, if we consider our previous example of 2 genes with 1600 states and
change the binarization thresholds to 10 rather than 20, then 10 × 10 = 100 states in the SME model
will be mapped to state[00] in the CSM model and10 × 30 = 300 states in the SME model will be
mapped to state[01] in the CSM model. LetP∆ = eA∆t represent theM dimensional discrete time
SME model (∆t is a suitable time period) andPr represent the reducedN dimensional CSM model.
For the SME model, letη represent theM dimensional steady state probability vector andζ represent
the N dimensional collapsed steady state vector. Here, collapsing refers to aggregation of states i.e.
ζ(i) =∑ai
i2=ai−1+1 η(i2) for i = 1, ..., N . In [13], a mapping from SME to CSM model is presented
where the steady state probability distribution of the reduced modelPr represented by Eq. 2 is equivalent
to the collapsed steady state probability distribution ofP∆.
Pr(i, j) =
∑aj
j1=aj−1+1
∑ai
i1=ai−1+1 P∆(i1, j1)η(i1)∑ai
i2=ai−1+1 η(i2)(2)
With Eq. 2 denoting the state transition probabilities of the CSM model, the steady state probability
distribution vector,π, of the CSM model is given by [13]:π(i) =∑ai
i2=ai−1+1 η(i2) = ζ(i) for i =
1, ..., N . Eq. 2 provides a mapping from a fine-scale SME model to a CSM model based on aggregation
of states that maintains the collapsed steady state distribution of the detailed model. If we revisit Eq.
2, we notice thatPr(i, j) can be calculated from the transitions of the network once ithas reached the
steady state. Thus, the mapping described by Eq. 2 exists between a network represented by a SME
model and a CSM model when the transition probabilities of the CSM are inferred based on coarse-scale
state transition data at steady-state.
3 MAXIMUM DEVIATION FOR SINGLE TIME STEP
The focus of this paper is analyzing the deviation of the state probability distribution of the CSM model
from the aggregated state probability distribution of the SME model during the transient phase. Let
x(t) = [x1(t), x2(t), ..., xM (t)] 1 denote the SME model’s probability distribution at any timet. Let
γj(t) denote the collapsed probability of statesaj−1 + 1 to aj of the SME model at any timet i.e.
γj(t) =∑aj
i1=aj−1+1 xi1(t). Let τj(t) denote the probability of statej at any timet for the CSM model.
We are interested in finding the difference at time instantt +∆t between the collapsed probability of
the states in the SME model that map to statej of the CSM model (i.e.γj(t+∆t)) and the probability
of statej in the CSM model (i.e.τj(t+∆t)) . Let us define the difference equation as
dj(t+∆t) = |γj(t+∆t)− τj(t+∆t)|. (3)
1. Note thatx(t) representsP (Θ; t) and [x1(t), x2(t), ..., xM (t)] = [p(θ1, t), p(θ2, t), · · · , p(θM , t)]
December 20, 2011 DRAFT
6
Theorem 3.1. The maximum deviation betweenγj(t+∆t) and τj(t+∆t) starting fromγj(t) = τj(t)
is given by the following equation:
dj(t+∆t) ≤ maxi1∈[1,...,N ]
(
maxi2∈Si1
q(i2, j)− mini2∈Si1
q(i2, j)
)
(4)
whereSi = [ai−1 + 1, ..., ai] and q(i2, j) =∑aj
j1=aj−1+1 P∆(i2, j1) for i2 ∈ [1, 2, ...,M ].
Proof: The value of the aggregated states of the SME model mapping toCSM statej at any time
t+∆t is given by:
γj(t+∆t) =
aj∑
j1=aj−1+1
M∑
i1=1
xi1 (t)P∆(i1, j1) (5)
The value of thejth state of the CSM model at any timet+∆t is given by:
τj(t+∆t) =
N∑
i2=1
τi2 (t)Pr(i2, j)
=
N∑
i2=1
τi2(t)
∑aj
j2=aj−1+1
∑ai2
i3=ai2−1+1 P∆(i3, j2)η(i3)∑ai2
i4=ai2−1+1 η(i4)
=N∑
i2=1
∑aj
j2=aj−1+1
∑ai2
i3=ai2−1+1 P∆(i3, j2)η(i3)τi2 (t)
ζ(i2)(6)
Assumingγj(t) = τj(t), dj(t +∆t) represents the difference betweenγj and τj during time stept to
t+∆t starting from equal distribution and is given by the following equation:
dj(t+∆t) = |γj(t+∆t)− τj(t+∆t)|
=
∣
∣
∣
∣
∣
∣
aj∑
j1=aj−1+1
M∑
i1=1
xi1 (t)P∆(i1, j1)−N∑
i2=1
∑aj
j2=aj−1+1
∑ai2
i3=ai2−1+1 P∆(i3, j2)η(i3)τi2(t)
ζ(i2)
∣
∣
∣
∣
∣
∣
=
∣
∣
∣
∣
∣
∣
aj∑
j1=aj−1+1
N∑
i1=1
ai1∑
i2=ai1−1+1
xi2 (t)P∆(i2, j1)−∑ai1
i3=ai1−1+1 P∆(i3, j1)η(i3)τi1 (t)
ζ(i1)
∣
∣
∣
∣
∣
∣
=
∣
∣
∣
∣
∣
∣
N∑
i1=1
∑
i2∈Si1
xi2 (t)ζ(i1)− τi1(t)η(i2)
ζ(i1)
∑
j1∈Sj
P∆(i2, j1))
∣
∣
∣
∣
∣
∣
(7)
whereSi = [ai−1 + 1, ..., ai]. Let q(i2, j) denote∑aj
j1=aj−1+1 P∆(i2, j1), then:
∑
i2∈Si1
xi2(t)ζ(i1)q(i2, j) ≤ γi1(t)ζ(i1) maxi2∈Si1
q(i2, j), (8)
∑
i2∈Si1
τi1(t)η(i2)q(i2, j) ≥ τi1 (t)ζ(i1) mini2∈Si1
q(i2, j) (9)
Substituting equations 8 and 9 into Eq. 7, and usingγj(t) = τj(t), the upper bound for the difference
December 20, 2011 DRAFT
7
equation is given by:
dj(t+∆t) ≤∣
∣
∣
∣
∣
N∑
i1=1
γi1(t)ζ(i1)maxi2∈Si1q(i2, j)− τi1(t)ζ(i1)mini2∈Si2
q(i2, j)
ζ(i1)
∣
∣
∣
∣
∣
≤N∑
i1=1
γi1(t)
(
maxi2∈Si1
q(i2, j)− mini2∈Si1
q(i2, j)
)
≤ maxi1∈[1,...,N ]
(
maxi2∈Si1
q(i2, j)− mini2∈Si1
q(i2, j)
)
(10)
Thus, theorem 3.1 provides an upper bound for the differenceequationdj for a single time step∆t
in terms of the row sumsq(i2, j) of entries of the state transition probability matrixP∆ . The bounds
on the deviation for multiple time steps is presented next.
3.1 Maximum Deviation for Multiple Time Steps
Theorem 3.2. The maximum deviation betweenγj(T∆t) and τj(T∆t) starting fromγj(0) = τj(0) is
given by the following equation:
dj(T∆t) ≤ β(1− µm
T
1− µm
) (11)
whereβ = maxj∈[1,··· ,N ] βj andβj = maxi1∈[1,...,N ]
(
maxi2∈Si1q(i2, j)−mini2∈Si1
q(i2, j))
is the
single time step upper bound derived in Theorem 3.1 andµm = maxj∈[1,··· ,N ](∑N
i1=1 |R(i1, j) − ρ|)whereρ is any constant andR(i1, j) = mini2∈[ai1−1+1,...,ai1 ]
q(i2, j).
Proof: Equation 4 bounds the deviation for a single time step∆t assuming the distributions
(collapsed probability distribution of the SME model and the probability distribution of the CSM
model) at timet to be equal. To arrive at the deviation of the transient probability distribution at time
t + ∆t without the equality assumption at timet, let us denoteτj(t) as τj(t) = γj(t) + φj(t) where∑N
j=1 φj(t) = 0. From Equations 7, 8 and 9, we have
dj(t+∆t) ≤∣
∣
∣
∣
∣
N∑
i1=1
γi1(t)ζ(ii)maxi2∈Si1q(i2, j)− τi1(t)ζ(ii)mini2∈Si2
q(i2, j)
ζ(ii)
∣
∣
∣
∣
∣
≤∣
∣
∣
∣
∣
N∑
i1=1
γi1(t)
(
maxi2∈Si1
q(i2, j)− mini2∈Si1
q(i2, j)
)
∣
∣
∣
∣
∣
+
∣
∣
∣
∣
∣
N∑
i1=1
(τi1(t)− γi1(t)) mini2∈Si1
q(i2, j)
∣
∣
∣
∣
∣
(12)
whereSi1 = [ai1−1 + 1, ..., ai1 ].
December 20, 2011 DRAFT
8
Let us denoteR(i1, j) = mini2∈Si1q(i2, j) for i1, j ∈ S whereS = [1, ..., N ], . For any constantρ,
∑Ni1=1 ρφi1(t) = ρ× 0 = 0. Then,
dj(t+∆t) ≤ maxi1∈S
(
maxi2∈Si1
q(i2, j)− mini2∈Si1
q(i2, j)
)
+ |N∑
i1=1
(R(i1, j)− ρ)φi1(t)|
≤ maxi1∈S
(
maxi2∈Si1
q(i2, j)− mini2∈Si1
q(i2, j)
)
+ (
N∑
i1=1
|R(i1, j)− ρ|)maxi1∈S
|φi1(t)| (13)
If we denote the summation∑N
i1=1 |R(i1, j) − ρ| by µ(j), and maxj∈S µ(j) by µm and Φ(t) =
maxi1∈S |φi1 (t)|, then the deviation for statej at t+∆t is upper bounded byβj + µmΦt whereβj is
the maximum deviation in one time step starting from the samedistribution. Furthermore,Φt denotes
the maximum deviation among all the states a timet and henceΦt ≤ β + µmΦt−∆t. Since at time
t = 0, Φ0 = 0 as τj(0) = γj(0) for j ∈ [1, ..., N ], the difference afterT∆t time can be bounded by
dj(T∆t) ≤ βj + µmΦ(T−1)∆t
≤ βj + µm(β + µmΦ(T−2)∆t)
≤ β(1 + µm + µ2m · · ·+ µT−1
m ) + µTmΦ0
≤ β(1− µm
T
1− µm
) (14)
4 STOCHASTIC BOUNDS
The calculation of the upper bound shown in Eq. 4 is specific toeach individualP∆ matrix and is not
straightforward. It involves generation of the entries of theP∆ matrix which has enormous computational
complexity for large M. Furthermore, our purpose is to use a coarse model when data is limited and
requirement on the knowledge of the fine scale probability transition matrix to upper bound our deviation
will defeat our purpose. Thus we would be interested in arriving at an estimate of the bound by exploiting
possible structures in the row sumsq(i, j) without the inference of the individual entries of theP∆
matrix. We will consider derivation of stochastic bounds assuming various distributions of the entries
of the transition probability matrix. If the structure of our SME model provides an estimate on the
distribution of the entries ofP∆, the stochastic bounds will provide us the maximum expecteddeviation.
To estimate the bound in Eq. 4, we need to estimate the range ofthe sum ofLj consecutive entries of
a row of a transition probability matrix whereLj representsaj − aj−1 for j ∈ [1, · · · , N ].
To approach this problem, we will first derive the distribution of the sum ofLj consecutive entries of
a row of a transition probability matrix based on the assumption of uniform distribution and distribution
centered around different means, for the entries ofP∆. Since the aggregation is over a large number of
states (i.e.Lj is large), the distribution of the sum ofLj entries (i.e. distribution ofq(i, j) ) will tend
December 20, 2011 DRAFT
9
towards a Gaussian distribution based on Central Limit Theorem. Even though, we consider only two
cases of uniform distribution and distribution centered around different means, for largeLj , we can get
an idea of the distribution ofq(i, j) based on the knowledge of the mean and variance of the distribution
of individual entries. Once, we derive the distribution ofq(i, j) in section 4.1, we calculate the bounds
on the expectation of the range ofLj random variables selected from the derived distribution ofq(i, j) in
section 4.2. This provides an estimate for the expected value of |maxi2∈Si1q(i2, j)−mini2∈Si1
q(i2, j)|which is the primary term in the bound derived in Eq. 4.
4.1 Distribution of the sum of L entries of a row of the probability transition matrix
We will consider two cases: firstly, we will assume that all the entries of the probability transition matrix
are independent and identically distributed belonging to auniform distribution. In the second case, we
will assume that the value of the diagonal entries ofP∆ are much higher than the other entries and the
non-diagonal entries are uniform, independent and identically distributed.
4.1.1 Uniform Distribution
If we consider generating a row of a probability transition matrix by selectingM random entries from
a uniform distribution of range[0, 1] and dividing each entry by the sum of theM selected entries, then
the expected value of each entry will be1/M and we can approximate the distribution of an entry in
the probability transition matrix by a uniform distribution with range[0, 2/M ]. The expectation of the
random variableZ is E(Z) = 1M
and varianceσ2 = 13M2 .
Let us consider the summation ofL entries of a row of a probability transition matrix where each
entry has distributionfZ . Then the random variableX = Z1+Z2+ · · ·+ZL has expectationLµ = L/M
and varianceLσ2 = L3M2 assumingZi’s are independent fori = 1, · · · , L.
Based on central limit theorem [19], for sufficiently largeL, the distribution ofX will approach
N ( LM, L3M2 ) ( Henceforth,N (µ, σ2) will denote a Normal distribution with meanµ and varianceσ2).
We should note that when we are reducing from a fine scale modelto a coarse scale model, the value
of L which represents the number of states of the finer model mapping to a single state of the coarser
model is usually large.
4.1.2 Distribution centered around different means
Let us consider the case whereP∆ = IM + C where IM is the M × M identity matrix and the
individual entries ofC are assumed to be small and bounded by|C(i, j)| ≤ ε for i, j ∈ [1, 2, · · · ,M ],
i 6= j and Mε/2 << 1. The mean of the distribution of the summation∑aj
j1=aj−1+1 P∆(i1, j1) for
i1 ∈ [ai−1 + 1, · · · , ai] will be centered around1 − (M − L) ε2 when i = j otherwise centered around
Lε/2 assuming a uniform distribution between0 and ε for P∆(i, j) for i 6= j. The variance of the
December 20, 2011 DRAFT
10
distribution of the summation∑aj
j1=aj−1+1 P∆(i1, j1) for i1 ∈ [ai−1 + 1, · · · , ai] is Lε2/12 for i 6= j.
For i = j,∑aj
j1=aj−1+1 P∆(i1, j1) = 1−∑
j2∈S2P∆(i1, j2) whereS2 = [1, 2, ...aj−1, aj +1, ...M ] and
thus the variance is(M − L)ε2/12 assuming theM − L to be independent. Therefore, distribution of
the summation∑aj
j1=aj−1+1 P∆(i1, j1) for i1 ∈ [ai−1 + 1, · · · , ai] can be approximated by
N (Lε2 , Lε2
12 ) if i 6= j
N (2−(M−L)ε2 , (M−L)ε2
12 ) if i = j
4.2 Bounds on the Expectation of the range of L random variables
To estimate the bound in Eq. 4, we need to estimate the range ofthe row sums of aL×L block of the
probability transition matrixP∆.
Let us considerL identically distributed random variablesX1, X2, · · ·XL with a common cumulative
distribution functionF , probability distribution functionf , meanµ and varianceσ2. The derivation in
section 4.1 is used to generateF, f, µ andσ. Let ML andmL denotes the maximum and minimum of
X1, X2, · · ·XL respectively. Based on the results of [20], we can arrive at the following theorem:
Theorem 4.1. The upper bound on the expected range is given by the follwingequation:
E(ML)− E(mL) ≤ rL − r1 + L
∫ ∞
rL
[1− F (x)]dx + L
∫ r1
−∞F (x)dx (15)
whererL = F−1(1− 1L) and r1 = F−1( 1
L).
Proof:
For any real numberr, we can show that [20]:
ML ≤ r +
L∑
i=1
(Xi − r)+. (16)
where(x)+ = x for x > 0 and (x)+ = 0 for x ≤ 0. Taking expectations on both sides, we have
E(ML) ≤ r +
L∑
i=1
E((Xi − r)+). (17)
We know that,
E((Xi − r)+) =
∫ ∞
r
(x − r)f(x)dx =
∫ ∞
r
[1− F (x)]dx (18)
Thus,
E(ML) ≤ r + L
∫ ∞
r
[1− F (x)]dx. (19)
To find the minimum of the functionh(r) = r + L∫∞r
[1− F (x)]dx, we equate
dh(r)
dr= 1− L[1− F (rmin)] = 0
=⇒ F (rmin) = 1− 1
L
December 20, 2011 DRAFT
11
Thush(r) is minimized atr = rL where
rL = F−1(1 − 1
L) (20)
andF−1(t) = inf(x|F (x) ≥ t) for 0 ≤ t ≤ 1. Thus,
E(ML) ≤ rL + L
∫ ∞
rL
[1− F (x)]dx. (21)
Similarly for the minimum, we have
E(mL) ≥ r1 − L
∫ r1
−∞F (x)dx. (22)
wherer1 = F−1( 1L).
Thus
E(ML)− E(mL) ≤ rL − r1 + L
∫ ∞
rL
[1− F (x)]dx + L
∫ r1
−∞F (x)dx (23)
4.3 Expected bound for single time step
Theorem 4.2. The expected bound on the deviationβj for single time step is given by the following
equation:
E(βj) < 2σ(2 logL− log logL)12 (24)
whereσ denotes the standard deviation for the distribution ofq(i, j) andL = maxi=1,··· ,N |ai− ai−1|.
Proof: As discussed in section 4.1, the distribution ofq(i, j) can be approximated by a Gaussian
distribution. For distributions such as Gaussian that are symmetric around the meanµ, we haveF (µ−x)+F (µ+x) = 1. Thus,r1 = F−1( 1
L) = 2µ−F−1(1− 1
L) = 2µ− rL and equation 23 can be written
as
E(ML)− E(mL) ≤ 2rL − 2µ+ 2L
∫ ∞
rL
[1− F (x)]dx (25)
The upper bound onrL + L∫∞rL
[1− F (x)]dx for standard normal distribution is given by [20]:
rL + L
∫ ∞
rL
[1− F (x)]dx < (2 logL− log logL)12 (26)
Thus for normal distribution with meanµ and standard deviationσ
E(βj) = E(ML)− E(mL) < 2σ(2 logL− log logL)12 (27)
As an example of the bound, let us consider the case of selection of transition probabilities from
uniform distribution as considered in section 4.1.1. We have σ = 1M
√
L3 , thus
December 20, 2011 DRAFT
12
E(ML)− E(mL) < 21
M
√
L
3(2 logL− log logL)
12 (28)
Numerically, forM = 3200 andL = 100 we haveE(ML)− E(mL) < 0.01.
For the case of unequal means in section 4.1.2, the maximum ofE(ML)−E(mL) will be achieved
for i = j and is given by:
E(ML)− E(mL) < 2
√
(M − L)ε2
12(2 logL− log logL)
12 (29)
4.4 Expected bound for multiple time steps
We will approximate the bound for multiple time steps by plugging in the expected values forβj andµm
in equation 11. The bound on the expected deviationE(βj) is given by theorem 4.2. In this subsection,
we will consider generating a stochastic upper-bound forµm = maxj∈S(∑N
i=1 |R(i, j) − ρ|). Let us
consider the case that the diagonal entries are much bigger than the other entries which is mostly true
for small ∆t. Since, eachR(i, j) is summation of L entries where L is large, we can approximatethe
distribution ofR(i, j) by a Gaussian distribution. Let the distribution ofR(i, j) for i 6= j be approximated
by N (ξ1, σ21) and the distribution ofR(i, j) for i = j be approximated byN (ξ2, σ
22).
The following theorem provides an upper bound on the expected value ofµm.
Theorem 4.3.
E(µm) ≤ (N − 1)
√
2
πσ1 + (ξ2 − ξ1)erf(
ξ2 − ξ1√2σ2
) +
√
2
πσ2e
−(ξ2−ξ1) (30)
whereerf(x) = 2√π
∫ x
0 e−t2 dt denotes the error function.
Proof: We haveµm = maxj∈[1,··· ,N ](∑N
i1=1 |R(i1, j) − ρ|) whereρ is any constant.R(i, j) has
distributionN (ξ1, σ21) for i 6= j and has distributionN (ξ2, σ
22) for i = j . For the following proof, we
will select ρ = ξ1 and assumeσ2 > σ1.
For i 6= j, we have
E(|R(i, j)− ρ|) =
∫ 1
0
|x− ρ| 1√2πσ1
e− (x−ξ1)2
2σ21 dx
≤∫ ∞
−∞|x− ρ| 1√
2πσ1
e− (x−ξ1)2
2σ21 dx
= 2
∫ ∞
0
u1√2πσ1
e− u2
2σ21 du
=
√
2
πσ1 (31)
December 20, 2011 DRAFT
13
Similarly for i = j , we have
E(|R(i, j)− ρ|) =
∫ 1
0
|x− ρ| 1√2πσ2
e− (x−ξ2)2
2σ22 dx
≤∫ ∞
−∞|x− ρ| 1√
2πσ2
e− (x−ξ2)2
2σ22 dx
≤∫ ∞
−∞|u+ ξ2 − ξ1|
1√2πσ2
e− u2
2σ22 du
≤∫ ξ1−ξ2
−∞−(u+ ξ2 − ξ1)
1√2πσ2
e− u2
2σ22 du +
∫ +∞
ξ1−ξ2
(u+ ξ2 − ξ1)1√2πσ2
e− u2
2σ22 du
≤ 2
∫ ξ2−ξ1
0
(ξ2 − ξ1)1√2πσ2
e− u2
2σ22 du + 2
∫ +∞
ξ2−ξ1
u1√2πσ2
e− u2
2σ22 du
≤ (ξ2 − ξ1)erf(ξ2 − ξ1√
2σ2
) +
√
2
πσ2e
−(ξ2−ξ1) (32)
Thus, combining the (N-1) cases ofi 6= j and single case ofi = j over the∑N
i=1 |R(i, j)− ρ|, we have
E(N∑
i=1
|R(i, j)− ρ|) ≤ (N − 1)
√
2
πσ1 + (ξ2 − ξ1)erf(
ξ2 − ξ1√2σ2
) +
√
2
πσ2e
−(ξ2−ξ1)
(33)
Since, we are assuming the same distribution for allj’s,
E(µm) ≤ (N − 1)
√
2
πσ1 + (ξ2 − ξ1)erf(
ξ2 − ξ1√2σ2
) +
√
2
πσ2e
−(ξ2−ξ1)
(34)
5 EFFECT ON THE TRANSIENT BEHAVIOR DURING APPLICATION OF STATION-
ARY CONTROL POLICY
5.1 Background
The motivation behind application of control theory is to devise optimal policies for manipulating
control variables that affect the transition probabilities of the network and can, therefore, be used to
desirably affect its dynamic evolution. In practice, intervention can be achieved by (a)targeted small
molecule kinase inhibitors (Imatinib, Gefitinib, Erlotinib, Sorafenib, Sunitinib etc.[21] ) (b) Monoclonal
antibodies altering the protein concentrations (Cetuximab, Alemtuzumab, Trastuzumab etc.[21]) or (c)
gene knockdowns[22] . The state desirability is determined by the values of genes/proteins associated
with phenotypes of interest. In this section, we will consider infinite-horizon control with the goal of
favorably altering the steady-state distribution of the network via a stationary control policy [11].
To explain infinite-horizon intervention in CSM, we next provide a brief mathematical description of
the control problem. A CSM with control can be modeled as a finite state Markov Chain [23], [11]
December 20, 2011 DRAFT
14
described by the control-dependent one-step transition probability pij(u) := P (zt+1 = j|zt = i, ut = u)
where, for allt, the statezt is an element of a spaceS and the control inputut is an element of a space
C. The states make transitions according toω := (Pu)u∈C . In this case, once a control input is chosen,
the resulting controlled transition probability matrix isuniquely determined.
Let µ = (u1, u2, ....) represent a generic control policy andΠ represent the set of all possibleµ’s,
i.e., the set of all possible control policies. LetJµ,ω denote the expected total cost for the discounted
cost infinite-horizon problem [11] under control policyµ and transitionsω:
Jµ,ω(z0) = limM→∞
Ezt+1{M−1∑
t=0
αtg(zt, ut, zt+1)}, (35)
where 0 < α < 1 denotes the discount factor andg(zt, ut, zt+1) represents the cost of going from
statezt to zt+1 under the control actionut. g is higher for undesirable destination states. For the same
destination states,g is higher when the control is active versus when it is not.
The control problem here corresponds to minimizing the costin Eq. 35. Consequently, the optimal
infinite-horizon discounted cost is given by:
Φ(Π, ω, z0) := minµ∈Π
Jµ,ω(z0). (36)
The interest in infinite-horizon control is based on the consideration that such policies allow us to
optimally alter the steady-state probability distribution of the network by driving its probability mass
into desirable states. Furthermore, they result in stationary control policies (i.e. independent of time and
dependent on state) which are easier to implement.
5.2 Transient Analysis
To analyze the effect on the transient dynamics of the stationary control policy designed on the reduced
order model when applied to the detailed SME model, we will consider a case of two control actions at
each time step. In reality, the two control actions will refer to whether to use a drug or not to apply the
drug. We will represent the SME models corresponding to the two control actions asP∆1 andP∆2 . As
mentioned,P∆1 might represent the model of the network with no drug delivery andP∆2 represent the
model of the network following a drug delivery. Let,Pr1 andPr2 represent the transition probability
matrices of the reduced order CSM models generated from models P∆1 andP∆2 respectively using Eq.
2.
Since, we will design the control policy based on the reducedorder model, our control design will be
based onPr1 andPr2 . Let the goal of our control action be to alter the steady state probability distribution
of the network and we design a stationary control policy using dynamic programming approaches based
on Pr1 andPr2 by minimizing Eq. 36. The steady state distribution of a GRN can reflect the phenotype
and thus alteration of it will be one of our primary control goals. However, we would also be interested
December 20, 2011 DRAFT
15
in the state probability distribution during the transientphase so as to reduce the probabilities of being
in a highly undesirable state during the transient phase of control policy application. The application
of the designed control policy on the reduced order model canprovide an estimate of the transient
dynamics during application of the control policy. In this section, we are interested in deriving the
maximum deviation in the transient phase of the aggregated state probability distribution of the SME
model from the state probability distribution of the reduced order model after application of the control
policy designed on the reduced order model.
A stationary control policy for the case of 2 control actionsfor the CSM will be a binary vector of
lengthN where0 at locationi denotes no control action when at statei (the model of the network for
statei will then be based onPr1 ) whereas a1 at locationi denotes control action when at statei (the
model of the network for statei will then be based onPr2 ). To illustrate it further, let us consider a
simple example from [13] whereN = 2 ,
Pr1 =
∣
∣
∣
∣
∣
∣
0.2 0.8
0.65 0.35
∣
∣
∣
∣
∣
∣
Pr2 =
∣
∣
∣
∣
∣
∣
0.45 0.55
0.25 0.75
∣
∣
∣
∣
∣
∣
and stationary control policy =[0 1]. Let Prd denote the controlled CSM transition probability matrix
for a specific stationary control policy. Thus for the stationary control policy of [0 1],Prd will have
its first row fromPr1 and second row fromPr2 and the controlled modelPrd can be represented as
Prd = TPr1 + (IN − T )Pr2 whereIN represent the identity matrix of sizeN ×N andT is aN ×N
matrix with all entries zero except the diagonal entries corresponding to no control action being equal
to 1. ThePrd andT matrices for stationary control policy[0 1] is as follows:
Prd =
∣
∣
∣
∣
∣
∣
0.2 0.8
0.25 0.75
∣
∣
∣
∣
∣
∣
T =
∣
∣
∣
∣
∣
∣
1 0
0 0
∣
∣
∣
∣
∣
∣
When a control policy designed using the CSM is applied to thefine-scale stochastic network model,
the control policy corresponding to statesai−1 + 1, · · · , ai of the SME model will be the same as the
control policy for statei of the CSM. Thus, the controlled fine scale model will beP∆c= T∆P∆1 +
(IM − T∆)P∆2 whereT∆ is a M × M matrix of all zeros except the states that map to CSM states
with no control action, equal to1. For the example before, if we considerM = 4 and the states1 and
2 of the fine-scale model map to state1 of the CSM, then
T∆ =
∣
∣
∣
∣
∣
∣
∣
∣
∣
∣
∣
∣
1 0 0 0
0 1 0 0
0 0 0 0
0 0 0 0
∣
∣
∣
∣
∣
∣
∣
∣
∣
∣
∣
∣
.
For the controlled SME modelP∆c, let ηc represent theM dimensional steady state probability
vector andζc represent theN dimensional collapsed steady state probability vector. Here, collapsing
December 20, 2011 DRAFT
16
refers to aggregation of probability of states i.e.ζc(i) =∑ai
i2=ai−1+1 ηc(i2) for i = 1, ..., N . Let xc(t) =
[xc1(t), x
c2(t), ..., x
cM (t)] denote the controlled SME model’s probability distribution at any timet. Let
γcj (t) denote the collapsed probability of statesaj−1 +1 to aj of the controlled SME model at any time
t. Let τcj (t) denote the probability of statej at any timet for the controlled CSM modelPrd . We are
interested in finding the difference between the collapsed probability of the states in the controlled SME
model that map to statej of the controlled CSM model and the probability of statej in the controlled
CSM model at time instantt+∆t.
Let us define the difference equation of interest as
dcj(t+∆t) = |γcj (t+∆t)− τcj (t+∆t)|. (37)
The following theorem provides an upper-bound on the deviation dcj(t + ∆t) starting fromγcj (t) =
τcj (t).
Theorem 5.1.
dcj(t+∆t) ≤ maxi1∈[1,...,N ]
(
maxi2∈Si1
qu(i1)(i2, j)− mini2∈Si1
qu(i1)(i2, j)
)
(38)
whereSi1 = [ai1−1 + 1, ..., ai1 ] and
u(i) =
1 if T (i, i) = 1
2 if T (i, i) = 0
Proof: The value of the collapsed states of the SME model mapping to CSM statej at any time
t+∆t is given by:
γcj (t+∆t) =
aj∑
j1=aj−1+1
M∑
i1=1
xci1(t)P∆c
(i1, j1) (39)
The transition probabilities of controlled CSM modelPrd can be represented as follows:
Prd(i, j) =
∑aj
j1=aj−1+1
∑ai
i1=ai−1+1 P∆u(i)(i1, j1)ηu(i)(i1)
∑ai
i2=ai−1+1 ηu(i)(i2)
=
∑aj
j1=aj−1+1
∑ai
i1=ai−1+1 P∆u(i)(i1, j1)ηu(i)(i1)
ζu(i)(i)(40)
Thus, the value of thejth state of the controlled CSM model at any timet+∆t is given by:
τcj (t+∆t) =
N∑
i2=1
τci2 (t)Prd(i2, j)
=N∑
i2=1
τci2(t)
∑aj
j2=aj−1+1
∑ai2
i3=ai2−1+1 P∆u(i2)(i3, j2)ηu(i2)(i3)
∑ai2
i4=ai2−1+1 ηu(i2)(i4)
=
N∑
i2=1
∑aj
j2=aj−1+1
∑ai2
i3=ai2−1+1 P∆u(i2)(i3, j2)ηu(i2)(i3)τ
ci2(t)
ζu(i2)(i2)(41)
December 20, 2011 DRAFT
17
If we consider the caseγcj (t) = τcj (t), d
cj(t+∆t) represents the difference betweenγc
j andτcj during
time stept to t+∆t starting from equal distribution at timet and is given by the following equation:
dcj(t+∆t)
= |γcj (t+∆t)− τcj (t+∆t)|
=
∣
∣
∣
∣
∣
∣
aj∑
j1=aj−1+1
M∑
i1=1
xci1(t)P∆c
(i1, j1)−N∑
i2=1
∑aj
j2=aj−1+1
∑ai2
i3=ai2−1+1 P∆u(i2)(i3, j2)ηu(i2)(i3)τ
ci2(t)
ζu(i2)(i2)
∣
∣
∣
∣
∣
∣
=
∣
∣
∣
∣
∣
∣
aj∑
j1=aj−1+1
N∑
i1=1
ai1∑
i2=ai1−1+1
xci2(t)P∆u(i1)
(i2, j1)−∑ai1
i3=ai1−1+1 P∆u(i1)(i3, j1)ηu(i1)(i3)τ
ci1(t)
ζu(i1)(i1)
∣
∣
∣
∣
∣
∣
=
∣
∣
∣
∣
∣
∣
N∑
i1=1
ai1∑
i2=ai1−1+1
xci2(t)
aj∑
j1=aj−1+1
P∆u(i1)(i2, j1)−
ηu(i1)(i2)τci1(t)
ζu(i1)(i1)
aj∑
j1=aj−1+1
P∆u(i1)(i2, j1)
∣
∣
∣
∣
∣
∣
=
∣
∣
∣
∣
∣
∣
N∑
i1=1
∑
i2∈Si1
xci2(t)ζu(i1)(i1)− τci1 (t)ηu(i1)(i2)
ζu(i1)(i1)
∑
j1∈Sj
P∆u(i1)(i2, j1))
∣
∣
∣
∣
∣
∣
(42)
whereSi = [ai−1 + 1, ..., ai]. If we let∑aj
j1=aj−1+1 P∆u(i1)(i2, j1) = qu(i1)(i2, j), we know that:
∑
i2∈Si1
xci2(t)ζu(i1)(ii)qu(i1)(i2, j) ≤ γc
i1(t)ζu(i1)(ii) max
i2∈Si1
qu(i1)(i2, j), (43)
∑
i2∈Si1
τci1 (t)ηu(i1)(i2)qu(i1)(i2, j) ≥ τci1(t)ζu(i1)(ii) mini2∈Si1
qu(i1)(i2, j) (44)
The upper bound for the controlled difference equation is then given by:
dcj(t+∆t) ≤∣
∣
∣
∣
∣
N∑
i1=1
γci1(t)ζu(i1)(i1)maxi2∈Si1
qu(i1)(i2, j)− τci1 (t)ζu(i1)(i1)mini2∈Si2qu(i1)(i2, j)
ζ(ii)
∣
∣
∣
∣
∣
≤N∑
i1=1
γci1(t)
(
maxi2∈Si1
qu(i1)(i2, j)− mini2∈Si1
qu(i1)(i2, j)
)
≤ maxi1∈[1,...,N ]
(
maxi2∈Si1
qu(i1)(i2, j)− mini2∈Si1
qu(i1)(i2, j)
)
(45)
Based on the results of theorem 5.1, we note that the single step deviation of the controlled system
is upper-bounded by the maximum of the single-step bounds ofthe individual SME models. Similarly,
the multi-step upper bound of the controlled system is the maximum of the multi-step bounds of the
individual systems.
6 RESULTS
In this section, we apply our analysis to randomly generatedSME models (section 6.1) , oscillating SME
models (section 6.2) and biological model of Genetic ToggleSwitch [24] (section 6.3).
December 20, 2011 DRAFT
18
6.1 Randomly Generated Models
In this sub-section, we present the simulation results of the shift in transient behavior of the SME model
and the reduced CSM model based on various random SME models.We generated random transition
probability matrices to represent the SME models and using the reduction mapping in Eq. 2 generated
the reduced CSM model. At first, we consider the cases of transition probability matrix for which the
diagonal entry is larger than all the non-diagonal entries.Let d ∈ [0, 1] represent the average value of the
diagonal entry. The non-diagonal entries of a row are selected from a uniform distributionU(0, 2(1−d)M
).
The diagonal entry is selected as1 − ∑
(non-diagonal entries). The generation of the diagonal entry
in this fashion guarantees that the variance of the diagonalentry is (M − 1) times the variance of each
individual non-diagonal entry. In actual biological transition probability matrices, the diagonal entry
represents the probability of remaining in the same state and that is equal to1 − ∑
( probabilities
of leaving the state). The results of the simulation are shown in Table 1 whereM and N represent
the number of states of the SME and CSM model respectively andd represents the average value of
the diagonal entry. We usedT = 100 as the number of time steps by which the system has reached
the steady state. For each set of parameters we created100 random transition probability matrices and
calculated the expected deviations and bounds. For the actual deviation calculations, the initial probability
distribution is taken to be[1, 0, 0, · · · , 0]. In Table 1,Ds represents the mean of the maximum difference
between the collapsed SME state probabilities and the CSM state probabilities in a single time step i.e.
Ds = E(maxi∈[1,...,N ],t∈[1,..T ] ||τj(t) − γj(t)| − |τj(t − 1) − γj(t − 1)||). Dm represents the mean of
the maximum difference between the collapsed SME state probabilities and the CSM state probabilities
in multiple time steps i.e.Ds = E(maxi∈[1,...,N ],t∈[0,1,..T ] |τj(t) − γj(t)|). Bs represents the mean of
the single step bounds calculated from the transition probability matrices using the bound in equation 4.
The mean is with respect to the100 randomly generated transition probability matrices.Bm represents
the mean of the multi-step bounds calculated from the transition probability matrices using the bound in
equation 11.SBs represents the mean of the single step stochastic bounds calculated using equation 29.
The ε used in Eq. 29 is2(1−d)/(M −1). SBm represents the mean of the multi-step stochastic bounds
calculated using equations 29, 11 and 30. For Eq. 30, theσ1 andσ2 used are2(1− d)/(M − 1)√
L/12
and2(1−d)/(M−1)√
(M − L)/12 respectively. The means of the distributionsξ1 andξ2 are(1−d)/N
and(1− d)/N + d respectively. The results of Table 1 shows that the generated bounds are quite close
to the actual maximum deviations. The bound for single time step Bs is quite close toDs, the actual
maximum deviation in a single time step. Similarly,Bm is of the same order asDm. Furthermore,
we note that the stochastic boundSBs is very close to the expectation of the boundBs, similarly the
stochastic bound for multiple stepSBm is very similar to the actual bound for multiple stepBm. This
shows that we can use the stochastic bounds for generating the deviation bounds when the distribution of
December 20, 2011 DRAFT
19
the entries of the transition probability matrix is known. The stochastic bounds can be calculated without
generating the actual transition probability matrices andhas extremely low computational complexity
involved in its calculation.
M N d Ds Dm Bs Bm SBs SBm
400 10 .5 0.035 0.036 0.059 0.119 0.068 0.147
400 4 .5 0.033 0.034 0.063 0.125 0.069 0.146
200 10 .5 0.049 0.049 0 .072 0.149 .086 0.194
200 10 .1 0.084 0.084 0.131 0.150 0.155 0.196
400 10 .1 0.060 0.060 0.107 0.120 0.122 0.148
400 10 .8 0.013 0.026 0.024 0.119 0.027 0.146
400 4 .8 0.014 0.027 0.025 0.124 0.028 0.145
800 10 .8 0.011 0.022 0.0185 0.094 0.021 0.111
TABLE 1
Transient response bounds starting from random SME models with diagonal entries greater than
non-diagonal entries
Table 2 shows the simulation results for transition probability matrices with all entries belonging to
a uniform distributionU(0, 2M). As before,M andN represent the number of states of the SME and
CSM model respectively. We usedT = 100 as the number of time steps to reach the steady state. For
each set of parameters we created100 random transition probability matrices and calculated theaverage
deviations and bounds. For the actual deviation calculations, the initial probability distribution is taken
to be [1, 0, 0, · · · , 0]. Ds represents the mean of the maximum difference between the collapsed SME
state probabilities and the CSM state probabilities in a single time step.Dm represents the mean of
the maximum difference between the collapsed SME state probabilities and the CSM state probabilities
in multiple time steps.Bs represents the mean of the single step bounds calculated from the transition
probability matrices using the bound in equation 4.Bm represents the mean of the multi-step bounds
calculated from the transition probability matrices usingthe bound in equation 11.SBs represents the
mean of the single step stochastic bounds calculated using equation 28.SBm represents the mean of
the multi-step stochastic bounds calculated using equations 28, 11 and 30. For Eq. 30, the standard
deviationsσ2 = σ1 = 2M
√
L12 and the means of the distributions areξ2 = ξ1 = 1/N .
From Table 2, we note that the proposed bounds are close to theactual deviations. The entries in
columnBs andBm are close to the entries of columnsDs andDm respectively. We also note that for
transition probability matrices with all entries selectedfrom same uniform distribution, the single step
deviation is quite close to multi-step deviation. The stochastic bounds indicated in columnsSBs and
SBm are very close to the actual boundsBs andBm and these stochastic bounds can be calculated in
December 20, 2011 DRAFT
20
M N Ds Dm Bs Bm SBs SBm
200 4 0.047 0.047 0.092 0.095 0.104 0.111
400 4 0.044 0.044 0.072 0.074 0.080 0.084
500 5 0.035 0.035 0.059 0.061 0.064 0.067
600 6 0.029 0.029 0.051 0.052 0.053 0.056
800 8 0.022 0.022 0.040 0.041 0.040 0.042
1000 4 0.023 0.023 0.049 0.050 0.056 0.057
TABLE 2
Transient response bounds starting from random SME models with uniform transition
probabilities
a computationally inexpensive manner. Thus for uniform distribution of entries of probability transition
matrix, the stochastic bounds provide a very good approximation of the upper bounds.
6.2 Oscillating SME models
In this subsection, we consider models with periodic behavior. We consider a two protein systemP1
andP2 and truncate the maximum number of protein molecules forP1 andP2 to 19 resulting in a total
number of20× 20 = 400 possible states for the SME. The CSM model contains2× 2 = 4 states and
the threshold used for binarization is10. To generate a SME model with oscillation, we considered a
transition probability matrix structure as shown in Fig 1(a) where the states in setS2i−1 andS2i of the
SME model map to statei of the CSM model fori = {1, 2, 3, 4}. Each setSi contain the same number
of 400/8 = 50 states. The blue squares contain non-zero transition probabilities and all blank squares
contain zeros. The non-zero transition probabilities are selected from a uniform random generator of
range[0, 1] and finally normalized to make the sum of rows equal to 1. For the figure 1, we note that
the state transitions can be grouped into the following transitions between the set of statesS1 to S8 :
S1 → S4 → S5 → S3 → S6 → S4
S1 → S8 → S4 → S5 → S3 → S6 → S4
S2 → S3 → S6 → S4 → S5 → S3
S2 → S7 → S4 → S5 → S3 → S6 → S4
The oscillating behavior of generated protein molecules isevident from Fig 1(b) that shows the
corresponding expected number of protein molecules forP1 and P2 vs time. As the SME transition
probability matrix is not ergodic in this example, we added asmall perturbation to calculate the steady
state probability to generate the reduced order CSM model using Eq 2. In this example, we had the sum
of transition probabilities from a state inS1 to any state inS4 or a state inS2 to any state inS3 fixed
at 0.5. The bound generated using Eq. 4 is zero as for alli1 and j ∈ [1, 2, 3, 4], maxi2∈Si1q(i2, j) =
December 20, 2011 DRAFT
21
0 2 4 6 8 10 12 14 16 18 208
8.5
9
9.5
10
10.5
11
time −>
Ava
rage
exp
ress
ion
Protein 1Protein 2
Fig. 1. (a) Transition Probability matrix description for 1st oscillating model (b) Expected protein
molecule generation vs time
mini2∈Si1q(i2, j). Thus a value of zero for the bound assures us that the transient probabilities of the
CSM and the collapsed SME model should match exactly. The time evolution of the CSM and the
collapsed SME model for the four states starting from a uniform distribution is shown in Fig. 2. From
Fig. 2, it is clear that the transient behavior of the CSM and the collapsed SME model matches exactly.
2 4 6 8 10 12 14 16 18 200
0.05
0.1
0.15
0.2
0.25
Time−>
prob
abili
ty
CSMcollapsed SME
(a) Probability vs time for state 00
2 4 6 8 10 12 14 16 18 200.25
0.3
0.35
0.4
0.45
0.5
0.55
0.6
0.65
Time−>
prob
abili
ty
CSMcollapsed SME
(b) Probability vs time for state 01
2 4 6 8 10 12 14 16 18 200.25
0.3
0.35
0.4
0.45
0.5
0.55
0.6
0.65
Time−>
prob
abili
ty
CSMcollapsed SME
(c) Probability vs time for state 10
2 4 6 8 10 12 14 16 18 200
0.05
0.1
0.15
0.2
0.25
Time−>
prob
abili
ty
CSMcollapsed SME
(d) Probability vs time for state 11
Fig. 2. Time evolution of probability starting from a uniform distribution for an oscillating model
described by Fig 1. Red * denotes the coarse scale model evolution and blue + denotes the
evolution of the collapsed SME model.
We next consider changes such that we have transient oscillations and the transition probabilities are
arranged in a manner to arrive at a non-zero value for the bound in Eq. 4. The transition probabilities
now are similar to Fig 1(a) except that we allow new transitions from states inS6 to states inS2 and
December 20, 2011 DRAFT
22
the sum of transition probabilities from a state inS6 to a state inS2 is fixed at0.25. The time evolution
of the CSM and the collapsed SME model for the four states starting from a uniform distribution is
shown in 3. TheDs for this model is0.03 andDm is 0.033. Based on the configuration of the transition
probability matrix, the single step bound using equation 4 is 0.25. We note that the bound is much higher
than the actual deviations. If we analyze the derivation of the bound, we will note that there are only
few blocks that havemaxi2∈Si1q(i2, j) 6= mini2∈Si1
q(i2, j). In this example, we can use the reduced
single step bound of0.25× γ3(t) if we have an estimate ofγ3(t).
5 10 15 20 25 30 35 40 45 500
0.05
0.1
0.15
0.2
0.25
Time−>
prob
abili
ty
CSMcollapsed SME
(a) Probability vs time for state 00
5 10 15 20 25 30 35 40 45 500.25
0.3
0.35
0.4
0.45
0.5
0.55
0.6
0.65
Time−>
prob
abili
ty
CSMcollapsed SME
(b) Probability vs time for state 01
5 10 15 20 25 30 35 40 45 500.2
0.25
0.3
0.35
0.4
0.45
0.5
0.55
0.6
0.65
Time−>
prob
abili
ty
CSMcollapsed SME
(c) Probability vs time for state 10
5 10 15 20 25 30 35 40 45 500
0.05
0.1
0.15
0.2
0.25
Time−>
prob
abili
ty
CSMcollapsed SME
(d) Probability vs time for state 11
Fig. 3. Time evolution of probability starting from a uniform distribution for the modified model
with transient oscillations. Red * denotes the coarse scale model evolution and blue + denotes
the evolution of the collapsed SME model.
6.3 Genetic Toggle Switch
We next apply our transient analysis on a genetic toggle switch where there are two competing proteins
U1 andU2 , each of which inhibits the transcription of the other [24],[25]. The decay reactionsU1 → ∅andU2 → ∅ have propensitiescΨ1 andcΨ2 respectively. The transcription of new copies of the proteins
are guided by the reactions∅ → U1 and∅ → U2 with propensitiesb/(b+Ψ2) andb/(b+Ψ1) respectively.
We considerb = 0.8 andc = 0.035 as the starting system. We truncate the maximum number of protein
molecules forU1 andU2 to 39 resulting in a total number of40 × 40 = 1600 possible states for the
SME. Based on the propensity functions and parameter valuesb = 0.8 andc = 0.035, the state reaction
matrix,A1, is generated and the continuous Markov Chain is approximated by a discrete chain using the
December 20, 2011 DRAFT
23
following equationP∆1 = eA1∆t with a time step of∆t = 0.5 seconds. The thresholds for binarization
of the proteinsU1 andU2 are selected to beT1 = T2 = 20 molecules2. After binarization, the two
protein reduced network has2× 2 = 4 states. We calculated the transition probabilities of the reduced
model (CSMPr1) using Equation 2. In our case, the Markov ChainP∆1 is ergodic as all the states are
communicating, recurrent and aperiodic. Due to the ergodicity, P∆1 has a unique steady state probability
distribution. The time evolution of the probability of the four states starting from a uniform distribution
is shown in Fig 4.
0 500 1000 1500 20000.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Time−>
prob
abili
ty
CSMcollapsed SME
(a) Probability vs time for state 00
0 500 1000 1500 20000
0.05
0.1
0.15
0.2
0.25
0.3
0.35
Time−>
prob
abili
ty
CSMcollapsed SME
(b) Probability vs time for state 01
0 500 1000 1500 20000
0.05
0.1
0.15
0.2
0.25
0.3
0.35
Time−>
prob
abili
ty
CSMcollapsed SME
(c) Probability vs time for state 10
0 500 1000 1500 20000
0.05
0.1
0.15
0.2
0.25
Time−>
prob
abili
ty
CSMcollapsed SME
(d) Probability vs time for state 11
Fig. 4. Time evolution of probability starting from a uniform distribution for the genetic switch with
parameters b = 0.8 and c = 0.035. Red * denotes the coarse scale model evolution and blue +
denotes the evolution of the collapsed SME model.
The actual deviation characteristics of the CSM probability and the SME model are shown in Table
2. The threshold levels selected here are arbitrary but in real systems medicine application, it will be decided based onthe
modes of the distribution corresponding to cancerous and non-cancerous states.
December 20, 2011 DRAFT
24
3. State denotes the4 states (00, 01, 10 and11) of the reduced model.Ds andDm denotes the actual
maximum deviation in a single time step and multiple time steps respectively.SBs andSBm denotes
the stochastic bounds for single time step and multiple timesteps respectively. Theε used to calculate
the stochastic bound is2 ∗ (1 − d)/(M − 1) with d = 0.4 whered is an approximation of a diagonal
entry. We are assuming that each diagonal entry is greater than 0.4 which is reasonable as we have a
small δt of 0.25 seconds. Theµm used to calculate the multiple step bound is estimated as follows.
The transition probability matrix is given byeA∗δt whereA is the reaction matrix. We approximate the
transition probability matrix byI +Aδt using the first two terms of the series expansion. Based on this
approximation, we generated the approximate transition probability matrix and calculatedµ’s from the
approximate matrix. For our case,δt = 0.25, c = 0.035, b = 0.8 and t1 = 20, t2 = 20. To calculate
the multi-step bound, we used the maximum ofµ’s 0.8154 as theµm. From Table 3, we note that the
stochastic bound for multi-steps (0.2596) is very close to the maximum actual deviations for each state
(0.2257, 0.0895, 0.0917, 0.0784).
State Ds Dm SBs SBm
00 0.0054 0.2257 0.0479 0.2596
01 0.0027 0.0895 0.0479 0.2596
10 0.0027 0.0917 0.0479 0.2596
11 0.0016 0.0784 0.0479 0.2596
TABLE 3
Transient response bounds for the Genetic Switch Model with parameters b = 0.8 and c = 0.035
To study the effect on the transient behavior of the control policy designed on the reduced network
when applied to the original network, we consider a control problem of reducing the probability of state
00. The control input is assumed to decrease the decay of the proteins by decreasing the value of the
parameterc from 0.035 to 0.025. Thus we have two systems, the original withb = 0.8 and c = 0.035
and the other corresponding to when control is switched on with parametersb = 0.4 and c = 0.025.
The time evolution of the probability of the four states starting from a uniform distribution for the new
system is shown in Fig 5.
The actual deviation characteristics of the CSM probability and the SME model for the new system
are shown in Table 4. As before,Ds andDm denotes the actual maximum deviation in a single time
step and multiple time steps respectively;SBs andSBm denotes the stochastic bounds for single time
step and multiple time steps respectively. Theε used to calculate the stochastic bound is2(1−d)M−1 with
d = 0.4. As in the previous case, theµm is considered to be the maximum of the approximateµ’s which
is 0.8654 for b = 0.8, c = 0.025. From Table 4, we note that the expected bound for multi-steps (0.356)
December 20, 2011 DRAFT
25
0 500 1000 1500 20000.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Time−>
prob
abili
ty
CSMcollapsed SME
(a) Probability vs time for state 00
0 500 1000 1500 20000.05
0.1
0.15
0.2
0.25
0.3
Time−>
prob
abili
ty
CSMcollapsed SME
(b) Probability vs time for state 01
0 500 1000 1500 20000.05
0.1
0.15
0.2
0.25
0.3
Time−>
prob
abili
ty
CSMcollapsed SME
(c) Probability vs time for state 10
0 500 1000 1500 20000
0.05
0.1
0.15
0.2
0.25
Time−>
prob
abili
ty
CSMcollapsed SME
(d) Probability vs time for state 11
Fig. 5. Time evolution of probability starting from a uniform distribution for the genetic switch with
parameters b = 0.8 and c = 0.025. Red * denotes the coarse scale model evolution and blue +
denotes the evolution of the collapsed SME model.
is very close to the maximum actual deviations for each state(0.2886, 0.1242, 0.1242, 0.0756).
The optimal stationary policy to reduce the steady state probability mass for state00 is 0, 1, 1, 0 i.e.
apply no control while in states00 and 11 and apply control while in states01 and 10. The optimal
control policy is derived using dynamic programming principles and is an example of sporadic control
based on state changes. The probability evolution for the controlled model is shown in Figure 6. Without
application of control, the steady state probability of state 00 is 0.795 for both the CSM and SME models
as shown in Figure 4(a). The equality of the steady-state probabilities for CSM and collapsed SME is
because of the steady state behavior maintaining property of the mapping [13]. With application of control
at every step, the steady state probability of state00 is 0.7043 for both the CSM and SME models as
shown in Figure 5(a). After application of the stationary control 0, 1, 1, 0, the steady state probability for
December 20, 2011 DRAFT
26
State Ds Dm SBs SBm
00 .0039 .2886 .0479 .356
01 .0019 .1242 .0479 .356
10 .0019 .1242 .0479 .356
11 .0011 .0756 .0479 .356
TABLE 4
Transient response bounds for the Genetic Switch Model with parameters b = 0.8 and c = 0.025
state00 is 0.6586 for the coarse-scale controlled model and0.6790 for the collapsed controlled SME
model as shown in Figure 6(a). We note that we have been able toachieve our objective of lowering
the steady state probability of state00. The maximum deviations for single step and multi-steps forthe
controlled model are shown in Table 5. The expected bounds for single time step and multi-time steps
shown in Table 5 are the maximum of the bounds for the uncontrolled system (b = 0.8, c = 0.035) and
the system with control always on (b = 0.8, c = 0.025). We note that the expected bounds are a good
indicator of the maximum deviation in the transient response.
State Ds Dm SBs SBm
00 .0043 .3155 .0479 .356
01 .0024 .1451 .0479 .356
10 .0024 .1482 .0479 .356
11 .0016 .0782 .0479 .356
TABLE 5
Transient response bounds for the controlled Genetic Switch Model with stationary policy 0110
7 CONCLUSIONS
In this paper, we provided bounds for the deviation of the transient state probability distribution of a
reduced order Markovian model as compared to the collapsed state probability distribution of the fine
scale SME model. Analysis of the predictive capability of a reduced order model is highly important
in systems biology to understand the effects of limited samples and data extraction noise. We showed
by simulations that our derived analytical bounds are of thesame order and quite close to the actual
differences in state probabilities. Our transient analysis in case of stationary control policies showed that
the deviations are bounded by the maximum of the bounds for the individual systems. In this article, we
also derived stochastic bounds for the transient deviations that are easier to calculate. Our simulations
December 20, 2011 DRAFT
27
0 500 1000 1500 20000.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Time−>
prob
abili
ty
CSMcollapsed SME
(a) Probability vs time for state 00
0 500 1000 1500 20000
0.05
0.1
0.15
0.2
0.25
0.3
0.35
Time−>
prob
abili
ty
CSMcollapsed SME
(b) Probability vs time for state 01
0 500 1000 1500 20000
0.05
0.1
0.15
0.2
0.25
0.3
0.35
Time−>
prob
abili
ty
CSMcollapsed SME
(c) Probability vs time for state 10
0 500 1000 1500 20000
0.05
0.1
0.15
0.2
0.25
Time−>
prob
abili
ty
CSMcollapsed SME
(d) Probability vs time for state 11
Fig. 6. Time evolution of probability starting from a uniform distribution for the controlled genetic
switch with stationary control of 0 1 1 0. Red * denotes the coarse scale model evolution and blue
+ denotes the evolution of the collapsed SME model.
illustrate that the stochastic bounds are a good measure of the maximum deviation in the transient
behavior.
REFERENCES
[1] N. Wiener,Cybernetics: Or the Control and Communication in the Animaland the Machine. MIT Press, Cambridge, MA,
1948.
[2] A. Arkin, J. Ross, and H. H. McAdams, “Stochastic kineticanalysis of developmental pathway bifurcation in phage-infected
escherichia coli cells,”Genetics, vol. 149, pp. 1633–1648, 1998.
[3] H. H. McAdams and A. Arkin, “Stochastic mechanisms in gene expression,”Proc. Natl. Acad. Sci, vol. 94, pp. 814–819,
1997.
[4] D. T. Gillespie, “A rigorous derivation of the chemical master equation,”Physica A, vol. 188, pp. 404–425, 1992.
[5] J. Goutsias, “A hidden markov model for transcriptionalregulation in single cells,”IEEE/ACM Transactions on Computational
Biology and Bioinformatics, vol. 3, pp. 57–71, 2006.
December 20, 2011 DRAFT
28
[6] Goutsias, John, “Classical versus Stochastic KineticsModeling of Biochemical Reaction Systems,”Biophys. J., vol. 92, no. 7,
pp. 2350–2365, April 2007.
[7] C. A. Gomez-Uribe and G. C. Verghese, “Mass fluctuation kinetics: capturing stochastic effects in systems of chemical
reactions through coupled mean-variance computations.”J Chem Phys, vol. 126, no. 2, p. 024109, 2007.
[8] Lee, Chang H., Kim, Kyeong H., and Kim, Pilwon, “A moment closure method for stochastic reaction networks,”The Journal
of Chemical Physics, vol. 130, no. 13, pp. 134 107+, 2009.
[9] R. Pal and M. U. Caglar, “Control of stochastic master equation models of genetic regulatory networks by approximating
their average behavior,” inIEEE International Workshop on Genomic Signal Processing and Statistics, 2010.
[10] A. T. Weeraratna, Y. Jiang, G. Hostetter, K. Rosenblatt, P. Duray, M. Bittner, and T. J. M., “Wnt5a signalling directly affects
cell motility and invasion of metastatic melanoma,”Cancer Cell, vol. 1, pp. 279–288, 2002.
[11] R. Pal, A. Datta, and E. R. Dougherty, “Optimal infinite horizon control for probabilistic boolean networks,”IEEE
Transactions on Signal Processing, vol. 54, pp. 2375– 2387, 2006.
[12] E. Yzerman, J. Den Boer, M. Caspers, A. Almal, B. Worzel,W. Van der Meer, R. Montijn, and F. Schuren, “Comparative
genome analysis of a large Dutch Legionella pneumophila strain collection identifies five markers highly correlated with
clinical strains,”BMC Genomics, vol. 11, pp. 433+, 2010.
[13] R. Pal and S. Bhattacharya, “Characterizing the effectof coarse-scale pbn modeling on dynamics and intervention performance
of genetic regulatory networks represented by stochastic master equation models,”IEEE Transactions on Signal Processing,
vol. 58, pp. 3341–3351, 2010.
[14] B. Munsky and M. Khammash, “The finite state projection algorithm for the solution of the chemical master equation,”The
Journal of Chemical Physics, vol. 124, no. 044104, 2006.
[15] S. Kauffman,The Origins of Order: Self-Organization and Selection in Evolution. New York: Oxford Univ. Press, 1993.
[16] I. Shmulevich, E. R. Dougherty, S. Kim, and W. Zhang, “Probabilistic boolean networks: A rule-based uncertainty model
for gene regulatory networks,”Bioinformatics, vol. 18, pp. 261–274, 2002.
[17] N. Friedman, M. Linial, I. Nachman, and D. Pe’er, “Bayesian networks to analyze expression data,”Proceedings of the
Fourth Annual International Conference on Computational Molecular Biology, pp. 127–135, 2000.
[18] H. Lahdesmaki, S. Hautaniemi, I. Shmulevich, and O. Yli-Harja, “Relationships between probabilistic boolean networks and
dynamic bayesian networks as models of gene regulatory networks,” Signal Processing, vol. 86, pp. 814–834, 2006.
[19] D. P. Bertsekas and J. N. Tsitsikilis,Introduction to Probability, 2nd ed. Athena Scientific, 2008.
[20] T. L. Lai and H. Robbins, “Maximally dependent random variables,” Proc. Nat. Acad. Sci. USA, vol. 73, pp. 286–288, 1976.
[21] J. W. Clark,Harrison’s Manual of Oncology. McGraw-Hill Professional, 2007, ch. 1.10 Molecular Targeted Drugs, pp.
67–75.
[22] R. A. Science, “Integrated solutions for gene knockdown,” Biochemica, no. 4, 2004.
[23] R. Pal, A. Datta, M. L. Bittner, and E. R. Dougherty, “Intervention in context-sensitive probabilistic boolean networks,”
Bioinformatics, vol. 21, pp. 1211–1218, 2005.
[24] M. Hegland, C. Burden, L. Santoso, S. MacNamara, and H. Booth, “A solver for the stochastic master equation applied to
gene regulatory networks,”Journal of Computational and Applied Mathematics, vol. 205, no. 2, pp. 708 – 724, 2007.
[25] T. Jahnke and W. Huisinga, “A dynamical low-rank approach to the chemical master equation,”Bull. Math. Biol., vol. 70,
pp. 2283–2302, 2008.
December 20, 2011 DRAFT
29
AUTHOR BIOS
Ranadip Pal received the B. Tech. degree in Electronics and ECE from the Indian Institute of Technology,
Kharagpur, India in 2002 and the M.S. and PhD degrees in Electrical Engineering from Texas A & M
University, College Station in 2004 and 2007 respectively.In August 2007, he joined the Department of
Electrical and Computer Engineering at Texas Tech University, Lubbock as an Assistant Professor. His
research areas are computational biology, genomic signal processing and control of genetic regulatory
networks. Pal is the recipient of NSF CAREER Award, 2010. While at Texas A&M, Pal was the recipient
of the Ebensberger/Fouraker Fellowship, a Distinguished Graduate Student Masters Research Award and
a National Instruments Fellowship. Pal was also an Indian National Math Olympiad Awardee.
Sonal Bhattacharya received the B.Sc. and M.Sc. degrees in physics from Calcutta University, Calcutta,
India, in 1998 and 2001, respectively, and the post M.S. diploma in physics from Saha Institute of
Nuclear Physics, Calcutta, in 2002. She is currently pursuing the Ph.D. degree in Electrical and Computer
Engineering at Texas Tech University, Lubbock, TX. Her research areas are genomic signal processing,
control of genetic regulatory networks, image processing,solid-state physics, and fluid mechanics.
December 20, 2011 DRAFT