- We are sorry, but NCBI web applications do not support your browser and may not function properly. More information

- Journal List
- J Chem Phys
- PMC2736569

# A rigorous framework for multiscale simulation of stochastic cellular networks

^{a)}Electronic mail: michael.chevalier/at/ucsf.edu.

^{b)}Electronic addresses: hana.el-samad/at/ucsf.edu and helsamad/at/biochem.ucsf.edu.

## Abstract

Noise and stochasticity are fundamental to biology and derive from the very nature of biochemical reactions where thermal motion of molecules translates into randomness in the sequence and timing of reactions. This randomness leads to cell-cell variability even in clonal populations. Stochastic biochemical networks are modeled as continuous time discrete state Markov processes whose probability density functions evolve according to a chemical master equation (CME). The CME is not solvable but for the simplest cases, and one has to resort to kinetic Monte Carlo techniques to simulate the stochastic trajectories of the biochemical network under study. A commonly used such algorithm is the stochastic simulation algorithm (SSA). Because it tracks every biochemical reaction that occurs in a given system, the SSA presents computational difficulties especially when there is a vast disparity in the timescales of the reactions or in the number of molecules involved in these reactions. This is common in cellular networks, and many approximation algorithms have evolved to alleviate the computational burdens of the SSA. Here, we present a rigorously derived modified CME framework based on the partition of a biochemically reacting system into restricted and unrestricted reactions. Although this modified CME decomposition is as analytically difficult as the original CME, it can be naturally used to generate a hierarchy of approximations at different levels of accuracy. Most importantly, some previously derived algorithms are demonstrated to be limiting cases of our formulation. We apply our methods to biologically relevant test systems to demonstrate their accuracy and efficiency.

## BACKGROUND AND MOTIVATION

In populations of organisms and cells, phenotypic differences between individuals are caused by a combination of genetic and environmental factors. However, significant variability persists between individuals of the same genotype in highly similar environments. For example, studies of gene expression variability in inbred mouse strains revealed as much interindividual variability within the same genotype as interstrain variability. In human genetic studies, there is substantial variability in the penetrance and expressivity of phenotypes of individuals who share a genotype. While subtle genetic or environmental differences might be at play, stochastic factors account for a significant portion of phenotypic variability. Such variability impacts many biological processes. In Drosophila embryos, variability in the gradient of the bicoid protein results in undesirable developmental alterations, and studies suggest that the system critically depends on molecular devices that are dedicated to filtering noise.^{1}^{, }^{2} Other biological systems use heterogeneity productively to allow populations to “hedge their bets” with respect to future environmental shifts. This is thought to explain the ability of subpopulations of bacteria to resist antibiotics and promote latency of infection by viruses.^{3}^{, }^{4} As a consequence, the thorough investigation of the principles underlying such nongenetic individuality is essential for understanding the differential susceptibility of cells and organisms to diseases, drugs, and pathogens.

Molecular noise, an important contributor to nongenetic individuality, is a direct consequence of the stochastic nature of biomolecular reactions. Indeed, biochemical reactions are probabilistic collisions of biological molecules whose occurrence depends on the channels of possible interactions between these molecules (structure of the interaction network), the interaction affinity (reaction rates), as well as the number of molecules available for such interactions. This inevitably results in fluctuations in the levels of the various molecular species and translates into cell-to-cell phenotypic differences in cellular behaviors.

Much experimental work has recently focused on the importance of fluctuations and noise in cell dynamics. Investigations ranged from probing the stochastic behaviors of the steps involved in gene expression^{5}^{, }^{6}^{, }^{7}^{, }^{8}^{, }^{9} to the effects of post-translational modifications. In some cases, modulating stochastic gene expression has been shown to percolate through the cellular networks, generating wide cell-to-cell differences in the chemotactic behaviors of genetically identical *E. coli*^{10} and influencing the excursion into competence in *B. subtilis*.^{11} Notably, in most of these studies, computational modeling has emerged as a key step for discovering, describing, understanding, and ultimately generalizing the observed stochastic phenomena.

The traditional description of the stochastic chemical reactions that implement biological functions relies on the formulation of the chemical master equation (CME), a differential equation for the time evolution of probabilities.^{12}^{, }^{13}^{, }^{14} The CME corresponds to a continuous time discrete state Markov process model, the sample paths of which can be simulated exactly using a simple Monte Carlo procedure known as the stochastic simulation algorithm (SSA) developed by Gillespie.^{15} The SSA has been used with great success for the study of a number of cellular networks. However, one of its most important drawbacks is computational inefficiency; since every reaction that occurs in the system is tracked, dramatic increases in simulation times are frequent when the system contains large concentrations and/or fast reaction rates for some or all of the species involved. This is rather the norm than the exception in biological networks. The number of molecules that interact in biological networks spans orders of magnitudes. For example, genes in unicellular eukaryotes (such as the yeast *S. cerevisiae*) are present in one to two copies per cell while mRNA and protein molecules can, on average, range between less than one to multiple thousands. Furthermore, typical biological timescales range from seconds/minutes for protein-protein interactions to minutes/hours for gene expressions. As a consequence, simulating realistic biological dynamics can rapidly become prohibitive, and efficient computational algorithms need to be devised to exploit the full predictive power of computational stochastic modeling.

A number of approaches, such as time-leaping methods and system-partitioning methods, have been developed to reduce the bottlenecks inherent in the SSA simulations of stochastic biological dynamics. All of these methods sacrifice some of the accuracy of the SSA for faster approximate Monte Carlo schemes.

Time-leaping methods, such as τ-leaping, originally developed by Gillespie,^{16} allows for numerous reactions to occur between each successive simulation times. These methods begin with picking a leaping-time τ over which the propensity functions are assumed to remain constant. In this case, the number of times a given reaction occurs in τ can be approximated by a *Poisson* random variable. The work by Gillespie^{16} gives indications as to the maximum value of τ that satisfies the constant-propensity assumption. A simple method developed by Rathinam,^{17} which accounts for the changing propensities of the system, results in an implicit τ-leaping algorithm. More recently, Rathinam and El-Samad^{18} proposed a τ-leaping algorithm for addressing small number and stiff stochastic chemical systems.

Another set of approximation techniques relies on multiscale methods in which the reacting system is typically partitioned into slow reactions and fast reactions. Notably, the work of Haseltine and Rawlings^{19} separates the joint probability of the CME into a conditional probability function of the slow/small populations reactions multiplied by a marginal probability density of the fast/large populations reactions. Based on the timescale of the slow reactions, the authors derive a CME for the fast reactions, which is then approximated by Langevin equations.^{20} These trajectories then serve as an input into an SSA algorithm for the slow reactions. The work of Rao and Arkin^{21} and Cao et al.^{22} develops similar algorithms, focusing on very stiff systems where the fast reactions typically reach an equilibrium before a slow reaction occurs. Similar to the work of Haseltine and Rawlings,^{19} they derive a CME involving only the fast reactions but set the time derivative of this CME to zero, allowing for stationary distribution calculations to be made. The propensities of the slow reactions, calculated from the average values of the stationary distribution for the fast reactions, are then used in an SSA for the slow reactions. More recently Lötstedt and Hellander^{23} proposed a hybrid method in which deterministic equations are derived for fast reactions with small variances and the SSA is applied to the slow reactions. The two partitions are coupled via Monte Carlo and quasi-Monte-Carlo summations. To maintain accuracy, the appearance of fast reactions with large variances causes the algorithm to revert to a nested-loop SSA algorithm.^{24} The method developed by Weinan et al.^{24} proposes a nested-SSA algorithm in which an inner SSA can be applied to the fast reactions by sampling a certain number of fast reaction trajectories. The approximate statistical properties of these trajectories are then used as an input to drive the SSA algorithm for the outer loop, i.e., the slow reactions. The nested-SSA is not exact as is the case for the SSA, but the authors provide error and convergence estimates for their algorithm.

In this paper, we devise a general multiscale approach to derive an algorithm that efficiently and accurately calculates stochastic realizations of the CME. We take an approach similar to the work of Haseltine and Rawlings,^{19} Rao and Arkin,^{21} and Cao et al.^{22} by partitioning our reactions into two groups, which we call restricted (low propensity) and unrestricted (high propensity). Based on this partition, we derive an exact modified CME for the unrestricted reactions, which explicitly includes time-dependent effects corresponding to the probability of occurrence of the restricted reactions. Although this modified CME decomposition is as analytically difficult as the original CME, it lends itself to natural approximations. We exploit this property to devise an algorithm that considers various approximations at different levels of accuracy. Using our results, we examine some specific test systems, all of which are biologically relevant, clearly demonstrating the accuracy and efficiency of our proposed methods.

## MARKOV PROCESS DESCRIPTION OF CHEMICAL REACTIONS AND THE CME

In this section we describe the discrete state, continuous time Markov process model for well-stirred chemical reaction systems, as well as an exact simulation algorithm for this model known as the SSA.^{15} Throughout the paper, **Z**_{+} denotes the set of nonnegative integers and **R**_{+} the set of nonnegative real numbers.

The formulation we consider consists of a system of well-stirred chemical reaction with *N* molecular species. We use the state $X\left(t\right)\u220a{\mathbf{Z}}_{+}^{N}$ to denote the vector whose elements *X*_{i}(*t*) are the number of molecules of the *i*th species at time *t*. If there are *M* elementary chemical reactions that can occur among these *N* species, then we associate with each reaction *r*_{j} (*j*=1,…,*M*) a non-negative *propensity function*${a}_{j}:{\mathbf{Z}}_{+}^{N}\to {\mathbf{R}}_{+}$ defined such that *a*_{j}(*X*(*t*))τ+*o*(τ) is the probability that reaction *r*_{j} will happen in the next small time interval (*t*,*t*+τ], as τ→0. The polynomial form of the propensities *a*_{j}(*x*) may be derived from fundamental principles under certain assumptions.^{15} In this paper, the propensities will be at most quadratic, that is, *a*_{j}(*x*)ϵ{*c*_{0},*c*_{1}*x*_{i},*c*_{2}*x*_{i}(*x*_{i}−1)/2!,*c*_{3}*x*_{i}*x*_{k}}, where *c*_{0},*c*_{1},*c*_{2},*c*_{3} are rate constants. The occurrence of a reaction *r*_{j} leads to a change in ν_{j}**Z**^{N} for the state *X*. ν_{j} is therefore a stoichiometric vector that reflects the integer change in reactant species due to a reaction *r*_{j}.

Based on these premises, it can be shown that the probability density function (PDF) for the waiting time τ for the next reaction is given by *a*_{0}(*x*)*e*^{−a0(x)τ}, where *x* is the state and ${a}_{0}\left(x\right)={\sum}_{j=1}^{M}{a}_{j}\left(x\right)$. Also, the probability that the next reaction is *r*_{j}, is given by *a*_{j}(*x*)/*a*_{0}(*x*), and is independent of τ. Knowing these two probability densities for the next reaction time and type, we can simulate the system one reaction event at a time. This method is known as the SSA and belongs to a wider class of numerical techniques known as kinetic Monte Carlo algorithms.

Given the initial conditions *n*_{0},*t*_{0}, the evolution of all possible SSA trajectories can be described by the joint PDF *P*(*x*,*t**n*_{0},*t*_{0}), whose evolution is given by

Equation 1 is the so-called CME for a single state initial condition, that is, *P*(*x*=*n*_{0},*t*=*t*_{0}*n*_{0},*t*_{0})=1. The CME is also generally derived using the Markov property by writing the Chapman–Kolmogorov equation, an identity that must be obeyed by the transition probability of any Markov process.^{12}^{, }^{13} Using stationarity and taking the limit for infinitesimally vanishing time intervals, one obtains the CME as the differential form of the Chapman–Kolmogorov equation.

## PARTITIONING A BIOCHEMICALLY REACTING SYSTEM INTO RESTRICTED AND UNRESTRICTED REACTIONS

Much of the computational inefficiency of the SSA derives from the interplay of propensities spanning a wide range of magnitudes. This can be the result of different biochemical rate constants and/or number of molecules. Biological networks always operate in this scenario. For example, transcription factors present at low molecular counts interact routinely with other abundant proteins. Furthermore, reactions involving interactions (association and dissociation) between biological molecules are typically very fast compared to gene expression. Irrespective, differences in propensity values will generally lead to disproportionate simulation time being devoted to the reactions with large propensities. This can result in severe degradation in computational efficiency. In this work, we present a method to circumvent this difficulty by leaping from one low propensity reaction to the next while estimating the change in the state due to the high propensity reactions.

Our starting point is a partitioning of the system of *M* biochemical reactions into two sets: *K**unrestricted* reactions (typically high propensity reactions) *U*={*U*_{1},…,*U*_{K}} with propensity set {*a*_{u1},…,*a*_{uK}} and *M*−*K**restricted* reactions (typically low propensity reactions) *R*={*R*_{1},…,*R*_{M−K}} with corresponding propensity set {*a*_{r1},…,*a*_{rM−K}}. Here, we will assume such a decomposition and defer the more detailed discussion of reaction partitioning to Sec. 7. When applying the SSA, one would expect that in any sequence of reactions, the relative frequency of occurrence of the restricted (*R*) and unrestricted (*U*) reactions will generally depend on the relative magnitude of the sum of each set’s propensity functions. To first order, the ratio of the average time between unrestricted reactions τ_{u}=1/∑_{i}*a*_{ui} relative to the average time between restricted reactions τ_{r}=1/∑_{i}*a*_{ri} approximates the relative average occurrence of each reaction type. For example, a system with τ_{r}/τ_{u}=1 might yield a reaction sequence of …*URUURRURRUURURR*…, while one with τ_{r}/τ_{u}=5 might yield …*RUUUUURUUUUUURUUUUR*… and so on. The larger the value of τ_{r}/τ_{u} is, the larger the sequence of unrestricted reactions would be before the occurrence of a restricted reaction. It is precisely in these situations that the SSA can become computationally cumbersome since it spends most of its time calculating the occurrence of the unrestricted reactions. In many cases of biological interest, it is essential to capture the slower-scale behavior of the restricted reactions while still properly accounting for the faster-scale unrestricted reaction statistics and their effect on the dynamics of the restricted reactions. For example, reactions involving transcription factor binding and unbinding to gene promoters occur fast and frequently. While it might not be important to account for every single one of these events, it is crucial to capture accurately the binding event that leads to productive gene expression, a rare and slow reaction.

Starting from a system that is in state *n*_{s} at time *t*_{s}, our general approach will be based on characterizing all possible SSA with restricted reaction (SSA-RR) trajectories. A SSA-RR trajectory is obtained by running the SSA starting from the state *n*_{s} at time *t*_{s} until the first occurrence of a restricted reaction. This results in a time-dependent trajectory due to a sequence of unrestricted reactions ending with a single restricted reaction. Analogous to the manner in which the CME describes the time evolution of all possible trajectories of the SSA in the form of a PDF, we will derive a similar equation for all possible SSA-RR trajectories. We will then use our derivations to form an algorithm that accurately samples a new state, *n*_{s+1} at time *t*_{s+1} given the current sampled state *n*_{s} at time *t*_{s} while skipping over the details of the many unrestricted reactions that might occur between times *t*_{s} and *t*_{s+1}.

## A MODIFIED CME FORMULATION

Given the partition of a biochemically reacting system into restricted and unrestricted, we can define the function *W*(*x*,*t**n*_{s},*t*_{s}) as the joint probability that the state assumes the value *X*(*t*)=*x* at time *t* starting from the initial state *X*(*t*_{s})=*n*_{s} at time *t*_{s}*and* that no restricted reaction has occurred. The time evolution equation for this function can be derived by using the laws of probability. Specifically,

The first term on the right side of the equation describes the probability that the trajectory remains in the state *x* over the interval *dt*, i.e., no reaction occurs. The second term describes the probability of the trajectories entering the state *x* from the states *x*−ν_{uj} (*j*=1,…,*K*) through unrestricted reactions. Note that there is no term describing the probability of entering *x* from the states *x*−ν_{r} (=1,…,*M*−*K*) through restricted reactions. It is absent in Eq. 2 since the definition of the SSA-RR trajectory requires that the trajectory terminates when a restricted reaction occurs. Rearranging the equation, then dividing by *dt*, and taking the limit as *dt*→0 yields

We will refer to Eq. 3 as the CME with restricted reactions (CME-RR). Since *W*(*x*,*t**n*_{s},*t*_{s}) is a joint PDF, it can be written as the product of the conditional probability *P*_{u}(*x*,*t**n*_{s},*t*_{s}) of being in state *X*(*t*)=*x* at time *t* given that no restricted reaction has occurred and the probability *P*_{nr,r}(*t*) that no restricted reaction has occurred by time *t*. Furthermore, *P*_{nr,r}(*t*)≤1 is a monotonically decreasing function given by

Therefore,

Replacing identity 5 into Eq. 3 generates the two coupled equations (for details see Appendix A),

and

Equations 6, 7 bear some resemblance to, but are more general than, those derived in other works.^{19}^{, }^{20}^{, }^{21}^{, }^{22} In fact, the exact equations derived here are generalizations of these previous works. Specifically, the last two terms in Eq. 7 are time-dependent effects due to the state-dependent restricted reaction probability flux that alters the shape of the distribution function. In the limit of very large timescale separations between the restricted and unrestricted reactions, the last two terms of Eq. 7 become negligible. In this case, Eq. 7 reduces to the equations used in the derivation of the slow-scale SSA (ssSSA) algorithm.^{22} Therefore, these equations recapitulate the formalism of the ssSSA as a limiting case when the propensity functions in the system are clearly separated in magnitude.

## A SIMULATION ALGORITHM

Equations 6, 7 form the basis for a general simulation procedure. Specifically, for a defined partition of a system into restricted and unrestricted reactions and given that *X*(*t*_{s})=*n*_{s} at time *t*_{s}, Eqs. 6, 7 are solved for until time *t*_{s}+τ_{s}, where τ_{s} is typically on the order of the expected time for a restricted reaction. *P*_{nr,r}(*t*_{s}+τ_{s}) is then used to determine whether or not a restricted reaction has occurred. If such a restricted reaction has not occurred, the state of the system at time *t*_{s}+τ_{s} is sampled from *P*_{u}(*x*,*t*_{s}+τ_{s}*n*_{s},*t*_{s}). However, if a restricted reaction has occurred, the restricted reaction type μ, the time *t*^{′} at which it occurred, and the state from which it occurred need to be sampled. The time and type of restricted reaction can be sampled from *P*(*t*^{′},μ*t*^{′}<*t*_{s}+τ_{s}), the joint PDF that a restricted reaction occurred at time *t*^{′} and that it is a μ reaction given that *t*^{′}<*t*_{s}+τ_{s}. It can be proven that *P*(*t*^{′},μ*t*^{′}<*t*_{s}+τ_{s}) is given by (see Appendix C)

This is simply the expected value of the propensity function of the μth restricted reaction at time *t*^{′}, multiplied by the probability that a restricted reaction has not occurred by time *t*^{′}, given that *t*^{′}<*t*_{s}+τ_{s}. The expression in Eq. 8 has the same functional form and rationale as the expression for the joint probability used to sample the time and type of reaction derived for the full SSA,^{15} but with two important differences. The first difference reflects the fact that the restricted reaction μ can occur from any state *x* which the system can assume at time *t*^{′}, necessitating summation over *x*. The second difference is the normalization factor 1−*P*_{nr,r}(*t*_{s}+τ_{s}), which reflects the condition that the time for the restricted reaction is bounded by τ_{s} and in this case, is not infinite.

To sample the state *x* from which the restricted reaction occurred, we can use the conditional PDF that the restricted reaction occurred from state *x*, given it occurred at time *t*^{′} and is a μ reaction,

We will call this procedure the restricted reaction algorithm (RRA). A block diagram illustrating the general workflow of the algorithm is depicted in Fig. Fig.1.1. The algorithm, based on propagating *P*_{nr,r} and *P*_{u}, then sampling from 8, 9, will yield identical statistics to the SSA since they share the same physical underpinnings. However, solving for Eqs. 6, 7 exactly is generally as computationally expensive as solving the CME. Nonetheless, the same procedure can be implemented if an efficient proxy of *P*_{u} is used instead. Below, we propose to use the moments of *P*_{u} to that effect and present an approximation of these moments, which can be used with a defined error.

## MOMENT EXPRESSIONS FOR *P*_{nr,r}(*t*) AND *P*_{u}(*x*,*t**n*_{s},*t*_{s})

Let *z*(*t*) be the time-dependent mean of *P*_{u}(*x*,*t**n*_{s},*t*_{s}). For the derivations presented below, it is convenient to express the propensities as a Taylor series around this mean of the form

In general, propensity functions for elementary reactions are at most quadratic (e.g., binding reactions). In this case, Eq. 10 is exact and can be used to express *P*_{nr,r}(*t*) in terms of the central moments of *P*_{u}(*x*,*t**n*_{s},*t*_{s}). Replacing Eq. 10 into Eq. 6 yields

Therefore,

Equation 11 states that for linear and quadratic propensities, *P*_{nr,r}(*t*) is a function of the central moments of *P*_{u}(*x*,*t**n*_{s},*t*_{s}) up through the covariances. Furthermore, this result is exact, and up to this point, has not involved any approximations. If the propensities are at most linear, the equations for means and covariances of *P*_{u}(*x*,*t**n*_{s},*t*_{s}) are self-contained and can be easily solved to generate *P*_{nr,r}(*t*). However, for quadratic propensity functions, the first two time-dependent central moments for *P*_{u}(*x*,*t**n*_{s},*t*_{s}) are given by^{25}

In these equations, the third and fourth order central moments are presented in summation form (see Appendix B for derivation). One can see that a difficulty arises when the covariances and means of *P*_{u}(*x*,*t**n*_{s},*t*_{s}) depend on higher order moments in the presence of quadratic propensities, i.e., nonzero terms involving the second derivatives of the unrestricted propensities. It is common practice in the literature to truncate the system after the *N*th moment (typically *N*=2) either by setting the higher than *N* moments to zero or by approximating their functional relationship to the first *N* moments. For example, a Gaussian approximation assumes that all odd central moments are zero and the even moments are functions of the second moment. In general, other moment truncation approximations such as those proposed by Singh and Hespanha^{26} could be used.

For time-dependent moments, the duration that a particular moment truncation method remains accurate is uncertain. Here, we can take advantage of the fact that by construction *P*_{u}(*x*=*n*_{s},*t*=*t*_{s}*n*_{s},*t*_{s})=1 since the system is in a single deterministic state at the time *t*_{s} when it is sampled. At this point, the first moment is nonzero, while all other central moments are zero. Evidently, as *P*_{u}(*x*,*t**n*_{s},*t*_{s}) evolves over time, dependence on higher order central moments increases. Therefore, in principle, one strategy is to ignore higher order moments with a quantifiable error until some time τ_{s} at which their contribution becomes important. In that sense, τ_{s} could be the maximum time-step for which moment truncation error is below a satisfactory tolerance. To quantify the error, we can exploit the functional form of Eq. 10 where the difference between the linear and quadratic terms reflects the relative contributions to the first order and second order unrestricted propensity derivative terms in Eqs. 12, 13. When the distribution is in a locally linear unrestricted reaction propensity landscape, the summation of the terms involving the second derivative of the unrestricted propensities will be negligible. These terms include covariances, third and higher central moments. We therefore truncate our equations at the covariances with the assumption that at time τ_{s}, the error is less or equal than some error bound ϵ_{s}. The benefit of this approach is that it ensures controllable accuracy in the moments of *P*_{u} without resorting to more complicated truncation methods. Indeed, the error made at every step is bounded by ϵ_{s}. This is reminiscent of the choice of error tolerance in ordinary differential equation (ODE) solvers. It is important to note here, however, that the choice of ϵ_{s} presents an intrinsic compromise between speed and accuracy; if ϵ_{s} is required to be very small, τ_{s} will be restricted to small values, and one might lose the benefits of this method as compared to the SSA.

A natural definition of the moment error *e*_{m} is

which is the maximum of the ratios of the second order propensity, covariance term to the mean propensity for each of the unrestricted reactions. The “max” function takes into account the worst error, which is a conservative measure. In our evaluation of *e*_{m}, we are only focused on the local nonlinear effects of the unrestricted propensities since, due to their high propensities, their contribution to the error will be the largest. Exploring other representations of *e*_{m} is a topic of future research.

With this approximation of the moments of *P*_{u}, the general simulation procedure outlined in the previous section can now be implemented. For τ_{s} that satisfies the prescribed tolerance ϵ_{s}, an expression for *P*(*t*^{′},μ*t*^{′}<*t*_{s}+τ_{s}), which is easily obtained by replacing Eq. 10 into Eq. 8, is given by

Likewise, a similar procedure can be used to derive an approximate expression for *P*(*x**t*^{′},μ) as

The numerator in Eq. 16 is the same as that in Eq. 9 with the difference that we have replaced *P*_{u} with the approximate version ${\overline{P}}_{u}$ fit to the calculated moments. The details of sampling from these distributions are given in Appendix D.

## A NOTE ABOUT PARTITIONING

The partitioning strategy for a system of reactions into restricted and unrestricted will necessarily affect the efficiency of the algorithm. Optimizing this partitioning beyond rules of thumb and intuition about a given system is a difficult but interesting problem, which we do not tackle in this work. Instead, here we discuss qualitatively a few typical partitioning examples, which will be further illustrated in the numerical examples.

### Large separation in propensity values

If the magnitudes of propensities are clearly separated by orders of magnitude, then an obvious partition would assign the low propensity values to be restricted reactions. In this case, the RRA effectively converges to the ssSSA algorithm. Specifically, the moments for *P*_{u} will reach an equilibrium well before the time at which a restricted reaction is expected to occur. An efficient ODE solver can still be used to determine both the dynamics and the convergence to steady-state of the moments in a few implicit iterations. The ssSSA, in which the stationary moments are iteratively solved for, can also be efficiently applied.

### Similar propensity values

If reaction propensities have similar magnitudes, one can choose to revert to SSA (all restricted reactions) or jump state through the evolution of the moments (all unrestricted reactions). The decision should depend on the efficiency of the SSA; if the concentrations of species and/or the propensities are low, the SSA will be optimal. On the other hand, if the concentrations and/or propensities are high, jumping to a new state by evolving the moments may be most efficient. Switching between these two regimes dynamically might also be required, for example, in a bistable system where high and low equilibriums are visited by stochastic trajectories.

### Spectrum of propensity values

Many biological systems assume a widespread spectrum of propensities, making separation between “fast” and “slow” timescales difficult. Consider, for example, a system of ten reactions and eight species. If there are four reactions with propensities near one, then *t*_{slow}1/(4×1)=1/4. If there are two reactions with propensities near 50, *t*_{middle}1/(2×50)=1/100. Finally, if the remaining four reactions each have propensities near 1000, then *t*_{fast}1/(4×1000)=1/4000. The timescales ratio are *t*_{slow}/*t*_{middle}=25 and *t*_{middle}/*t*_{fast}=40. If the four lowest propensity reactions are considered to be restricted, then the ODE solver needs to integrate the moments for a time, on average, of *t*_{slow} seconds where the moment system has a stiffness of approximately *t*_{middle}/*t*_{fast}=40. In this case it would be most efficient to apply an implicit ODE solver which can take much larger timesteps than explicit solvers, especially for stiff systems. If, instead, both low and middle propensity reactions are considered to be restricted, then the update time for a restricted reaction is *t*_{middle} seconds and the moment system stiffness is approximately 1. Here, it would be most efficient to apply an explicit ODE solver. However, there will be 25 restricted reactions before the reacting system has evolved through a time of *t*_{slow} seconds. For the two strategies described, the most computationally efficient strategy would be to partition both the middle and fast reactions as unrestricted and apply an implicit ODE solver. In general, one has to be aware of the fact that the largest computational expense of the RRA stems from the ODE solution of the moment equations and, therefore, the choice of implicit versus explicit integration strategies will either enhance or inhibit efficiency.

For the numerical results in this paper, we will only use the implicit ODE solver CVODE^{27}^{, }^{28} from the SUNDIALS (Ref. ^{29}) package, which can be interfaced with the Portable Extensible Toolkit for Scientific Computation (PETSc).^{30}^{, }^{31} Throughout our numerical examples below, we will investigate the issues associated with partitioning in the three cases presented above and explore the tradeoffs between numerical efficiency and accuracy.

## BIOLOGICAL EXAMPLES

In this section, we apply the RRA algorithm to three biological network examples, a simple linear system, a genetic oscillator, and a bistable system. The examples are chosen to scrutinize various properties of the method and allow us to gain some practical understanding of the tradeoffs between its accuracy and computational expediency. We therefore compare the performance of a number of approximate algorithms, all based on Eqs. 6, 7 and the general strategy outline in Fig. Fig.1,1, but which involve different accuracy in the approximation of *P*_{u}. These include the following.

### A simple linear reacting system

For this example we will examine the time evolution of *P*_{nr,r}(*t*) and *P*_{u}(*x*,*t**n*_{s},*t*_{s}) for a simple linear reacting system. The system consists of *N*=2 species and *M*=4 reaction pathways given by

The propensities are given by *a*(*x*)=[*k*_{1},α_{1}*x*_{1},*k*_{2}*x*_{1},α_{2}*x*_{2}]. We define the unrestricted reaction propensities to be [*k*_{1},α_{1}*x*_{1}] and the restricted reaction propensities to be [*k*_{2}*x*_{1},α_{2}*x*_{2}]. Here, a numerical solution for Eq. 3 can directly be obtained using the finite state projection.^{32} This is possible and accurate since in our case (*K*=2), *P*_{nr,r}(*t*), converges toward zero quickly, allowing us to represent the state space with only 400 states. This method will provide a solution used to gauge the accuracy of the RRA. The explicit formulas for *P*_{nr,r}(*t*) and the moments are given by

Equations 18b, 18c form a closed set of equations that can be solved for mean and variance [*z*_{1}(*t*),*C*_{11}(*t*)] for species one. For the initial state *n*_{s}=[256;5] and parameter values [*k*_{1}=49, α_{1}=.5, *k*_{2}=.0031, α_{2}=.055], we evolve both the analytical system and the moment system [Eqs. 18a, 18b, 18c] over time. Snapshots of *P*_{u}(*x*,*t**n*_{s},*t*_{s}) for both the analytical method (solid) and the moment method (dashed) are plotted for different time points in Fig. Fig.2a.2a. For this example, the Gaussian function approximation *P*_{u}(*x*,*t**n*_{s},*t*_{s})=(1/(2π*C*_{11}(*t*)))^{1/2} exp(−(*x*_{1}−*z*_{1}(*t*))^{2}/(2*C*_{11}(*t*))) agrees very well with the analytical solution.

*P*

_{u}(

*x*,

*t*

*n*

_{s},

*t*

_{s}) and

*P*

_{nr,r}(

*t*) for a simple reacting system. (a) Plot of

*P*

_{u}(

*x*,

*t*

*n*

_{s},

*t*

_{s}) at different times. (b) Plot of

*P*

_{nr,r}(

*t*). Analytic method, moment method, and the results from the compilation of 14 000 SSA-RR runs all agree very closely.

**...**

*P*_{nr,r}(*t*), plotted in Fig. Fig.2b,2b, shows perfect overlap between the analytic method and the moment approximation. The combined results from 14 000 SSA-RR runs also show excellent agreement. Figure Figure2b2b also shows an approximation of *P*_{nr,r}(*t*) in the case where the restricted reaction propensities are held constant at *a*_{r}(*n*_{s}). This approximations result in a purely exponential curve, which differs from the exact solution. This clearly underscores the importance of accounting for the restricted reaction propensity changes due to the unrestricted reactions distribution, *P*_{u}(*x*,*t**n*_{s},*t*_{s}).

### A genetic oscillator

In this example, we simulate a model of a system developed by Vilar et al.^{33} to approximate the essential functionality of a circadian oscillator. The system is described by the following biochemical reactions:

Briefly, transcription factor *A* positively regulates both itself and the repressor protein *R*, resulting in nested positive and negative feedback loops. For the parameters used in this paper, *A* is produced at a faster rate than *R* but has a shorter half-life. As *R* is produced, it negatively regulates *A* by combining with *A* to form the complex *C*. Eventually, the population of *A* is reduced to near zero, thereby downregulating the transcription of *A* and *R*. As the population of *R* decays toward zero, as long as the reactions are stochastic, the resulting fluctuations in the concentration of *A* and *R* enable the process to repeat (oscillate). It is during the periods of near-zero numbers of *A* that the transcription factor binding of *A* and consequently the transcription of *A* and *R* are rare events. We simulate the oscillator model for three sets of parameter values, where time is measured in hours, given in Table Table11.

For case 1, the deterministic (mean path) behavior of the system plotted as the dashed line in Fig. Fig.3a3a indicates that repressor protein *R* converges to a stable steady state. However, accounting for the stochasticity in the reactions by applying the SSA results in oscillations [Fig. [Fig.3a,3a, solid curve for *R* and inset for *A*]. We define the period of oscillation as a random variable τ that captures the time difference between two successive peaks. As a metric for comparison, we will use histograms of τ produced by the different algorithms. All histograms are calculated from the results of 300 simulations where each simulation is run for 6000 h.

*R*for the mean path equations and one SSA simulation. (b) Histrograms of τ for a fixed partitioning of the

*unrestricted*reactions (gene state transitions). (c) Histrograms

**...**

In a first set of tests whose purpose is to demonstrate the equivalence of the SSA and RRA-exact algorithms and to compare the performance of RRA-mean and RRA-cov, we investigate static partitioning (partitioning is preassigned and remains constant throughout the simulation). We model the reactions that dictate the transcription factor-bound states of the genes, i.e., 19a, 19b, as unrestricted reactions and consider all other reactions to be restricted. RRA-exact and RRA-exact-m2 are easily implemented since for one copy of gene promoters, there are only four states in *P*_{u}(*x*,*t**n*_{s},*t*_{s}) to be solved for over time. This particular restricted reaction partitioning also allows all of the algorithms to use the exact discrete sampling techniques given in Appendix D, therefore, eliminating sources of error for the RRA-cov and RRA-mean algorithms. Figure Figure3b3b shows PDFs of τ, the period of oscillation. As expected, the SSA and the RRA-exact overlap. The RRA-exact-m2, which neglects the last two terms of *P*_{u} in Eq. 7, results in a thinner distribution and a lower mean. The RRA-mean algorithm has a similar distribution width to the SSA but is shifted to a higher mean. In contrast, the RRA-cov algorithm generates a PDF that agrees very well with the SSA and RRA-exact. This test again demonstrates that neglecting the effect of restricted reactions on *P*_{u} can generate error. It also demonstrates that, at least for this particular static partitioning, accounting for the covariances is both necessary and sufficient to reproduce the PDF of τ.

Next, we applied dynamic partitioning of propensities to the RRA-cov for the three difference cases described in Table Table1.1. We tested two simple partitioning methods. The first partitioning method (type I) uses a propensity threshold value *a*_{t}, below which a reaction is partitioned as restricted and above which it is considered unrestricted. We set the ODE integration time for the moments of *P*_{u} to be τ_{s}=2.0(1/∑_{}*a*_{r}(*n*_{s})) and the error threshold to ϵ_{s}=0.15. To gauge the performance of the approximation, we measure the maximum error produced by the nonlinear terms within the moments equations as a function of state update/time. A threshold value of *a*_{t}=18 h^{−1} was chosen since it optimized the run times for the three test cases. For comparison, the second partitioning method (type II) is based on the value of the protein *A*. When *A*<10, we partition all the reactions as restricted; otherwise all reactions are partitioned as unrestricted.

Figure Figure3c3c shows the results for case 1. Period distributions for RRA-cov (I) and RRA-cov (II) exhibit equally accurate distributions compared to the SSA. The differences in the distributions between the SSA vs. the RRA-cov algorithms are now mostly due to the sampling technique for *P*_{u}, specifically to the extraction of a value for *A* mRNA. In this case, when protein *A* concentration is at near-zero levels, the mRNA also exists in low numbers. At these low values, the assumptions made for the sampling technique will be inaccurate (see Appendix D for further details). In general, sampling from a multivariate distribution *P*_{u} is a challenging problem and a focus of much current research.

In case 2, the transcription rate for *A* and *R* was increased by a factor of 20. A plot of the repressor protein *R* as a function of time is shown in Fig. Fig.3d.3d. In this case, protein levels are about a factor of 20 higher than in case 1. The results of the mean-path equations are also plotted in Fig. Fig.3d3d and still converge to a unique stable equilibrium as in case 1. Period distributions for RRA-cov (I) and RRA-cov (II) are plotted in Fig. Fig.3e.3e. Here, distributions generated by the RRA-cov algorithms agree exquisitely well with the SSA, indicating that the algorithms are still applicable even for this substantially “stiffer” problem.

For case 3, the transcription rate for *A* and *R* was increased from case 1 by a factor of 200. The distributions (not shown) for the RRA-cov algorithms show a similar remarkable accuracy as in case 2.

The log of the moment error *e*_{m}(*t*), which was found to be similar for the RRA-cov algorithms in all three cases, is plotted in Fig. Fig.3f3f for a portion of a trajectory of the RRA-cov (I). The error is greatest during the pulses of the oscillator and only on occasion reaches the error threshold ϵ_{s}=0.15. One notices that there are regions in between the pulses where the error is vanished. These regions are where all unrestricted reactions are at most linear propensities, i.e., all bimolecular reactions are partitioned as restricted. An interesting feature, emphasized by these results, is that the number of state updates and the run time duration for both SSA and RRA-cov (II) scaled directly with the change in transcription rates (see Table Table2).2). In contrast, the number of state updates and run times for the RRA-cov (I) algorithm remained relatively constant as parameters changed. This is not unexpected since spreading of the timescales in the system results mainly in an increase in the number of unrestricted reactions, which are handled efficiently by the implicit ODE solver. For the less-stiff system in case 1, the gain from the reduction in the number of reactions is outweighed by the cost of the implicit ODE solver. However, as the problem becomes stiffer, the computational cost of the SSA scales up, while that of RRA, particularly RRA (I), stays relatively constant. Optimizing the interface between the RRA and the ODE solver and improving the dynamic partitioning, which was not attempted in this paper, should make this contrast all the more salient.

To probe further the computational efficiency of the RRA (I), we plotted the log of ∑_{j}*a*_{uj}(*z*(*t*)), min(*a*_{u}(*z*(*t*))), and ∑_{j}*a*_{r}(*z*(*t*)) versus time as the simulation proceeded for cases 1 and 3 [see Figs. Figs.3g,3g, ,3h].3h]. These quantities represent the fastest timescale of the unrestricted reactions, the slowest timescale of the unrestricted reactions, and the fastest timescale of the restricted reactions, respectively. The difference in the log plots of min(*a*_{u}(*z*(*t*))) (long dashed) versus ∑_{}*a*_{r}(*z*(*t*)) (short dashed) represents a measure of the minimum separation of timescales between the unrestricted and the restricted partitions. More precisely, the value of the log difference represents the order of magnitude in the ratios of the quantities. For case 1, the difference is between two and three for the early part of the oscillation and is very small for the rest of the cycle. In contrast, for case 3, the separation is two and three for most of the cycle. Furthermore, the difference in the log plots of min(*a*_{u}(*z*(*t*))) (long dashed) versus ∑_{j}*a*_{uj}(*z*(*t*)) (solid) represents a measure of the maximum separation of timescales within the unrestricted partition and is an approximate gauge for the stiffness within the unrestricted partition. For case 1, the difference is one or less, while for case 3, the difference is two to four. These results are in clear agreement with our numerical studies indicating that the SSA should be very efficient for case 1 and that the RRA should more efficient in case 3 with the use of an implicit ODE solver. These results also show that the oscillator, especially in case 3, exhibits a spectrum of timescales throughout the cycle for which the RRA is ideally suited.

### A bistable switch

Next, we use the RRA to simulate a bistable system developed by Hasty et al.^{34} as an example to discuss noise-based switches and amplifiers for gene expression. The biochemical reactions are given by

For this system, the transcription/translation of a protein *X* is aggregated into one step. It can be shown that for a deterministic mass action model, the equation for the protein abundance *x* at steady-state reduces to

where *f*(*x*)=(*k*_{1}/(*k*_{−1}+*k*_{d}))*x*(*x*−1)/2, *p*_{0} is the RNA polymerase concentration, *d*_{t} is the total number of DNA sites, *R*_{2}=*k*_{2}/*k*_{−2}, *R*_{3}=*k*_{3}/*k*_{−3}, *R*_{4}=*k*_{4}/*k*_{−4}, and *n*=2. Solving Eq. 21 yields the fixed points for the system for parameter values, where time is measured in seconds, given in Table Table33.

For case 1, the stable fixed points are *x*_{ss}=[1.45,13.23]. In the first set of tests for this case, we adopt a static partitioning where reactions that dictate the states of the genes 20b, 20c, 20d are considered to be unrestricted, while all other reactions are restricted. Figure Figure4a4a shows an example of a time plot of protein *X* for the SSA where the dashed lines represent the fixed points. Figures Figures4b,4b, ,4c4c show typical results for the RRA-mean and RRA-cov algorithms, respectively. Obviously, RRA-mean generates fluctuations of *X* around its low steady-state and does not exhibit the switching seen either by SSA or RRA-cov. This is further corroborated in Fig. Fig.4d4d where the stationary distribution of the protein *X* is plotted. The distributions calculated from the SSA, RRA-exact, and RRA-cov all agree very closely. Therefore, in order to capture the noise induced switching in this example, it is necessary to include the higher order moments. Importantly, this strongly points to the fact that such switching behavior is inherently dependent on the noise in the binding of *X*_{2} to DNA since switching is eliminated when only the mean behavior of the process is accounted for. As in the oscillator example, the results for the RRA-exact-m2 are again very poor, and the stationary distribution does not display bistable behavior [distribution is near zero for *X*>10 (Fig. (Fig.4d4d inset)].

**...**

Next, we applied two simple partitioning methods for the RRA-cov to the two cases given in Table Table3.3. For the first method (type I), we partition all reactions as unrestricted and set the maximum integration time to be τ_{s}=10 h and an error threshold of ϵ_{s}=0.15. The second method (type II) is based on the value of the protein *A*; when *A*<10, we partition the reactions as all restricted, while for *A*>=10 all reactions are partitioned as unrestricted. The maximum integration time is still τ_{s}=10 h.

For case 1, the SSA is efficient and there is no gain in computational expediency from using RRA-cov (I) or RRA-cov (II) (see Table Table44).

In case 2, we scale *k*_{1}, *k*_{2}, *k*_{3}, *k*_{4}, and *k*_{d} by α where α=1/7 (Table (Table3).3). The stable fixed points are *x*_{ss}=[15.4,90.4]. Figure Figure4e4e shows a time plot of the SSA where dashed lines represent the fixed points. Figure Figure4f4f shows the stationary distribution for the SSA, RRA-cov (I), and RRA-cov (II). All methods capture the noise-based switching. Distributions generated by SSA and RRA-cov (II) match perfectly, while that generated by RRA-cov (I) shows a perfect match to the high-state distribution but an offset in the distribution of the low state. Error in RRA-cov (I) is likely due to the moment error, as well as from the sampling of the state from *P*_{u} while the system at low concentrations.

The log of the moment error *e*_{m}(*t*) generated by RRA-cov (I) for case 2 is plotted in Fig. Fig.4f.4f. It can easily be seen that the error in the low state reaches or is near ϵ_{s}=0.15 much of the time. The error in the high state is an order of magnitude lower. Reducing the error threshold will decrease the speed of the RRA-cov (I) algorithm. This compromise is captured by the performance differences between RRA-cov (I) (faster and less accurate) and RRA-cov (II) (slower and more accurate) partitioning schemes (see Table Table44).

## SUMMARY AND FUTURE WORK

In this paper, we introduce a general formulation that decomposes a system of biochemical reactions into restricted and unrestricted sets, then rigorously derives evolution equations for both. This representation gives rise to a modified form of the CME, the CME-RR, which untangles the probability of occurrence for a restricted reaction [Eq. 6] from the probability distribution due to the unrestricted reactions [Eq. 7]. Importantly, the result of this decomposition unfolds as a set of equations that generalize some previous approximations^{19}^{, }^{21} and converge to others as a limiting case.^{22} In that sense, we have succeeded in rigorously deriving the exact equations and have illustrated how and when previous approximations arise.

Some of the previous work has ignored the coupling of the restricted reactions in Eq. 7. Our framework accounts for this coupling, a feature that was demonstrated through example to be important. However, we do recognize that in the limit of a large separation in timescales, the contribution of this coupling becomes negligible and the cost of computing it might not outweigh the gain in accuracy. In this case, using established algorithms such as the ssSSA may be more efficient.

Starting from the exact derivation, we devised the RRA algorithm for the exact system and the approximate moment system. The accuracy and speed of the moment system are presented in a series of examples. We found that the algorithm is crucially dependent on accurate sampling from *P*_{u}. In the case where *P*_{u} is analytically or numerically solvable, the accuracy of the algorithm is excellent, i.e., the static partition tests for the oscillator and bistable examples. Our process of sampling from the distribution *P*_{u} knowing only its moments (Appendix D) has not been optimized, and we are currently investigating more general methods. We are also investigating the applicability of alternative moment truncation approximations based on log-normal closure methods^{26} while still truncating at the covariances.

The bistable and oscillator examples show many regions of parameter space where the RRA algorithm is more efficient than, but has comparable accuracy, to the SSA. There are parameter regimes, however, where it seems to be less efficient. This is strongly dependent on the optimization of the partitioning. For the oscillator example, the simple propensity threshold method (type I) was the most robust in terms of accuracy and efficiency with respect to the reaction rate scaling. In the bistable system, partitioning all reactions as unrestricted (type I) was the most efficient, while type II partitioning method provided the most accuracy. Ultimately, for a given level of accuracy, optimizing the dynamic partitioning by minimizing the number of computations/operations per unit time will be the most efficient.

Despite these sources of inefficiency in the implementation of the RRA, there are two important features of the algorithm that we would like to emphasize. First, the specific formulation makes the RRA method dynamically applicable, limiting to the SSA on one extreme and the ssSSA on the other but also being able to tackle situations where timescales are spread over multiple scales. Second, while the computational cost of the SSA seems to scale directly with the increase in the number of molecular species and stiffness of the problem, the cost of the RRA algorithm stays relatively constant by construction. These are valuable properties that underlie the applicability of our method to the study of realistic complex biological problems.

In this work, we have not attempted to compare our results to those of τ-leaping algorithms. We believe that there will be cases when these algorithms may be more efficient/accurate and vice versa. Ultimately, different simulations algorithms should be coupled and utilized depending on the specifics of any given problem. However, we would like to point out a crucial distinction that differentiates our framework. While τ-leaping algorithms jump from one state to the next and generate a single trajectory, the RRA algorithm evolves the distribution *P*_{u} between any two states, therefore generating information that can be exploited to derive various characteristics of the system.

## ACKNOWLEDGMENTS

This work was supported by the UCSF Program for breakthrough biomedical research and the NIH to H.E. We would like to thank members of the El-Samad Laboratory for critical reading of the manuscript.

#### APPENDIX A: DERIVATION OF EQUATIONS 6, 7

In this section we derive Eqs. 6, 7. Since *W*(*x*,*t**n*_{s},*t*_{s})=*P*_{nr,r}(*t*)*P*_{u}(*x*,*t**n*_{s},*t*_{s}), then

where we have replaced *W*(*x*,*t**n*_{s},*t*_{s}) in Eq. 3 with *P*_{nr,r}(*t*)*P*_{u}(*x*,*t**n*_{s},*t*_{s}) throughout.

Summing over *x* (∑_{x}) and using the fact that ∑_{x}*P*_{u}(*x*,*t**n*_{s},*t*_{s})=1, we obtain

Equation 6 can be easily be seen as the solution to Eq. A2. Substituting the explicit expression for *P*_{nr,r}(*t*)/*t* from Eq. A2 into Eq. A1 yields

Dividing through by *P*_{nr,r}(*t*) and rearranging the terms yields Eq. 7. The effect of the final term in Eq. 7 is essentially to renormalize *P*_{u}(*x*,*t**n*_{s},*t*_{s}) and enforce that ∑_{x}*P*_{u}(*x*,*t**n*_{s},*t*_{s})=1. This term also makes Eq. 7 a nonlinear equation.

#### APPENDIX B: MOMENT DERIVATIONS

To calculate the time-dependent equations for moments, we use the method by Engblom^{25} where moment equations for the CME were derived by assuming quadratic propensities. Time-dependent moment equations can be computed using test functions *T*(*x*) and a summation over *x* in Eq. 7. The general form of the time-dependent moment equation is

For our purposes *T*(*x*) will either be *T*(*x*)=*x*_{k}, the first moment of the *k*th species, or *T*(*x*)=[*x*_{k}−*z*_{k}(*t*)][*x*_{k′}−*z*_{k′}(*t*)], the second central moment of the *k*th and *k*^{′}th species. Obtaining a given time-dependent moment equation involves transforming the propensities in Eq. B1 to their Taylor series form from Eq. 10. After some manipulation, one obtains terms for moments of various orders. Terms of the form ∑_{x}*x*_{j}*P*_{u}(*x*,*t**n*_{s},*t*_{s}) are then represented by *z*_{j}(*t*). Terms of the form ∑_{x}[*x*_{j}−*z*_{j}(*t*)]*P*_{u}(*x*,*t**n*_{s},*t*_{s}) are zero. Second order central moments ∑_{x}[*x*_{j}−*z*_{j}(*t*)][*x*_{j′}−*z*_{j′}(*t*)]*P*_{u}(*x*,*t**n*_{s},*t*_{s}) can be represented by the covariance *C*_{jj′}. Any third or fourth order terms are left in their integral form. The terms in Eqs. 12, 13 involving *unrestricted* reactions are taken directly from the work of Engblom.^{25} However, the terms involving *restricted* reactions, i.e., the last two terms on the right in Eq. B1, have to be derived for each time-dependent moment of interest. We first analyze the term ${\sum}_{x}T\left(x\right){\sum}_{\ell =1}^{M-K}{a}_{{r}_{\ell}}\left(x\right){P}_{u}\left(x,t\mid {n}_{s},{t}_{s}\right)$. When *T*(*x*)=*x*_{k}, we have

which contributes to Eq. 12. When *T*(*x*)=[*x*_{k}−*z*_{k}(*t*)][_{xk′}−_{zk′}(*t*)], the same term becomes

which contributes to Eq. 13. For second term on the right hand side of Eq. B1 we have

which then contributes to the time-dependent moment equation for a particular *T*(*x*). Equations 12, 13 represent the final contributions of adding up all the derived moment terms.

#### APPENDIX C: DERIVATION OF *P*(*t*^{′},μ*t*^{′}<*t*_{s}+τ_{s})

The probability that a restricted reaction occurred before *t*^{′} would simply be *P*_{r,r}(*t*^{′})=1−*P*_{nr,r}(*t*^{′}), both of which are cumulative distribution functions. Therefore the PDF for a restricted reaction to occur as a function of *t*^{′} would simply by *P*(*t*^{′})=*P*_{r,r}(*t*^{′})/*t*^{′}=−*P*_{nr,r}(*t*^{′})/*t*^{′}. Directly applying Eq. A2 yields

Next, the conditional PDF for a restricted reaction to occur as a function of *t*^{′}, given that it occurred before *t*_{s}+τ_{s}, would be

Finally, the probability that a restricted reaction of type μ occurs given a restricted reaction occurred at time *t*^{′}, by construction, is

which represents the weighted sum over all possible states from which a restricted reaction of type μ could occur, divided by the weighted sum over all possible states from which a restricted reaction could occur. Therefore *P*(*t*^{′},μ*t*^{′}<*t*_{s}+τ_{s})=*P*(μ*t*^{′})*P*(*t*^{′}*t*^{′}<*t*_{s}+τ_{s}), yielding Eq. 8. To sample *P*(*t*^{′},μ*t*^{′}<*t*_{s}+τ_{s}), we first sample from the time of the restricted reaction *P*(*t*^{′}*t*^{′}<*t*_{s}+τ_{s}) and then we sample the restricted reaction type from *P*(μ*t*^{′}).

#### APPENDIX D: SAMPLING FROM THE VARIOUS DISTRIBUTIONS

Below we discuss sampling from all the distributions involved in the RRA algorithm.

##### Sampling the time of the restricted reaction

To sample *t*^{′}, we must sample from the cumulative distribution function of *P*(*t*^{′}*t*^{′}<*t*_{s}+τ_{s}), which we know to be (1−*P*_{nr,r}(*t*^{′}))/(1−*P*_{nr,r}(*t*_{s}+τ_{s})). We therefore generate random number *p* and solve *P*_{nr,t} and *P*_{u} or its moments until the time *t*^{′} that satisfies *p*=(1−*P*_{nr,r}(*t*^{′}))/(1−*P*_{nr,r}(*t*_{s}+τ_{s})).

##### Sampling the restricted reaction

We discuss sampling from *P*(μ*t*^{′}). Let *G*(μ*t*^{′}) be the cumulative distribution function of *P*(μ*t*^{′}). We generate a random number *p* from a uniform distribution between zero and one and then choose μ that satisfies *G*(μ−1*t*^{′})<*p*<=*G*(μ*t*^{′}). For the exact case

and for the moment system

##### Sampling a state from *P*(*x**t*^{′},μ) for the discrete distribution

Given the time of the restricted reaction *t*^{′} and the restricted reaction type μ, we now can sample the state *n*_{s+1−} from *P*(*x**t*^{′},μ). We define *G*(*x**t*^{′},μ) as the cumulative distribution function of *P*(*x**t*^{′},μ). This is applicable when *P*_{u}(*x*,*t*^{′}*n*_{s},*t*_{s}) can be represented by a finite number of *N* discrete states. To sample a state we generate a random number *p* from a uniform distribution between zero and one and then choose the state with index *I* that satisfies the inequality *G*(*I*−1*t*^{′},μ)<*p*<=*G*(*I**t*^{′},μ), where

##### Sampling a state from *P*(*x**t*^{′},μ) when there is only moment information for *P*_{u}

Sampling from *P*(*x**t*^{′},μ) when only the moment information for *P*_{u} is available is a difficult problem and an important topic of current research. We apply a heuristic method. The first step is to impose constraints on the distribution based on the restricted reaction that occurred. For instance, let us assume that the restricted reaction is a transcription of mRNA. Let us also assume that the transcription could only occur from the gene state *J* of *N* possible gene states and that the transition between gene states are unrestricted reactions. We would then impose that the gene is in the state *J* at time *t*^{′}.

For this paper, the genes that we model use positively regulating transcription factors. The transitions between gene states are also modeled as unrestricted reactions only when transcription factor numbers are relatively abundant. When these conditions hold, the states of a given gene are independent of the states of another gene. This is because each gene is relatively independent of fluctuations in the transcription factor. This can be verified by checking the covariances between the various genes and transcription factors, ensuring they are near zero. Given that these conditions hold, we can justifiably sample the state of each gene independently. For each gene, the probability that the gene is in a particular state is simply that state’s mean value. Accordingly, we can setup a cumulative distribution function for each gene and sample from it.

For the remaining species with their corresponding mean and covariances, we assume that they fit a multivariate Gaussian distribution *P*_{r}(*x*). Technically, we must sample from the distribution, *a*_{rμ}(*x*)*P*_{r}(*x*) (or rather a normalized version of it). However, the function *a*_{rμ}(*x*) is up to quadratic and complicates the sampling. To simplify the sampling procedure, in this paper, we make an assumption that *a*_{rμ}(*x*) is relatively constant over the range of *P*_{r}(*x*) allowing one to sample just from the multivariate Gaussian distribution *P*_{r}(*x*). For the multivariate Gaussian, we apply the standard sampling through the Cholesky factoring of the covariance matrix.^{35} Each species sampled from *P*_{r}(*x*) is then rounded to the nearest non-negative integer to ensure that ${n}_{s+1-}\u220a{\mathbf{Z}}_{+}^{N}$. For future work, we aim to devise methods to sample the distribution more systematically.

##### Sampling a state from *P*_{u} for the discrete distribution

Given that no restricted reaction occurred by time *t*_{s}+τ_{s}, we must sample a state from *P*_{u}. We define *G*_{u} as the cumulative distribution function of *P*_{u}. This is applicable when *P*_{u}(*x*,*t*_{s}+τ_{s}*n*_{s},*t*_{s}) can be represented by a finite number of *N* discrete states. To sample a state from *P*_{u} we generate a random number *p* from a uniform distribution between zero and one and than choose the state with index *I* that satisfies the inequality *G*_{u}(*I*−1,*t*_{s}+τ_{s}*n*_{s},*t*_{s})<*p*<=*G*_{u}(*I*,*t*_{s}+τ_{s}*n*_{s},*t*_{s}), where

##### Sampling a state from *P*_{u} when there is only moment information

Given that no restricted reaction occurred by time *t*_{s}+τ_{s}, we must sample a state from an approximate version of *P*_{u}, i.e., ${\overline{P}}_{u}$, given only that we have only solved through the covariances. The basic procedure is the same as that from Appendix D4, but where we do not weight the distribution by the restricted reaction propensity since a restricted reaction did not occur.

## References

- Gregor T., Wieschaus E., McGregor A., Bialek W., and Tank D., Cell 130, 141 (2007).10.1016/j.cell.2007.05.026 [PMC free article] [PubMed] [Cross Ref]
- Gregor T., Tank D., Wieschaus E., and Bialek W., Cell 130, 153 (2007).10.1016/j.cell.2007.05.025 [PMC free article] [PubMed] [Cross Ref]
- Balaban N., Merrin J., Chait R., Kowalik L., and Leibler S., Science 305, 1622 (2004).10.1126/science.1099390 [PubMed] [Cross Ref]
- Kaern M., Elston T., Blake W., and Collins J., Nat. Rev. Genet. 6, 451 (2005).10.1038/nrg1615 [PubMed] [Cross Ref]
- McAdams H. and Arkin A., Proc. Natl. Acad. Sci. U.S.A. 94, 814 (1997).10.1073/pnas.94.3.814 [PMC free article] [PubMed] [Cross Ref]
- Kepler T. and Elston T., Biophys. J. 81, 3116 (2001).10.1016/S0006-3495(01)75949-8 [PMC free article] [PubMed] [Cross Ref]
- Swain P., Elowitz M., and Siggia E., Proc. Natl. Acad. Sci. U.S.A. 99, 12795 (2002).10.1073/pnas.162041399 [PMC free article] [PubMed] [Cross Ref]
- Blake W., Kærn N., Cantor C., and Collins J., Nature (London) 422, 633 (2003).10.1038/nature01546 [PubMed] [Cross Ref]
- Raser J. and O’Shea E., Science 304, 1811 (2004).10.1126/science.1098641 [PMC free article] [PubMed] [Cross Ref]
- Levin M., FEBS Lett. 550, 135 (2003).10.1016/S0014-5793(03)00857-3 [PubMed] [Cross Ref]
- Suel G., Kulkarni R., Dworkin J., Garcia-Ojalvo J., and Elowitz M., Science 315, 1716 (2007).10.1126/science.1137455 [PubMed] [Cross Ref]
- McQuarrie D., J. Appl. Probab. 4, 413 (1967).10.2307/3212214 [Cross Ref]
- Gillespie D., Physica A 188, 404 (1992).10.1016/0378-4371(92)90283-V [Cross Ref]
- van Kampen N., Stochastic Processes in Physics and Chemistry (North-Holland, Amsterdam, 1992).
- Gillespie D., J. Comput. Phys. 22, 403 (1976).10.1016/0021-9991(76)90041-3 [Cross Ref]
- Gillespie D., J. Chem. Phys. 115, 1716 (2001).10.1063/1.1378322 [Cross Ref]
- Rathinam M., Petzold L., Cao Y., and Gillespie D., J. Chem. Phys. 119, 12784 (2003).10.1063/1.1627296 [Cross Ref]
- Rathinam M. and El-Samad H., J. Comput. Phys. 224, 897 (2007).10.1016/j.jcp.2006.10.034 [Cross Ref]
- Haseltine E. and Rawlings J., J. Chem. Phys. 117, 6959 (2002).10.1063/1.1505860 [Cross Ref]
- Gillespie D., J. Chem. Phys. 113, 297 (2000).10.1063/1.481811 [Cross Ref]
- Rao C. and Arkin A., J. Comput. Phys. 227, 100 (2003).
- Cao Y., Gillespie D., and Petzold L., J. Chem. Phys. 122, 014116 (2005).10.1063/1.1824902 [PubMed] [Cross Ref]
- Lötstedt P. and Hellander A., J. Comput. Phys. 227, 100 (2007).10.1016/j.jcp.2007.07.020 [Cross Ref]
- Weinan E., Liu D., and Vanden-Eijnden E., J. Comput. Phys. 221, 158 (2007).10.1016/j.jcp.2006.06.019 [Cross Ref]
- Engblom S., Appl. Math. Comput. 180, 498 (2006).10.1016/j.amc.2005.12.032 [Cross Ref]
- Singh A. and Hespanha J., Bull. Math. Biol. 69, 1909 (2007).10.1007/s11538-007-9198-9 [PubMed] [Cross Ref]
- Byrne G. and Hindmarsh A., Int. J. High Perform. Comput. Appl. 13, 354 (1999).10.1177/109434209901300405 [Cross Ref]
- Cohen S. and Hindmarsh A., Comput. Phys. 10, 138 (1996).
- Hindmarsh A., Brown P., Grant K., Lee S., Serban R., Shumaker D., and Woodward C., ACM Trans. Math. Softw. 31, 363 (2005).10.1145/1089014.1089020 [Cross Ref]
- Balay S., Gropp W., McInnes L., and Smith B., Modern Software Tools in Scientific Computing (Birkhäuser, Basel, 1997).
- Balay S., Buschelman K., Eijkhout V., Gropp W., Kaushik D., Knepley M., McInnes L., Smith B., and Zhang H., PETSc User’s Manual, ANL-95/11-Revision 2.1.5, Argonne National Laboratory (2004).
- Munsky B. and Khammash M., J. Chem. Phys. 124, 044104 (2006).10.1063/1.2145882 [PubMed] [Cross Ref]
- Vilar J., Kueh H., Barkai N., and Leibler S., Proc. Natl. Acad. Sci. U.S.A. 99, 5988 (2002).10.1073/pnas.092133899 [PMC free article] [PubMed] [Cross Ref]
- Hasty J., Pradines J., Dolnik M., and Collins J., Proc. Natl. Acad. Sci. U.S.A. 97, 2075 (2000).10.1073/pnas.040411297 [PMC free article] [PubMed] [Cross Ref]
- Barr D. and Slezak N., Commun. ACM 15, 1048 (1972).10.1145/361598.361620 [Cross Ref]

**American Institute of Physics**

## Formats:

- Article |
- PubReader |
- ePub (beta) |
- PDF (660K)

- Stochastic modeling of cellular networks.[Methods Cell Biol. 2012]
*Stewart-Ornstein J, El-Samad H.**Methods Cell Biol. 2012; 110:111-37.* - Stochastic approaches in systems biology.[Wiley Interdiscip Rev Syst Biol Med. 2010]
*Ullah M, Wolkenhauer O.**Wiley Interdiscip Rev Syst Biol Med. 2010 Jul-Aug; 2(4):385-97.* - The finite state projection algorithm for the solution of the chemical master equation.[J Chem Phys. 2006]
*Munsky B, Khammash M.**J Chem Phys. 2006 Jan 28; 124(4):044104.* - Discrete-time stochastic modeling and simulation of biochemical networks.[Comput Biol Chem. 2008]
*Sandmann W.**Comput Biol Chem. 2008 Aug; 32(4):292-7. Epub 2008 Apr 10.* - Modeling cell-to-cell stochastic variability in intrinsic apoptosis pathway.[Conf Proc IEEE Eng Med Biol Soc. 2012]
*Ooi HK, Ma L.**Conf Proc IEEE Eng Med Biol Soc. 2012; 2012:5498-501.*

- Constructing stochastic models from deterministic process equations by propensity adjustment[BMC Systems Biology. ]
*Wu J, Vidakovic B, Voit EO.**BMC Systems Biology. 5187* - Stochastic hybrid modeling of intracellular calcium dynamics[The Journal of Chemical Physics. 2010]
*Choi T, Maurya MR, Tartakovsky DM, Subramaniam S.**The Journal of Chemical Physics. 2010 Oct 28; 133(16)165101*

- MedGenMedGenRelated information in MedGen
- PubMedPubMedPubMed citations for these articles
- TaxonomyTaxonomyRelated taxonomy entry
- Taxonomy TreeTaxonomy Tree

- A rigorous framework for multiscale simulation of stochastic cellular networksA rigorous framework for multiscale simulation of stochastic cellular networksThe Journal of Chemical Physics. Aug 7, 2009; 131(5)PMC

Your browsing activity is empty.

Activity recording is turned off.

See more...