How accurate are the nonlinear chemical Fokker-Planck and chemical Langevin equations?

Optimal control of bioproduction in the presence of population heterogeneity

Cell-to-cell variability, born of stochastic chemical kinetics, persists even in large isogenic populations. In the study of single-cell dynamics this is typically accounted for. However, on the population level this source of heterogeneity is often sidelined to avoid the inevitable complexity it introduces. The homogeneous models used instead are more tractable but risk disagreeing with their heterogeneous counterparts and may thus lead to severely suboptimal control of bioproduction. In this work, we introduce a comprehensive mathematical framework for solving bioproduction optimal control problems in the presence of heterogeneity. We study population-level models in which such heterogeneity is retained, and propose order-reduction approximation techniques. The reduced-order models take forms typical of homogeneous bioproduction models, making them a useful benchmark by which to study the importance of heterogeneity. Moreover, the derivation from the heterogeneous setting sheds light on parameter selection in ways a direct homogeneous outlook cannot, and reveals the source of approximation error. With view to optimally controlling bioproduction in microbial communities, we ask the question: when does optimising the reduced-order models produce strategies that work well in the presence of population heterogeneity? We show that, in some cases, homogeneous approximations provide remarkably accurate surrogate models. Nevertheless, we also demonstrate that this is not uniformly true: overlooking the heterogeneity can lead to significantly suboptimal control strategies. In these cases, the heterogeneous tools and perspective are crucial to optimise bioproduction.

Mathematical Modelling in Plant Synthetic Biology

Mathematical modelling techniques are integral to current research in plant synthetic biology. Modelling approaches can provide mechanistic understanding of a system, allowing predictions of behaviour and thus providing a tool to help design and analyse biological circuits. In this chapter, we provide an overview of mathematical modelling methods and their significance for plant synthetic biology. Starting with the basics of dynamics, we describe the process of constructing a model over both temporal and spatial scales and highlight crucial approaches, such as stochastic modelling and model-based design. Next, we focus on the model parameters and the techniques required in parameter analysis. We then describe the process of selecting a model based on tests and criteria and proceed to methods that allow closer analysis of the system's behaviour. Finally, we highlight the importance of uncertainty in modelling approaches and how to deal with a lack of knowledge, noisy data, and biological variability; all aspects that play a crucial role in the cooperation between the experimental and modelling components. Overall, this chapter aims to illustrate the importance of mathematical modelling in plant synthetic biology, providing an introduction for those researchers who are working with or working on modelling techniques.

Stochastic pH Oscillations in a Model of the Urea-Urease Reaction Confined to Lipid Vesicles

The urea-urease clock reaction is a pH switch from acid to basic that can turn into a pH oscillator if it occurs inside a suitable open reactor. We numerically study the confinement of the reaction to lipid vesicles, which permit the exchange with an external reservoir by differential transport, enabling the recovery of the pH level and yielding a constant supply of urea molecules. For microscopically small vesicles, the discreteness of the number of molecules requires a stochastic treatment of the reaction dynamics. Our analysis shows that intrinsic noise induces a significant statistical variation of the oscillation period, which increases as the vesicles become smaller. The mean period, however, is found to be remarkably robust for vesicle sizes down to approximately 200 nm, but the periodicity of the rhythm is gradually destroyed for smaller vesicles. The observed oscillations are explained as a canard-like limit cycle that differs from the wide class of conventional feedback oscillators.

Solving the chemical master equation for monomolecular reaction systems and beyond: a Doi-Peliti path integral view

The chemical master equation (CME) is a fundamental description of interacting molecules commonly used to model chemical kinetics and noisy gene regulatory networks. Exact time-dependent solutions of the CME-which typically consists of infinitely many coupled differential equations-are rare, and are valuable for numerical benchmarking and getting intuition for the behavior of more complicated systems. Jahnke and Huisinga's landmark calculation of the exact time-dependent solution of the CME for monomolecular reaction systems is one of the most general analytic results known; however, it is hard to generalize, because it relies crucially on special properties of monomolecular reactions. In this paper, we rederive Jahnke and Huisinga's result on the time-dependent probability distribution and moments of monomolecular reaction systems using the Doi-Peliti path integral approach, which reduces solving the CME to evaluating many integrals. While the Doi-Peliti approach is less intuitive, it is also more mechanical, and hence easier to generalize. To illustrate how the Doi-Peliti approach can go beyond the method of Jahnke and Huisinga, we also find an explicit and exact time-dependent solution to a problem involving an autocatalytic reaction that Jahnke and Huisinga identified as not solvable using their method. Most interestingly, we are able to find a formal exact time-dependent solution for any CME whose list of reactions involves only zero and first order reactions, which may be the most general result currently known. This formal solution also yields a useful algorithm for efficiently computing numerical solutions to CMEs of this type.

Three-dimensional stochastic simulation of chemoattractant-mediated excitability in cells

During the last decade, a consensus has emerged that the stochastic triggering of an excitable system drives pseudopod formation and subsequent migration of amoeboid cells. The presence of chemoattractant stimuli alters the threshold for triggering this activity and can bias the direction of migration. Though noise plays an important role in these behaviors, mathematical models have typically ignored its origin and merely introduced it as an external signal into a series of reaction-diffusion equations. Here we consider a more realistic description based on a reaction-diffusion master equation formalism to implement these networks. In this scheme, noise arises naturally from a stochastic description of the various reaction and diffusion terms. Working on a three-dimensional geometry in which separate compartments are divided into a tetrahedral mesh, we implement a modular description of the system, consisting of G-protein coupled receptor signaling (GPCR), a local excitation-global inhibition mechanism (LEGI), and signal transduction excitable network (STEN). Our models implement detailed biochemical descriptions whenever this information is available, such as in the GPCR and G-protein interactions. In contrast, where the biochemical entities are less certain, such as the LEGI mechanism, we consider various possible schemes and highlight the differences between them. Our simulations show that even when the LEGI mechanism displays perfect adaptation in terms of the mean level of proteins, the variance shows a dose-dependence. This differs between the various models considered, suggesting a possible means for determining experimentally among the various potential networks. Overall, our simulations recreate temporal and spatial patterns observed experimentally in both wild-type and perturbed cells, providing further evidence for the excitable system paradigm. Moreover, because of the overall importance and ubiquity of the modules we consider, including GPCR signaling and adaptation, our results will be of interest beyond the field of directed migration.

Though the term noise usually carries negative connotations, it can also contribute positively to the characteristic dynamics of a system. In biological systems, where noise arises from the stochastic interactions between molecules, its study is usually confined to genetic regulatory systems in which copy numbers are small and fluctuations large. However, noise can have important roles when the number of signaling molecules is large. The extension of pseudopods and the subsequent motion of amoeboid cells arises from the noise-induced trigger of an excitable system. Chemoattractant signals bias this triggering thereby directing cell motion. To date, this paradigm has not been tested by mathematical models that account accurately for the noise that arises in the corresponding reactions. In this study, we employ a reaction-diffusion master equation approach to investigate the effects of noise. Using a modular approach and a three-dimensional cell model with specific subdomains attributed to the cell membrane and cortex, we explore the spatiotemporal dynamics of the system. Our simulations recreate many experimentally-observed cell behaviors thereby supporting the biased-excitable network hypothesis.

Thermodynamic Optimality of Glycolytic Oscillations

Temporal order in living matters reflects the self-organizing nature of dynamical processes driven out of thermodynamic equilibrium. Because of functional reasons, the period of a biochemical oscillation must be tuned to a specific value with precision; however, according to the thermodynamic uncertainty relation (TUR), the precision of the oscillatory period is constrained by the thermodynamic cost of generating it. After reviewing the basics of chemical oscillations using the Brusselator as a model system, we study the glycolytic oscillation generated by octameric phosphofructokinase (PFK), which is known to display a period of several minutes. By exploring the phase space of glycolytic oscillations, we find that the glycolytic oscillation under the cellular condition is realized in a cost-effective manner. Specifically, over the biologically relevant range of parameter values of glycolysis and octameric PFK, the entropy production from the glycolytic oscillation is minimal when the oscillation period is (5-10) min. Furthermore, the glycolytic oscillation is found at work near the phase boundary of limit cycles, suggesting that a moderate increase of glucose injection rate leads to the loss of oscillatory dynamics, which is reminiscent of the loss of pulsatile insulin release resulting from elevated blood glucose level.

Data-driven model reduction of agent-based systems using the Koopman generator

The dynamical behavior of social systems can be described by agent-based models. Although single agents follow easily explainable rules, complex time-evolving patterns emerge due to their interaction. The simulation and analysis of such agent-based models, however, is often prohibitively time-consuming if the number of agents is large. In this paper, we show how Koopman operator theory can be used to derive reduced models of agent-based systems using only simulation data. Our goal is to learn coarse-grained models and to represent the reduced dynamics by ordinary or stochastic differential equations. The new variables are, for instance, aggregated state variables of the agent-based model, modeling the collective behavior of larger groups or the entire population. Using benchmark problems with known coarse-grained models, we demonstrate that the obtained reduced systems are in good agreement with the analytical results, provided that the numbers of agents is sufficiently large.

Effects of noise and time delay on E2F's expression level in a bistable Rb-E2F gene's regulatory network

The bistable Rb-E2F gene regulatory network plays a central role in regulating cellular proliferation-quiescence transition. Based on Gillespie's chemical Langevin method, the stochastic bistable Rb-E2F gene's regulatory network with time delays is proposed. It is found that under the moderate intensity of internal noise, delay in the Cyclin E synthesis rate can greatly increase the average concentration value of E2F. When the delay is considered in both E2F-related positive feedback loops, within a specific range of delay (3-13) hr , the average expression of E2F is significantly increased. Also, this range is in the scope with that experimentally given by Dong et al. [65]. By analysing the quasi-potential curves at different delay times, simulation results show that delay regulates the dynamic behaviour of the system in the following way: small delay stabilises the bistable system; the medium delay is conducive to a high steady-state, making the system fluctuate near the high steady-state; large delay induces approximately periodic transitions between high and low steady-state. Therefore, by regulating noise and time delay, the cell itself can control the expression level of E2F to respond to different situations. These findings may provide an explanation of some experimental result intricacies related to the cell cycle.

On rapid oscillations driving biological processes at disparate timescales

We consider a generic biological process described by a dynamical system, subject to an input signal with a high-frequency periodic component. The rapid oscillations of the input signal induce inherently multiscale dynamics, motivating order-reduction techniques. It is intuitive that the system behaviour is well approximated by its response to the averaged input signal. However, changes to the high-frequency component that preserve the average signal are beyond the reach of such intuitive reasoning. In this study, we explore system response under the influence of such an input signal by exploiting the timescale separation between high-frequency input variations and system response time. Employing the asymptotic method of multiple scales, we establish that, in some circumstances, the intuitive approach is simply the leading-order asymptotic contribution. We focus on higher-order corrections that capture the response to the details of the high-frequency component beyond its average. This approach achieves a reduction in system complexity while providing valuable insight into the structure of the response to the oscillations. We develop the general theory for nonlinear systems, while highlighting the important case of systems affine in the state and the input signal, presenting examples of both discrete and continuum state spaces. Importantly, this class of systems encompasses biochemical reaction networks described by the chemical master equation and its continuum approximations. Finally, we apply the framework to a nonlinear system describing mRNA translation and protein expression previously studied in the literature. The analysis shines new light on several aspects of the system quantification and both extends and simplifies results previously obtained.

Agent-based modeling: Population limits and large timescales

Modeling, simulation, and analysis of interacting agent systems is a broad field of research, with existing approaches reaching from informal descriptions of interaction dynamics to more formal, mathematical models. In this paper, we study agent-based models (ABMs) given as continuous-time stochastic processes and their pathwise approximation by ordinary and stochastic differential equations (SDEs) for medium to large populations. By means of an appropriately adapted transfer operator approach, we study the behavior of the ABM process on long time scales. We show that, under certain conditions, the transfer operator approach allows us to bridge the gap between the pathwise results for large populations on finite timescales, i.e., the SDE limit model, and approaches built to study dynamical behavior on long time scales like large deviation theory. The latter provides a rigorous analysis of rare events including the associated asymptotic rates on timescales that scale exponentially with the population size. We demonstrate that it is possible to reveal metastable structures and timescales of rare events of the ABM process by finite-length trajectories of the SDE process for large enough populations. This approach has the potential to drastically reduce computational effort for the analysis of ABMs.

Theoretical study of the impact of adaptation on cell-fate heterogeneity and fractional killing

Fractional killing illustrates the cell propensity to display a heterogeneous fate response over a wide range of stimuli. The interplay between the nonlinear and stochastic dynamics of biochemical networks plays a fundamental role in shaping this probabilistic response and in reconciling requirements for heterogeneity and controllability of cell-fate decisions. The stress-induced fate choice between life and death depends on an early adaptation response which may contribute to fractional killing by amplifying small differences between cells. To test this hypothesis, we consider a stochastic modeling framework suited for comprehensive sensitivity analysis of dose response curve through the computation of a fractionality index. Combining bifurcation analysis and Langevin simulation, we show that adaptation dynamics enhances noise-induced cell-fate heterogeneity by shifting from a saddle-node to a saddle-collision transition scenario. The generality of this result is further assessed by a computational analysis of a detailed regulatory network model of apoptosis initiation and by a theoretical analysis of stochastic bifurcation mechanisms. Overall, the present study identifies a cooperative interplay between stochastic, adaptation and decision intracellular processes that could promote cell-fate heterogeneity in many contexts.

Predicting properties of the stationary probability currents for two-species reaction systems without solving the Fokker-Planck equation

We derive methods for estimating the topology of the stationary probability current j[over ⃗]_{s} of the two-species Fokker-Planck equation (FPE) without the need to solve the FPE. These methods are chosen such that they become exact in certain limits, such as infinite system size or vanishing coupling between species in the diffusion matrix. The methods make predictions about the fixed points of j[over ⃗]_{s} and their relation to extrema of the stationary probability distribution and to fixed points of the convective field, which is related to the deterministic drift of the system. Furthermore, they predict the rotation sense of j[over ⃗]_{s} around extrema of the stationary probability distribution. Even though these methods cannot be proven to be valid away from extrema, the boundary lines between regions with different rotation senses are obtained with surprising accuracy. We illustrate and test these methods, using simple reaction systems with only one coupling term between the two species as well as a few generic reaction networks taken from the literature. We use it also to investigate the shape of nonphysical probability currents occurring in reaction systems with detailed balance due to the approximations involved in deriving the Fokker-Planck equation.

Chemical Langevin equation: A path-integral view of Gillespie's derivation

In 2000, Gillespie rehabilitated the chemical Langevin equation (CLE) by describing two conditions that must be satisfied for it to yield a valid approximation of the chemical master equation (CME). In this work, we construct an original path-integral description of the CME and show how applying Gillespie's two conditions to it directly leads to a path-integral equivalent to the CLE. We compare this approach to the path-integral equivalent of a large system size derivation and show that they are qualitatively different. In particular, both approaches involve converting many sums into many integrals, and the difference between the two methods is essentially the difference between using the Euler-Maclaurin formula and using Riemann sums. Our results shed light on how path integrals can be used to conceptualize coarse-graining biochemical systems and are readily generalizable.

Stochastic Modeling of Autoregulatory Genetic Feedback Loops: A Review and Comparative Study

Autoregulatory feedback loops are one of the most common network motifs. A wide variety of stochastic models have been constructed to understand how the fluctuations in protein numbers in these loops are influenced by the kinetic parameters of the main biochemical steps. These models differ according to 1) which subcellular processes are explicitly modeled, 2) the modeling methodology employed (discrete, continuous, or hybrid), and 3) whether they can be analytically solved for the steady-state distribution of protein numbers. We discuss the assumptions and properties of the main models in the literature, summarize our current understanding of the relationship between them, and highlight some of the insights gained through modeling.

Thermodynamics of switching in multistable non-equilibrium systems

Multistable non-equilibrium systems are abundant outcomes of nonlinear dynamics with feedback, but still relatively little is known about what determines the stability of the steady states and their switching rates in terms of entropy and entropy production. Here, we will link fluctuation theorems for the entropy production along trajectories with the action obtainable from the Freidlin-Wentzell theorem to elucidate the thermodynamics of switching between states in the large volume limit of multistable systems. We find that the entropy production at steady state plays no role, but the entropy production during switching is key. Steady-state entropy and diffusive noise strength can be neglected in this limit. The relevance to biological, ecological, and climate models is apparent.

Discrete and continuous models of probability flux of switching dynamics: Uncovering stochastic oscillations in a toggle-switch system

The probability flux and velocity in stochastic reaction networks can help in characterizing dynamic changes in probability landscapes of these networks. Here, we study the behavior of three different models of probability flux, namely, the discrete flux model, the Fokker-Planck model, and a new continuum model of the Liouville flux. We compare these fluxes that are formulated based on, respectively, the chemical master equation, the stochastic differential equation, and the ordinary differential equation. We examine similarities and differences among these models at the nonequilibrium steady state for the toggle switch network under different binding and unbinding conditions. Our results show that at a strong stochastic condition of weak promoter binding, continuum models of Fokker-Planck and Liouville fluxes deviate significantly from the discrete flux model. Furthermore, we report the discovery of stochastic oscillation in the toggle-switch system occurring at weak binding conditions, a phenomenon captured only by the discrete flux model.

Frequency spectrum of chemical fluctuation: A probe of reaction mechanism and dynamics

Even in the steady-state, the number of biomolecules in living cells fluctuates dynamically, and the frequency spectrum of this chemical fluctuation carries valuable information about the dynamics of the reactions creating these biomolecules. Recent advances in single-cell techniques enable direct monitoring of the time-traces of the protein number in each cell; however, it is not yet clear how the stochastic dynamics of these time-traces is related to the reaction mechanism and dynamics. Here, we derive a rigorous relation between the frequency-spectrum of the product number fluctuation and the reaction mechanism and dynamics, starting from a generalized master equation. This relation enables us to analyze the time-traces of the protein number and extract information about dynamics of mRNA number and transcriptional regulation, which cannot be directly observed by current experimental techniques. We demonstrate our frequency spectrum analysis of protein number fluctuation, using the gene network model of luciferase expression under the control of the Bmal 1a promoter in mouse fibroblast cells. We also discuss how the dynamic heterogeneity of transcription and translation rates affects the frequency-spectra of the mRNA and protein number.

Mitochondrial Network State Scales mtDNA Genetic Dynamics

Mitochondrial DNA (mtDNA) mutations cause severe congenital diseases but may also be associated with healthy aging. mtDNA is stochastically replicated and degraded, and exists within organelles which undergo dynamic fusion and fission. The role of the resulting mitochondrial networks in the time evolution of the cellular proportion of mutated mtDNA molecules (heteroplasmy), and cell-to-cell variability in heteroplasmy (heteroplasmy variance), remains incompletely understood. Heteroplasmy variance is particularly important since it modulates the number of pathological cells in a tissue. Here, we provide the first wide-reaching theoretical framework which bridges mitochondrial network and genetic states. We show that, under a range of conditions, the (genetic) rate of increase in heteroplasmy variance and de novo mutation are proportionally modulated by the (physical) fraction of unfused mitochondria, independently of the absolute fission-fusion rate. In the context of selective fusion, we show that intermediate fusion:fission ratios are optimal for the clearance of mtDNA mutants. Our findings imply that modulating network state, mitophagy rate, and copy number to slow down heteroplasmy dynamics when mean heteroplasmy is low could have therapeutic advantages for mitochondrial disease and healthy aging.

On the Properties of the Reaction Counts Chemical Master Equation

The reaction counts chemical master equation (CME) is a high-dimensional variant of the classical population counts CME. In the reaction counts CME setting, we count the reactions which have fired over time rather than monitoring the population state over time. Since a reaction either fires or not, the reaction counts CME transitions are only forward stepping. Typically there are more reactions in a system than species, this results in the reaction counts CME being higher in dimension, but simpler in dynamics. In this work, we revisit the reaction counts CME framework and its key theoretical results. Then we will extend the theory by exploiting the reactions counts' forward stepping feature, by decomposing the state space into independent continuous-time Markov chains (CTMC). We extend the reaction counts CME theory to derive analytical forms and estimates for the CTMC decomposition of the CME. This new theory gives new insights into solving hitting times-, rare events-, and a priori domain construction problems.

Extending the linear-noise approximation to biochemical systems influenced by intrinsic noise and slow lognormally distributed extrinsic noise

It is well known that the kinetics of an intracellular biochemical network is stochastic. This is due to intrinsic noise arising from the random timing of biochemical reactions in the network as well as due to extrinsic noise stemming from the interaction of unknown molecular components with the network and from the cell's changing environment. While there are many methods to study the effect of intrinsic noise on the system dynamics, few exist to study the influence of both types of noise. Here we show how one can extend the conventional linear-noise approximation to allow for the rapid evaluation of the molecule numbers statistics of a biochemical network influenced by intrinsic noise and by slow lognormally distributed extrinsic noise. The theory is applied to simple models of gene regulatory networks and its validity confirmed by comparison with exact stochastic simulations. In particular, we consider three important biological examples. First, we investigate how extrinsic noise modifies the dependence of the variance of the molecule number fluctuations on the rate constants. Second, we show how the mutual information between input and output of a network motif is affected by extrinsic noise. And third, we study the robustness of the ubiquitously found feed-forward loop motifs when subjected to extrinsic noise.

Accuracy of parameter estimation for auto-regulatory transcriptional feedback loops from noisy data

Bayesian and non-Bayesian moment-based inference methods are commonly used to estimate the parameters defining stochastic models of gene regulatory networks from noisy single cell or population snapshot data. However, a systematic investigation of the accuracy of the predictions of these methods remains missing. Here, we present the results of such a study using synthetic noisy data of a negative auto-regulatory transcriptional feedback loop, one of the most common building blocks of complex gene regulatory networks. We study the error in parameter estimation as a function of (i) number of cells in each sample; (ii) the number of time points; (iii) the highest-order moment of protein fluctuations used for inference; (iv) the moment-closure method used for likelihood approximation. We find that for sample sizes typical of flow cytometry experiments, parameter estimation by maximizing the likelihood is as accurate as using Bayesian methods but with a much reduced computational time. We also show that the choice of moment-closure method is the crucial factor determining the maximum achievable accuracy of moment-based inference methods. Common likelihood approximation methods based on the linear noise approximation or the zero cumulants closure perform poorly for feedback loops with large protein-DNA binding rates or large protein bursts; this is exacerbated for highly heterogeneous cell populations. By contrast, approximating the likelihood using the linear-mapping approximation or conditional derivative matching leads to highly accurate parameter estimates for a wide range of conditions.

First-passage time statistics of stochastic transcription process for time-dependent reaction rates

Transcription in gene expression is an intrinsically noisy process which involves production and degradation of mRNAs. An important quantity to describe this stochastic process is the first-passage time (FPT), i.e., the time taken by mRNAs to reach a particular threshold. The process of transcription can be modelled as a simple birth-death process, assuming that the promoter is always in an active state and to encode the stochastic environment we consider the transcription rate to be time dependent. This generalization is suitable to capture bursty mRNA dynamics usually modelled as an ON-Off model and simplifies the calculation of FPT statistics for a cell population. We study the role of periodic modulation of the transcription rate on different moments of FPT distribution of a population of cells. Our calculation shows that for sinusoidal modulation there exists an extremal value of mean FPT as a function of the time period and phase of the transcription signal. However, for the square wave modulation of transcription rates simulation results show that the extremal value of the MFPT behaves monotonically with the variation of the phase.

Intrinsic and extrinsic noise of gene expression in lineage trees

Cell-to-cell heterogeneity is driven by stochasticity in intracellular reactions and the population dynamics. While these sources are usually studied separately, we develop an agent-based framework that accounts for both factors while tracking every single cell of a growing population. Apart from the common intrinsic variability, the framework also predicts extrinsic noise without the need to introduce fluctuating rate constants. Instead, extrinsic fluctuations are explained by cell cycle fluctuations and differences in cell age. We provide explicit formulas to quantify mean molecule numbers, intrinsic and extrinsic noise statistics in two-colour experiments. We find that these statistics differ significantly depending on the experimental setup used to observe the cells. We illustrate this fact using (i) averages over an isolated cell lineage tracked over many generations as observed in the mother machine, (ii) population snapshots with known cell ages as recorded in time-lapse microscopy, and (iii) snapshots with unknown cell ages as measured from static images or flow cytometry. Applying the method to models of stochastic gene expression and feedback regulation elucidates that isolated lineages, as compared to snapshot data, can significantly overestimate the mean number of molecules, overestimate extrinsic noise but underestimate intrinsic noise and have qualitatively different sensitivities to cell cycle fluctuations.

Discrete flux and velocity fields of probability and their global maps in reaction systems

Stochasticity plays important roles in reaction systems. Vector fields of probability flux and velocity characterize time-varying and steady-state properties of these systems, including high probability paths, barriers, checkpoints among different stable regions, as well as mechanisms of dynamic switching among them. However, conventional fluxes on continuous space are ill-defined and are problematic when at the boundaries of the state space or when copy numbers are small. By re-defining the derivative and divergence operators based on the discrete nature of reactions, we introduce new formulations of discrete fluxes. Our flux model fully accounts for the discreetness of both the state space and the jump processes of reactions. The reactional discrete flux satisfies the continuity equation and describes the behavior of the system evolving along directions of reactions. The species discrete flux directly describes the dynamic behavior in the state space of the reactants such as the transfer of probability mass. With the relationship between these two fluxes specified, we show how to construct time-evolving and steady-state global flow-maps of probability flux and velocity in the directions of every species at every microstate and how they are related to the outflow and inflow of probability fluxes when tracing out reaction trajectories. We also describe how to impose proper conditions enabling exact quantification of flux and velocity in the boundary regions, without the difficulty of enforcing artificial reflecting conditions. We illustrate the computation of probability flux and velocity using three model systems, namely, the birth-death process, the bistable Schlögl model, and the oscillating Schnakenberg model.

On Differences between Deterministic and Stochastic Models of Chemical Reactions: Schlögl Solved with ZI-Closure

Deterministic and stochastic models of chemical reaction kinetics can give starkly different results when the deterministic model exhibits more than one stable solution. For example, in the stochastic Schlögl model, the bimodal stationary probability distribution collapses to a unimodal distribution when the system size increases, even for kinetic constant values that result in two distinct stable solutions in the deterministic Schlögl model. Using zero-information (ZI) closure scheme, an algorithm for solving chemical master equations, we compute stationary probability distributions for varying system sizes of the Schlögl model. With ZI-closure, system sizes can be studied that have been previously unattainable by stochastic simulation algorithms. We observe and quantify paradoxical discrepancies between stochastic and deterministic models and explain this behavior by postulating that the entropy of non-equilibrium steady states (NESS) is maximum.

Analysis of stochastic bifurcations with phase portraits

We propose a method to obtain phase portraits for stochastic systems. Starting from the Fokker-Planck equation, we separate the dynamics into a convective and a diffusive part. We show that stable and unstable fixed points of the convective field correspond to maxima and minima of the stationary probability distribution if the probability current vanishes at these points. Stochastic phase portraits, which are vector plots of the convective field, therefore indicate the extrema of the stationary distribution and can be used to identify stochastic bifurcations that change the number and stability of these extrema. We show that limit cycles in stochastic phase portraits can indicate ridges of the probability distribution, and we identify a novel type of stochastic bifurcation, where the probability maximum moves to the edge of the system through a gap between the two nullclines of the convective field.

Fundamental limits on dynamic inference from single-cell snapshots

Seeing a snapshot of individuals at different stages of a dynamic process can reveal what the process would look like for a single individual over time. Biologists apply this principle to infer temporal sequences of gene expression states in cells from measurements made at a single moment in time. However, the sparsity and high dimensionality of single-cell data have made inference difficult using formal approaches. Here, we apply recent innovations in spectral graph theory to devise a simple and asymptotically exact algorithm for inferring the unique dynamic solution under defined approximations and apply it to data from bone marrow stem cells.

Single-cell expression profiling reveals the molecular states of individual cells with unprecedented detail. Because these methods destroy cells in the process of analysis, they cannot measure how gene expression changes over time. However, some information on dynamics is present in the data: the continuum of molecular states in the population can reflect the trajectory of a typical cell. Many methods for extracting single-cell dynamics from population data have been proposed. However, all such attempts face a common limitation: for any measured distribution of cell states, there are multiple dynamics that could give rise to it, and by extension, multiple possibilities for underlying mechanisms of gene regulation. Here, we describe the aspects of gene expression dynamics that cannot be inferred from a static snapshot alone and identify assumptions necessary to constrain a unique solution for cell dynamics from static snapshots. We translate these constraints into a practical algorithmic approach, population balance analysis (PBA), which makes use of a method from spectral graph theory to solve a class of high-dimensional differential equations. We use simulations to show the strengths and limitations of PBA, and then apply it to single-cell profiles of hematopoietic progenitor cells (HPCs). Cell state predictions from this analysis agree with HPC fate assays reported in several papers over the past two decades. By highlighting the fundamental limits on dynamic inference faced by any method, our framework provides a rigorous basis for dynamic interpretation of a gene expression continuum and clarifies best experimental designs for trajectory reconstruction from static snapshot measurements.

Motif analysis for small-number effects in chemical reaction dynamics

The number of molecules involved in a cell or subcellular structure is sometimes rather small. In this situation, ordinary macroscopic-level fluctuations can be overwhelmed by non-negligible large fluctuations, which results in drastic changes in chemical-reaction dynamics and statistics compared to those observed under a macroscopic system (i.e., with a large number of molecules). In order to understand how salient changes emerge from fluctuations in molecular number, we here quantitatively define small-number effect by focusing on a "mesoscopic" level, in which the concentration distribution is distinguishable both from micro- and macroscopic ones and propose a criterion for determining whether or not such an effect can emerge in a given chemical reaction network. Using the proposed criterion, we systematically derive a list of motifs of chemical reaction networks that can show small-number effects, which includes motifs showing emergence of the power law and the bimodal distribution observable in a mesoscopic regime with respect to molecule number. The list of motifs provided herein is helpful in the search for candidates of biochemical reactions with a small-number effect for possible biological functions, as well as for designing a reaction system whose behavior can change drastically depending on molecule number, rather than concentration.

Statistical physics approaches to subnetwork dynamics in biochemical systems

We apply a Gaussian variational approximation to model reduction in large biochemical networks of unary and binary reactions. We focus on a small subset of variables (subnetwork) of interest, e.g. because they are accessible experimentally, embedded in a larger network (bulk). The key goal is to write dynamical equations reduced to the subnetwork but still retaining the effects of the bulk. As a result, the subnetwork-reduced dynamics contains a memory term and an extrinsic noise term with non-trivial temporal correlations. We first derive expressions for this memory and noise in the linearized (Gaussian) dynamics and then use a perturbative power expansion to obtain first order nonlinear corrections. For the case of vanishing intrinsic noise, our description is explicitly shown to be equivalent to projection methods up to quadratic terms, but it is applicable also in the presence of stochastic fluctuations in the original dynamics. An example from the epidermal growth factor receptor signalling pathway is provided to probe the increased prediction accuracy and computational efficiency of our method.

An open-loop approach to calculate noise-induced transitions

Bistability permits the co-existence of two distinct cell fates in a population of genetically identical cells. Noise induced transitions between two fates of a bistable system are difficult to calculate due to the intricate interplay between nonlinear dynamics and noise in bistable positive feedback loops. Here we opened multivariable feedback loops at the slowest variable to obtain the open-loop function and the fluctuations in the open-loop output. By the subsequent reclosing of the loop, we calculated the mean first passage time (MFPT) using the Fokker-Planck equation in good agreement with the exact stochastic simulation. When an external component interacts with a feedback component, it amplifies the extrinsic noise in the loop. Consequently, the open-loop function is shifted and the transition rates between the two states in the closed loop are increased. Despite this shift, the open-loop output reflects the system faithfully to predict the MFPT in the feedback loop. Therefore, the open-loop approach can help theoretical analysis. Furthermore, the measurement of the mean value, variance, and the reaction time-scale of the open-loop output permits the prediction of MFPT simply from experimental data, which underscores the practical value of the stochastic open-loop approach.

Dichotomous noise models of gene switches

Molecular noise in gene regulatory networks has two intrinsic components, one part being due to fluctuations caused by the birth and death of protein or mRNA molecules which are often present in small numbers and the other part arising from gene state switching, a single molecule event. Stochastic dynamics of gene regulatory circuits appears to be largely responsible for bifurcations into a set of multi-attractor states that encode different cell phenotypes. The interplay of dichotomous single molecule gene noise with the nonlinear architecture of genetic networks generates rich and complex phenomena. In this paper, we elaborate on an approximate framework that leads to simple hybrid multi-scale schemes well suited for the quantitative exploration of the steady state properties of large-scale cellular genetic circuits. Through a path sum based analysis of trajectory statistics, we elucidate the connection of these hybrid schemes to the underlying master equation and provide a rigorous justification for using dichotomous noise based models to study genetic networks. Numerical simulations of circuit models reveal that the contribution of the genetic noise of single molecule origin to the total noise is significant for a wide range of kinetic regimes.

ACCURATE CHEMICAL MASTER EQUATION SOLUTION USING MULTI-FINITE BUFFERS

The discrete chemical master equation (dCME) provides a fundamental framework for studying stochasticity in mesoscopic networks. Because of the multi-scale nature of many networks where reaction rates have large disparity, directly solving dCMEs is intractable due to the exploding size of the state space. It is important to truncate the state space effectively with quantified errors, so accurate solutions can be computed. It is also important to know if all major probabilistic peaks have been computed. Here we introduce the Accurate CME (ACME) algorithm for obtaining direct solutions to dCMEs. With multi-finite buffers for reducing the state space by O(n!), exact steady-state and time-evolving network probability landscapes can be computed. We further describe a theoretical framework of aggregating microstates into a smaller number of macrostates by decomposing a network into independent aggregated birth and death processes, and give an a priori method for rapidly determining steady-state truncation errors. The maximal sizes of the finite buffers for a given error tolerance can also be pre-computed without costly trial solutions of dCMEs. We show exactly computed probability landscapes of three multi-scale networks, namely, a 6-node toggle switch, 11-node phage-lambda epigenetic circuit, and 16-node MAPK cascade network, the latter two with no known solutions. We also show how probabilities of rare events can be computed from first-passage times, another class of unsolved problems challenging for simulation-based techniques due to large separations in time scales. Overall, the ACME method enables accurate and efficient solutions of the dCME for a large class of networks.

State Space Truncation with Quantified Errors for Accurate Solutions to Discrete Chemical Master Equation

The discrete chemical master equation (dCME) provides a general framework for studying stochasticity in mesoscopic reaction networks. Since its direct solution rapidly becomes intractable due to the increasing size of the state space, truncation of the state space is necessary for solving most dCMEs. It is therefore important to assess the consequences of state space truncations so errors can be quantified and minimized. Here we describe a novel method for state space truncation. By partitioning a reaction network into multiple molecular equivalence groups (MEGs), we truncate the state space by limiting the total molecular copy numbers in each MEG. We further describe a theoretical framework for analysis of the truncation error in the steady-state probability landscape using reflecting boundaries. By aggregating the state space based on the usage of a MEG and constructing an aggregated Markov process, we show that the truncation error of a MEG can be asymptotically bounded by the probability of states on the reflecting boundary of the MEG. Furthermore, truncating states of an arbitrary MEG will not undermine the estimated error of truncating any other MEGs. We then provide an overall error estimate for networks with multiple MEGs. To rapidly determine the appropriate size of an arbitrary MEG, we also introduce an a priori method to estimate the upper bound of its truncation error. This a priori estimate can be rapidly computed from reaction rates of the network, without the need of costly trial solutions of the dCME. As examples, we show results of applying our methods to the four stochastic networks of (1) the birth and death model, (2) the single gene expression model, (3) the genetic toggle switch model, and (4) the phage lambda bistable epigenetic switch model. We demonstrate how truncation errors and steady-state probability landscapes can be computed using different sizes of the MEG(s) and how the results validate our theories. Overall, the novel state space truncation and error analysis methods developed here can be used to ensure accurate direct solutions to the dCME for a large number of stochastic networks.

Promoters Architecture-Based Mechanism for Noise-Induced Oscillations in a Single-Gene Circuit

It is well known that single-gene circuits with negative feedback loop can lead to oscillatory gene expression when they operate with time delay. In order to generate these oscillations many processes can contribute to properly timing such delay. Here we show that the time delay coming from the transitions between internal states of the cis-regulatory system (CRS) can drive sustained oscillations in an auto-repressive single-gene circuit operating in a small volume like a cell. We found that the cooperative binding of repressor molecules is not mandatory for a oscillatory behavior if there are enough binding sites in the CRS. These oscillations depend on an adequate balance between the CRS kinetic, and the synthesis/degradation rates of repressor molecules. This finding suggest that the multi-site CRS architecture can play a key role for oscillatory behavior of gene expression. Finally, our results can also help to synthetic biologists on the design of the promoters architecture for new genetic oscillatory circuits.

Stochasticity in the miR-9/Hes1 oscillatory network can account for clonal heterogeneity in the timing of differentiation

Recent studies suggest that cells make stochastic choices with respect to differentiation or division. However, the molecular mechanism underlying such stochasticity is unknown. We previously proposed that the timing of vertebrate neuronal differentiation is regulated by molecular oscillations of a transcriptional repressor, HES1, tuned by a post-transcriptional repressor, miR-9. Here, we computationally model the effects of intrinsic noise on the Hes1/miR-9 oscillator as a consequence of low molecular numbers of interacting species, determined experimentally. We report that increased stochasticity spreads the timing of differentiation in a population, such that initially equivalent cells differentiate over a period of time. Surprisingly, inherent stochasticity also increases the robustness of the progenitor state and lessens the impact of unequal, random distribution of molecules at cell division on the temporal spread of differentiation at the population level. This advantageous use of biological noise contrasts with the view that noise needs to be counteracted.

Linear-noise approximation and the chemical master equation agree up to second-order moments for a class of chemical systems

It is well known that the linear-noise approximation (LNA) agrees with the chemical master equation, up to second-order moments, for chemical systems composed of zero and first-order reactions. Here we show that this is also a property of the LNA for a subset of chemical systems with second-order reactions. This agreement is independent of the number of interacting molecules.

Approximate probability distributions of the master equation

Master equations are common descriptions of mesoscopic systems. Analytical solutions to these equations can rarely be obtained. We here derive an analytical approximation of the time-dependent probability distribution of the master equation using orthogonal polynomials. The solution is given in two alternative formulations: a series with continuous and a series with discrete support, both of which can be systematically truncated. While both approximations satisfy the system size expansion of the master equation, the continuous distribution approximations become increasingly negative and tend to oscillations with increasing truncation order. In contrast, the discrete approximations rapidly converge to the underlying non-Gaussian distributions. The theory is shown to lead to particularly simple analytical expressions for the probability distributions of molecule numbers in metabolic reactions and gene expression systems.

General transient solution of the one-step master equation in one dimension

Exact analytical solutions of the master equation are limited to special cases and exact numerical methods are inefficient. Even the generic one-dimensional, one-step master equation has evaded exact solution, aside from the steady-state case. This type of master equation describes the dynamics of a continuous-time Markov process whose range consists of positive integers and whose transitions are allowed only between adjacent sites. The solution of any master equation can be written as the exponential of a (typically huge) matrix, which requires the calculation of the eigenvalues and eigenvectors of the matrix. Here we propose a linear algebraic method for simplifying this exponential for the general one-dimensional, one-step process. In particular, we prove that the calculation of the eigenvectors is actually not necessary for the computation of exponential, thereby we dramatically cut the time of this calculation. We apply our new methodology to examples from birth-death processes and biochemical networks. We show that the computational time is significantly reduced compared to existing methods.

The complex chemical Langevin equation

The chemical Langevin equation (CLE) is a popular simulation method to probe the stochastic dynamics of chemical systems. The CLE's main disadvantage is its break down in finite time due to the problem of evaluating square roots of negative quantities whenever the molecule numbers become sufficiently small. We show that this issue is not a numerical integration problem, rather in many systems it is intrinsic to all representations of the CLE. Various methods of correcting the CLE have been proposed which avoid its break down. We show that these methods introduce undesirable artefacts in the CLE's predictions. In particular, for unimolecular systems, these correction methods lead to CLE predictions for the mean concentrations and variance of fluctuations which disagree with those of the chemical master equation. We show that, by extending the domain of the CLE to complex space, break down is eliminated, and the CLE's accuracy for unimolecular systems is restored. Although the molecule numbers are generally complex, we show that the "complex CLE" predicts real-valued quantities for the mean concentrations, the moments of intrinsic noise, power spectra, and first passage times, hence admitting a physical interpretation. It is also shown to provide a more accurate approximation of the chemical master equation of simple biochemical circuits involving bimolecular reactions than the various corrected forms of the real-valued CLE, the linear-noise approximation and a commonly used two moment-closure approximation.

Role of relaxation time scale in noisy signal transduction

Intra-cellular fluctuations, mainly triggered by gene expression, are an inevitable phenomenon observed in living cells. It influences generation of phenotypic diversity in genetically identical cells. Such variation of cellular components is beneficial in some contexts but detrimental in others. To quantify the fluctuations in a gene product, we undertake an analytical scheme for studying few naturally abundant linear as well as branched chain network motifs. We solve the Langevin equations associated with each motif under the purview of linear noise approximation and derive the expressions for Fano factor and mutual information in close analytical form. Both quantifiable expressions exclusively depend on the relaxation time (decay rate constant) and steady state population of the network components. We investigate the effect of relaxation time constraints on Fano factor and mutual information to indentify a time scale domain where a network can recognize the fluctuations associated with the input signal more reliably. We also show how input population affects both quantities. We extend our calculation to long chain linear motif and show that with increasing chain length, the Fano factor value increases but the mutual information processing capability decreases. In this type of motif, the intermediate components act as a noise filter that tune up input fluctuations and maintain optimum fluctuations in the output. For branched chain motifs, both quantities vary within a large scale due to their network architecture and facilitate survival of living system in diverse environmental conditions.

Negative feedback contributes to the stochastic expression of the interferon-β gene in virus-triggered type I interferon signaling pathways

Type I interferon (IFN) signaling pathways play an essential role in the defense against early viral infections; however, the diverse and intricate molecular mechanisms of virus-triggered type I IFN responses are still poorly understood. In this study, we analyzed and compared two classes of models i.e., deterministic ordinary differential equations (ODEs) and stochastic models to elucidate the dynamics and stochasticity of type I IFN signaling pathways. Bifurcation analysis based on an ODE model reveals that the system exhibits a bistable switch and a one-way switch at high or low levels when the strengths of the negative and positive feedbacks are tuned. Furthermore, we compared the stochastic simulation results under the Master and Langevin equations. Both of the stochastic equations generate the bistable switch phenomenon, and the distance between two stable states are smaller than normal under the simulation of the Langevin equation. The quantitative computations also show that a moderate ratio between positive and negative feedback strengths is required to ensure a reliable switch between the different IFN concentrations that regulate the immune response. Moreover, we propose a multi-state stochastic model based on the above deterministic model to describe the multi-cellular system coupled with the diffusion of IFNs. The perturbation and inhibition analysis showed that the positive feedback, as well as noises, has little effect on the stochastic expression of IFNs, but the negative feedback of ISG56 on the activation of IRF7 has a great influence on IFN stochastic expression. Together, these results reveal that positive feedback stabilizes IFN gene expression, and negative feedback may be the main contribution to the stochastic expression of the IFN gene in the virus-triggered type I IFN response. These findings will provide new insight into the molecular mechanisms of virus-triggered type I IFN signaling pathways.

Noise-induced multistability in chemical systems: Discrete versus continuum modeling

The noisy dynamics of chemical systems is commonly studied using either the chemical master equation (CME) or the chemical Fokker-Planck equation (CFPE). The latter is a continuum approximation of the discrete CME approach. It has recently been shown that for a particular system, the CFPE captures noise-induced multistability predicted by the CME. This phenomenon involves the CME's marginal probability distribution changing from unimodal to multimodal as the system size decreases below a critical value. We here show that the CFPE does not always capture noise-induced multistability. In particular we find simple chemical systems for which the CME predicts noise-induced multistability, whereas the CFPE predicts monostability for all system sizes.