Numerical simulation of stochastic gene circuits

Niblack Binarization on Document Images: Area Efficient, Low Cost, and Noise Tolerant Stochastic Architecture

Binarization plays a crucial role in Optical Character Recognition (OCR) ancillary domains, such as recovery of degraded document images. In Document Image Analysis (DIA), selecting threshold is not trivial since it differs from one problem (dataset) to another. Instead of trying several different thresholds for one dataset to another, we consider noise inherency of document images in our proposed binarization scheme. The proposed stochastic architecture implements the local thresholding technique: Niblack’s binarization algorithm. We introduce a stochastic comparator circuit that works on unipolar stochastic numbers. Unlike the conventional stochastic circuit, it is simple and easy to deploy. We implemented it on the Xilinx Virtex6 XC6VLX760-2FF1760 FPGA platform and received encouraging experimental results. The complete set of results are available upon request. Besides, compared to conventional designs, the proposed stochastic implementation is better in terms of time complexity as well as fault-tolerant capacity.

Maximum Entropy Prediction of Non-Equilibrium Stationary Distributions for Stochastic Reaction Networks with Oscillatory Dynamics

Many chemical reaction networks in biological systems present complex oscillatory dynamics. In systems such as regulatory gene networks, cell cycle, and enzymatic processes, the number of molecules involved is often far from the thermodynamic limit. Although stochastic models based on the probabilistic approach of the Chemical Master Equation (CME) have been proposed, studies in the literature have been limited by the challenges of solving the CME and the lack of computational power to perform large-scale stochastic simulations. In this paper, we show that the infinite set of stationary moment equations describing the stochastic Brusselator and Schnakenberg oscillatory reactions networks can be truncated and solved using maximization of the entropy of the distributions. The results from our numerical experiments compare with the distributions obtained from well-established kinetic Monte Carlo methods and suggest that the accuracy of the prediction increases exponentially with the closure order chosen for the system. We conclude that maximum entropy models can be used as an efficient closure scheme alternative for moment equations to predict the non-equilibrium stationary distributions of stochastic chemical reactions with oscillatory dynamics. This prediction is accomplished without any prior knowledge of the system dynamics and without imposing any biased assumptions on the mathematical relations among species involved.

Chemical master equation closure for computer-aided synthetic biology

With inexpensive DNA synthesis technologies, we can now construct biological systems by quickly piecing together DNA sequences. Synthetic biology is the promising discipline that focuses on the construction of these new biological systems. Synthetic biology is an engineering discipline, and as such, it can benefit from mathematical modeling. This chapter focuses on mathematical models of biological systems. These models take the form of chemical reaction networks. The importance of stochasticity is discussed and methods to simulate stochastic reaction networks are reviewed. A closure scheme solution is also presented for the master equation of chemical reaction networks. The master equation is a complete model of randomly evolving molecular populations. Because of its ambitious character, the master equation remained unsolved for all but the simplest of molecular interaction networks for over 70 years. With the first complete solution of chemical master equations, a wide range of experimental observations of biomolecular interactions may be mathematically conceptualized. We anticipate that models based on the closure scheme described herein may assist in rationally designing synthetic biological systems.

Solution of Chemical Master Equations for Nonlinear Stochastic Reaction Networks

Stochasticity in the dynamics of small reacting systems requires discrete-probabilistic models of reaction kinetics instead of traditional continuous-deterministic ones. The master probability equation is a complete model of randomly evolving molecular populations. Because of its ambitious character, the master equation remained unsolved for all but the simplest of molecular interaction networks. With the first solution of chemical master equations, a wide range of experimental observations of small-system interactions may be mathematically conceptualized.

Stochastic model reduction using a modified Hill-type kinetic rate law

In the present work, we address a major challenge facing the modeling of biochemical reaction networks: when using stochastic simulations, the computational load and number of unknown parameters may dramatically increase with system size and complexity. A proposed solution to this challenge is the reduction of models by utilizing nonlinear reaction rate laws in place of a complex multi-reaction mechanism. This type of model reduction in stochastic systems often fails when applied outside of the context in which it was initially conceived. We hypothesize that the use of nonlinear rate laws fails because a single reaction is inherently Poisson distributed and cannot match higher order statistics. In this study we explore the use of Hill-type rate laws as an approximation for gene regulation, specifically transcription repression. We matched output data for several simple gene networks to determine Hill-type parameters. We show that the models exhibit inaccuracies when placed into a simple feedback repression model. By adding an additional abstract reaction to the models we account for second-order statistics. This split Hill rate law matches higher order statistics and demonstrates that the new model is able to more accurately describe the mean protein output. Finally, the modified Hill model is shown to be modular and models retain accuracy when placed into a larger multi-gene network. The work as presented may be used in gene regulatory or cell-signaling networks, where multiple binding events can be captured by Hill kinetics. The added benefit of the proposed split-Hill kinetics is the improved accuracy in modeling stochastic effects. We demonstrate these benefits with a few specific reaction network examples.

Stochastic simulations of a synthetic bacteria-yeast ecosystem

BACKGROUND

The field of synthetic biology has greatly evolved and numerous functions can now be implemented by artificially engineered cells carrying the appropriate genetic information. However, in order for the cells to robustly perform complex or multiple tasks, co-operation between them may be necessary. Therefore, various synthetic biological systems whose functionality requires cell-cell communication are being designed. These systems, microbial consortia, are composed of engineered cells and exhibit a wide range of behaviors. These include yeast cells whose growth is dependent on one another, or bacteria that kill or rescue each other, synchronize, behave as predator-prey ecosystems or invade cancer cells.

RESULTS

In this paper, we study a synthetic ecosystem comprising of bacteria and yeast that communicate with and benefit from each other using small diffusible molecules. We explore the behavior of this heterogeneous microbial consortium, composed of Saccharomyces cerevisiae and Escherichia coli cells, using stochastic modeling. The stochastic model captures the relevant intra-cellular and inter-cellular interactions taking place in and between the eukaryotic and prokaryotic cells. Integration of well-characterized molecular regulatory elements into these two microbes allows for communication through quorum sensing. A gene controlling growth in yeast is induced by bacteria via chemical signals and vice versa. Interesting dynamics that are common in natural ecosystems, such as obligatory and facultative mutualism, extinction, commensalism and predator-prey like dynamics are observed. We investigate and report on the conditions under which the two species can successfully communicate and rescue each other.

CONCLUSIONS

This study explores the various behaviors exhibited by the cohabitation of engineered yeast and bacterial cells. The way that the model is built allows for studying the dynamics of any system consisting of two species communicating with one another via chemical signals. Therefore, key information acquired by our model may potentially drive the experimental design of various synthetic heterogeneous ecosystems.

proTeOn and proTeOff, new protein devices that inducibly activate bacterial gene expression

Using an original workflow, we have modeled, constructed, and characterized two new molecular devices that inducibly activate gene expression in Escherichia coli. The devices, prokaryotic-TetOn and prokaryotic-TetOff, were built by fusing an inducible DNA-binding protein domain to a transcription activation domain and constructing a complementary synthetic promoter sequence through which they could control downstream gene expression. In particular, the transactivators were built using variants of the tetracycline repressor, TetR, and the transactivating domain of the LuxR activator. The complementary promoter sequence included TetR's operator, tetO, and elements of the lux promoter. These specific protein domains and their operator sites were chosen as they have been thoroughly studied and well characterized. First, our methodology began with optimizing the geometry of the molecular components using molecular modeling. We did so to achieve an unprecedented combination of controllable and transactivating function in bacterial organisms. The devices were then built to activate the expression of green fluorescent protein. Their unique function was found to be robustly tight and activating many-fold increases of expressed gene levels, as measured by flow cytometry experiments. The devices were further characterized with stochastic kinetic models. The new devices presented herein may become useful additions to the molecular toolboxes used by biologists to control bacterial gene expression. The methodology used may also be a foundation for the design, development, and characterization of a library of such devices and more complex gene regulatory networks.

SynBioSS-aided design of synthetic biological constructs

We present walkthrough examples of using SynBioSS to design, model, and simulate synthetic gene regulatory networks. SynBioSS stands for Synthetic Biology Software Suite, a platform that is publicly available with Open Licenses at www.synbioss.org. An important aim of computational synthetic biology is the development of a mathematical modeling formalism that is applicable to a wide variety of simple synthetic biological constructs. SynBioSS-based modeling of biomolecular ensembles that interact away from the thermodynamic limit and not necessarily at steady state affords for a theoretical framework that is generally applicable to known synthetic biological systems, such as bistable switches, AND gates, and oscillators. Here, we discuss how SynBioSS creates links between DNA sequences and targeted dynamic phenotypes of these simple systems.

Mathematical models in biology: from molecules to life

A vexing question in the biological sciences is the following: can biological phenotypes be explained with mathematical models of molecules that interact according to physical laws? At the crux of the matter lies the doubt that humans can develop physically faithful mathematical representations of living organisms. We discuss advantages that synthetic biological systems confer that may help us describe life's distinctiveness with tractable mathematics that are grounded on universal laws of thermodynamics and molecular biology.

Stochastic simulations of the tetracycline operon

Background

The tetracycline operon is a self-regulated system. It is found naturally in bacteria where it confers resistance to antibiotic tetracycline. Because of the performance of the molecular elements of the tetracycline operon, these elements are widely used as parts of synthetic gene networks where the protein production can be efficiently turned on and off in response to the presence or the absence of tetracycline. In this paper, we investigate the dynamics of the tetracycline operon. To this end, we develop a mathematical model guided by experimental findings. Our model consists of biochemical reactions that capture the biomolecular interactions of this intriguing system. Having in mind that small biological systems are subjects to stochasticity, we use a stochastic algorithm to simulate the tetracycline operon behavior. A sensitivity analysis of two critical parameters embodied this system is also performed providing a useful understanding of the function of this system.

Results

Simulations generate a timeline of biomolecular events that confer resistance to bacteria against tetracycline. We monitor the amounts of intracellular TetR2 and TetA proteins, the two important regulatory and resistance molecules, as a function of intrecellular tetracycline. We find that lack of one of the promoters of the tetracycline operon has no influence on the total behavior of this system inferring that this promoter is not essential for Escherichia coli. Sensitivity analysis with respect to the binding strength of tetracycline to repressor and of repressor to operators suggests that these two parameters play a predominant role in the behavior of the system. The results of the simulations agree well with experimental observations such as tight repression, fast gene expression, induction with tetracycline, and small intracellular TetR2 amounts.

Conclusions

Computer simulations of the tetracycline operon afford augmented insight into the interplay between its molecular components. They provide useful explanations of how the components and their interactions have evolved to best serve bacteria carrying this operon. Therefore, simulations may assist in designing novel gene network architectures consisting of tetracycline operon components.

Multiscale models for synthetic biology

Reacting systems away from the thermodynamic limit cannot be accurately modeled with ordinary differential equations. These continuous-deterministic modeling formalisms, traditionally developed and used by chemical engineers can be distinctly false if the number of molecules of reacting chemical species is very small, or if reaction events are very rare. Then stochastic-discrete representations are appropriate. Importantly, in cases where in a network of reactions there are some parts that must be modeled discretely and stochastically, yet others can be modeled continuously and deterministically, the need for development of multiscale models emerges naturally. In computational synthetic biology, such cases arise often. In this work we present the development of multiscale models for synthetic biology applications, demonstrating accuracy, computational efficiency and utility.

The origins of time-delay in template biopolymerization processes

Time-delays are common in many physical and biological systems and they give rise to complex dynamic phenomena. The elementary processes involved in template biopolymerization, such as mRNA and protein synthesis, introduce significant time delays. However, there is not currently a systematic mapping between the individual mechanistic parameters and the time delays in these networks. We present here the development of mathematical, time-delay models for protein translation, based on PDE models, which in turn are derived through systematic approximations of first-principles mechanistic models. Theoretical analysis suggests that the key features that determine the time-delays and the agreement between the time-delay and the mechanistic models are ribosome density and distribution, i.e., the number of ribosomes on the mRNA chain relative to their maximum and their distribution along the mRNA chain. Based on analytical considerations and on computational studies, we show that the steady-state and dynamic responses of the time-delay models are in excellent agreement with the detailed mechanistic models, under physiological conditions that correspond to uniform ribosome distribution and for ribosome density up to 70%. The methodology presented here can be used for the development of reduced time-delay models of mRNA synthesis and large genetic networks. The good agreement between the time-delay and the mechanistic models will allow us to use the reduced model and advanced computational methods from nonlinear dynamics in order to perform studies that are not practical using the large-scale mechanistic models.

Computational methods in synthetic biology

We discuss how a theoretical synthetic biology research programme may liberate empiricism in biological sciences beyond the unaided human brain. Because synthetic biological systems are relatively small and largely independent of evolutionary contexts, they can be represented with mathematical models strongly founded on first principles of molecular biology and laws of statistical thermodynamics. A universal mathematical formalism for describing synthetic constructs may then be plausibly used to explain in unambiguous, quantitative terms how biological phenotypic complexity emerges as a result of well-defined biomolecular interactions. SynBioSS, a publicly available software package, is described that implements this mathematical formalism.

Model reduction of multiscale chemical langevin equations: a numerical case study

Two very important characteristics of biological reaction networks need to be considered carefully when modeling these systems. First, models must account for the inherent probabilistic nature of systems far from the thermodynamic limit. Often, biological systems cannot be modeled with traditional continuous-deterministic models. Second, models must take into consideration the disparate spectrum of time scales observed in biological phenomena, such as slow transcription events and fast dimerization reactions. In the last decade, significant efforts have been expended on the development of stochastic chemical kinetics models to capture the dynamics of biomolecular systems, and on the development of robust multiscale algorithms, able to handle stiffness. In this paper, the focus is on the dynamics of reaction sets governed by stiff chemical Langevin equations, i.e., stiff stochastic differential equations. These are particularly challenging systems to model, requiring prohibitively small integration step sizes. We describe and illustrate the application of a semianalytical reduction framework for chemical Langevin equations that results in significant gains in computational cost.

Synthetic tetracycline-inducible regulatory networks: computer-aided design of dynamic phenotypes

BACKGROUND

Tightly regulated gene networks, precisely controlling the expression of protein molecules, have received considerable interest by the biomedical community due to their promising applications. Among the most well studied inducible transcription systems are the tetracycline regulatory expression systems based on the tetracycline resistance operon of Escherichia coli, Tet-Off (tTA) and Tet-On (rtTA). Despite their initial success and improved designs, limitations still persist, such as low inducer sensitivity. Instead of looking at these networks statically, and simply changing or mutating the promoter and operator regions with trial and error, a systematic investigation of the dynamic behavior of the network can result in rational design of regulatory gene expression systems. Sophisticated algorithms can accurately capture the dynamical behavior of gene networks. With computer aided design, we aim to improve the synthesis of regulatory networks and propose new designs that enable tighter control of expression.

RESULTS

In this paper we engineer novel networks by recombining existing genes or part of genes. We synthesize four novel regulatory networks based on the Tet-Off and Tet-On systems. We model all the known individual biomolecular interactions involved in transcription, translation, regulation and induction. With multiple time-scale stochastic-discrete and stochastic-continuous models we accurately capture the transient and steady state dynamics of these networks. Important biomolecular interactions are identified and the strength of the interactions engineered to satisfy design criteria. A set of clear design rules is developed and appropriate mutants of regulatory proteins and operator sites are proposed.

CONCLUSION

The complexity of biomolecular interactions is accurately captured through computer simulations. Computer simulations allow us to look into the molecular level, portray the dynamic behavior of gene regulatory networks and rationally engineer novel ones with useful applications. We are able to propose, test and accept or reject design principles for each network. Guided by simulations, we develop a set of design principles for novel tetracycline-inducible networks.

Computer-aided design of modular protein devices: Boolean AND gene activation

Many potentially useful synthetic gene networks require the expression of an engineered gene if and only if two different DNA-binding proteins exist in sufficient concentration. While some natural and engineered systems activate gene expression according to a logical AND-like behavior, they often utilize allosteric or cooperative protein-protein interactions, rendering their components unsuitable for a toolbox of modular parts for use in multiple applications. Here, we develop a quantitative model to demonstrate that a small system of interacting fusion proteins, called a protein device, can activate an engineered gene according to the Boolean AND behavior while using only modular protein domains and DNA sites. The fusion proteins are created from transactivating, DNA-binding, non-DNA binding, and protein-protein interaction domains along with the corresponding peptide ligands. Using a combined kinetic and thermodynamic model, we identify the characteristics of the molecular components and their rates of constitutive production that maximize the fidelity of AND behavior. These AND protein devices facilitate the creation of complex genetic programs and may be used to create gene therapies, biosensors and other biomedical and biotechnological applications that turn on gene expression only when multiple DNA-binding proteins are simultaneously present.

Multiscale Hy3S: hybrid stochastic simulation for supercomputers

Background

Stochastic simulation has become a useful tool to both study natural biological systems and design new synthetic ones. By capturing the intrinsic molecular fluctuations of "small" systems, these simulations produce a more accurate picture of single cell dynamics, including interesting phenomena missed by deterministic methods, such as noise-induced oscillations and transitions between stable states. However, the computational cost of the original stochastic simulation algorithm can be high, motivating the use of hybrid stochastic methods. Hybrid stochastic methods partition the system into multiple subsets and describe each subset as a different representation, such as a jump Markov, Poisson, continuous Markov, or deterministic process. By applying valid approximations and self-consistently merging disparate descriptions, a method can be considerably faster, while retaining accuracy. In this paper, we describe Hy3S, a collection of multiscale simulation programs.

Results

Building on our previous work on developing novel hybrid stochastic algorithms, we have created the Hy3S software package to enable scientists and engineers to both study and design extremely large well-mixed biological systems with many thousands of reactions and chemical species. We have added adaptive stochastic numerical integrators to permit the robust simulation of dynamically stiff biological systems. In addition, Hy3S has many useful features, including embarrassingly parallelized simulations with MPI; special discrete events, such as transcriptional and translation elongation and cell division; mid-simulation perturbations in both the number of molecules of species and reaction kinetic parameters; combinatorial variation of both initial conditions and kinetic parameters to enable sensitivity analysis; use of NetCDF optimized binary format to quickly read and write large datasets; and a simple graphical user interface, written in Matlab, to help users create biological systems and analyze data. We demonstrate the accuracy and efficiency of Hy3S with examples, including a large-scale system benchmark and a complex bistable biochemical network with positive feedback. The software itself is open-sourced under the GPL license and is modular, allowing users to modify it for their own purposes.

Conclusion

Hy3S is a powerful suite of simulation programs for simulating the stochastic dynamics of networks of biochemical reactions. Its first public version enables computational biologists to more efficiently investigate the dynamics of realistic biological systems.

An equation-free probabilistic steady-state approximation: Dynamic application to the stochastic simulation of biochemical reaction networks

Stochastic chemical kinetics more accurately describes the dynamics of "small" chemical systems, such as biological cells. Many real systems contain dynamical stiffness, which causes the exact stochastic simulation algorithm or other kinetic Monte Carlo methods to spend the majority of their time executing frequently occurring reaction events. Previous methods have successfully applied a type of probabilistic steady-state approximation by deriving an evolution equation, such as the chemical master equation, for the relaxed fast dynamics and using the solution of that equation to determine the slow dynamics. However, because the solution of the chemical master equation is limited to small, carefully selected, or linear reaction networks, an alternate equation-free method would be highly useful. We present a probabilistic steady-state approximation that separates the time scales of an arbitrary reaction network, detects the convergence of a marginal distribution to a quasi-steady-state, directly samples the underlying distribution, and uses those samples to accurately predict the state of the system, including the effects of the slow dynamics, at future times. The numerical method produces an accurate solution of both the fast and slow reaction dynamics while, for stiff systems, reducing the computational time by orders of magnitude. The developed theory makes no approximations on the shape or form of the underlying steady-state distribution and only assumes that it is ergodic. We demonstrate the accuracy and efficiency of the method using multiple interesting examples, including a highly nonlinear protein-protein interaction network. The developed theory may be applied to any type of kinetic Monte Carlo simulation to more efficiently simulate dynamically stiff systems, including existing exact, approximate, or hybrid stochastic simulation techniques.