Parameter estimation using Simulated Annealing for S-system models of biochemical networks

Estimation of kinetic parameters in dynamic FDG PET imaging based on shortened protocols: a virtual clinical study

This study investigated the estimation of kinetic parameters and production of related parametric Ki images in FDG PET imaging using the proposed shortened protocol (three 3-min/bed routine static images) by means of the simulated annealing (SA) algorithm. Six realistic heterogeneous tumors and various levels of [18F] FDG uptake were simulated by the XCAT phantom. An irreversible two-tissue compartment model (2TCM) using population-based input function was employed. By keeping two routine clinical scans fixed (60-min and 90-min post injection), the effect of the early scan time on optimizing the estimation of the pharmacokinetic parameters was investigated. The SA optimization algorithm was applied to estimate micro- and macro-parameters (K1, k2, k3, Ki). The minimum bias for most parameters was observed at a scan time of 20-min, which was < 10%. A highly significant correlation (> 0.9) as well as limited bias (< 10%) were observed between kinetic parameters generated from two methods [two-tissue compartment full dynamic scan (2TCM-full) and two-tissue compartment by SA algorithm (2TCM-SA)]. The analysis showed a strong correlation (> 0.8) between (2TCM-SA) Ki and SUV images. In addition, the tumor-to-background ratio (TBR) metric in the parametric (2TCM-SA) Ki images was significantly higher than SUV, although the SUV images provide better Contrast-to-noise ratio relative to parametric (2TCM-SA) Ki images. The proposed shortened protocol by the SA algorithm can estimate the kinetic parameters in FDG PET scan with high accuracy and robustness. It was also concluded that the parametric Ki images obtained from the 2TCM-SA as a complementary image of the SUV possess more quantification information than SUV images and can be used by the nuclear medicine specialist. This method has the potential to be an alternative to a full dynamic PET scan.

Artificial Bee Colony algorithm in estimating kinetic parameters for yeast fermentation pathway

Analyzing metabolic pathways in systems biology requires accurate kinetic parameters that represent the simulated in vivo processes. Simulation of the fermentation pathway in the Saccharomyces cerevisiae kinetic model help saves much time in the optimization process. Fitting the simulated model into the experimental data is categorized under the parameter estimation problem. Parameter estimation is conducted to obtain the optimal values for parameters related to the fermentation process. This step is essential because insufficient identification of model parameters can cause erroneous conclusions. The kinetic parameters cannot be measured directly. Therefore, they must be estimated from the experimental data either in vitro or in vivo. Parameter estimation is a challenging task in the biological process due to the complexity and nonlinearity of the model. Therefore, we propose the Artificial Bee Colony algorithm (ABC) to estimate the parameters in the fermentation pathway of S. cerevisiae to obtain more accurate values. A metabolite with a total of six parameters is involved in this article. The experimental results show that ABC outperforms other estimation algorithms and gives more accurate kinetic parameter values for the simulated model. Most of the estimated kinetic parameter values obtained from the proposed algorithm are the closest to the experimental data.

Symbolic kinetic models in python (SKiMpy): intuitive modeling of large-scale biological kinetic models

MOTIVATION

Large-scale kinetic models are an invaluable tool to understand the dynamic and adaptive responses of biological systems. The development and application of these models have been limited by the availability of computational tools to build and analyze large-scale models efficiently. The toolbox presented here provides the means to implement, parameterize and analyze large-scale kinetic models intuitively and efficiently.

RESULTS

We present a Python package (SKiMpy) bridging this gap by implementing an efficient kinetic modeling toolbox for the semiautomatic generation and analysis of large-scale kinetic models for various biological domains such as signaling, gene expression and metabolism. Furthermore, we demonstrate how this toolbox is used to parameterize kinetic models around a steady-state reference efficiently. Finally, we show how SKiMpy can implement multispecies bioreactor simulations to assess biotechnological processes.

AVAILABILITY AND IMPLEMENTATION

The software is available as a Python 3 package on GitHub: https://github.com/EPFL-LCSB/SKiMpy, along with adequate documentation.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

PGRNIG: novel parallel gene regulatory network identification algorithm based on GPU

Molecular biology has revealed that complex life phenomena can be treated as the result of many gene interactions. Investigating these interactions and understanding the intrinsic mechanisms of biological systems using gene expression data have attracted a lot of attention. As a typical gene regulatory network (GRN) inference method, the S-system has been utilized to deal with small-scale network identification. However, it is extremely difficult to optimize it to infer medium-to-large networks. This paper proposes a novel parallel swarm intelligent algorithm, PGRNIG, to optimize the parameters of the S-system. We employed the clone selection strategy to improve the whale optimization algorithm (CWOA). To enhance the time efficiency of CWOA optimization, we utilized a parallel CWOA (PCWOA) based on the compute unified device architecture (CUDA) platform. Decomposition strategy and L1 regularization were utilized to reduce the search space and complexity of GRN inference. We applied the PGRNIG algorithm on three synthetic datasets and two real time-series expression datasets of the species of Escherichia coli and Saccharomyces cerevisiae. Experimental results show that PGRNIG could infer the gene regulatory network more accurately than other state-of-the-art methods with a convincing computational speed-up. Our findings show that CWOA and PCWOA have faster convergence performances than WOA.

Gene Regulatory Identification Based on the Novel Hybrid Time-Delayed Method

Gene regulatory network (GRN) inference with biology data is a difficult and serious issue in the field of system biology. In order to detect the direct associations of GRN more accurately, a novel two-step GRN inference technique based on the time-delayed correlation coefficient (TDCC) and time-delayed complex-valued S-system model (TDCVSS) is proposed. First, a TDCC algorithm is utilized to construct an initial network. Second, a TDCVSS model is utilized to prune the network topology in order to delete false-positive regulatory relationships for each target gene. The complex-valued restricted additive tree and complex-valued differential evolution are proposed to approximate the optimal TDCVSS model. Finally, the overall network could be inferred by integrating the regulations of all target genes. Two real gene expression datasets from E. coli and S. cerevisiae gene networks are utilized to evaluate the performances of our proposed two-step GRN inference algorithm. The results demonstrated that the proposed algorithm could infer GRN more correct than classical methods and time-delayed methods.

Root locus-based stability analysis for biological systems

Background: The first objective for realizing and handling biological systems is to choose a suitable model prototype and then perform structure and parameter identification. Afterwards, a theoretical analysis is needed to understand the characteristics, abilities, and limitations of the underlying systems. Generalized Michaelis-Menten kinetics (MM) and S-systems are two well-known biochemical system theory-based models. Research on steady-state estimation of generalized MM systems is difficult because of their complex structure. Further, theoretical analysis of S-systems is still difficult because of the power-law structure, and even the estimation of steady states can be easily achieved via algebraic equations. Aim: We focus on how to flexibly use control technologies to perform deeper biological system analysis. Methods: For generalized MM systems, the root locus method (proposed by Walter R. Evans) is used to predict the direction and rate (flux) limitations of the reaction and to estimate the steady states and stability margins (relative stability). Mode analysis is additionally introduced to discuss the transient behavior and the setting time. For S-systems, the concept of root locus, mode analysis, and the converse theorem are used to predict the dynamic behavior, to estimate the setting time and to analyze the relative stability of systems. Theoretical results were examined via simulation in a Simulink/MATLAB environment. Results: Four kinds of small functional modules (a system with reversible MM kinetics, a system with a singular or nearly singular system matrix and systems with cascade or branch pathways) are used to describe the proposed strategies clearly. For the reversible MM kinetics system, we successfully predict the direction and the rate (flux) limitations of reactions and obtain the values of steady state and net flux. We observe that theoretically derived results are consistent with simulation results. Good prediction is observed ([Formula: see text]% accuracy). For the system with a (nearly) singular matrix, we demonstrate that the system is neither globally exponentially stable nor globally asymptotically stable but globally semistable. The system possesses an infinite gain margin (GM denoting how much the gain can increase before the system becomes unstable) regardless of how large or how small the values of independent variables are, but the setting time decreases and then increases or always decreases as the values of independent variables increase. For S-systems, we first demonstrate that the stability of S-systems can be determined by linearized systems via root loci, mode analysis, and block diagram-based simulation. The relevant S-systems possess infinite GM for the values of independent variables varying from zero to infinity, and the setting time increases as the values of independent variables increase. Furthermore, the branch pathway maintains oscillation until a steady state is reached, but the oscillation phenomenon does not exist in the cascade pathway because in this system, all of the root loci are located on real lines. The theoretical predictions of dynamic behavior for these two systems are consistent with the simulation results. This study provides a guideline describing how to choose suitable independent variables such that systems possess satisfactory performance for stability margins, setting time and dynamic behavior. Conclusion: The proposed root locus-based analysis can be applied to any kind of differential equation-based biological system. This research initiates a method to examine system dynamic behavior and to discuss operating principles.

Systems approach reveals photosensitivity and PER2 level as determinants of clock-modulator efficacy

In mammals, the master circadian clock synchronizes daily rhythms of physiology and behavior with the day-night cycle. Failure of synchrony, which increases the risk for numerous chronic diseases, can be treated by phase adjustment of the circadian clock pharmacologically, for example, with melatonin, or a CK1δ/ε inhibitor. Here, using in silico experiments with a systems pharmacology model describing molecular interactions, and pharmacokinetic and behavioral experiments in cynomolgus monkeys, we find that the circadian phase delay caused by CK1δ/ε inhibition is more strongly attenuated by light in diurnal monkeys and humans than in nocturnal mice, which are common preclinical models. Furthermore, the effect of CK1δ/ε inhibition strongly depends on endogenous PER2 protein levels, which differs depending on both the molecular cause of the circadian disruption and the patient's lighting environment. To circumvent such large interindividual variations, we developed an adaptive chronotherapeutics to identify precise dosing regimens that could restore normal circadian phase under different conditions. Our results reveal the importance of photosensitivity in the clinical efficacy of clock-modulating drugs, and enable precision medicine for circadian disruption.

Simulink-Based Analysis for Coupled Metabolic Systems

Stability analysis and dynamic simulation are important for researchers to capture the performance and the properties of underling systems. S-systems have good potential for characterizing dynamic interactive behaviour of large scale metabolic and genetic systems. It is important to develop a platform to achieve timely dynamic behaviour of S-systems to various situations. In this study, we first set up the respective block diagrams of S-systems for module-based simulation. We then derive reasonable theorems to examine the stability of S-systems and find out what kinds of environmental situations will make systems stable. Three canonical systems are used to examine the results which are carried out in the Matlab/Simulink environments.

A modified particle swarm optimization algorithm for parameter estimation of a biological system

Background

Mathematical modeling has achieved a broad interest in the field of biology. These models represent the associations among the metabolism of the biological phenomenon with some mathematical equations such that the observed time course profile of the biological data fits the model. However, the estimation of the unknown parameters of the model is a challenging task. Many algorithms have been developed for parameter estimation, but none of them is entirely capable of finding the best solution. The purpose of this paper is to develop a method for precise estimation of parameters of a biological model.

Methods

In this paper, a novel particle swarm optimization algorithm based on a decomposition technique is developed. Then, its root mean square error is compared with simple particle swarm optimization, Iterative Unscented Kalman Filter and Simulated Annealing algorithms for two different simulation scenarios and a real data set related to the metabolism of CAD system.

Results

Our proposed algorithm results in 54.39% and 26.72% average reduction in root mean square error when applied to the simulation and experimental data, respectively.

Conclusion

The results show that the metaheuristic approaches such as the proposed method are very wise choices for finding the solution of nonlinear problems with many unknown parameters.

Kernel Generalized Likelihood Ratio Test for Fault Detection of Biological Systems

In this paper, we develop an improved fault detection (FD) technique in order to enhance the monitoring abilities of nonlinear biological processes. Generalized likelihood ratio test (GLRT)-based kernel principal component analysis (KPCA) (called also kernel GLRT) is an effective data-driven technique for monitoring nonlinear processes. However, it is well known that the data collected from complex and multivariate processes are multiscale due to the variety of changes that could occur in process with different localization in time and frequency. Thus, to enhance the process monitoring abilities, we propose to combine the advantages of kernel GLRT and multiscale representation using wavelets by developing a multiscale kernel GLRT (MS-KGLRT) detection chart. The proposed fault detection approach is addressed so that the KPCA is used to compute the model in the feature space and the MS-KGLRT chart is applied to detect the faults. The detection performance of the new chart is studied using two examples, one using synthetic data and the other using biological process representing a Cad System in E. Coli (CSEC) model for detecting small and moderate shifts (offset or bias and drift). The MS-KGLRT chart is used to enhance fault detection of the CSEC model through monitoring some of the key variables involved in this model such as enzymes, lysine, and cadaverine.

Improved Statistical Fault Detection Technique and Application to Biological Phenomena Modeled by S-Systems

In our previous work, we have demonstrated the effectiveness of the linear multiscale principal component analysis (PCA)-based moving window (MW)-generalized likelihood ratio test (GLRT) technique over the classical PCA and multiscale principal component analysis (MSPCA)-based GLRT methods. The developed fault detection algorithm provided optimal properties by maximizing the detection probability for a particular false alarm rate (FAR) with different values of windows, and however, most real systems are nonlinear, which make the linear PCA method not able to tackle the issue of non-linearity to a great extent. Thus, in this paper, first, we apply a nonlinear PCA to obtain an accurate principal component of a set of data and handle a wide range of nonlinearities using the kernel principal component analysis (KPCA) model. The KPCA is among the most popular nonlinear statistical methods. Second, we extend the MW-GLRT technique to one that utilizes exponential weights to residuals in the moving window (instead of equal weightage) as it might be able to further improve fault detection performance by reducing the FAR using exponentially weighed moving average (EWMA). The developed detection method, which is called EWMA-GLRT, provides improved properties, such as smaller missed detection and FARs and smaller average run length. The idea behind the developed EWMA-GLRT is to compute a new GLRT statistic that integrates current and previous data information in a decreasing exponential fashion giving more weight to the more recent data. This provides a more accurate estimation of the GLRT statistic and provides a stronger memory that will enable better decision making with respect to fault detection. Therefore, in this paper, a KPCA-based EWMA-GLRT method is developed and utilized in practice to improve fault detection in biological phenomena modeled by S-systems and to enhance monitoring process mean. The idea behind a KPCA-based EWMA-GLRT fault detection algorithm is to combine the advantages brought forward by the proposed EWMA-GLRT fault detection chart with the KPCA model. Thus, it is used to enhance fault detection of the Cad System in E. coli model through monitoring some of the key variables involved in this model such as enzymes, transport proteins, regulatory proteins, lysine, and cadaverine. The results demonstrate the effectiveness of the proposed KPCA-based EWMA-GLRT method over Q , GLRT, EWMA, Shewhart, and moving window-GLRT methods. The detection performance is assessed and evaluated in terms of FAR, missed detection rates, and average run length (ARL₁) values.

First principle-based models in plant suspension cell cultures: a review

In this work, the development and application of published models for describing the behavior of plant cell cultures is reviewed. The structure of each type of model is analyzed and the new tendencies for the modeling of biotechnological processes that can be applied in plant cell cultures are presented. This review is a tool for clarifying the main features that characterize each type of model in the field of plant cell cultures and can be used as a support on the selection of the more suitable model type, taking into account the purpose of specific research.

Reverse engineering gene regulatory networks from measurement with missing values

Background

Gene expression time series data are usually in the form of high-dimensional arrays. Unfortunately, the data may sometimes contain missing values: for either the expression values of some genes at some time points or the entire expression values of a single time point or some sets of consecutive time points. This significantly affects the performance of many algorithms for gene expression analysis that take as an input, the complete matrix of gene expression measurement. For instance, previous works have shown that gene regulatory interactions can be estimated from the complete matrix of gene expression measurement. Yet, till date, few algorithms have been proposed for the inference of gene regulatory network from gene expression data with missing values.

Results

We describe a nonlinear dynamic stochastic model for the evolution of gene expression. The model captures the structural, dynamical, and the nonlinear natures of the underlying biomolecular systems. We present point-based Gaussian approximation (PBGA) filters for joint state and parameter estimation of the system with one-step or two-step missing measurements. The PBGA filters use Gaussian approximation and various quadrature rules, such as the unscented transform (UT), the third-degree cubature rule and the central difference rule for computing the related posteriors. The proposed algorithm is evaluated with satisfying results for synthetic networks, in silico networks released as a part of the DREAM project, and the real biological network, the in vivo reverse engineering and modeling assessment (IRMA) network of yeast Saccharomyces cerevisiae.

Conclusion

PBGA filters are proposed to elucidate the underlying gene regulatory network (GRN) from time series gene expression data that contain missing values. In our state-space model, we proposed a measurement model that incorporates the effect of the missing data points into the sequential algorithm. This approach produces a better inference of the model parameters and hence, more accurate prediction of the underlying GRN compared to when using the conventional Gaussian approximation (GA) filters ignoring the missing data points.

Electronic supplementary material

The online version of this article (doi:10.1186/s13637-016-0055-8) contains supplementary material, which is available to authorized users.

Model-driven experimental approach reveals the complex regulatory distribution of p53 by the circadian factor Period 2

The circadian clock and cell cycle networks are interlocked on the molecular level, with the core clock loop exerting a multilevel regulatory role over cell cycle components. This is particularly relevant to the circadian factor Period 2 (Per2), which modulates the stability of the tumor suppressor p53 in unstressed cells and transcriptional activity in response to genotoxic stress. Per2 binding prevents Mdm2-mediated ubiquitination of p53 and, therefore, its degradation, and oscillations in the peaks of Per2 and p53 were expected to correspond. However, our findings showed that Per2 and p53 rhythms were significantly out-of-phase relative to each other in cell lysates and in purified cytoplasmic fractions. These seemingly conflicting experimental data motivated the use of a combined theoretical and experimental approach focusing on the role played by Per2 in dictating the phase of p53 oscillations. Systematic modeling of all possible regulatory scenarios predicted that the observed phase relationship between Per2 and p53 could be simulated if (i) p53 was more stable in the nucleus than in the cytoplasm, (ii) Per2 associates to various ubiquitinated forms of p53, and (iii) Per2 mediated p53 nuclear import. These predictions were supported by a sevenfold increase in p53's half-life in the nucleus and by in vitro binding of Per2 to the various ubiquitinated forms of p53. Last, p53's nuclear shuttling was significantly favored by ectopic expression of Per2 and reduced because of Per2 down-regulation. Our combined theoretical/mathematical approach reveals how clock regulatory nodes can be inferred from oscillating time course data.

Reverse engineering of gene regulatory network using restricted gene expression programming

Inference of gene regulatory networks has been becoming a major area of interest in the field of systems biology over the past decade. In this paper, we present a novel representation of S-system model, named restricted gene expression programming (RGEP), to infer gene regulatory network. A new hybrid evolutionary algorithm based on structure-based evolutionary algorithm and cuckoo search (CS) is proposed to optimize the architecture and corresponding parameters of model, respectively. Two synthetic benchmark datasets and one real biological dataset from SOS DNA repair network in E. coli are used to test the validity of our method. Experimental results demonstrate that our proposed method performs better than previously proposed popular methods.

Construction of Gene Regulatory Networks Using Recurrent Neural Networks and Swarm Intelligence

We have proposed a methodology for the reverse engineering of biologically plausible gene regulatory networks from temporal genetic expression data. We have used established information and the fundamental mathematical theory for this purpose. We have employed the Recurrent Neural Network formalism to extract the underlying dynamics present in the time series expression data accurately. We have introduced a new hybrid swarm intelligence framework for the accurate training of the model parameters. The proposed methodology has been first applied to a small artificial network, and the results obtained suggest that it can produce the best results available in the contemporary literature, to the best of our knowledge. Subsequently, we have implemented our proposed framework on experimental (in vivo) datasets. Finally, we have investigated two medium sized genetic networks (in silico) extracted from GeneNetWeaver, to understand how the proposed algorithm scales up with network size. Additionally, we have implemented our proposed algorithm with half the number of time points. The results indicate that a reduction of 50% in the number of time points does not have an effect on the accuracy of the proposed methodology significantly, with a maximum of just over 15% deterioration in the worst case.

Inverse problem studies of biochemical systems with structure identification of S-systems by embedding training functions in a genetic algorithm

An efficient inverse problem approach for parameter estimation, state and structure identification from dynamic data by embedding training functions in a genetic algorithm methodology (ETFGA) is proposed for nonlinear dynamical biosystems using S-system canonical models. Use of multiple shooting and decomposition approach as training functions has been shown for handling of noisy datasets and computational efficiency in studying the inverse problem. The advantages of the methodology are brought out systematically by studying it for three biochemical model systems of interest. By studying a small-scale gene regulatory system described by a S-system model, the first example demonstrates the use of ETFGA for the multifold aims of the inverse problem. The estimation of a large number of parameters with simultaneous state and network identification is shown by training a generalized S-system canonical model with noisy datasets. The results of this study bring out the superior performance of ETFGA on comparison with other metaheuristic approaches. The second example studies the regulation of cAMP oscillations in Dictyostelium cells now assuming limited availability of noisy data. Here, flexibility of the approach to incorporate partial system information in the identification process is shown and its effect on accuracy and predictive ability of the estimated model are studied. The third example studies the phenomenological toy model of the regulation of circadian oscillations in Drosophila that follows rate laws different from S-system power-law. For the limited noisy data, using a priori information about properties of the system, we could estimate an alternate S-system model that showed robust oscillatory behavior with predictive abilities.

SYNTHETIC BIOLOGY. Emergent genetic oscillations in a synthetic microbial consortium

A challenge of synthetic biology is the creation of cooperative microbial systems that exhibit population-level behaviors. Such systems use cellular signaling mechanisms to regulate gene expression across multiple cell types. We describe the construction of a synthetic microbial consortium consisting of two distinct cell types—an "activator" strain and a "repressor" strain. These strains produced two orthogonal cell-signaling molecules that regulate gene expression within a synthetic circuit spanning both strains. The two strains generated emergent, population-level oscillations only when cultured together. Certain network topologies of the two-strain circuit were better at maintaining robust oscillations than others. The ability to program population-level dynamics through the genetic engineering of multiple cooperative strains points the way toward engineering complex synthetic tissues and organs with multiple cell types.

Automated parameter estimation for biological models using Bayesian statistical model checking

Background

Probabilistic models have gained widespread acceptance in the systems biology community as a useful way to represent complex biological systems. Such models are developed using existing knowledge of the structure and dynamics of the system, experimental observations, and inferences drawn from statistical analysis of empirical data. A key bottleneck in building such models is that some system variables cannot be measured experimentally. These variables are incorporated into the model as numerical parameters. Determining values of these parameters that justify existing experiments and provide reliable predictions when model simulations are performed is a key research problem.

Domain experts usually estimate the values of these parameters by fitting the model to experimental data. Model fitting is usually expressed as an optimization problem that requires minimizing a cost-function which measures some notion of distance between the model and the data. This optimization problem is often solved by combining local and global search methods that tend to perform well for the specific application domain. When some prior information about parameters is available, methods such as Bayesian inference are commonly used for parameter learning. Choosing the appropriate parameter search technique requires detailed domain knowledge and insight into the underlying system.

Results

Using an agent-based model of the dynamics of acute inflammation, we demonstrate a novel parameter estimation algorithm by discovering the amount and schedule of doses of bacterial lipopolysaccharide that guarantee a set of observed clinical outcomes with high probability. We synthesized values of twenty-eight unknown parameters such that the parameterized model instantiated with these parameter values satisfies four specifications describing the dynamic behavior of the model.

Conclusions

We have developed a new algorithmic technique for discovering parameters in complex stochastic models of biological systems given behavioral specifications written in a formal mathematical logic. Our algorithm uses Bayesian model checking, sequential hypothesis testing, and stochastic optimization to automatically synthesize parameters of probabilistic biological models.

Bottom-up modeling approach for the quantitative estimation of parameters in pathogen-host interactions

Opportunistic fungal pathogens can cause bloodstream infection and severe sepsis upon entering the blood stream of the host. The early immune response in human blood comprises the elimination of pathogens by antimicrobial peptides and innate immune cells, such as neutrophils or monocytes. Mathematical modeling is a predictive method to examine these complex processes and to quantify the dynamics of pathogen-host interactions. Since model parameters are often not directly accessible from experiment, their estimation is required by calibrating model predictions with experimental data. Depending on the complexity of the mathematical model, parameter estimation can be associated with excessively high computational costs in terms of run time and memory. We apply a strategy for reliable parameter estimation where different modeling approaches with increasing complexity are used that build on one another. This bottom-up modeling approach is applied to an experimental human whole-blood infection assay for Candida albicans. Aiming for the quantification of the relative impact of different routes of the immune response against this human-pathogenic fungus, we start from a non-spatial state-based model (SBM), because this level of model complexity allows estimating a priori unknown transition rates between various system states by the global optimization method simulated annealing. Building on the non-spatial SBM, an agent-based model (ABM) is implemented that incorporates the migration of interacting cells in three-dimensional space. The ABM takes advantage of estimated parameters from the non-spatial SBM, leading to a decreased dimensionality of the parameter space. This space can be scanned using a local optimization approach, i.e., least-squares error estimation based on an adaptive regular grid search, to predict cell migration parameters that are not accessible in experiment. In the future, spatio-temporal simulations of whole-blood samples may enable timely stratification of sepsis patients by distinguishing hyper-inflammatory from paralytic phases in immune dysregulation.

MP-GeneticSynth: inferring biological network regulations from time series

MP-GeneticSynth is a Java tool for discovering the logic and regulation mechanisms responsible for observed biological dynamics in terms of finite difference recurrent equations. The software makes use of: (i) metabolic P systems as a modeling framework, (ii) an evolutionary approach to discover flux regulation functions as linear combinations of given primitive functions, (iii) a suitable reformulation of the least squares method to estimate function parameters considering simultaneously all the reactions involved in complex dynamics. The tool is available as a plugin for the virtual laboratory MetaPlab. It has graphical and interactive interfaces for data preparation, a priori knowledge integration, and flux regulator analysis. Availability and implementation: Source code, binaries, documentation (including quick start guide and videos) and case studies are freely available at http://mplab.sci.univr.it/plugins/mpgs/index.html.

Approximate Bayesian computation schemes for parameter inference of discrete stochastic models using simulated likelihood density

BACKGROUND

Mathematical modeling is an important tool in systems biology to study the dynamic property of complex biological systems. However, one of the major challenges in systems biology is how to infer unknown parameters in mathematical models based on the experimental data sets, in particular, when the data are sparse and the regulatory network is stochastic.

RESULTS

To address this issue, this work proposed a new algorithm to estimate parameters in stochastic models using simulated likelihood density in the framework of approximate Bayesian computation. Two stochastic models were used to demonstrate the efficiency and effectiveness of the proposed method. In addition, we designed another algorithm based on a novel objective function to measure the accuracy of stochastic simulations.

CONCLUSIONS

Simulation results suggest that the usage of simulated likelihood density improves the accuracy of estimates substantially. When the error is measured at each observation time point individually, the estimated parameters have better accuracy than those obtained by a published method in which the error is measured using simulations over the entire observation time period.

A Parameter Estimation Method for Biological Systems modelled by ODE/DDE Models Using Spline Approximation and Differential Evolution Algorithm

The inverse problem of identifying unknown parameters of known structure dynamical biological systems, which are modelled by ordinary differential equations or delay differential equations, from experimental data is treated in this paper. A two stage approach is adopted: first, combine spline theory and Nonlinear Programming (NLP), the parameter estimation problem is formulated as an optimization problem with only algebraic constraints; then, a new differential evolution (DE) algorithm is proposed to find a feasible solution. The approach is designed to handle problem of realistic size with noisy observation data. Three cases are studied to evaluate the performance of the proposed algorithm: two are based on benchmark models with priori-determined structure and parameters; the other one is a particular biological system with unknown model structure. In the last case, only a set of observation data available and in this case a nominal model is adopted for the identification. All the test systems were successfully identified by using a reasonable amount of experimental data within an acceptable computation time. Experimental evaluation reveals that the proposed method is capable of fast estimation on the unknown parameters with good precision.

Biochemical systems identification by a random drift particle swarm optimization approach

BACKGROUND

Finding an efficient method to solve the parameter estimation problem (inverse problem) for nonlinear biochemical dynamical systems could help promote the functional understanding at the system level for signalling pathways. The problem is stated as a data-driven nonlinear regression problem, which is converted into a nonlinear programming problem with many nonlinear differential and algebraic constraints. Due to the typical ill conditioning and multimodality nature of the problem, it is in general difficult for gradient-based local optimization methods to obtain satisfactory solutions. To surmount this limitation, many stochastic optimization methods have been employed to find the global solution of the problem.

RESULTS

This paper presents an effective search strategy for a particle swarm optimization (PSO) algorithm that enhances the ability of the algorithm for estimating the parameters of complex dynamic biochemical pathways. The proposed algorithm is a new variant of random drift particle swarm optimization (RDPSO), which is used to solve the above mentioned inverse problem and compared with other well known stochastic optimization methods. Two case studies on estimating the parameters of two nonlinear biochemical dynamic models have been taken as benchmarks, under both the noise-free and noisy simulation data scenarios.

CONCLUSIONS

The experimental results show that the novel variant of RDPSO algorithm is able to successfully solve the problem and obtain solutions of better quality than other global optimization methods used for finding the solution to the inverse problems in this study.

An approach for dynamical network reconstruction of simple network motifs

Background

One of the most important projects in the post-genome-era is the systemic identification of biological network. The almost of studies for network identification focused on the improvement of computational efficiency in large-scale network inference of complex system with cyclic relations and few attempted have been done for answering practical problem occurred in real biological systems. In this study, we focused to evaluate inferring performance of our previously proposed method for inferring biological network on simple network motifs.

Results

We evaluated the network inferring accuracy and efficiency of our previously proposed network inferring algorithm, by using 6 kinds of repeated appearance of highly significant network motifs in the regulatory network of E. coli proposed by Shen-Orr et al and Herrgård et al, and 2 kinds of network motif in S. cerevisiae proposed by Lee et. al. As a result, our method could reconstruct about 40% of interactions in network motif from time-series data set. Moreover the introduction of time-series data of one-factor disrupted model could remarkably improved the performance of network inference.

Conclusions

The results of network inference examination of E. coli network motif shows that our network inferring algorithm was able to apply to typical topology of biological network. A continuous examination of inferring well established network motif in biology would strengthen the applicability of our algorithm to the realistic biological network.

Modeling of nonlinear biological phenomena modeled by S-systems

A central challenge in computational modeling of biological systems is the determination of the model parameters. In such cases, estimating these variables or parameters from other easily obtained measurements can be extremely useful. For example, time-series dynamic genomic data can be used to develop models representing dynamic genetic regulatory networks, which can be used to design intervention strategies to cure major diseases and to better understand the behavior of biological systems. Unfortunately, biological measurements are usually highly infected by errors that hide the important characteristics in the data. Therefore, these noisy measurements need to be filtered to enhance their usefulness in practice. This paper addresses the problem of state and parameter estimation of biological phenomena modeled by S-systems using Bayesian approaches, where the nonlinear observed system is assumed to progress according to a probabilistic state space model. The performances of various conventional and state-of-the-art state estimation techniques are compared. These techniques include the extended Kalman filter (EKF), unscented Kalman filter (UKF), particle filter (PF), and the developed variational Bayesian filter (VBF). Specifically, two comparative studies are performed. In the first comparative study, the state variables (the enzyme CadA, the model cadBA, the cadaverine Cadav and the lysine Lys for a model of the Cad System in Escherichia coli (CSEC)) are estimated from noisy measurements of these variables, and the various estimation techniques are compared by computing the estimation root mean square error (RMSE) with respect to the noise-free data. In the second comparative study, the state variables as well as the model parameters are simultaneously estimated. In this case, in addition to comparing the performances of the various state estimation techniques, the effect of the number of estimated model parameters on the accuracy and convergence of these techniques is also assessed. The results of both comparative studies show that the UKF provides a higher accuracy than the EKF due to the limited ability of EKF to accurately estimate the mean and covariance matrix of the estimated states through lineralization of the nonlinear process model. The results also show that the VBF provides a relative improvement over PF. This is because, unlike the PF which depends on the choice of sampling distribution used to estimate the posterior distribution, the VBF yields an optimum choice of the sampling distribution, which also utilizes the observed data. The results of the second comparative study show that, for all techniques, estimating more model parameters affects the estimation accuracy as well as the convergence of the estimated states and parameters. The VBF, however, still provides advantages over other methods with respect to estimation accuracy as well convergence.

Bayesian model comparison and parameter inference in systems biology using nested sampling

Inferring parameters for models of biological processes is a current challenge in systems biology, as is the related problem of comparing competing models that explain the data. In this work we apply Skilling's nested sampling to address both of these problems. Nested sampling is a Bayesian method for exploring parameter space that transforms a multi-dimensional integral to a 1D integration over likelihood space. This approach focuses on the computation of the marginal likelihood or evidence. The ratio of evidences of different models leads to the Bayes factor, which can be used for model comparison. We demonstrate how nested sampling can be used to reverse-engineer a system's behaviour whilst accounting for the uncertainty in the results. The effect of missing initial conditions of the variables as well as unknown parameters is investigated. We show how the evidence and the model ranking can change as a function of the available data. Furthermore, the addition of data from extra variables of the system can deliver more information for model comparison than increasing the data from one variable, thus providing a basis for experimental design.

Inference of Vohradský's models of genetic networks by solving two-dimensional function optimization problems

The inference of a genetic network is a problem in which mutual interactions among genes are inferred from time-series of gene expression levels. While a number of models have been proposed to describe genetic networks, this study focuses on a mathematical model proposed by Vohradský. Because of its advantageous features, several researchers have proposed the inference methods based on Vohradský's model. When trying to analyze large-scale networks consisting of dozens of genes, however, these methods must solve high-dimensional non-linear function optimization problems. In order to resolve the difficulty of estimating the parameters of the Vohradský's model, this study proposes a new method that defines the problem as several two-dimensional function optimization problems. Through numerical experiments on artificial genetic network inference problems, we showed that, although the computation time of the proposed method is not the shortest, the method has the ability to estimate parameters of Vohradský's models more effectively with sufficiently short computation times. This study then applied the proposed method to an actual inference problem of the bacterial SOS DNA repair system, and succeeded in finding several reasonable regulations.

Parameter discovery in stochastic biological models using simulated annealing and statistical model checking

Stochastic models are increasingly used to study the behaviour of biochemical systems. While the structure of such models is often readily available from first principles, unknown quantitative features of the model are incorporated into the model as parameters. Algorithmic discovery of parameter values from experimentally observed facts remains a challenge for the computational systems biology community. We present a new parameter discovery algorithm that uses simulated annealing, sequential hypothesis testing, and statistical model checking to learn the parameters in a stochastic model. We apply our technique to a model of glucose and insulin metabolism used for in-silico validation of artificial pancreata and demonstrate its effectiveness by developing parallel CUDA-based implementation for parameter synthesis in this model.

Differential simulated annealing: a robust and efficient global optimization algorithm for parameter estimation of biological networks

DSA outperformed five other algorithms in parameter estimation of 95 biological networks and showed significant advantage in large networks.

Perturbation biology: inferring signaling networks in cellular systems

We present a powerful experimental-computational technology for inferring network models that predict the response of cells to perturbations, and that may be useful in the design of combinatorial therapy against cancer. The experiments are systematic series of perturbations of cancer cell lines by targeted drugs, singly or in combination. The response to perturbation is quantified in terms of relative changes in the measured levels of proteins, phospho-proteins and cellular phenotypes such as viability. Computational network models are derived de novo, i.e., without prior knowledge of signaling pathways, and are based on simple non-linear differential equations. The prohibitively large solution space of all possible network models is explored efficiently using a probabilistic algorithm, Belief Propagation (BP), which is three orders of magnitude faster than standard Monte Carlo methods. Explicit executable models are derived for a set of perturbation experiments in SKMEL-133 melanoma cell lines, which are resistant to the therapeutically important inhibitor of RAF kinase. The resulting network models reproduce and extend known pathway biology. They empower potential discoveries of new molecular interactions and predict efficacious novel drug perturbations, such as the inhibition of PLK1, which is verified experimentally. This technology is suitable for application to larger systems in diverse areas of molecular biology.

Drugs that target specific effects of signaling proteins are promising agents for treating cancer. One of the many obstacles facing optimal drug design is inadequate quantitative understanding of the coordinated interactions between signaling proteins. De novo model inference of network or pathway models refers to the algorithmic construction of mathematical predictive models from experimental data without dependence on prior knowledge. De novo inference is difficult because of the prohibitively large number of possible sets of interactions that may or may not be consistent with observations. Our new method overcomes this difficulty by adapting a method from statistical physics, called Belief Propagation, which first calculates probabilistically the most likely interactions in the vast space of all possible solutions, then derives a set of individual, highly probable solutions in the form of executable models. In this paper, we test this method on artificial data and then apply it to model signaling pathways in a BRAF-mutant melanoma cancer cell line based on a large set of rich output measurements from a systematic set of perturbation experiments using drug combinations. Our results are in agreement with established biological knowledge, predict novel interactions, and predict efficacious drug targets that are specific to the experimental cell line and potentially to related tumors. The method has the potential, with sufficient systematic perturbation data, to model, de novo and quantitatively, the effects of hundreds of proteins on cellular responses, on a scale that is currently unreachable in diverse areas of cell biology. In a disease context, the method is applicable to the computational design of novel combination drug treatments.

Dynamical and topological robustness of the mammalian cell cycle network: a reverse engineering approach

A common gene regulatory network model is the threshold Boolean network, used for example to model the Arabidopsis thaliana floral morphogenesis network or the fission yeast cell cycle network. In this paper, we analyze a logical model of the mammalian cell cycle network and its threshold Boolean network equivalent. Firstly, the robustness of the network was explored with respect to update perturbations, in particular, what happened to the attractors for all the deterministic updating schemes. Results on the number of different limit cycles, limit cycle lengths, basin of attraction size, for all the deterministic updating schemes were obtained through mathematical and computational tools. Secondly, we analyzed the topology robustness of the network, by reconstructing synthetic networks that contained exactly the same attractors as the original model by means of a swarm intelligence approach. Our results indicate that networks may not be very robust given the great variety of limit cycles that a network can obtain depending on the updating scheme. In addition, we identified an omnipresent network with interactions that match with the original model as well as the discovery of new interactions. The techniques presented in this paper are general, and can be used to analyze other logical or threshold Boolean network models of gene regulatory networks.

Machines vs. ensembles: effective MAPK signaling through heterogeneous sets of protein complexes

Despite the importance of intracellular signaling networks, there is currently no consensus regarding the fundamental nature of the protein complexes such networks employ. One prominent view involves stable signaling machines with well-defined quaternary structures. The combinatorial complexity of signaling networks has led to an opposing perspective, namely that signaling proceeds via heterogeneous pleiomorphic ensembles of transient complexes. Since many hypotheses regarding network function rely on how we conceptualize signaling complexes, resolving this issue is a central problem in systems biology. Unfortunately, direct experimental characterization of these complexes has proven technologically difficult, while combinatorial complexity has prevented traditional modeling methods from approaching this question. Here we employ rule-based modeling, a technique that overcomes these limitations, to construct a model of the yeast pheromone signaling network. We found that this model exhibits significant ensemble character while generating reliable responses that match experimental observations. To contrast the ensemble behavior, we constructed a model that employs hierarchical assembly pathways to produce scaffold-based signaling machines. We found that this machine model could not replicate the experimentally observed combinatorial inhibition that arises when the scaffold is overexpressed. This finding provides evidence against the hierarchical assembly of machines in the pheromone signaling network and suggests that machines and ensembles may serve distinct purposes in vivo. In some cases, e.g. core enzymatic activities like protein synthesis and degradation, machines assembled via hierarchical energy landscapes may provide functional stability for the cell. In other cases, such as signaling, ensembles may represent a form of weak linkage, facilitating variation and plasticity in network evolution. The capacity of ensembles to signal effectively will ultimately shape how we conceptualize the function, evolution and engineering of signaling networks.

Intracellular signaling networks are central to a cell's ability to adapt to its environment. Developing the capacity to effectively manipulate such networks would have a wide range of applications, from cancer therapy to synthetic biology. This requires a thorough understanding of the mechanisms of signal transduction, particularly the kinds of protein complexes that are formed during transmission of extracellular information to the nucleus. Traditionally, signaling complexes have been largely perceived (albeit often implicitly) as machine-like structures. However, the number of molecular complexes that could theoretically be formed by complex signaling networks is astronomically large. This has led to the pleiomorphic ensemble hypothesis, which posits that diverse and rapidly changing sets of transient protein complexes can transmit and process information. Our goal was to use computational approaches, specifically rule-based modeling, to test these hypotheses. We constructed a model of the prototypical yeast mating pathway and found significant ensemble-like behavior. Our results thus demonstrated that ensembles can in fact transmit extracellular signals with minimal noise. Additionally, a comparison of this model with one tailored to generate machine-like complexes displayed notable phenotypic differences, revealing potential advantages for ensemble-like signaling. Our demonstration that ensembles can function effectively will have a significant impact on how we conceptualize signaling and other processes inside cells.

Identification of parameter correlations for parameter estimation in dynamic biological models

Background

One of the challenging tasks in systems biology is parameter estimation in nonlinear dynamic models. A biological model usually contains a large number of correlated parameters leading to non-identifiability problems. Although many approaches have been developed to address both structural and practical non-identifiability problems, very few studies have been made to systematically investigate parameter correlations.

Results

In this study we present an approach that is able to identify both pairwise parameter correlations and higher order interrelationships among parameters in nonlinear dynamic models. Correlations are interpreted as surfaces in the subspaces of correlated parameters. Based on the correlation information obtained in this way both structural and practical non-identifiability can be clarified. Moreover, it can be concluded from the correlation analysis that a minimum number of data sets with different inputs for experimental design are needed to relieve the parameter correlations, which corresponds to the maximum number of correlated parameters among the correlation groups.

Conclusions

The information of pairwise and higher order interrelationships among parameters in biological models gives a deeper insight into the cause of non-identifiability problems. The result of our correlation analysis provides a necessary condition for experimental design in order to acquire suitable measurement data for unique parameter estimation.

Modeling and validating chronic pharmacological manipulation of circadian rhythms

Circadian rhythms can be entrained by a light-dark (LD) cycle and can also be reset pharmacologically, for example, by the CK1δ/ε inhibitor PF-670462. Here, we determine how these two independent signals affect circadian timekeeping from the molecular to the behavioral level. By developing a systems pharmacology model, we predict and experimentally validate that chronic CK1δ/ε inhibition during the earlier hours of a LD cycle can produce a constant stable delay of rhythm. However, chronic dosing later during the day, or in the presence of longer light intervals, is not predicted to yield an entrained rhythm. We also propose a simple method based on phase response curves (PRCs) that predicts the effects of a LD cycle and chronic dosing of a circadian drug. This work indicates that dosing timing and environmental signals must be carefully considered for accurate pharmacological manipulation of circadian phase.

A mechanism for robust circadian timekeeping via stoichiometric balance

An accurate mathematical model of the mammalian circadian clock provides novel insights into the mechanisms that generate 24-h rhythms. A double-negative feedback loop design is proposed for biological clocks whose period needs to be tightly regulated.

A 1–1 stoichiometric balance and tight binding between activators (PER–CRY) and repressors (BMAL1–CLOCK/NPAS2) is required for sustained rhythmicity.

Stoichiometry is balanced by an additional negative feedback loop consisting of a stable activator.

Our detailed model can explain more experimental data than previous models.

Mathematical analysis of a simple model supports our claims.

Circadian (∼24 h) timekeeping is essential for the lives of many organisms. To understand the biochemical mechanisms of this timekeeping, we have developed a detailed mathematical model of the mammalian circadian clock. Our model can accurately predict diverse experimental data including the phenotypes of mutations or knockdown of clock genes as well as the time courses and relative expression of clock transcripts and proteins. Using this model, we show how a universal motif of circadian timekeeping, where repressors tightly bind activators rather than directly binding to DNA, can generate oscillations when activators and repressors are in stoichiometric balance. Furthermore, we find that an additional slow negative feedback loop preserves this stoichiometric balance and maintains timekeeping with a fixed period. The role of this mechanism in generating robust rhythms is validated by analysis of a simple and general model and a previous model of the Drosophila circadian clock. We propose a double-negative feedback loop design for biological clocks whose period needs to be tightly regulated even with large changes in gene dosage.

Parameter estimation of biological phenomena: an unscented Kalman filter approach

Recent advances in high-throughput technologies for biological data acquisition have spurred a broad interest in the construction of mathematical models for biological phenomena. The development of such mathematical models relies on the estimation of unknown parameters of the system using the time-course profiles of different metabolites in the system. One of the main challenges in the parameter estimation of biological phenomena is the fact that the number of unknown parameters is much more than the number of metabolites in the system. Moreover, the available metabolite measurements are corrupted by noise. In this paper, a new parameter estimation algorithm is developed based on the stochastic estimation framework for nonlinear systems, namely the unscented Kalman filter (UKF). A new iterative UKF algorithm with covariance resetting is developed in which the UKF algorithm is applied iteratively to the available noisy time profiles of the metabolites. The proposed estimation algorithm is applied to noisy time-course data synthetically produced from a generic branched pathway as well as real time-course profile for the Cad system of E. coli. The simulation results demonstrate the effectiveness of the proposed scheme.