PyDREAM: high-dimensional parameter inference for biological models in python

From FAIR to CURE: Guidelines for Computational Models of Biological Systems

Guidelines for managing scientific data have been established under the FAIR principles requiring that data be Findable, Accessible, Interoperable, and Reusable. In many scientific disciplines, especially computational biology, both data and models are key to progress. For this reason, and recognizing that such models are a very special type of "data", we argue that computational models, especially mechanistic models prevalent in medicine, physiology and systems biology, deserve a complementary set of guidelines. We propose the CURE principles, emphasizing that models should be Credible, Understandable, Reproducible, and Extensible. We delve into each principle, discussing verification, validation, and uncertainty quantification for model credibility; the clarity of model descriptions and annotations for understandability; adherence to standards and open science practices for reproducibility; and the use of open standards and modular code for extensibility and reuse. We outline recommended and baseline requirements for each aspect of CURE, aiming to enhance the impact and trustworthiness of computational models, particularly in biomedical applications where credibility is paramount. Our perspective underscores the need for a more disciplined approach to modeling, aligning with emerging trends such as Digital Twins and emphasizing the importance of data and modeling standards for interoperability and reuse. Finally, we emphasize that given the non-trivial effort required to implement the guidelines, the community moves to automate as many of the guidelines as possible.

Parallelization of particle-based reaction-diffusion simulations using MPI

Particle-based reaction-diffusion models offer a high-resolution alternative to the continuum reaction-diffusion approach, capturing the discrete and volume-excluding nature of molecules undergoing stochastic dynamics. These methods are thus uniquely capable of simulating explicit self-assembly of particles into higher-order structures like filaments, spherical cages, or heterogeneous macromolecular complexes, which are ubiquitous across living systems and in materials design. The disadvantage of these high-resolution methods is their increased computational cost. Here we present a parallel implementation of the particle-based NERDSS software using the Message Passing Interface (MPI) and spatial domain decomposition, achieving close to linear scaling for up to 96 processors in the largest simulation systems. The scalability of parallel NERDSS is evaluated for bimolecular reactions in 3D and 2D, for self-assembly of trimeric and hexameric complexes, and for protein lattice assembly from 3D to 2D, with all parallel test cases producing accurate solutions. We demonstrate how parallel efficiency depends on the system size, the reaction network, and the limiting timescales of the system, showing optimal scaling only for smaller assemblies with slower timescales. The formation of very large assemblies represents a challenge in evaluating reaction updates across processors, and here we restrict assembly sizes to below the spatial decomposition size. We provide the parallel NERDSS code open source, with detailed documentation for developers and extension to other particle-based reaction-diffusion software.

Signal execution modes emerge in biochemical reaction networks calibrated to experimental data

Mathematical models of biomolecular networks are commonly used to study cellular processes; however, their usefulness to explain and predict dynamic behaviors is often questioned due to the unclear relationship between parameter uncertainty and network dynamics. In this work, we introduce PyDyNo (Python dynamic analysis of biochemical networks), a non-equilibrium reaction-flux based analysis to identify dominant reaction paths within a biochemical reaction network calibrated to experimental data. We first show, in a simplified apoptosis execution model, that despite the thousands of parameter vectors with equally good fits to experimental data, our framework identifies the dynamic differences between these parameter sets and outputs three dominant execution modes, which exhibit varying sensitivity to perturbations. We then apply our methodology to JAK2/STAT5 network in colony-forming unit-erythroid (CFU-E) cells and provide previously unrecognized mechanistic explanation for the survival responses of CFU-E cell population that would have been impossible to deduce with traditional protein-concentration based analyses.

Mutation of p53 increases the competitive ability of pluripotent stem cells

During development, the rate of tissue growth is determined by the relative balance of cell division and cell death. Cell competition is a fitness quality-control mechanism that contributes to this balance by eliminating viable cells that are less fit than their neighbours. The mutations that confer cells with a competitive advantage and the dynamics of the interactions between winner and loser cells are not well understood. Here, we show that embryonic cells lacking the tumour suppressor p53 are 'super-competitors' that eliminate their wild-type neighbours through the direct induction of apoptosis. This elimination is context dependent, as it does not occur when cells are pluripotent and it is triggered by the onset of differentiation. Furthermore, by combining mathematical modelling and cell-based assays we show that the elimination of wild-type cells is not through competition for space or nutrients, but instead is mediated by short-range interactions that are dependent on the local cell neighbourhood. This highlights the importance of the local cell neighbourhood and the competitive interactions within this neighbourhood for the regulation of proliferation during early embryonic development.

Concepts of multi-level dynamical modelling: understanding mechanisms of squamous cell carcinoma development in Fanconi anemia

Fanconi anemia (FA) is a rare disease (incidence of 1:300,000) primarily based on the inheritance of pathogenic variants in genes of the FA/BRCA (breast cancer) pathway. These variants ultimately reduce the functionality of different proteins involved in the repair of DNA interstrand crosslinks and DNA double-strand breaks. At birth, individuals with FA might present with typical malformations, particularly radial axis and renal malformations, as well as other physical abnormalities like skin pigmentation anomalies. During the first decade of life, FA mostly causes bone marrow failure due to reduced capacity and loss of the hematopoietic stem and progenitor cells. This often makes hematopoietic stem cell transplantation necessary, but this therapy increases the already intrinsic risk of developing squamous cell carcinoma (SCC) in early adult age. Due to the underlying genetic defect in FA, classical chemo-radiation-based treatment protocols cannot be applied. Therefore, detecting and treating the multi-step tumorigenesis process of SCC in an early stage, or even its progenitors, is the best option for prolonging the life of adult FA individuals. However, the small number of FA individuals makes classical evidence-based medicine approaches based on results from randomized clinical trials impossible. As an alternative, we introduce here the concept of multi-level dynamical modelling using large, longitudinally collected genome, proteome- and transcriptome-wide data sets from a small number of FA individuals. This mechanistic modelling approach is based on the "hallmarks of cancer in FA", which we derive from our unique database of the clinical history of over 750 FA individuals. Multi-omic data from healthy and diseased tissue samples of FA individuals are to be used for training constituent models of a multi-level tumorigenesis model, which will then be used to make experimentally testable predictions. In this way, mechanistic models facilitate not only a descriptive but also a functional understanding of SCC in FA. This approach will provide the basis for detecting signatures of SCCs at early stages and their precursors so they can be efficiently treated or even prevented, leading to a better prognosis and quality of life for the FA individual.

The tumor microenvironment shows a hierarchy of cell-cell interactions dominated by fibroblasts

The tumor microenvironment (TME) is comprised of non-malignant cells that interact with each other and with cancer cells, critically impacting cancer biology. The TME is complex, and understanding it requires simplifying approaches. Here we provide an experimental-mathematical approach to decompose the TME into small circuits of interacting cell types. We find, using female breast cancer single-cell-RNA-sequencing data, a hierarchical network of interactions, with cancer-associated fibroblasts (CAFs) at the top secreting factors primarily to tumor-associated macrophages (TAMs). This network is composed of repeating circuit motifs. We isolate the strongest two-cell circuit motif by culturing fibroblasts and macrophages in-vitro, and analyze their dynamics and transcriptomes. This isolated circuit recapitulates the hierarchy of in-vivo interactions, and enables testing the effect of ligand-receptor interactions on cell dynamics and function, as we demonstrate by identifying a mediator of CAF-TAM interactions - RARRES2, and its receptor CMKLR1. Thus, the complexity of the TME may be simplified by identifying small circuits, facilitating the development of strategies to modulate the TME.

A biochemical necroptosis model explains cell-type-specific responses to cell death cues

Necroptosis is a form of regulated cell death associated with degenerative disorders, autoimmune and inflammatory diseases, and cancer. To better understand the biochemical mechanisms regulating necroptosis, we constructed a detailed computational model of tumor necrosis factor-induced necroptosis based on known molecular interactions from the literature. Intracellular protein levels, used as model inputs, were quantified using label-free mass spectrometry, and the model was calibrated using Bayesian parameter inference to experimental protein time course data from a well-established necroptosis-executing cell line. The calibrated model reproduced the dynamics of phosphorylated mixed lineage kinase domain-like protein, an established necroptosis reporter. A subsequent dynamical systems analysis identified four distinct modes of necroptosis signal execution, distinguished by rate constant values and the roles of the RIP1 deubiquitinating enzymes A20 and CYLD. In one case, A20 and CYLD both contribute to RIP1 deubiquitination, in another RIP1 deubiquitination is driven exclusively by CYLD, and in two modes either A20 or CYLD acts as the driver with the other enzyme, counterintuitively, inhibiting necroptosis. We also performed sensitivity analyses of initial protein concentrations and rate constants to identify potential targets for modulating necroptosis sensitivity within each mode. We conclude by associating numerous contrasting and, in some cases, counterintuitive experimental results reported in the literature with one or more of the model-predicted modes of necroptosis execution. In all, we demonstrate that a consensus pathway model of tumor necrosis factor-induced necroptosis can provide insights into unresolved controversies regarding the molecular mechanisms driving necroptosis execution in numerous cell types under different experimental conditions.

SBbadger: biochemical reaction networks with definable degree distributions

MOTIVATION

An essential step in developing computational tools for the inference, optimization and simulation of biochemical reaction networks is gauging tool performance against earlier efforts using an appropriate set of benchmarks. General strategies for the assembly of benchmark models include collection from the literature, creation via subnetwork extraction and de novo generation. However, with respect to biochemical reaction networks, these approaches and their associated tools are either poorly suited to generate models that reflect the wide range of properties found in natural biochemical networks or to do so in numbers that enable rigorous statistical analysis.

RESULTS

In this work, we present SBbadger, a python-based software tool for the generation of synthetic biochemical reaction or metabolic networks with user-defined degree distributions, multiple available kinetic formalisms and a host of other definable properties. SBbadger thus enables the creation of benchmark model sets that reflect properties of biological systems and generate the kinetics and model structures typically targeted by computational analysis and inference software. Here, we detail the computational and algorithmic workflow of SBbadger, demonstrate its performance under various settings, provide sample outputs and compare it to currently available biochemical reaction network generation software.

AVAILABILITY AND IMPLEMENTATION

SBbadger is implemented in Python and is freely available at https://github.com/sys-bio/SBbadger and via PyPI at https://pypi.org/project/SBbadger/. Documentation can be found at https://SBbadger.readthedocs.io.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Bayesian parameter estimation for dynamical models in systems biology

Dynamical systems modeling, particularly via systems of ordinary differential equations, has been used to effectively capture the temporal behavior of different biochemical components in signal transduction networks. Despite the recent advances in experimental measurements, including sensor development and '-omics' studies that have helped populate protein-protein interaction networks in great detail, modeling in systems biology lacks systematic methods to estimate kinetic parameters and quantify associated uncertainties. This is because of multiple reasons, including sparse and noisy experimental measurements, lack of detailed molecular mechanisms underlying the reactions, and missing biochemical interactions. Additionally, the inherent nonlinearities with respect to the states and parameters associated with the system of differential equations further compound the challenges of parameter estimation. In this study, we propose a comprehensive framework for Bayesian parameter estimation and complete quantification of the effects of uncertainties in the data and models. We apply these methods to a series of signaling models of increasing mathematical complexity. Systematic analysis of these dynamical systems showed that parameter estimation depends on data sparsity, noise level, and model structure, including the existence of multiple steady states. These results highlight how focused uncertainty quantification can enrich systems biology modeling and enable additional quantitative analyses for parameter estimation.

'Neighbourhood watch' model: embryonic epiblast cells assess positional information in relation to their neighbours

In many developing and regenerating systems, tissue pattern is established through gradients of informative morphogens, but we know little about how cells interpret these. Using experimental manipulation of early chick embryos, including misexpression of an inducer (VG1 or ACTIVIN) and an inhibitor (BMP4), we test two alternative models for their ability to explain how the site of primitive streak formation is positioned relative to the rest of the embryo. In one model, cells read morphogen concentrations cell-autonomously. In the other, cells sense changes in morphogen status relative to their neighbourhood. We find that only the latter model can account for the experimental results, including some counter-intuitive predictions. This mechanism (which we name the ‘neighbourhood watch’ model) illuminates the classic ‘French Flag Problem’ and how positional information is interpreted by a sheet of cells in a large developing system.

Summary: In a large developing system, the chick embryo before gastrulation, cells may interpret gradients of positional signals relative to their neighbours to position the primitive streak, establishing bilateral symmetry.

Implementation of a practical Markov chain Monte Carlo sampling algorithm in PyBioNetFit

SUMMARY

Bayesian inference in biological modeling commonly relies on Markov chain Monte Carlo (MCMC) sampling of a multidimensional and non-Gaussian posterior distribution that is not analytically tractable. Here, we present the implementation of a practical MCMC method in the open-source software package PyBioNetFit (PyBNF), which is designed to support parameterization of mathematical models for biological systems. The new MCMC method, am, incorporates an adaptive move proposal distribution. For warm starts, sampling can be initiated at a specified location in parameter space and with a multivariate Gaussian proposal distribution defined initially by a specified covariance matrix. Multiple chains can be generated in parallel using a computer cluster. We demonstrate that am can be used to successfully solve real-world Bayesian inference problems, including forecasting of new Coronavirus Disease 2019 case detection with Bayesian quantification of forecast uncertainty.

AVAILABILITY AND IMPLEMENTATION

PyBNF version 1.1.9, the first stable release with am, is available at PyPI and can be installed using the pip package-management system on platforms that have a working installation of Python 3. PyBNF relies on libRoadRunner and BioNetGen for simulations (e.g. numerical integration of ordinary differential equations defined in SBML or BNGL files) and Dask.Distributed for task scheduling on Linux computer clusters. The Python source code can be freely downloaded/cloned from GitHub and used and modified under terms of the BSD-3 license (https://github.com/lanl/pybnf). Online documentation covering installation/usage is available (https://pybnf.readthedocs.io/en/latest/). A tutorial video is available on YouTube (https://www.youtube.com/watch?v=2aRqpqFOiS4&t=63s).

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Determinants of Ligand Specificity and Functional Plasticity in Type I Interferon Signaling

The Type I Interferon family of cytokines all act through the same cell surface receptor and induce phosphorylation of the same subset of response regulators of the STAT family. Despite their shared receptor, different Type I Interferons have different functions during immune response to infection. In particular, they differ in the potency of their induced anti-viral and anti-proliferative responses in target cells. It remains not fully understood how these functional differences can arise in a ligand-specific manner both at the level of STAT phosphorylation and the downstream function. We use a minimal computational model of Type I Interferon signaling, focusing on Interferon-α and Interferon-β. We validate the model with quantitative experimental data to identify the key determinants of specificity and functional plasticity in Type I Interferon signaling. We investigate different mechanisms of signal discrimination, and how multiple system components such as binding affinity, receptor expression levels and their variability, receptor internalization, short-term negative feedback by SOCS1 protein, and differential receptor expression play together to ensure ligand specificity on the level of STAT phosphorylation. Based on these results, we propose phenomenological functional mappings from STAT activation to downstream anti-viral and anti-proliferative activity to investigate differential signal processing steps downstream of STAT phosphorylation. We find that the negative feedback by the protein USP18, which enhances differences in signaling between Interferons via ligand-dependent refractoriness, can give rise to functional plasticity in Interferon-α and Interferon-β signaling, and explore other factors that control functional plasticity. Beyond Type I Interferon signaling, our results have a broad applicability to questions of signaling specificity and functional plasticity in signaling systems with multiple ligands acting through a bottleneck of a small number of shared receptors.

Programmatic modeling for biological systems

Computational modeling has become an established technique to encode mathematical representations of cellular processes and gain mechanistic insights that drive testable predictions. These models are often constructed using graphical user interfaces or domain-specific languages, with community standards used for interchange. Models undergo steady state or dynamic analysis, which can include simulation and calibration within a single application, or transfer across various tools. Here, we describe a novel programmatic modeling paradigm, whereby modeling is augmented with software engineering best practices. We focus on Python - a popular programming language with a large scientific package ecosystem. Models can be encoded as programs, adding benefits such as modularity, testing, and automated documentation generators, while still being extensible and exportable to standardized formats for use with external tools if desired. Programmatic modeling is a key technology to enable collaborative model development and enhance dissemination, transparency, and reproducibility.

Revealing dynamic regulations and the related key proteins of myeloma-initiating cells by integrating experimental data into a systems biological model

MOTIVATION

The growth and survival of myeloma cells are greatly affected by their surrounding microenvironment. To understand the molecular mechanism and the impact of stiffness on the fate of myeloma-initiating cells (MICs), we develop a systems biological model to reveal the dynamic regulations by integrating reverse-phase protein array data and the stiffness-associated pathway.

RESULTS

We not only develop a stiffness-associated signaling pathway to describe the dynamic regulations of the MICs, but also clearly identify three critical proteins governing the MIC proliferation and death, including FAK, mTORC1 and NFκB, which are validated to be related with multiple myeloma by our immunohistochemistry experiment, computation and manually reviewed evidences. Moreover, we demonstrate that the systematic model performs better than widely used parameter estimation algorithms for the complicated signaling pathway.

AVAILABILITY AND IMPLEMENTATION

We can not only use the systems biological model to infer the stiffness-associated genetic signaling pathway and locate the critical proteins, but also investigate the important pathways, proteins or genes for other type of the cancer. Thus, it holds universal scientific significance.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Predictive modeling of parafoveal information processing during reading

Skilled reading requires information processing of the fixated and the not-yet-fixated words to generate precise control of gaze. Over the last 30 years, experimental research provided evidence that word processing is distributed across the perceptual span, which permits recognition of the fixated (foveal) word as well as preview of parafoveal words to the right of fixation. However, theoretical models have been unable to differentiate the specific influences of foveal and parafoveal information on saccade control. Here we show how parafoveal word difficulty modulates spatial and temporal control of gaze in a computational model to reproduce experimental results. In a fully Bayesian framework, we estimated model parameters for different models of parafoveal processing and carried out large-scale predictive simulations and model comparisons for a gaze-contingent reading experiment. We conclude that mathematical modeling of data from gaze-contingent experiments permits the precise identification of pathways from parafoveal information processing to gaze control, uncovering potential mechanisms underlying the parafoveal contribution to eye-movement control.

Unsupervised logic-based mechanism inference for network-driven biological processes

Modern analytical techniques enable researchers to collect data about cellular states, before and after perturbations. These states can be characterized using analytical techniques, but the inference of regulatory interactions that explain and predict changes in these states remains a challenge. Here we present a generalizable, unsupervised approach to generate parameter-free, logic-based models of cellular processes, described by multiple discrete states. Our algorithm employs a Hamming-distance based approach to formulate, test, and identify optimized logic rules that link two states. Our approach comprises two steps. First, a model with no prior knowledge except for the mapping between initial and attractor states is built. We then employ biological constraints to improve model fidelity. Our algorithm automatically recovers the relevant dynamics for the explored models and recapitulates key aspects of the biochemical species concentration dynamics in the original model. We present the advantages and limitations of our work and discuss how our approach could be used to infer logic-based mechanisms of signaling, gene-regulatory, or other input-output processes describable by the Boolean formalism.

Epidemiological modeling in StochSS Live!

Summary

We present StochSS Live!, a web-based service for modeling, simulation and analysis of a wide range of mathematical, biological and biochemical systems. Using an epidemiological model of COVID-19, we demonstrate the power of StochSS Live! to enable researchers to quickly develop a deterministic or a discrete stochastic model, infer its parameters and analyze the results.

Availability and implementation

StochSS Live! is freely available at https://live.stochss.org/

Supplementary information

Supplementary data are available at Bioinformatics online.

Quantifying the roles of space and stochasticity in computer simulations for cell biology and cellular biochemistry

Most of the fascinating phenomena studied in cell biology emerge from interactions among highly organized multimolecular structures embedded into complex and frequently dynamic cellular morphologies. For the exploration of such systems, computer simulation has proved to be an invaluable tool, and many researchers in this field have developed sophisticated computational models for application to specific cell biological questions. However, it is often difficult to reconcile conflicting computational results that use different approaches to describe the same phenomenon. To address this issue systematically, we have defined a series of computational test cases ranging from very simple to moderately complex, varying key features of dimensionality, reaction type, reaction speed, crowding, and cell size. We then quantified how explicit spatial and/or stochastic implementations alter outcomes, even when all methods use the same reaction network, rates, and concentrations. For simple cases, we generally find minor differences in solutions of the same problem. However, we observe increasing discordance as the effects of localization, dimensionality reduction, and irreversible enzymatic reactions are combined. We discuss the strengths and limitations of commonly used computational approaches for exploring cell biological questions and provide a framework for decision making by researchers developing new models. As computational power and speed continue to increase at a remarkable rate, the dream of a fully comprehensive computational model of a living cell may be drawing closer to reality, but our analysis demonstrates that it will be crucial to evaluate the accuracy of such models critically and systematically.

MASSpy: Building, simulating, and visualizing dynamic biological models in Python using mass action kinetics

Mathematical models of metabolic networks utilize simulation to study system-level mechanisms and functions. Various approaches have been used to model the steady state behavior of metabolic networks using genome-scale reconstructions, but formulating dynamic models from such reconstructions continues to be a key challenge. Here, we present the Mass Action Stoichiometric Simulation Python (MASSpy) package, an open-source computational framework for dynamic modeling of metabolism. MASSpy utilizes mass action kinetics and detailed chemical mechanisms to build dynamic models of complex biological processes. MASSpy adds dynamic modeling tools to the COnstraint-Based Reconstruction and Analysis Python (COBRApy) package to provide an unified framework for constraint-based and kinetic modeling of metabolic networks. MASSpy supports high-performance dynamic simulation through its implementation of libRoadRunner: the Systems Biology Markup Language (SBML) simulation engine. Three examples are provided to demonstrate how to use MASSpy: (1) a validation of the MASSpy modeling tool through dynamic simulation of detailed mechanisms of enzyme regulation; (2) a feature demonstration using a workflow for generating ensemble of kinetic models using Monte Carlo sampling to approximate missing numerical values of parameters and to quantify biological uncertainty, and (3) a case study in which MASSpy is utilized to overcome issues that arise when integrating experimental data with the computation of functional states of detailed biological mechanisms. MASSpy represents a powerful tool to address challenges that arise in dynamic modeling of metabolic networks, both at small and large scales.

Species-specific pace of development is associated with differences in protein stability

Although many molecular mechanisms controlling developmental processes are evolutionarily conserved, the speed at which the embryo develops can vary substantially between species. For example, the same genetic program, comprising sequential changes in transcriptional states, governs the differentiation of motor neurons in mouse and human, but the tempo at which it operates differs between species. Using in vitro directed differentiation of embryonic stem cells to motor neurons, we show that the program runs more than twice as fast in mouse as in human. This is not due to differences in signaling, nor the genomic sequence of genes or their regulatory elements. Instead, there is an approximately two-fold increase in protein stability and cell cycle duration in human cells compared with mouse cells. This can account for the slower pace of human development and suggests that differences in protein turnover play a role in interspecies differences in developmental tempo.

Best Practices for Making Reproducible Biochemical Models

Like many scientific disciplines, dynamical biochemical modeling is hindered by irreproducible results. This limits the utility of biochemical models by making them difficult to understand, trust, or reuse. We comprehensively list the best practices that biochemical modelers should follow to build reproducible biochemical model artifacts-all data, model descriptions, and custom software used by the model-that can be understood and reused. The best practices provide advice for all steps of a typical biochemical modeling workflow in which a modeler collects data; constructs, trains, simulates, and validates the model; uses the predictions of a model to advance knowledge; and publicly shares the model artifacts. The best practices emphasize the benefits obtained by using standard tools and formats and provides guidance to modelers who do not or cannot use standards in some stages of their modeling workflow. Adoption of these best practices will enhance the ability of researchers to reproduce, understand, and reuse biochemical models.

A Probabilistic Approach to Explore Signal Execution Mechanisms With Limited Experimental Data

Mathematical models of biochemical reaction networks are central to the study of dynamic cellular processes and hypothesis generation that informs experimentation and validation. Unfortunately, model parameters are often not available and sparse experimental data leads to challenges in model calibration and parameter estimation. This can in turn lead to unreliable mechanistic interpretations of experimental data and the generation of poorly conceived hypotheses for experimental validation. To address this challenge, we evaluate whether a Bayesian-inspired probability-based approach, that relies on expected values for quantities of interest calculated from available information regarding the reaction network topology and parameters can be used to qualitatively explore hypothetical biochemical network execution mechanisms in the context of limited available data. We test our approach on a model of extrinsic apoptosis execution to identify preferred signal execution modes across varying conditions. Apoptosis signal processing can take place either through a mitochondria independent (Type I) mode or a mitochondria dependent (Type II) mode. We first show that in silico knockouts, represented by model subnetworks, successfully identify the most likely execution mode for specific concentrations of key molecular regulators. We then show that changes in molecular regulator concentrations alter the overall reaction flux through the network by shifting the primary route of signal flow between the direct caspase and mitochondrial pathways. Our work thus demonstrates that probabilistic approaches can be used to explore the qualitative dynamic behavior of model biochemical systems even with missing or sparse data.

Controlling spatiotemporal pattern formation in a concentration gradient with a synthetic toggle switch

The formation of spatiotemporal patterns of gene expression is frequently guided by gradients of diffusible signaling molecules. The toggle switch subnetwork, composed of two cross-repressing transcription factors, is a common component of gene regulatory networks in charge of patterning, converting the continuous information provided by the gradient into discrete abutting stripes of gene expression. We present a synthetic biology framework to understand and characterize the spatiotemporal patterning properties of the toggle switch. To this end, we built a synthetic toggle switch controllable by diffusible molecules in Escherichia coli. We analyzed the patterning capabilities of the circuit by combining quantitative measurements with a mathematical reconstruction of the underlying dynamical system. The toggle switch can produce robust patterns with sharp boundaries, governed by bistability and hysteresis. We further demonstrate how the hysteresis, position, timing, and precision of the boundary can be controlled, highlighting the dynamical flexibility of the circuit.

Markov Chain Monte Carlo Methods for Fluid Flow Forecasting in the Subsurface

Accurate predictions in subsurface flows require the forecasting of quantities of interest by applying models of subsurface fluid flow with very little available data. In general a Bayesian statistical approach along with a Markov Chain Monte Carlo (MCMC) algorithm can be used for quantifying the uncertainties associated with subsurface parameters. However, the complex nature of flow simulators presents considerable challenges to accessing inherent uncertainty in all flow simulator parameters of interest. In this work we focus on the transport of contaminants in a heterogeneous permeability field of a aquifer. In our problem the limited data comes in the form of contaminant fractional flow curves at monitoring wells of the aquifer. We then employ a Karhunen-Loève expansion to truncate the stochastic dimension of the permeability field and thus the expansion helps reducing the computational burden. Aiming to reduce the computational burden further, we code our numerical simulator using parallel programming procedures on Graphics Processing Units (GPUs). In this paper we mainly present a comparative study of two well-known MCMC methods, namely, two-stage and DiffeRential Evolution Adaptive Metropolis (DREAM), for the characterization of the two-dimensional aquifer. A thorough statistical analysis of ensembles of the contaminant fractional flow curves from both MCMC methods is presented. The analysis indicates that although the average fractional flow curves are quite similar, both time-dependent ensemble variances and posterior analysis are considerably distinct for both methods.

Modeling heterogeneous tumor growth dynamics and cell-cell interactions at single-cell and cell-population resolution

Cancer is a complex, dynamic disease that despite recent advances remains mostly incurable. Inter- and intratumoral heterogeneity are generally considered major drivers of therapy resistance, metastasis, and treatment failure. Recent advances in high-throughput experimentation have produced a wealth of data on tumor heterogeneity and researchers are increasingly turning to mathematical modeling to aid in the interpretation of these complex datasets. In this mini-review, we discuss three important classes of approaches for modeling cellular dynamics within heterogeneous tumors: agent-based models, population dynamics, and multiscale models. An important new focus, for which we provide an example, is the role of intratumoral cell-cell interactions.

PyBioNetFit and the Biological Property Specification Language

In systems biology modeling, important steps include model parameterization, uncertainty quantification, and evaluation of agreement with experimental observations. To help modelers perform these steps, we developed the software PyBioNetFit, which in addition supports checking models against known system properties and solving design problems. PyBioNetFit introduces Biological Property Specification Language (BPSL) for the formal declaration of system properties. BPSL allows qualitative data to be used alone or in combination with quantitative data. PyBioNetFit performs parameterization with parallelized metaheuristic optimization algorithms that work directly with existing model definition standards: BioNetGen Language (BNGL) and Systems Biology Markup Language (SBML). We demonstrate PyBioNetFit's capabilities by solving various example problems, including the challenging problem of parameterizing a 153-parameter model of cell cycle control in yeast based on both quantitative and qualitative data. We demonstrate the model checking and design applications of PyBioNetFit and BPSL by analyzing a model of targeted drug interventions in autophagy signaling.

Signal integration and information transfer in an allosterically regulated network

A biological reaction network may serve multiple purposes, processing more than one input and impacting downstream processes via more than one output. These networks operate in a dynamic cellular environment in which the levels of network components may change within cells and across cells. Recent evidence suggests that protein concentration variability could explain cell fate decisions. However, systems with multiple inputs, multiple outputs, and changing input concentrations have not been studied in detail due to their complexity. Here, we take a systems biochemistry approach, combining physiochemical modeling and information theory, to investigate how cyclooxygenase-2 (COX-2) processes simultaneous input signals within a complex interaction network. We find that changes in input levels affect the amount of information transmitted by the network, as does the correlation between those inputs. This, and the allosteric regulation of COX-2 by its substrates, allows it to act as a signal integrator that is most sensitive to changes in relative input levels.

Evaluation of a Straight-Ray Forward Model for Bayesian Inversion of Crosshole Ground Penetrating Radar Data

Bayesian inversion of crosshole ground penetrating radar (GPR) data is capable of characterizing the subsurface dielectric properties and qualifying the associated uncertainties. Markov chain Monte Carlo (MCMC) simulations within the Bayesian inversion usually require thousands to millions of forward model evaluations for the parameters to hit their posterior distributions. Therefore, the CPU cost of the forward model is a key issue that influences the efficiency of the Bayesian inversion method. In this paper we implement a widely used straight-ray forward model within our Bayesian inversion framework. Based on a synthetic unit square relative permittivity model, we simulate the crosshole GPR first-arrival traveltime data using the finite-difference time-domain (FDTD) and straight-ray solver, respectively, and find that the straight-ray simulator runs 450 times faster than its FDTD counterpart, yet suffers from a modeling error that is more than 7 times larger. We also perform a series of numerical experiments to evaluate the performance of the straight-ray model within the Bayesian inversion framework. With modeling error disregarded, the inverted posterior models fit the measurement data nicely, yet converge to the wrong set of parameters at the expense of unreasonably large number of iterations. When the modeling error is accounted for, with a quarter of the computational burden, the main features of the true model can be identified from the posterior realizations although there still exist some unwanted artifacts. Finally, a smooth constraint on the model structure improves the inversion results considerably, to the extent that it enhances the inversion accuracy approximating to those of the FDTD model, and further reduces the CPU demand. Our results demonstrate that the use of the straight-ray forward model in the Bayesian inversion saves computational cost tremendously, and the modeling error correction together with the model structure constraint are the necessary amendments that ensure that the model parameters converge correctly.

Plant Synthetic Biology: Quantifying the "Known Unknowns" and Discovering the "Unknown Unknowns"

Biosensors, advanced microscopy, and single- cell transcriptomics are advancing plant synthetic biology.

Arrestin-3 scaffolding of the JNK3 cascade suggests a mechanism for signal amplification

Scaffold proteins tether and orient components of a signaling cascade to facilitate signaling. Although much is known about how scaffolds colocalize signaling proteins, it is unclear whether scaffolds promote signal amplification. Here, we used arrestin-3, a scaffold of the ASK1-MKK4/7-JNK3 cascade, as a model to understand signal amplification by a scaffold protein. We found that arrestin-3 exhibited >15-fold higher affinity for inactive JNK3 than for active JNK3, and this change involved a shift in the binding site following JNK3 activation. We used systems biochemistry modeling and Bayesian inference to evaluate how the activation of upstream kinases contributed to JNK3 phosphorylation. Our combined experimental and computational approach suggested that the catalytic phosphorylation rate of JNK3 at Thr-221 by MKK7 is two orders of magnitude faster than the corresponding phosphorylation of Tyr-223 by MKK4 with or without arrestin-3. Finally, we showed that the release of activated JNK3 was critical for signal amplification. Collectively, our data suggest a "conveyor belt" mechanism for signal amplification by scaffold proteins. This mechanism informs on a long-standing mystery for how few upstream kinase molecules activate numerous downstream kinases to amplify signaling.

Intrinsic limits of information transmission in biochemical signalling motifs

All living things have evolved to sense changes in their environment in order to respond in adaptive ways. At the cellular level, these sensing systems generally involve receptor molecules at the cell surface, which detect changes outside the cell and relay those changes to the appropriate response elements downstream. With the advent of experimental technologies that can track signalling at the single-cell level, it has become clear that many signalling systems exhibit significant levels of 'noise,' manifesting as differential responses of otherwise identical cells to the same environment. This noise has a large impact on the capacity of cell signalling networks to transmit information from the environment. Application of information theory to experimental data has found that all systems studied to date encode less than 2.5 bits of information, with the majority transmitting significantly less than 1 bit. Given the growing interest in applying information theory to biological data, it is crucial to understand whether the low values observed to date represent some sort of intrinsic limit on information flow given the inherently stochastic nature of biochemical signalling events. In this work, we used a series of computational models to explore how much information a variety of common 'signalling motifs' can encode. We found that the majority of these motifs, which serve as the basic building blocks of cell signalling networks, can encode far more information (4-6 bits) than has ever been observed experimentally. In addition to providing a consistent framework for estimating information-theoretic quantities from experimental data, our findings suggest that the low levels of information flow observed so far in living system are not necessarily due to intrinsic limitations. Further experimental work will be needed to understand whether certain cell signalling systems actually can approach the intrinsic limits described here, and to understand the sources and purpose of the variation that reduces information flow in living cells.

Evaluation of Parallel Tempering to Accelerate Bayesian Parameter Estimation in Systems Biology

Models of biological systems often have many unknown parameters that must be determined in order for model behavior to match experimental observations. Commonly-used methods for parameter estimation that return point estimates of the best-fit parameters are insufficient when models are high dimensional and under-constrained. As a result, Bayesian methods, which treat model parameters as random variables and attempt to estimate their probability distributions given data, have become popular in systems biology. Bayesian parameter estimation often relies on Markov Chain Monte Carlo (MCMC) methods to sample model parameter distributions, but the slow convergence of MCMC sampling can be a major bottleneck. One approach to improving performance is parallel tempering (PT), a physics-based method that uses swapping between multiple Markov chains run in parallel at different temperatures to accelerate sampling. The temperature of a Markov chain determines the probability of accepting an unfavorable move, so swapping with higher temperatures chains enables the sampling chain to escape from local minima. In this work we compared the MCMC performance of PT and the commonly-used Metropolis-Hastings (MH) algorithm on six biological models of varying complexity. We found that for simpler models PT accelerated convergence and sampling, and that for more complex models, PT often converged in cases MH became trapped in non-optimal local minima. We also developed a freely-available MATLAB package for Bayesian parameter estimation called PTEMPEST (http://github.com/RuleWorld/ptempest), which is closely integrated with the popular BioNetGen software for rule-based modeling of biological systems.