1
|
Ahmadi M, Thomas PJ, Buecherl L, Winstead C, Myers CJ, Zheng H. A Comparison of Weighted Stochastic Simulation Methods for the Analysis of Genetic Circuits. ACS Synth Biol 2023; 12:287-304. [PMID: 36583529 DOI: 10.1021/acssynbio.2c00553] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
Abstract
Rare events are of particular interest in synthetic biology because rare biochemical events may be catastrophic to a biological system by, for example, triggering irreversible events such as off-target drug delivery. To estimate the probability of rare events efficiently, several weighted stochastic simulation methods have been developed. Under optimal parameters and model conditions, these methods can greatly improve simulation efficiency in comparison to traditional stochastic simulation. Unfortunately, the optimal parameters and conditions cannot be deduced a priori. This paper presents a critical survey of weighted stochastic simulation methods. It shows that the methods considered here cannot consistently, efficiently, and exactly accomplish the task of rare event simulation without resorting to a computationally expensive calibration procedure, which undermines their overall efficiency. The results suggest that further development is needed before these methods can be deployed for general use in biological simulations.
Collapse
Affiliation(s)
- Mohammad Ahmadi
- Department of Computer Science and Engineering, University of South Florida, Tampa, Florida33620-9951, United States
| | - Payton J Thomas
- Department of Biomedical Engineering, University of Utah, Salt Lake City, Utah84112, United States
| | - Lukas Buecherl
- Department of Electrical, Computer, and Energy Engineering, University of Colorado Boulder, Boulder, Colorado80309-0401, United States
| | - Chris Winstead
- Department of Electrical and Computer Engineering, Utah State University, Logan, Utah84322-1400, United States
| | - Chris J Myers
- Department of Electrical, Computer, and Energy Engineering, University of Colorado Boulder, Boulder, Colorado80309-0401, United States
| | - Hao Zheng
- Department of Computer Science and Engineering, University of South Florida, Tampa, Florida33620-9951, United States
| |
Collapse
|
2
|
Sherlock C, Golightly A. Exact Bayesian inference for discretely observed Markov Jump Processes using finite rate matrices. J Comput Graph Stat 2022. [DOI: 10.1080/10618600.2022.2093886] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/17/2022]
Affiliation(s)
- Chris Sherlock
- Department of Mathematics and Statistics, Lancaster University, UK
| | | |
Collapse
|
3
|
Variational inference for Markovian queueing networks. ADV APPL PROBAB 2021. [DOI: 10.1017/apr.2020.72] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
AbstractQueueing networks are stochastic systems formed by interconnected resources routing and serving jobs. They induce jump processes with distinctive properties, and find widespread use in inferential tasks. Here, service rates for jobs and potential bottlenecks in the routing mechanism must be estimated from a reduced set of observations. However, this calls for the derivation of complex conditional density representations, over both the stochastic network trajectories and the rates, which is considered an intractable problem. Numerical simulation procedures designed for this purpose do not scale, because of high computational costs; furthermore, variational approaches relying on approximating measures and full independence assumptions are unsuitable. In this paper, we offer a probabilistic interpretation of variational methods applied to inference tasks with queueing networks, and show that approximating measure choices routinely used with jump processes yield ill-defined optimization problems. Yet we demonstrate that it is still possible to enable a variational inferential task, by considering a novel space expansion treatment over an analogous counting process for job transitions. We present and compare exemplary use cases with practical queueing networks, showing that our framework offers an efficient and improved alternative where existing variational or numerically intensive solutions fail.
Collapse
|
4
|
Sherlock C. Direct statistical inference for finite Markov jump processes via the matrix exponential. Comput Stat 2021; 36:2863-2887. [PMID: 33897113 PMCID: PMC8054858 DOI: 10.1007/s00180-021-01102-6] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2020] [Accepted: 03/23/2021] [Indexed: 11/27/2022]
Abstract
Given noisy, partial observations of a time-homogeneous, finite-statespace Markov chain, conceptually simple, direct statistical inference is available, in theory, via its rate matrix, or infinitesimal generator, \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$${\mathsf {Q}}$$\end{document}Q, since \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$\exp ({\mathsf {Q}}t)$$\end{document}exp(Qt) is the transition matrix over time t. However, perhaps because of inadequate tools for matrix exponentiation in programming languages commonly used amongst statisticians or a belief that the necessary calculations are prohibitively expensive, statistical inference for continuous-time Markov chains with a large but finite state space is typically conducted via particle MCMC or other relatively complex inference schemes. When, as in many applications \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$${\mathsf {Q}}$$\end{document}Q arises from a reaction network, it is usually sparse. We describe variations on known algorithms which allow fast, robust and accurate evaluation of the product of a non-negative vector with the exponential of a large, sparse rate matrix. Our implementation uses relatively recently developed, efficient, linear algebra tools that take advantage of such sparsity. We demonstrate the straightforward statistical application of the key algorithm on a model for the mixing of two alleles in a population and on the Susceptible-Infectious-Removed epidemic model.
Collapse
Affiliation(s)
- Chris Sherlock
- Department of Mathematics and Statistics, Lancaster University, Lancaster, UK
| |
Collapse
|
5
|
Nguyen-Van-Yen B, Del Moral P, Cazelles B. Stochastic Epidemic Models inference and diagnosis with Poisson Random Measure Data Augmentation. Math Biosci 2021; 335:108583. [PMID: 33713696 DOI: 10.1016/j.mbs.2021.108583] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2020] [Revised: 12/22/2020] [Accepted: 02/28/2021] [Indexed: 11/24/2022]
Abstract
We present a new Bayesian inference method for compartmental models that takes into account the intrinsic stochasticity of the process. We show how to formulate a SIR-type Markov jump process as the solution of a stochastic differential equation with respect to a Poisson Random Measure (PRM), and how to simulate the process trajectory deterministically from a parameter value and a PRM realization. This forms the basis of our Data Augmented MCMC, which consists of augmenting parameter space with the unobserved PRM value. The resulting simple Metropolis-Hastings sampler acts as an efficient simulation-based inference method, that can easily be transferred from model to model. Compared with a recent Data Augmentation method based on Gibbs sampling of individual infection histories, PRM-augmented MCMC scales much better with epidemic size and is far more flexible. It is also found to be competitive with Particle MCMC for moderate epidemics when using approximate simulations. PRM-augmented MCMC also yields a posteriori estimates of the PRM, that represent process stochasticity, and which can be used to validate the model. A pattern of deviation from the PRM prior distribution will indicate that the model underfits the data and help to understand the cause. We illustrate this by fitting a non-seasonal model to some simulated seasonal case count data. Applied to the Zika epidemic of 2013 in French Polynesia, our approach shows that a simple SEIR model cannot correctly reproduce both the initial sharp increase in the number of cases as well as the final proportion of seropositive. PRM augmentation thus provides a coherent story for Stochastic Epidemic Model inference, where explicitly inferring process stochasticity helps with model validation.
Collapse
Affiliation(s)
- Benjamin Nguyen-Van-Yen
- Institut Pasteur, Unité de Génétique Fonctionnelle des Maladies Infectieuses, UMR 2000 CNRS, Paris, France; Institut de Biologie de l'ENS (IBENS), Ecole Normale Supérieure, CNRS, INSERM, Université PSL, 75005 Paris, France.
| | | | - Bernard Cazelles
- Institut de Biologie de l'ENS (IBENS), Ecole Normale Supérieure, CNRS, INSERM, Université PSL, 75005 Paris, France; International Center for Mathematical and Computational Modeling of Complex Systems (UMMISCO), UMI 209, Sorbonne Université, France; iGLOBE, UMI CNRS 3157, University of Arizona, Tucson, AZ, United States of America
| |
Collapse
|
6
|
Mikelson J, Khammash M. Likelihood-free nested sampling for parameter inference of biochemical reaction networks. PLoS Comput Biol 2020; 16:e1008264. [PMID: 33035218 PMCID: PMC7577508 DOI: 10.1371/journal.pcbi.1008264] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2019] [Revised: 10/21/2020] [Accepted: 08/16/2020] [Indexed: 12/03/2022] Open
Abstract
The development of mechanistic models of biological systems is a central part of Systems Biology. One major challenge in developing these models is the accurate inference of model parameters. In recent years, nested sampling methods have gained increased attention in the Systems Biology community due to the fact that they are parallelizable and provide error estimates with no additional computations. One drawback that severely limits the usability of these methods, however, is that they require the likelihood function to be available, and thus cannot be applied to systems with intractable likelihoods, such as stochastic models. Here we present a likelihood-free nested sampling method for parameter inference which overcomes these drawbacks. This method gives an unbiased estimator of the Bayesian evidence as well as samples from the posterior. We derive a lower bound on the estimators variance which we use to formulate a novel termination criterion for nested sampling. The presented method enables not only the reliable inference of the posterior of parameters for stochastic systems of a size and complexity that is challenging for traditional methods, but it also provides an estimate of the obtained variance. We illustrate our approach by applying it to several realistically sized models with simulated data as well as recently published biological data. We also compare our developed method with the two most popular other likeliood-free approaches: pMCMC and ABC-SMC. The C++ code of the proposed methods, together with test data, is available at the github web page https://github.com/Mijan/LFNS_paper. The behaviour of mathematical models of biochemical reactions is governed by model parameters encoding for various reaction rates, molecule concentrations and other biochemical quantities. As the general purpose of these models is to reproduce and predict the true biological response to different stimuli, the inference of these parameters, given experimental observations, is a crucial part of Systems Biology. While plenty of methods have been published for the inference of model parameters, most of them require the availability of the likelihood function and thus cannot be applied to models that do not allow for the computation of the likelihood. Further, most established methods do not provide an estimate of the variance of the obtained estimator. In this paper, we present a novel inference method that accurately approximates the posterior distribution of parameters and does not require the evaluation of the likelihood function. Our method is based on the nested sampling algorithm and approximates the likelihood with a particle filter. We show that the resulting posterior estimates are unbiased and provide a way to estimate not just the posterior distribution, but also an error estimate of the final estimator. We illustrate our method on several stochastic models with simulated data as well as one model of transcription with real biological data.
Collapse
|
7
|
Roesch E, Stumpf MPH. Parameter inference in dynamical systems with co-dimension 1 bifurcations. ROYAL SOCIETY OPEN SCIENCE 2019; 6:190747. [PMID: 31824698 PMCID: PMC6837231 DOI: 10.1098/rsos.190747] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/07/2019] [Accepted: 09/20/2019] [Indexed: 05/03/2023]
Abstract
Dynamical systems with intricate behaviour are all-pervasive in biology. Many of the most interesting biological processes indicate the presence of bifurcations, i.e. phenomena where a small change in a system parameter causes qualitatively different behaviour. Bifurcation theory has become a rich field of research in its own right and evaluating the bifurcation behaviour of a given dynamical system can be challenging. An even greater challenge, however, is to learn the bifurcation structure of dynamical systems from data, where the precise model structure is not known. Here, we study one aspects of this problem: the practical implications that the presence of bifurcations has on our ability to infer model parameters and initial conditions from empirical data; we focus on the canonical co-dimension 1 bifurcations and provide a comprehensive analysis of how dynamics, and our ability to infer kinetic parameters are linked. The picture thus emerging is surprisingly nuanced and suggests that identification of the qualitative dynamics-the bifurcation diagram-should precede any attempt at inferring kinetic parameters.
Collapse
|
8
|
Golightly A, Bradley E, Lowe T, Gillespie CS. Correlated pseudo-marginal schemes for time-discretised stochastic kinetic models. Comput Stat Data Anal 2019. [DOI: 10.1016/j.csda.2019.01.006] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
|
9
|
Gillespie CS, Golightly A. Guided proposals for efficient weighted stochastic simulation. J Chem Phys 2019; 150:224103. [DOI: 10.1063/1.5090979] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
Affiliation(s)
- Colin S. Gillespie
- School of Mathematics, Statistics and Physics, Newcastle University, Newcastle upon Tyne NE1 7RU, United Kingdom
| | - Andrew Golightly
- School of Mathematics, Statistics and Physics, Newcastle University, Newcastle upon Tyne NE1 7RU, United Kingdom
| |
Collapse
|
10
|
Filippi S, Barnes CP, Kirk PDW, Kudo T, Kunida K, McMahon SS, Tsuchiya T, Wada T, Kuroda S, Stumpf MPH. Robustness of MEK-ERK Dynamics and Origins of Cell-to-Cell Variability in MAPK Signaling. Cell Rep 2016; 15:2524-35. [PMID: 27264188 PMCID: PMC4914773 DOI: 10.1016/j.celrep.2016.05.024] [Citation(s) in RCA: 33] [Impact Index Per Article: 4.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2016] [Revised: 04/25/2016] [Accepted: 05/04/2016] [Indexed: 12/27/2022] Open
Abstract
Cellular signaling processes can exhibit pronounced cell-to-cell variability in genetically identical cells. This affects how individual cells respond differentially to the same environmental stimulus. However, the origins of cell-to-cell variability in cellular signaling systems remain poorly understood. Here, we measure the dynamics of phosphorylated MEK and ERK across cell populations and quantify the levels of population heterogeneity over time using high-throughput image cytometry. We use a statistical modeling framework to show that extrinsic noise, particularly that from upstream MEK, is the dominant factor causing cell-to-cell variability in ERK phosphorylation, rather than stochasticity in the phosphorylation/dephosphorylation of ERK. We furthermore show that without extrinsic noise in the core module, variable (including noisy) signals would be faithfully reproduced downstream, but the within-module extrinsic variability distorts these signals and leads to a drastic reduction in the mutual information between incoming signal and ERK activity. Active MEK and ERK levels differ profoundly among genetically identical cells A statistical framework is developed to identify the causes of this variability Analysis shows that extrinsic noise upstream MEK-ERK module causes cell variability Within-module extrinsic variability distorts signals
Collapse
Affiliation(s)
- Sarah Filippi
- Centre for Integrative Systems Biology and Bioinformatics, Imperial College London, London SW7 2AZ, UK
| | - Chris P Barnes
- Department of Cell and Developmental Biology, University College London, London WC1E 6BT, UK; Department of Genetics, Evolution and Environment, University College London, London WC1E 6BT, UK
| | - Paul D W Kirk
- Centre for Integrative Systems Biology and Bioinformatics, Imperial College London, London SW7 2AZ, UK
| | - Takamasa Kudo
- Department of Biological Sciences, Graduate School of Science, University of Tokyo, Tokyo 113-8654, Japan
| | - Katsuyuki Kunida
- Department of Biological Sciences, Graduate School of Science, University of Tokyo, Tokyo 113-8654, Japan
| | - Siobhan S McMahon
- Centre for Integrative Systems Biology and Bioinformatics, Imperial College London, London SW7 2AZ, UK
| | - Takaho Tsuchiya
- Department of Biological Sciences, Graduate School of Science, University of Tokyo, Tokyo 113-8654, Japan
| | - Takumi Wada
- Department of Biological Sciences, Graduate School of Science, University of Tokyo, Tokyo 113-8654, Japan
| | - Shinya Kuroda
- Department of Biological Sciences, Graduate School of Science, University of Tokyo, Tokyo 113-8654, Japan; CREST, Japan Science and Technology Agency, Bunkyo-ku, Tokyo 113-0033, Japan
| | - Michael P H Stumpf
- Centre for Integrative Systems Biology and Bioinformatics, Imperial College London, London SW7 2AZ, UK; Institute of Chemical Biology, Imperial College London, London SW7 2AZ, UK.
| |
Collapse
|
11
|
Miasojedow B, Niemiro W. Geometric ergodicity of Rao and Teh’s algorithm for homogeneous Markov jump processes. Stat Probab Lett 2016. [DOI: 10.1016/j.spl.2016.02.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
|