1
|
Tönsing C, Steiert B, Timmer J, Kreutz C. Likelihood-ratio test statistic for the finite-sample case in nonlinear ordinary differential equation models. PLoS Comput Biol 2023; 19:e1011417. [PMID: 37738254 PMCID: PMC10550180 DOI: 10.1371/journal.pcbi.1011417] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2023] [Revised: 10/04/2023] [Accepted: 08/08/2023] [Indexed: 09/24/2023] Open
Abstract
Likelihood ratios are frequently utilized as basis for statistical tests, for model selection criteria and for assessing parameter and prediction uncertainties, e.g. using the profile likelihood. However, translating these likelihood ratios into p-values or confidence intervals requires the exact form of the test statistic's distribution. The lack of knowledge about this distribution for nonlinear ordinary differential equation (ODE) models requires an approximation which assumes the so-called asymptotic setting, i.e. a sufficiently large amount of data. Since the amount of data from quantitative molecular biology is typically limited in applications, this finite-sample case regularly occurs for mechanistic models of dynamical systems, e.g. biochemical reaction networks or infectious disease models. Thus, it is unclear whether the standard approach of using statistical thresholds derived for the asymptotic large-sample setting in realistic applications results in valid conclusions. In this study, empirical likelihood ratios for parameters from 19 published nonlinear ODE benchmark models are investigated using a resampling approach for the original data designs. Their distributions are compared to the asymptotic approximation and statistical thresholds are checked for conservativeness. It turns out, that corrections of the likelihood ratios in such finite-sample applications are required in order to avoid anti-conservative results.
Collapse
Affiliation(s)
- Christian Tönsing
- Institute of Physics, University of Freiburg, Germany
- CIBSS Centre for Integrative Biological Signalling Studies, University of Freiburg, Germany
- FDM Freiburg Center for Data Analysis and Modeling, University of Freiburg, Germany
| | | | - Jens Timmer
- Institute of Physics, University of Freiburg, Germany
- CIBSS Centre for Integrative Biological Signalling Studies, University of Freiburg, Germany
- FDM Freiburg Center for Data Analysis and Modeling, University of Freiburg, Germany
| | - Clemens Kreutz
- CIBSS Centre for Integrative Biological Signalling Studies, University of Freiburg, Germany
- FDM Freiburg Center for Data Analysis and Modeling, University of Freiburg, Germany
- Faculty of Medicine and Medical Center, Institute of Medical Biometry and Statistics, University of Freiburg, Germany
| |
Collapse
|
2
|
Quinn KN, Abbott MC, Transtrum MK, Machta BB, Sethna JP. Information geometry for multiparameter models: new perspectives on the origin of simplicity. REPORTS ON PROGRESS IN PHYSICS. PHYSICAL SOCIETY (GREAT BRITAIN) 2022; 86:10.1088/1361-6633/aca6f8. [PMID: 36576176 PMCID: PMC10018491 DOI: 10.1088/1361-6633/aca6f8] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/03/2021] [Accepted: 11/29/2022] [Indexed: 05/20/2023]
Abstract
Complex models in physics, biology, economics, and engineering are oftensloppy, meaning that the model parameters are not well determined by the model predictions for collective behavior. Many parameter combinations can vary over decades without significant changes in the predictions. This review uses information geometry to explore sloppiness and its deep relation to emergent theories. We introduce themodel manifoldof predictions, whose coordinates are the model parameters. Itshyperribbonstructure explains why only a few parameter combinations matter for the behavior. We review recent rigorous results that connect the hierarchy of hyperribbon widths to approximation theory, and to the smoothness of model predictions under changes of the control variables. We discuss recent geodesic methods to find simpler models on nearby boundaries of the model manifold-emergent theories with fewer parameters that explain the behavior equally well. We discuss a Bayesian prior which optimizes the mutual information between model parameters and experimental data, naturally favoring points on the emergent boundary theories and thus simpler models. We introduce a 'projected maximum likelihood' prior that efficiently approximates this optimal prior, and contrast both to the poor behavior of the traditional Jeffreys prior. We discuss the way the renormalization group coarse-graining in statistical mechanics introduces a flow of the model manifold, and connect stiff and sloppy directions along the model manifold with relevant and irrelevant eigendirections of the renormalization group. Finally, we discuss recently developed 'intensive' embedding methods, allowing one to visualize the predictions of arbitrary probabilistic models as low-dimensional projections of an isometric embedding, and illustrate our method by generating the model manifold of the Ising model.
Collapse
Affiliation(s)
- Katherine N Quinn
- Center for the Physics of Biological Function, Princeton University, Princeton, NJ, United States of America
| | - Michael C Abbott
- Department of Physics, Yale University, New Haven, CT, United States of America
| | - Mark K Transtrum
- Department of Physics and Astronomy, Brigham Young University, Provo, UT, United States of America
| | - Benjamin B Machta
- Department of Physics and Systems Biology Institute, Yale University, New Haven, CT, United States of America
| | - James P Sethna
- Department of Physics, Cornell University, Ithaca, NY, United States of America
| |
Collapse
|
3
|
Sharp JA, Browning AP, Burrage K, Simpson MJ. Parameter estimation and uncertainty quantification using information geometry. J R Soc Interface 2022; 19:20210940. [PMID: 35472269 PMCID: PMC9042578 DOI: 10.1098/rsif.2021.0940] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022] Open
Abstract
In this work, we: (i) review likelihood-based inference for parameter estimation and the construction of confidence regions; and (ii) explore the use of techniques from information geometry, including geodesic curves and Riemann scalar curvature, to supplement typical techniques for uncertainty quantification, such as Bayesian methods, profile likelihood, asymptotic analysis and bootstrapping. These techniques from information geometry provide data-independent insights into uncertainty and identifiability, and can be used to inform data collection decisions. All code used in this work to implement the inference and information geometry techniques is available on GitHub.
Collapse
Affiliation(s)
- Jesse A Sharp
- School of Mathematical Sciences, Queensland University of Technology, Brisbane, Queensland, Australia.,ARC Centre of Excellence for Mathematical and Statistical Frontiers, Queensland University of Technology, Brisbane, Queensland, Australia
| | - Alexander P Browning
- School of Mathematical Sciences, Queensland University of Technology, Brisbane, Queensland, Australia.,ARC Centre of Excellence for Mathematical and Statistical Frontiers, Queensland University of Technology, Brisbane, Queensland, Australia
| | - Kevin Burrage
- School of Mathematical Sciences, Queensland University of Technology, Brisbane, Queensland, Australia.,ARC Centre of Excellence for Mathematical and Statistical Frontiers, Queensland University of Technology, Brisbane, Queensland, Australia.,Department of Computer Science, University of Oxford, Oxford, UK
| | - Matthew J Simpson
- School of Mathematical Sciences, Queensland University of Technology, Brisbane, Queensland, Australia.,Centre for Data Science, Queensland University of Technology, Brisbane, Queensland, Australia
| |
Collapse
|
4
|
Browning AP, Warne DJ, Burrage K, Baker RE, Simpson MJ. Identifiability analysis for stochastic differential equation models in systems biology. J R Soc Interface 2020; 17:20200652. [PMID: 33323054 PMCID: PMC7811582 DOI: 10.1098/rsif.2020.0652] [Citation(s) in RCA: 30] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2020] [Accepted: 11/24/2020] [Indexed: 12/26/2022] Open
Abstract
Mathematical models are routinely calibrated to experimental data, with goals ranging from building predictive models to quantifying parameters that cannot be measured. Whether or not reliable parameter estimates are obtainable from the available data can easily be overlooked. Such issues of parameter identifiability have important ramifications for both the predictive power of a model, and the mechanistic insight that can be obtained. Identifiability analysis is well-established for deterministic, ordinary differential equation (ODE) models, but there are no commonly adopted methods for analysing identifiability in stochastic models. We provide an accessible introduction to identifiability analysis and demonstrate how existing ideas for analysis of ODE models can be applied to stochastic differential equation (SDE) models through four practical case studies. To assess structural identifiability, we study ODEs that describe the statistical moments of the stochastic process using open-source software tools. Using practically motivated synthetic data and Markov chain Monte Carlo methods, we assess parameter identifiability in the context of available data. Our analysis shows that SDE models can often extract more information about parameters than deterministic descriptions. All code used to perform the analysis is available on Github.
Collapse
Affiliation(s)
- Alexander P. Browning
- School of Mathematical Sciences, Queensland University of Technology, Brisbane, Australia
- ARC Centre of Excellence for Mathematical and Statistical Frontiers, Queensland University of Technology, Brisbane, Australia
| | - David J. Warne
- School of Mathematical Sciences, Queensland University of Technology, Brisbane, Australia
- ARC Centre of Excellence for Mathematical and Statistical Frontiers, Queensland University of Technology, Brisbane, Australia
| | - Kevin Burrage
- School of Mathematical Sciences, Queensland University of Technology, Brisbane, Australia
- ARC Centre of Excellence for Mathematical and Statistical Frontiers, Queensland University of Technology, Brisbane, Australia
- ARC Centre of Excellence for Plant Success in Nature and Agriculture, Queensland University of Technology, Brisbane, Australia
- Department of Computer Science, University of Oxford, Oxford, UK
| | - Ruth E. Baker
- Mathematical Institute, University of Oxford, Oxford, UK
| | - Matthew J. Simpson
- School of Mathematical Sciences, Queensland University of Technology, Brisbane, Australia
- ARC Centre of Excellence for Mathematical and Statistical Frontiers, Queensland University of Technology, Brisbane, Australia
| |
Collapse
|