Collocation based training of neural ordinary differential equations

Predicting Covid-19 pandemic waves with biologically and behaviorally informed universal differential equations

During the COVID-19 pandemic, it became clear that pandemic waves and population responses were locked in a mutual feedback loop in a classic example of a coupled behavior-disease system. We demonstrate for the first time that universal differential equation (UDE) models are able to extract this interplay from data. We develop a UDE model for COVID-19 and test its ability to make predictions of second pandemic waves. We find that UDEs are capable of learning coupled behavior-disease dynamics and predicting second waves in a variety of populations, provided they are supplied with learning biases describing simple assumptions about disease transmission and population response. Though not yet suitable for deployment as a policy-guiding tool, our results demonstrate potential benefits, drawbacks, and useful techniques when applying universal differential equations to coupled systems.

Assessing the performance of QSP models: biology as the driver for validation

Validation of a quantitative model is a critical step in establishing confidence in the model's suitability for whatever analysis it was designed. While processes for validation are well-established in the statistical sciences, the field of quantitative systems pharmacology (QSP) has taken a more piecemeal approach to defining and demonstrating validation. Although classical statistical methods can be used in a QSP context, proper validation of a mechanistic systems model requires a more nuanced approach to what precisely is being validated, and what role said validation plays in the larger context of the analysis. In this review, we summarize current thoughts of QSP validation in the scientific community, contrast the aims of statistical validation from several contexts (including inference, pharmacometrics analysis, and machine learning) with the challenges faced in QSP analysis, and use examples from published QSP models to define different stages or levels of validation, any of which may be sufficient depending on the context at hand.

The lost art of mathematical modelling

We provide a critique of mathematical biology in light of rapid developments in modern machine learning. We argue that out of the three modelling activities - (1) formulating models; (2) analysing models; and (3) fitting or comparing models to data - inherent to mathematical biology, researchers currently focus too much on activity (2) at the cost of (1). This trend, we propose, can be reversed by realising that any given biological phenomenon can be modelled in an infinite number of different ways, through the adoption of an pluralistic approach, where we view a system from multiple, different points of view. We explain this pluralistic approach using fish locomotion as a case study and illustrate some of the pitfalls - universalism, creating models of models, etc. - that hinder mathematical biology. We then ask how we might rediscover a lost art: that of creative mathematical modelling.

Interpretable polynomial neural ordinary differential equations

Neural networks have the ability to serve as universal function approximators, but they are not interpretable and do not generalize well outside of their training region. Both of these issues are problematic when trying to apply standard neural ordinary differential equations (ODEs) to dynamical systems. We introduce the polynomial neural ODE, which is a deep polynomial neural network inside of the neural ODE framework. We demonstrate the capability of polynomial neural ODEs to predict outside of the training region, as well as to perform direct symbolic regression without using additional tools such as SINDy.

Open problems in mathematical biology

Biology is data-rich, and it is equally rich in concepts and hypotheses. Part of trying to understand biological processes and systems is therefore to confront our ideas and hypotheses with data using statistical methods to determine the extent to which our hypotheses agree with reality. But doing so in a systematic way is becoming increasingly challenging as our hypotheses become more detailed, and our data becomes more complex. Mathematical methods are therefore gaining in importance across the life- and biomedical sciences. Mathematical models allow us to test our understanding, make testable predictions about future behaviour, and gain insights into how we can control the behaviour of biological systems. It has been argued that mathematical methods can be of great benefit to biologists to make sense of data. But mathematics and mathematicians are set to benefit equally from considering the often bewildering complexity inherent to living systems. Here we present a small selection of open problems and challenges in mathematical biology. We have chosen these open problems because they are of both biological and mathematical interest.

Inferring structural and dynamical properties of gene networks from data with deep learning

The reconstruction of gene regulatory networks (GRNs) from data is vital in systems biology. Although different approaches have been proposed to infer causality from data, some challenges remain, such as how to accurately infer the direction and type of interactions, how to deal with complex network involving multiple feedbacks, as well as how to infer causality between variables from real-world data, especially single cell data. Here, we tackle these problems by deep neural networks (DNNs). The underlying regulatory network for different systems (gene regulations, ecology, diseases, development) can be successfully reconstructed from trained DNN models. We show that DNN is superior to existing approaches including Boolean network, Random Forest and partial cross mapping for network inference. Further, by interrogating the ensemble DNN model trained from single cell data from dynamical system perspective, we are able to unravel complex cell fate dynamics during preimplantation development. We also propose a data-driven approach to quantify the energy landscape for gene regulatory systems, by combining DNN with the partial self-consistent mean field approximation (PSCA) approach. We anticipate the proposed method can be applied to other fields to decipher the underlying dynamical mechanisms of systems from data.