Learning mean-field equations from particle data using WSINDy

Random walk models in the life sciences: including births, deaths and local interactions

Random walks and related spatial stochastic models have been used in a range of application areas, including animal and plant ecology, infectious disease epidemiology, developmental biology, wound healing and oncology. Classical random walk models assume that all individuals in a population behave independently, ignoring local physical and biological interactions. This assumption simplifies the mathematical description of the population considerably, enabling continuum-limit descriptions to be derived and used in model analysis and fitting. However, interactions between individuals can have a crucial impact on population-level behaviour. In recent decades, research has increasingly been directed towards models that include interactions, including physical crowding effects and local biological processes such as adhesion, competition, dispersal, predation and adaptive directional bias. In this article, we review the progress that has been made with models of interacting individuals. We aim to provide an overview that is accessible to researchers in application areas, as well as to specialist modellers. We focus particularly on derivation of asymptotically exact or approximate continuum-limit descriptions and simplified deterministic models of mean-field behaviour and resulting spatial patterns. We provide worked examples and illustrative results of selected models. We conclude with a discussion of current areas of focus and future challenges.

Weak-form inference for hybrid dynamical systems in ecology

Species subject to predation and environmental threats commonly exhibit variable periods of population boom and bust over long timescales. Understanding and predicting such behaviour, especially given the inherent heterogeneity and stochasticity of exogenous driving factors over short timescales, is an ongoing challenge. A modelling paradigm gaining popularity in the ecological sciences for such multi-scale effects is to couple short-term continuous dynamics to long-term discrete updates. We develop a data-driven method utilizing weak-form equation learning to extract such hybrid governing equations for population dynamics and to estimate the requisite parameters using sparse intermittent measurements of the discrete and continuous variables. The method produces a set of short-term continuous dynamical system equations parametrized by long-term variables, and long-term discrete equations parametrized by short-term variables, allowing direct assessment of interdependencies between the two timescales. We demonstrate the utility of the method on a variety of ecological scenarios and provide extensive tests using models previously derived for epizootics experienced by the North American spongy moth (Lymantria dispar dispar).

Forecasting and Predicting Stochastic Agent-Based Model Data with Biologically-Informed Neural Networks

Collective migration is an important component of many biological processes, including wound healing, tumorigenesis, and embryo development. Spatial agent-based models (ABMs) are often used to model collective migration, but it is challenging to thoroughly predict these models' behavior throughout parameter space due to their random and computationally intensive nature. Modelers often coarse-grain ABM rules into mean-field differential equation (DE) models. While these DE models are fast to simulate, they suffer from poor (or even ill-posed) ABM predictions in some regions of parameter space. In this work, we describe how biologically-informed neural networks (BINNs) can be trained to learn interpretable BINN-guided DE models capable of accurately predicting ABM behavior. In particular, we show that BINN-guided partial DE (PDE) simulations can (1) forecast future spatial ABM data not seen during model training, and (2) predict ABM data at previously-unexplored parameter values. This latter task is achieved by combining BINN-guided PDE simulations with multivariate interpolation. We demonstrate our approach using three case study ABMs of collective migration that imitate cell biology experiments and find that BINN-guided PDEs accurately forecast and predict ABM data with a one-compartment PDE when the mean-field PDE is ill-posed or requires two compartments. This work suggests that BINN-guided PDEs allow modelers to efficiently explore parameter space, which may enable data-driven tasks for ABMs, such as estimating parameters from experimental data. All code and data from our study is available at https://github.com/johnnardini/Forecasting_predicting_ABMs .

Coarse-graining Hamiltonian systems using WSINDy

Weak form equation learning and surrogate modeling has proven to be computationally efficient and robust to measurement noise in a wide range of applications including ODE, PDE, and SDE discovery, as well as in coarse-graining applications, such as homogenization and mean-field descriptions of interacting particle systems. In this work we extend this coarse-graining capability to the setting of Hamiltonian dynamics which possess approximate symmetries associated with timescale separation. A smooth ε -dependent Hamiltonian vector field X ε possesses an approximate symmetry if the limiting vector fieldX 0 = lim ε → 0 X ε possesses an exact symmetry. Such approximate symmetries often lead to the existence of a Hamiltonian system of reduced dimension that may be used to efficiently capture the dynamics of the symmetry-invariant dependent variables. Deriving such reduced systems, or approximating them numerically, is an ongoing challenge. We demonstrate that WSINDy can successfully identify this reduced Hamiltonian system in the presence of large perturbations imparted in the ε > 0 regime, while remaining robust to extrinsic noise. This is significant in part due to the nontrivial means by which such systems are derived analytically. WSINDy naturally preserves the Hamiltonian structure by restricting to a trial basis of Hamiltonian vector fields. The methodology is computationally efficient, often requiring only a single trajectory to learn the global reduced Hamiltonian, and avoiding forward solves in the learning process. In this way, we argue that weak-form equation learning is particularly well-suited for Hamiltonian coarse-graining. Using nearly-periodic Hamiltonian systems as a prototypical class of systems with approximate symmetries, we show that WSINDy robustly identifies the correct leading-order system, with dimension reduced by at least two, upon observation of the relevant degrees of freedom. While our main contribution is computational, we also provide a contribution to the literature on averaging theory by proving that first-order averaging at the level of vector fields preserves Hamiltonian structure in nearly-periodic Hamiltonian systems. This provides theoretical justification for our approach as WSINDy's computations occur at the level of Hamiltonian vector fields. We illustrate the efficacy of our proposed method using physically relevant examples, including coupled oscillator dynamics, the Hénon-Heiles system for stellar motion within a galaxy, and the dynamics of charged particles.

Variable-moment fluid closures with Hamiltonian structure

Based on ideas due to Scovel-Weinstein, I present a general framework for constructing fluid moment closures of the Vlasov-Poisson system that exactly preserve that system's Hamiltonian structure. Notably, the technique applies in any space dimension and produces closures involving arbitrarily-large finite collections of moments. After selecting a desired collection of moments, the Poisson bracket for the closure is uniquely determined. Therefore data-driven fluid closures can be constructed by adjusting the closure Hamiltonian for compatibility with kinetic simulations.

Direct Estimation of Parameters in ODE Models Using WENDy: Weak-Form Estimation of Nonlinear Dynamics

We introduce the Weak-form Estimation of Nonlinear Dynamics (WENDy) method for estimating model parameters for non-linear systems of ODEs. Without relying on any numerical differential equation solvers, WENDy computes accurate estimates and is robust to large (biologically relevant) levels of measurement noise. For low dimensional systems with modest amounts of data, WENDy is competitive with conventional forward solver-based nonlinear least squares methods in terms of speed and accuracy. For both higher dimensional systems and stiff systems, WENDy is typically both faster (often by orders of magnitude) and more accurate than forward solver-based approaches. The core mathematical idea involves an efficient conversion of the strong form representation of a model to its weak form, and then solving a regression problem to perform parameter inference. The core statistical idea rests on the Errors-In-Variables framework, which necessitates the use of the iteratively reweighted least squares algorithm. Further improvements are obtained by using orthonormal test functions, created from a set of [Formula: see text] bump functions of varying support sizes.We demonstrate the high robustness and computational efficiency by applying WENDy to estimate parameters in some common models from population biology, neuroscience, and biochemistry, including logistic growth, Lotka-Volterra, FitzHugh-Nagumo, Hindmarsh-Rose, and a Protein Transduction Benchmark model. Software and code for reproducing the examples is available at https://github.com/MathBioCU/WENDy .

Learning anisotropic interaction rules from individual trajectories in a heterogeneous cellular population

Interacting particle system (IPS) models have proven to be highly successful for describing the spatial movement of organisms. However, it is challenging to infer the interaction rules directly from data. In the field of equation discovery, the weak-form sparse identification of nonlinear dynamics (WSINDy) methodology has been shown to be computationally efficient for identifying the governing equations of complex systems from noisy data. Motivated by the success of IPS models to describe the spatial movement of organisms, we develop WSINDy for the second-order IPS to learn equations for communities of cells. Our approach learns the directional interaction rules for each individual cell that in aggregate govern the dynamics of a heterogeneous population of migrating cells. To sort a cell according to the active classes present in its model, we also develop a novel ad hoc classification scheme (which accounts for the fact that some cells do not have enough evidence to accurately infer a model). Aggregated models are then constructed hierarchically to simultaneously identify different species of cells present in the population and determine best-fit models for each species. We demonstrate the efficiency and proficiency of the method on several test scenarios, motivated by common cell migration experiments.

Online Weak-form Sparse Identification of Partial Differential Equations

This paper presents an online algorithm for identification of partial differential equations (PDEs) based on the weak-form sparse identification of nonlinear dynamics algorithm (WSINDy). The algorithm is online in the sense that if performs the identification task by processing solution snapshots that arrive sequentially. The core of the method combines a weak-form discretization of candidate PDEs with an online proximal gradient descent approach to the sparse regression problem. In particular, we do not regularize the ℓ 0 -pseudo-norm, instead finding that directly applying its proximal operator (which corresponds to a hard thresholding) leads to efficient online system identification from noisy data. We demonstrate the success of the method on the Kuramoto-Sivashinsky equation, the nonlinear wave equation with time-varying wavespeed, and the linear wave equation, in one, two, and three spatial dimensions, respectively. In particular, our examples show that the method is capable of identifying and tracking systems with coefficients that vary abruptly in time, and offers a streaming alternative to problems in higher dimensions.