A unifying view of synchronization for data assimilation in complex nonlinear networks

Master-slave coupling scheme for synchronization and parameter estimation in the generalized Kuramoto-Sivashinsky equation

The problem of estimating the constant parameters of the Kuramoto-Sivashinsky (KS) equation from observed data has received attention from researchers in physics, applied mathematics, and statistics. This is motivated by the various physical applications of the equation and also because it often serves as a test model for the study of space-time pattern formation. Remarkably, most existing inference techniques rely on statistical tools, which are computationally very costly yet do not exploit the dynamical features of the system. In this paper, we introduce a simple, online parameter estimation method that relies on the synchronization properties of the KS equation. In particular, we describe a master-slave setup where the slave model is driven by observations from the master system. The slave dynamics are data-driven and designed to continuously adapt the model parameters until identical synchronization with the master system is achieved. We provide a simple analysis that supports the proposed approach and also present and discuss the results of an extensive set of computer simulations. Our numerical study shows that the proposed method is computationally fast and also robust to initialization errors, observational noise, and variations in the spatial resolution of the numerical scheme used to integrate the KS equation.

Predicting the Behavior of Sparsely-Sampled Systems Across Neurobiology and Epidemiology

Inference is a term that encompasses many techniques including statistical data assimilation (SDA). Unlike machine learning, which is designed to harness predictive power from extremely large data sets, SDA is designed for sparsely-sampled systems. This is the realm of study of nonlinear dynamical systems in nature. Formulated as an optimization procedure, SDA can be considered a path-integral approach to state and parameter estimation. Within this formulation, we can use the physical principle of least action to identify optimal solutions: solutions that are consistent with both measurements and a dynamical model assumed to give rise to those measurements. I review examples from neurobiology and an epidemiological model tailored to the coronavirus SARS-CoV-2, to demonstrate the versatility of SDA across the sciences, and how these distinct applications possess commonalities that can inform one another.

Asymptotic behavior of the forecast-assimilation process with unstable dynamics

Extensive numerical evidence shows that the assimilation of observations has a stabilizing effect on unstable dynamics, in numerical weather prediction, and elsewhere. In this paper, we apply mathematically rigorous methods to show why this is so. Our stabilization results do not assume a full set of observations and we provide examples where it suffices to observe the model's unstable degrees of freedom.

Optimal control methods for nonlinear parameter estimation in biophysical neuron models

Functional forms of biophysically-realistic neuron models are constrained by neurobiological and anatomical considerations, such as cell morphologies and the presence of known ion channels. Despite these constraints, neuron models still contain unknown static parameters which must be inferred from experiment. This inference task is most readily cast into the framework of state-space models, which systematically takes into account partial observability and measurement noise. Inferring only dynamical state variables such as membrane voltages is a well-studied problem, and has been approached with a wide range of techniques beginning with the well-known Kalman filter. Inferring both states and fixed parameters, on the other hand, is less straightforward. Here, we develop a method for joint parameter and state inference that combines traditional state space modeling with chaotic synchronization and optimal control. Our methods are tailored particularly to situations with considerable measurement noise, sparse observability, very nonlinear or chaotic dynamics, and highly uninformed priors. We illustrate our approach both in a canonical chaotic model and in a phenomenological neuron model, showing that many unknown parameters can be uncovered reliably and accurately from short and noisy observed time traces. Our method holds promise for estimation in larger-scale systems, given ongoing improvements in calcium reporters and genetically-encoded voltage indicators.

Systems neuroscience aims to understand how individual neurons and neural networks process external stimuli into behavioral responses. Underlying this characterization are mathematical models intimately shaped by experimental observations. But neural systems are high-dimensional and contain highly nonlinear interactions, so developing accurate models remains a challenge given current experimental capabilities. In practice, this means that the dynamical equations characterizing neural activity have many unknown parameters, and these parameters must be inferred from data. This inference problem is nontrivial owing to model nonlinearity, system and measurement noise, and the sparsity of observations from electrode recordings. Here, we present a novel method for inferring model parameters of neural systems. Our technique combines ideas from control theory and optimization, and amounts to using data to “control” estimates toward the best fit. Our method compares well in accuracy against other state-of-the-art inference methods, both in phenomenological chaotic systems and biophysical neuron models. Our work shows that many unknown model parameters of interest can be inferred from voltage measurements, despite signaling noise, instrument noise, and low observability.

Model selection of chaotic systems from data with hidden variables using sparse data assimilation

Many natural systems exhibit chaotic behavior, including the weather, hydrology, neuroscience, and population dynamics. Although many chaotic systems can be described by relatively simple dynamical equations, characterizing these systems can be challenging due to sensitivity to initial conditions and difficulties in differentiating chaotic behavior from noise. Ideally, one wishes to find a parsimonious set of equations that describe a dynamical system. However, model selection is more challenging when only a subset of the variables are experimentally accessible. Manifold learning methods using time-delay embeddings can successfully reconstruct the underlying structure of the system from data with hidden variables, but not the equations. Recent work in sparse-optimization based model selection has enabled model discovery given a library of possible terms, but regression-based methods require measurements of all state variables. We present a method combining variational annealing-a technique previously used for parameter estimation in chaotic systems with hidden variables-with sparse-optimization methods to perform model identification for chaotic systems with unmeasured variables. We applied the method to ground-truth time-series simulated from the classic Lorenz system and experimental data from an electrical circuit with Lorenz-system like behavior. In both cases, we successfully recover the expected equations with two measured and one hidden variable. Application to simulated data from the Colpitts oscillator demonstrates successful model selection of terms within nonlinear functions. We discuss the robustness of our method to varying noise.

Intrinsic neuronal properties represent song and error in zebra finch vocal learning

Neurons regulate their intrinsic physiological properties, which could influence network properties and contribute to behavioral plasticity. Recording from adult zebra finch brain slices we show that within each bird basal ganglia Area X-projecting (HVCX) neurons share similar spike waveform morphology and timing of spike trains, with modeling indicating similar magnitudes of five principal ion currents. These properties vary among birds in lawful relation to acoustic similarity of the birds' songs, with adult sibling pairs (same songs) sharing similar waveforms and spiking characteristics. The properties are maintained dynamically: HVCX within juveniles learning to sing show variable properties, whereas the uniformity rapidly degrades within hours in adults singing while exposed to abnormal (delayed) auditory feedback. Thus, within individual birds the population of current magnitudes covary over the arc of development, while rapidly responding to changes in feedback (in adults). This identifies network interactions with intrinsic properties that affect information storage and processing of learned vocalizations.

Statistical data assimilation for estimating electrophysiology simultaneously with connectivity within a biological neuronal network

A method of data assimilation (DA) is employed to estimate electrophysiological parameters of neurons simultaneously with their synaptic connectivity in a small model biological network. The DA procedure is cast as an optimization, with a cost function consisting of both a measurement error and a model error term. An iterative reweighting of these terms permits a systematic method to identify the lowest minimum, within a local region of state space, on the surface of a nonconvex cost function. In the model, two sets of parameter values are associated with two particular functional modes of network activity: simultaneous firing of all neurons and a pattern-generating mode wherein the neurons burst in sequence. The DA procedure is able to recover these modes if: (i) the stimulating electrical currents have chaotic waveforms and (ii) the measurements consist of the membrane voltages of all neurons in the circuit. Further, this method is able to prune a model of unnecessarily high dimensionality to a representation that contains the maximum dimensionality required to reproduce the provided measurements. This paper offers a proof-of-concept that DA has the potential to inform laboratory designs for estimating properties in small and isolatable functional circuits.

Data Assimilation Methods for Neuronal State and Parameter Estimation

This tutorial illustrates the use of data assimilation algorithms to estimate unobserved variables and unknown parameters of conductance-based neuronal models. Modern data assimilation (DA) techniques are widely used in climate science and weather prediction, but have only recently begun to be applied in neuroscience. The two main classes of DA techniques are sequential methods and variational methods. We provide computer code implementing basic versions of a method from each class, the Unscented Kalman Filter and 4D-Var, and demonstrate how to use these algorithms to infer several parameters of the Morris-Lecar model from a single voltage trace. Depending on parameters, the Morris-Lecar model exhibits qualitatively different types of neuronal excitability due to changes in the underlying bifurcation structure. We show that when presented with voltage traces from each of the various excitability regimes, the DA methods can identify parameter sets that produce the correct bifurcation structure even with initial parameter guesses that correspond to a different excitability regime. This demonstrates the ability of DA techniques to perform nonlinear state and parameter estimation and introduces the geometric structure of inferred models as a novel qualitative measure of estimation success. We conclude by discussing extensions of these DA algorithms that have appeared in the neuroscience literature.

Introduction to focus issue: Synchronization in large networks and continuous media-data, models, and supermodels

The synchronization of loosely coupled chaotic systems has increasingly found applications to large networks of differential equations and to models of continuous media. These applications are at the core of the present Focus Issue. Synchronization between a system and its model, based on limited observations, gives a new perspective on data assimilation. Synchronization among different models of the same system defines a supermodel that can achieve partial consensus among models that otherwise disagree in several respects. Finally, novel methods of time series analysis permit a better description of synchronization in a system that is only observed partially and for a relatively short time. This Focus Issue discusses synchronization in extended systems or in components thereof, with particular attention to data assimilation, supermodeling, and their applications to various areas, from climate modeling to macroeconomics.