Data-adaptive harmonic spectra and multilayer Stuart-Landau models

A Martingale-Free Introduction to Conditional Gaussian Nonlinear Systems

The conditional Gaussian nonlinear system (CGNS) is a broad class of nonlinear stochastic dynamical systems. Given the trajectories for a subset of state variables, the remaining follow a Gaussian distribution. Despite the conditionally linear structure, the CGNS exhibits strong nonlinearity, thus capturing many non-Gaussian characteristics observed in nature through its joint and marginal distributions. Desirably, it enjoys closed analytic formulae for the time evolution of its conditional Gaussian statistics, which facilitate the study of data assimilation and other related topics. In this paper, we develop a martingale-free approach to improve the understanding of CGNSs. This methodology provides a tractable approach to proving the time evolution of the conditional statistics by deriving results through time discretization schemes, with the continuous-time regime obtained via a formal limiting process as the discretization time-step vanishes. This discretized approach further allows for developing analytic formulae for optimal posterior sampling of unobserved state variables with correlated noise. These tools are particularly valuable for studying extreme events and intermittency and apply to high-dimensional systems. Moreover, the approach improves the understanding of different sampling methods in characterizing uncertainty. The effectiveness of the framework is demonstrated through a physics-constrained, triad-interaction climate model with cubic nonlinearity and state-dependent cross-interacting noise.

Conditional Gaussian nonlinear system: A fast preconditioner and a cheap surrogate model for complex nonlinear systems

Developing suitable approximate models for analyzing and simulating complex nonlinear systems is practically important. This paper aims at exploring the skill of a rich class of nonlinear stochastic models, known as the conditional Gaussian nonlinear system (CGNS), as both a cheap surrogate model and a fast preconditioner for facilitating many computationally challenging tasks. The CGNS preserves the underlying physics to a large extent and can reproduce intermittency, extreme events, and other non-Gaussian features in many complex systems arising from practical applications. Three interrelated topics are studied. First, the closed analytic formulas of solving the conditional statistics provide an efficient and accurate data assimilation scheme. It is shown that the data assimilation skill of a suitable CGNS approximate forecast model outweighs that by applying an ensemble method even to the perfect model with strong nonlinearity, where the latter suffers from filter divergence. Second, the CGNS allows the development of a fast algorithm for simultaneously estimating the parameters and the unobserved variables with uncertainty quantification in the presence of only partial observations. Utilizing an appropriate CGNS as a preconditioner significantly reduces the computational cost in accurately estimating the parameters in the original complex system. Finally, the CGNS advances rapid and statistically accurate algorithms for computing the probability density function and sampling the trajectories of the unobserved state variables. These fast algorithms facilitate the development of an efficient and accurate data-driven method for predicting the linear response of the original system with respect to parameter perturbations based on a suitable CGNS preconditioner.

Data-driven stochastic model for cross-interacting processes with different time scales

In this work, we propose a new data-driven method for modeling cross-interacting processes with different time scales represented by time series with different sampling steps. It is a generalization of a nonlinear stochastic model of an evolution operator based on neural networks and designed for the case of time series with a constant sampling step. The proposed model has a more complex structure. First, it describes each process by its own stochastic evolution operator with its own time step. Second, it takes into account possible nonlinear connections within each pair of processes in both directions. These connections are parameterized asymmetrically, depending on which process is faster and which process is slower. They make this model essentially different from the set of independent stochastic models constructed individually for each time scale. All evolution operators and connections are trained and optimized using the Bayesian framework, forming a multi-scale stochastic model. We demonstrate the performance of the model on two examples. The first example is a pair of coupled oscillators, with the couplings in both directions which can be turned on and off. Here, we show that inclusion of the connections into the model allows us to correctly reproduce observable effects related to coupling. The second example is a spatially distributed data generated by a global climate model running in the middle 19th century external conditions. In this case, the multi-scale model allows us to reproduce the coupling between the processes which exists in the observed data but is not captured by the model constructed individually for each process.

Stochastic rectification of fast oscillations on slow manifold closures

Slow–fast systems arise in many scientific applications, in particular in atmospheric and oceanic flows with fast inertia–gravity waves and slow geostrophic motions. When the slow and fast variables are strongly coupled—symptomatic of breakdown of slow-to-fast scales deterministic parameterizations—it remains a challenge to derive reduced systems able to capture the dynamics. Here, generic ingredients for successful reduction of such systems are identified and illustrated for the paradigmatic atmospheric Lorenz 80 model. The approach relies on a filtering operated through a nonlinear parameterization that separates the full dynamics into its slow motion and fast residual dynamics. The latter is mainly orthogonal to the former and is modeled via networks of stochastic nonlinear oscillators, independent of the slow dynamics.

The problems of identifying the slow component (e.g., for weather forecast initialization) and of characterizing slow–fast interactions are central to geophysical fluid dynamics. In this study, the related rectification problem of slow manifold closures is addressed when breakdown of slow-to-fast scales deterministic parameterizations occurs due to explosive emergence of fast oscillations on the slow, geostrophic motion. For such regimes, it is shown on the Lorenz 80 model that if 1) the underlying manifold provides a good approximation of the optimal nonlinear parameterization that averages out the fast variables and 2) the residual dynamics off this manifold is mainly orthogonal to it, then no memory terms are required in the Mori–Zwanzig full closure. Instead, the noise term is key to resolve, and is shown to be, in this case, well modeled by a state-independent noise, obtained by means of networks of stochastic nonlinear oscillators. This stochastic parameterization allows, in turn, for rectifying the momentum-balanced slow manifold, and for accurate recovery of the multiscale dynamics. The approach is promising to be further applied to the closure of other more complex slow–fast systems, in strongly coupled regimes.

Harmonic cross-correlation decomposition for multivariate time series

We introduce harmonic cross-correlation decomposition (HCD) as a tool to detect and visualize features in the frequency structure of multivariate time series. HCD decomposes multivariate time series into spatiotemporal harmonic modes with the leading modes representing dominant oscillatory patterns in the data. HCD is closely related to data-adaptive harmonic decomposition (DAHD) [Chekroun and Kondrashov, Chaos 27, 093110 (2017)10.1063/1.4989400] in that it performs an eigendecomposition of a grand matrix containing lagged cross-correlations. As for DAHD, each HCD mode is uniquely associated with a Fourier frequency, which allows for the definition of multidimensional power and phase spectra. Unlike in DAHD, however, HCD does not exhibit a systematic dependency on the ordering of the channels within the grand matrix. Further, HCD phase spectra can be related to the phase relations in the data in an intuitive way. We compare HCD with DAHD and multivariate singular spectrum analysis, a third related correlation-based decomposition, and we give illustrative applications to a simple traveling wave, as well as to simulations of three coupled Stuart-Landau oscillators and to human EEG recordings.

Reduced-order models for coupled dynamical systems: Data-driven methods and the Koopman operator

Providing efficient and accurate parameterizations for model reduction is a key goal in many areas of science and technology. Here, we present a strong link between data-driven and theoretical approaches to achieving this goal. Formal perturbation expansions of the Koopman operator allow us to derive general stochastic parameterizations of weakly coupled dynamical systems. Such parameterizations yield a set of stochastic integrodifferential equations with explicit noise and memory kernel formulas to describe the effects of unresolved variables. We show that the perturbation expansions involved need not be truncated when the coupling is additive. The unwieldy integrodifferential equations can be recast as a simpler multilevel Markovian model, and we establish an intuitive connection with a generalized Langevin equation. This connection helps setting up a parallelism between the top-down, equation-based methodology herein and the well-established empirical model reduction (EMR) methodology that has been shown to provide efficient dynamical closures to partially observed systems. Hence, our findings, on the one hand, support the physical basis and robustness of the EMR methodology and, on the other hand, illustrate the practical relevance of the perturbative expansion used for deriving the parameterizations.

Analysis of 20th century surface air temperature using linear dynamical modes

A Bayesian Linear Dynamical Mode (LDM) decomposition method is applied to isolate robust modes of climate variability in the observed surface air temperature (SAT) field. This decomposition finds the optimal number of internal modes characterized by their own time scales, which enter the cost function through a specific choice of prior probabilities. The forced climate response, with time dependence estimated from state-of-the-art climate-model simulations, is also incorporated in the present LDM decomposition and shown to increase its optimality from a Bayesian standpoint. On top of the forced signal, the decomposition identifies five distinct LDMs of internal climate variability. The first three modes exhibit multidecadal scales, while the remaining two modes are attributable to interannual-to-decadal variability associated with El Niño-Southern oscillation; all of these modes contribute to the secular climate signal-the so-called global stadium wave-missing in the climate-model simulations. One of the multidecadal LDMs is associated with Atlantic multidecadal oscillation. The two remaining slow modes have secular time scales and patterns exhibiting regional-to-global similarities to the forced-signal pattern. These patterns have a global scale and contribute significantly to SAT variability over the Southern and Pacific Oceans. In combination with low-frequency modulation of the fast LDMs, they explain the vast majority of the variability associated with interdecadal Pacific oscillation. The global teleconnectivity of the secular climate modes and their possible crucial role in shaping the forced climate response are the two key dynamical questions brought about by the present analysis.

Data-Driven Model Reduction for Stochastic Burgers Equations

We present a class of efficient parametric closure models for 1D stochastic Burgers equations. Casting it as statistical learning of the flow map, we derive the parametric form by representing the unresolved high wavenumber Fourier modes as functionals of the resolved variable’s trajectory. The reduced models are nonlinear autoregression (NAR) time series models, with coefficients estimated from data by least squares. The NAR models can accurately reproduce the energy spectrum, the invariant densities, and the autocorrelations. Taking advantage of the simplicity of the NAR models, we investigate maximal space-time reduction. Reduction in space dimension is unlimited, and NAR models with two Fourier modes can perform well. The NAR model’s stability limits time reduction, with a maximal time step smaller than that of the K-mode Galerkin system. We report a potential criterion for optimal space-time reduction: the NAR models achieve minimal relative error in the energy spectrum at the time step, where the K-mode Galerkin system’s mean Courant–Friedrichs–Lewy (CFL) number agrees with that of the full model.

Data-adaptive harmonic analysis of oceanic waves and turbulent flows

We introduce new features of data-adaptive harmonic decomposition (DAHD) that are showcased to characterize spatiotemporal variability in high-dimensional datasets of complex and mutsicale oceanic flows, offering new perspectives and novel insights. First, we present a didactic example with synthetic data for identification of coherent oceanic waves embedded in high amplitude noise. Then, DAHD is applied to analyze turbulent oceanic flows simulated by the Regional Oceanic Modeling System and an eddy-resolving three-layer quasigeostrophic ocean model, where resulting spectra exhibit a thin line capturing nearly all the energy at a given temporal frequency and showing well-defined scaling behavior across frequencies. DAHD thus permits sparse representation of complex, multiscale, and chaotic dynamics by a relatively few data-inferred spatial patterns evolving with simple temporal dynamics, namely, oscillating harmonically in time at a given single frequency. The detection of this low-rank behavior is facilitated by an eigendecomposition of the Hermitian cross-spectral matrix and resulting eigenvectors that represent an orthonormal set of global spatiotemporal modes associated with a specific temporal frequency, which in turn allows to rank these modes by their captured energy and across frequencies, and allow accurate space-time reconstruction. Furthermore, by using a correlogram estimator of the Hermitian cross-spectral density matrix, DAHD is both closely related and distinctly different from the spectral proper orthogonal decomposition that relies on Welch's periodogram as its estimator method.

Efficient reduction for diagnosing Hopf bifurcation in delay differential systems: Applications to cloud-rain models

By means of Galerkin-Koornwinder (GK) approximations, an efficient reduction approach to the Stuart-Landau (SL) normal form and center manifold is presented for a broad class of nonlinear systems of delay differential equations that covers the cases of discrete as well as distributed delays. The focus is on the Hopf bifurcation as a consequence of the critical equilibrium's destabilization resulting from an eigenpair crossing the imaginary axis. The nature of the resulting Hopf bifurcation (super- or subcritical) is then characterized by the inspection of a Lyapunov coefficient easy to determine based on the model's coefficients and delay parameters. We believe that our approach, which does not rely too much on functional analysis considerations but more on analytic calculations, is suitable to concrete situations arising in physics applications. Thus, using this GK approach to the Lyapunov coefficient and the SL normal form, the occurrence of Hopf bifurcations in the cloud-rain delay models of Koren and Feingold (KF) on one hand and Koren, Tziperman, and Feingold on the other are analyzed. Noteworthy is the existence of the KF model of large regions of the parameter space for which subcritical and supercritical Hopf bifurcations coexist. These regions are determined, in particular, by the intensity of the KF model's nonlinear effects. "Islands" of supercritical Hopf bifurcations are shown to exist within a subcritical Hopf bifurcation "sea"; these islands being bordered by double-Hopf bifurcations occurring when the linearized dynamics at the critical equilibrium exhibit two pairs of purely imaginary eigenvalues.

Introduction to focus issue: Synchronization in large networks and continuous media-data, models, and supermodels

The synchronization of loosely coupled chaotic systems has increasingly found applications to large networks of differential equations and to models of continuous media. These applications are at the core of the present Focus Issue. Synchronization between a system and its model, based on limited observations, gives a new perspective on data assimilation. Synchronization among different models of the same system defines a supermodel that can achieve partial consensus among models that otherwise disagree in several respects. Finally, novel methods of time series analysis permit a better description of synchronization in a system that is only observed partially and for a relatively short time. This Focus Issue discusses synchronization in extended systems or in components thereof, with particular attention to data assimilation, supermodeling, and their applications to various areas, from climate modeling to macroeconomics.