Constraining chaos: Enforcing dynamical invariants in the training of reservoir computers

Drawing on ergodic theory, we introduce a novel training method for machine learning based forecasting methods for chaotic dynamical systems. The training enforces dynamical invariants-such as the Lyapunov exponent spectrum and the fractal dimension-in the systems of interest, enabling longer and more stable forecasts when operating with limited data. The technique is demonstrated in detail using reservoir computing, a specific kind of recurrent neural network. Results are given for the Lorenz 1996 chaotic dynamical system and a spectral quasi-geostrophic model of the atmosphere, both typical test cases for numerical weather prediction.

A systematic exploration of reservoir computing for forecasting complex spatiotemporal dynamics

A reservoir computer (RC) is a type of recurrent neural network architecture with demonstrated success in the prediction of spatiotemporally chaotic dynamical systems. A further advantage of RC is that it reproduces intrinsic dynamical quantities essential for its incorporation into numerical forecasting routines such as the ensemble Kalman filter-used in numerical weather prediction to compensate for sparse and noisy data. We explore here the architecture and design choices for a "best in class" RC for a number of characteristic dynamical systems. Our analysis points to the importance of large scale parameter optimization. We also note in particular the importance of including input bias in the RC design, which has a significant impact on the forecast skill of the trained RC model. In our tests, the use of a nonlinear readout operator does not affect the forecast time or the stability of the forecast. The effects of the reservoir dimension, spinup time, amount of training data, normalization, noise, and the RC time step are also investigated. Finally, we detail how our investigation leads to optimal design choices for a parallel RC scheme applied to the 40 dimensional spatiotemporally chaotic Lorenz 1996 dynamics.

Robust forecasting using predictive generalized synchronization in reservoir computing

Reservoir computers (RCs) are a class of recurrent neural networks (RNNs) that can be used for forecasting the future of observed time series data. As with all RNNs, selecting the hyperparameters in the network to yield excellent forecasting presents a challenge when training on new inputs. We analyze a method based on predictive generalized synchronization (PGS) that gives direction in designing and evaluating the architecture and hyperparameters of an RC. To determine the occurrences of PGS, we rely on the auxiliary method to provide a computationally efficient pre-training test that guides hyperparameter selection. We provide a metric for evaluating the RC using the reproduction of the input system's Lyapunov exponents that demonstrates robustness in prediction.

Observational Needs for Improving Ocean and Coupled Reanalysis, S2S Prediction, and Decadal Prediction

Developments in observing system technologies and ocean data assimilation (DA) are symbiotic. New observation types lead to new DA methods and new DA methods, such as coupled DA, can change the value of existing observations or indicate where new observations can have greater utility for monitoring and prediction. Practitioners of DA are encouraged to make better use of observations that are already available, for example, taking advantage of strongly coupled DA so that ocean observations can be used to improve atmospheric analyses and vice versa. Ocean reanalyses are useful for the analysis of climate as well as the initialization of operational long-range prediction models. There are many remaining challenges for ocean reanalyses due to biases and abrupt changes in the ocean-observing system throughout its history, the presence of biases and drifts in models, and the simplifying assumptions made in DA solution methods. From a governance point of view, more support is needed to bring the ocean-observing and DA communities together. For prediction applications, there is wide agreement that protocols are needed for rapid communication of ocean-observing data on numerical weather prediction (NWP) timescales. There is potential for new observation types to enhance the observing system by supporting prediction on multiple timescales, ranging from the typical timescale of NWP, covering hours to weeks, out to multiple decades. Better communication between DA and observation communities is encouraged in order to allow operational prediction centers the ability to provide guidance for the design of a sustained and adaptive observing network.

Mathematical foundations of hybrid data assimilation from a synchronization perspective

The state-of-the-art data assimilation methods used today in operational weather prediction centers around the world can be classified as generalized one-way coupled impulsive synchronization. This classification permits the investigation of hybrid data assimilation methods, which combine dynamic error estimates of the system state with long time-averaged (climatological) error estimates, from a synchronization perspective. Illustrative results show how dynamically informed formulations of the coupling matrix (via an Ensemble Kalman Filter, EnKF) can lead to synchronization when observing networks are sparse and how hybrid methods can lead to synchronization when those dynamic formulations are inadequate (due to small ensemble sizes). A large-scale application with a global ocean general circulation model is also presented. Results indicate that the hybrid methods also have useful applications in generalized synchronization, in particular, for correcting systematic model errors.

Causality Analysis: Identifying the Leading Element in a Coupled Dynamical System

Physical systems with time-varying internal couplings are abundant in nature. While the full governing equations of these systems are typically unknown due to insufficient understanding of their internal mechanisms, there is often interest in determining the leading element. Here, the leading element is defined as the sub-system with the largest coupling coefficient averaged over a selected time span. Previously, the Convergent Cross Mapping (CCM) method has been employed to determine causality and dominant component in weakly coupled systems with constant coupling coefficients. In this study, CCM is applied to a pair of coupled Lorenz systems with time-varying coupling coefficients, exhibiting switching between dominant sub-systems in different periods. Four sets of numerical experiments are carried out. The first three cases consist of different coupling coefficient schemes: I) Periodic-constant, II) Normal, and III) Mixed Normal/Non-normal. In case IV, numerical experiment of cases II and III are repeated with imposed temporal uncertainties as well as additive normal noise. Our results show that, through detecting directional interactions, CCM identifies the leading sub-system in all cases except when the average coupling coefficients are approximately equal, i.e., when the dominant sub-system is not well defined.